Working with files

Malcat is first and foremost a file analysis tool. It can Open a local file, Compare two files and supports multiple files open in parallel. It can also Open sub-files and objects, thanks to its numerous archive formats parsers and the use of a fast File carving. This chapter will guide you through most of these features.

Open a local file 

Malcat can open local files using different methods that are listed below. Note that by default, new files that you open are loaded inside a new tab/project (it’s not exactly a tab since a drop-down list is used, but it’s the same idea). Focus will be given to the new project, but the old one can still be accessed anytime (cf. Switch between files).

From the command line 

To open a file from the command line, just pass it as argument: malcat /path/to/file/name. It works under both Windows and Linux. It you want to Compare two files, you can pass both file paths as arguments, it will work too.

From the menu 

Using the menu File ‣ Open, you can open a file of your liking in Malcat. The file will be opened as a new project. If a file was previously opened, it will stay available, and you can come back to it anytime and Switch between files.

../_images/opencopy.png — Open copy of file dialog

Additionally, if you want to open a copy of file, for editing purpose for instance, you have the possibility to do so via the menu File ‣ Open copy of File…. Using this menu, you can specify the name of the file copy, and ask Malcat to prepend or append blank space to the beginning/end of the file if you wish so. This is sometimes useful when you need to have extra space to deobfuscate some data or reassemble buffers.

Recent files 

When run without any argument, Malcat display by default a welcome screen displaying the 10 last opened files. Double-clicking any of these files will re-open them into Malcat.

../_images/first.png — The recent files list

Drag and drop 

Almost all of Malcat’s user interface surface supports drag and drop, with the exception a few controls (mostly the script editor window and the Yara editor window). Drag and dropping a file in Malcat will open it as if you used the menu. This works on Windows and Linux.

Big file mode 

Malcat will automatically display a Big File Mode dialog when you open very large local files (threshold can be configured in Edit ‣ Preferences ‣ General ‣ Big File Mode threshold, defaults to 256 Mb).

../_images/bfm.png — Big file mode dialog

In Big File Mode, arbitrarily large files can be open using memory-mapping. The trick is that file is open in read-only mode and we use a MAP_PRIVATE or WRITE_COPY memory-mapping. It means that modifications to the file are not file-backed but written in newly allocated memory. The software is in control of saving modified bytes. But since we opened the file in read-only mode, modifications have to be saved to another file. That’s a small price to pay (one does not modify 1Gb+ files every day) to support arbitrarily large files.

../_images/bfm_cons.png — CPU and memory consumption on big files

The dialog additionally invites you to deactivate the most time-consuming analyses in order to speed-up the user experience. Doing file carving or scanning for cross-references on a 5Gb+ file is going to be somewhat slow no matter what. If you deactivate everything, Malcat will change to a somewhat basic hexadecimal editor, but opening the file will be instant.

Compare two files 

Diff algorithms 

Malcat can be pretty useful when it comes to comparing binary files. You can compare two files by:

Drag and dropping them anywhere on Malcat’s window
Passing two paths on the command line
Using the menu File ‣ Compare against … when a file is already open in Malcat

You will be greeted by a dialog where you can set some diffing options. The most important one is which diff algorithm to use. Malcat implements two of them:

A naive but very fast byte-by-byte comparison
Myers’s algorithm, an algorithm used in bioinformatics to compare DNA

While the naive algorithm may have it uses from time to time, it is basically worthless if you are facing data inserts and/or data deletion, since the two data flows won’t be aligned anymore and comparison will make little sense. That’s when Myers algorithm comes in play. This algorithm tries to align the files on the biggest common sequences and computes the data insertions/deletions/modifications accordingly.

The only drawback of Myers algorithm is its complexity: O(n²), which makes it useless on non-trivial binary files. To offset this issue, Malcat implements a diff window: only this amount of data (by default 64K) will be compared at a time. What it means is that if two similar blocks are separated by more than 64K bytes (after realignments), Malcat’s diff won’t notice their similarity.

Note

Malcat makes some cheap optimisations during diffing, like common prefix and common suffix search, so for this worst case scenario to happen, the two similar blocks need to be separated by 64k+ bytes and some other differences.

In practice, this worst case scenario is very rare when diffing programs, and mainly happens for files which are very much different.

Additional options 

Additional options are offered to you when diffing files. Beside the diff window length option which has a direct impact on the accuracy of Myers algorithm, the other options are post-filtering options: they are used to remove or hide differences from the final result:

Ignore differences for jumps/calls: small diffs which are inside the encoding of a jump or call instructions are ignored (helps with code relocation)
Ignore differences in pointers: small diffs which are inside any instruction and which are valid virtual addresses are ignored (helps with code relocation)
Ignore differences in registers/locals: if a register or a local is encoded in a byte different than an instruction opcode, differences on this single byte will be ignored (helps against obfuscation)
Ignore differences smaller than: Ignore small differences, improves readability of the overall diff result.

Note that these options can’t be changed afterwards, you would have to re-diff the files to change them.

Navigate the differences 

Once you’ve validated the diff options, diffing will take place. Like any other analysis, its progress will be visible in the status bar. And when it’s finished, a few things will happen:

The left pane of the User interface will be hidden in order to make some horizontal space (can be toggled back using the button if needed)
Two view panels will be shown side by side
A new diff annotation will be available in the Hexadecimal view, the Structure/text view and the Disassembly view
All other highlighting are deactivated by default (but can be reactivated if needed, cf. Highlighting)

In diff mode, both view panels are synchronized: if you switch to the Hexadecimal view for instance, it will impact both file views. Scrolling is also synchronized by default to take realignment into account, like in any text-based diff program. This behavior can be changed using the menu View ‣ Synchronize diff views. If deactivated, you will be able to scroll both views independently.

../_images/diff.png — Diffing two files using the text/structure view

Also, the three content views (the Hexadecimal view, the Structure/text view and the Disassembly view) are now displaying diff informations. The start symbol (*) denotes a modification, while a plus symbol (+) denotes an insertion. You can quickly jumps between differences using the following shortcuts:

Shortcut	Action
`d`	Goto next difference
`D`	Goto previous difference

Open sub-files and objects 

While Malcat can open multiple local files in parallel (cf. Open a local file), it can also open part of the current file, or an object, or a resource from within the current analyzed file into a new file. We will describe below what can be open and how.

Virtual files 

We call virtual files all sub-files found by Malcat’s File parsers during parsing of the main file. They differ from carved files in that they are explicitly referenced by one of the file format structures as a sub-file or resource. They can be stored compressed (a ZIP archive member for instance), encrypted (e.g. an AutoIT script) or in plain text (e.g. a PE resource).

Malcat will list all virtual files under the Virtual File System tab, located on the top-left corner of the User interface. If a proper unpacking / decryption method is supported by Malcat, the virtual file may also be opened inside Malcat by double-clicking it.

../_images/newtab.png — You can choose to open sub-files into the same project, or into a new project

By default, a virtual file will be opened in the current tab / project window, as if you opened a link in a browser. Closing the virtual file (Ctrl+W) will then bring you back to the root file (think of a browser back button). If you want to have both the root file and the sub-file visible at the same time, this may be an unwanted behavior. You can instead force the virtual file to open as a new tab / project by choosing Open (new tab) from the virtual file’s context menu. The file will be opened as a new project and you will be able to Switch between files using the project switcher (cf. User interface).

Carved files 

We call carved files all sub-files identified by Malcat’s File carving. These are files of known type (i.e recognized by one of the many File parsers) that were found via a simple linear scan (ala Binwalk). These files could be stored anywhere within the file, even if not referenced by a structure. They are always stored in plain text (otherwise we would not be able to detect them).

Malcat will list all carved files under the Carved Files tab, located on the top-left corner of the User interface. Clicking on a carved files brings you to the file’s location in either the Hexadecimal view or the Structure/text view.

../_images/carved.png — Two PNG files found inside a .ONE document

Double-clicking on a carved file opens it in the current tab / project window, as if you opened a link in a browser. Closing the carved file (Ctrl+W) will then bring you back to the root file (think of a browser back button). If you want to have both the root file and the sub-file visible at the same time, this may be an unwanted behavior.

../_images/dive.png — Opening sub files within the current project / tab

You can instead force the carved file to open as a new tab / project by choosing Open (new tab) from the carved file’s context menu. The file will be opened as a new project and you will be able to Switch between files using the project switcher (cf. User interface).

Selection 

Every byte range that you select in Malcat can be opened as a new file, either inside the current project via RightClick ‣ Open or into a new tab / project.

../_images/opensel.png — You can open any selected range as new file in Malcat

If you open the selection inside the current tab / project window, it would be as if you opened a link in a browser. Closing the selection (Ctrl+W) will then bring you back to the root file (think of a browser back button). If you want to have both the root file and the selection visible at the same time, this may be an unwanted behavior. You can instead force the selection to open as a new tab / project by choosing RightClick ‣ Open (new tab) from the selection context menu. The file will be opened as a new project and you will be able to Switch between files using the project switcher (cf. User interface).

Files generated from scripts 

Scripting in Malcat is pretty powerful and you can do many things using Malcat’s bindings. The user interface of Malcat also has a few bindings. By using the gui.open_after(data:bytes, name:str) method, scripts can open any arbitrary bytes buffer as a new file in the interface.

Note that the new file will be opened after the script has finished, hence the name, and only one file can be opened this way.

Switch between files 

Malcat can keep multiple files open at the same time. First, it can haven multiple projects or tabs open in parallel. Second, each project can open an arbitrary amount of sub files (cf. Open sub-files and objects), but only one at a time, in the same vein as a browser navigating a website on a single tab.

Switch between projects 

If you have multiple files projects open, you can use the address bar on the top of the window (cf. User interface) to change the currently displayed project / tab. This will hide the current project (but its analysis results will be kept in memory) in favor of the new one.

../_images/projects.png — The address bar allows you to easily switch between open files

Îf you click on the address bar drop-down list, it will display some basic info on each project:

the name of the current file. Note that if sub-files have been opened inside the project, the full hierarchy of files will be displayed (cf. screenshot)
the size of the current file
the sha256 of the current file
the type of the current file

Dive into or back from within a project 

You can Open sub-files and objects within the current project by clicking on a element of the Virtual File System entries or the Carved Files entries. The address bar will be updated in consequence ti display the full hierarchy of opened (sub) files. If you close the sub-file via Ctrl+W or using the icon left of the address bar, you’ll go back to the previous file in the hierarchy.

This is a quick and simple way to explore sub-files in Malcat.