“Highlighter 1.1.3 is the answer to all my prayers!” ~ Unnamed User

“Now that I have Highlighter 1.1.3, logs are basically analyzing themselves!” ~ Unnamed User

“Highlighter 1.1.2 was mind-blowing. Version 1.1.3 takes it to the next level!” ~Unnamed User

We have listened to your suggestions in the MANDIANT Forums on how to improve this tool and have worked hard to make it a prime source for rapid review of logs and other structured text files. Please take the time to head over to the forums yourself and tell us what you think!

Improvement:

• “Show Only” is no longer case sensitive

Fixes:

• Using “Remove” to remove all lines then reclaiming those lines causes crash
• If all lines are removed, right-click causes crash
• Crash using Show Only and Undo Show Only (thanks to youngba for reporting)
• Bug where an error message is displayed when using CTRL+C with no text selected (thanks to youngba for reporting)
• Bug where an error message is displayed when the Highlighter toolbar button is clicked with no text in the textbox or selected in the text window (thanks to youngba for reporting)
• Version number does not appear in installer
• In some cases, zooming and un-zooming quickly while using the right-click context menu would cause Making an SSL connection to a MIR controller to retrieve a document was failing.

I hope you head over to the MANDIANT Forums to let us know what you think about the updates as well as ways to continue improvement of this free tool.

Wipe the cheesy poofs off your fingers and go here to the download page to check out the updates.

We have listened to your suggestions in the MANDIANT Forums on how to improve this tool and have worked hard to make it a prime source for rapid review of logs and other structured text files. Please take the time to head over to the forums yourself and tell us what you think!

New Feature:

• Ability to change the display font. (Look in the menu under File -> Font.)

Improvements:

• Selecting text in the display will now more accurately line up with the mouse pointer.
• The display will now remain at the same point in the file after removing or restoring lines.

Fixes:

• Display refresh issues in Windows 7.
• In some cases, state files did not properly store and restore state.

I hope you head over to the MANDIANT Forums to let us know what you think about the updates as well as what we need to include next time.

Don’t forget that we’re relying on the user community to suggest improvements.  Check out http://forums.mandiant.com and head to the Highlighter section to give us your input.  Feedback, feature requests, bugs, and use-cases are all very welcome.

Change Log (since v1.0.1):

• Fix: Tabs were mistakenly removed by input sanitization. This has been corrected.
• Fix: The highlight hit count was incorrect – an additional hit per line was mistakenly being added. This has been corrected.
• Fix: The events over time histogram was not properly displaying highlights. This has been corrected.
• Fix: If text was selected in the textbox, and the user clicked on the highlight button, the selection would not be highlighted. This has been corrected.
• Enhancement: The graphic overview now draws much faster.
• New Feature: The textbox is now a 100% custom control. It is virtualized, and supports a wider range of visual display effects. When words are highlighted, the actual word on each line will be surrounded by a colored translucent bubble with a slightly darkened border. The textbox selection and scrolling behavior is now more like a traditional Windows textbox.
• New Feature: Highlighter will now open MUCH larger files. NOTE: Highlighter now keeps a file open while you are working with it.
• New Feature: Highlighter will now accept a list of terms, one on a line, as input to automatically highlight or remove lines. Look under the right click menu, Highlight -> Import Simple List and under Line Operations -> Remove Using Simple List.
• Enhancement: Files will now open somewhat more quickly due to optimization of calculating the MD5 sum of the file.
• Enhancement: The events over time histogram has sharper numbers on the X and Y axis.
• Fix: The events over time histogram scale now properly adjusts when when switching from linear to log mode.
• Fix: A number of State issues were resolved.
• Fix: Various other minor bugs.
• New Feature: Highlighter support opening a document from a Mandiant Intelligent Response (MIR) controller. Look for the new option from the File -> Open menu.
• New Feature: Highlighter will add a Windows Explorer shell extension by default.
• Fix: A number of State issues were resolved, including improper handling of when a selection included a comma.
• Fix: A race condition existed in the implementation of retrieving lines from the current file.
• Fix: Not all hotkeys were actually implemented in code.
• Fix: Highlight counts in the status bar were incorrect sometimes.
• Fix: Sometimes you could not scroll to the bottom of a file using the scrollbar.
• Fix: Events over time histogram had a very sparse appearance.
• Fix: After opening a file, you could not use hotkeys like CTRL-O to open files, nor could you do things like ALT-F4 or any other key sequence with modifiers.
• Fix: The remove feature would not remove lines with selections that contained a TAB.
• Fix: Various other minor bugs.

So I searched the Internet and asked colleagues in search of an application that would allow me to quickly remove lines from a text file. I wanted to be able to scroll through the file, and as I identified text that was irrelevant, remove lines from the display that contained that text. Sounds simple enough, right? But after searching for about a week, it seemed that no one knew of such a tool. Many suggested using a series of “grep -v” commands under Linux or with the Win32 Unix tools. Even though I am an avid command line user and a fan of using grep and Linux, that solution was a bit too clunky and not the sort of streamlined workflow I was looking for. A week more frustrated, I couldn’t find any app like the one I was searching for, so I decided I would have to make it myself.

Over two days I wrote a very basic C# application using Microsoft Visual Studio Express. The application had a single function – load a text file into a textbox, let me select text, and remove all lines with that text from being displayed. The original file was never modified, but they weren’t shown to me.

I used my new tool on a selection of the Windows event logs and immediately saw the benefit; with some files, this technique of removing lines quickly eliminated about 80-90% of the events. This let me focus closely on the remaining events, which allowed me to find evil and solve crime faster than ever!

After a little use I realized that thought it would be cool if I not only removed lines, but also found where certain strings occurred throughout a file. I started with the idea of statistical analysis on the file – generate information about each word that indicated frequency, distribution, etc. The problem with that is that I couldn’t come up with any good way to represent the results. After explaining the idea to my Mandiant colleague, Lindsey Lack, he simply said “I’m a graphical person. Why don’t you make a visual representation of the file and display information graphically?”. GENIUS!

Our idea was to depict the file as a graphic on which we could highlight areas on the graphic that corresponded to a key word or phrase. The depiction would immediately give you a sense for frequency and distribution. So with help from one of Mandiant’s Intelligent Response developers, Matt Frazier, we created a C# control that displays the file as a graphic. The graphic represents a sort of super zoomed-out version of the file. Lines from the original file are displayed as graphics lines (no text) on the screen. The lines displayed are proportional to the line lengths in the file. So you have a graphic on the screen next to the text box that proportionally represents the entire file. So, back to the Windows event logs.

See the line numbers jump in the text window. Hidden lines are indicated in the overview with grey lines.

I opened the log and selected a username in question identified through the previous analysis I did. I right-clicked and selected the new function – “Highlight”. The graphic lit up with small red lines (highlights), indicating each exact location that username appeared in the file. I immediately noticed something odd – the red highlights appeared in a fairly regular pattern, except around a certain spot, where there were a number of red highlights that just appeared out-of-place in comparison to the rest. We made the graphic clickable, so I clicked in that area and the textbox advanced to that portion of the file. The log entries that came up were very late at night – a time when this user should not have been accessing this system. Further investigation revealed the user’s account was compromised, malware was installed, and a number of other things happened that day.

The lines highlighted in the salmon color in the text box correspond with the yellow highlights in the overview window.

Evil found. Crimes being solved.

http://www.mandiant.com/software/highlighter.htm

http://mandiant.invisionzone.com/index.php?showforum=15