Web Historian: Reloaded
Written by Aaron LeMasters
We’ve been busy here on team agent at MANDIANT. In the spirit of our long-standing support of free software in the Incident Response community, we are happy to announce the release of Web Historian 2.0. This release is a complete rewrite and revamp of our very popular web history extraction tool. This version of Web Historian comes packed with features and supports Firefox 2/3+, Chrome 3+, and Internet Explorer versions 5 through 8. Here is a quick run-down of some of the new features:
- Collects web history, cookie history, file download history, and form history into data sets
- Simple/powerful UI based on tabbed organization of datasets
- Perform a live artifact scan of the local system
- Perform an artifact scan of one or more arbitrary history files from all supported browsers
- Import results from existing XML scan documents
- Data displayed in gridview style with full search, sort, and filter capabilities
- Custom filters can be created and applied to one or more data sets
- Export data sets to XML, HTML or CSV
- Extract and export history files used in live artifact scan
- Quick copy/paste selected gridview rows to clipboard
- Customizable scan settings can tweak the scan to target specific browsers and data sets
- Right-click context menu for narrowing gridview data instantly
- Select which columns to display in each dataset
- View page thumbnails and indexed content
- Export sanitized version of history results to distribute to others
- Website Analyzer provides visualization of datasets using bar graphs, pie charts and timelines
- Website Profiler shows a quick “report card” of artifacts for various websites
The custom filters mentioned above are extremely useful for narrowing the scope of your web history investigation. Web Historian ships with several pre-defined filters that allow you to quickly cull through large web history data sets. For example, you can instantly filter the web history data by visit type to only show hidden page views caused by ads; or, filter the file download history data to only show downloaded media (movies, images, etc.), PDF’s, or plain text files. You can easily create your own filters using the filter editor and configure Web Historian to automatically save any of your searches as filters. Finally, more filters are accessible with a simple right-click on any web history item.
Also new in Web Historian 2.0 are the Website Analyzer and Website Profiler features. The Website Analyzer allows you to visualize web history data (rather than scrolling through pages of records) and generate useful bar graphs, pie charts and timeline plots that can be used in an external report. The Website Profiler generates a quick “report card” summary of any domain in your web history data, showing all artifacts created on your system when it was visited (page titles, cookies, cached files, form data, etc). This feature allows you to get a quick impression of how a site behaves. The screenshot below shows the profile of CNN.com:
We hope you enjoy the new features in this release of Web Historian. As usual, if you have any questions, comments or feedback, please head on over to the user forum.
Stay tuned for even more exciting features coming soon! If you would like a demo or talk to me about features, I will be at Blackhat USA in Las Vegas this summer and hope to be accepted to demo Web Historian 2.0 at Blackhat Arsenal. And finally, don’t miss out on our memory forensics training at Blackhat: Advanced Memory Forensics in Incident Response.
Tags: blackhat, browser forensics, free tools, MIR 1.4, Web Historian
New Memoryze, Audit Viewer, and Training
Written by Jamie Butler
For those who are not on our mailing list for Memoryze or Audit Viewer, we released a new version a little over a week ago. The new version of the software includes all of the memory analysis features that are available in the newly released MANDIANT Intelligent Response (MIR) 1.4.
So what is included in Memoryze and Audit Viewer 1.4? Well, here is the short of it.
Memoryze:
- Support for Windows 2003 x64 SP2
- Improved support of Vista SP1 and SP2 including port enumeration and a better installer
- Enumeration of digital signatures for all loaded modules in a processes’ address space, hooked and hooking drivers, and all drivers found by driver signature scans
- Enumeration of MD5/SHA1/SHA256 hash on disk for all loaded modules in a process’ address space and all drivers found by driver signature scans
- Updated documentation
- Single installer for 64-bit and 32-bit versions
Audit Viewer:
- Improvements to the Malware Rating Index (MRI)
- Report visualization of MRI results
- MRI rule editors that will allow users to graphically edit the MRI rule file
- Handle Trust view to help identify suspicious handles
- Ability to search results within a specific process
- Multi-select with copy
- Multi-select and export to a CSV file
Those who attended the CanSecWest Training in March have already been enjoying many of these features in beta form for months, and we are committed to ensuring that those who attend the Advanced Memory Forensics in Incident Response class at Black Hat will get early access to the next version of Memorzye, which will support Windows 7 64-bit.
As for the Black Hat training, there is a lot of new and updated content for 2010.
- Coverage of 64-bit operating systems
- New section on malware covering different malware techniques and how they stand out in memory
- Four new case studies ranging from real Advanced Persistent Threat (APT) incidents, to spear phishing attacks, and everything in between
- Student receive early access Memoryze and Audit Viewer for Windows 7 64-bit
- Students receive the only free tool to analyze Windows Vista
- Students receive the only free tool to analyze Windows 2003 64-bit
- Better data collection to help identify processes and drivers as malicious or not
- Added the Malware Rating Index (MRI), which helps automatically identify many malware behaviors discussed in the class. Through a simple user interface, students learn how to write rules to identify malware in their own work environments. MRI then uses those rules to score processes as suspicious or not.
I would like to thank James Long who pointed out an issue with the batch scripts* and Peter Villadsen who worked so hard to improve the build process and installation for Memoryze. Peter and I would also like to thank all our loyal users. We appreciate all your feedback, and we hope to see you in Las Vegas.
* When specifying an output directory from the command line with the batch scripts in Memoryze, the directory must already exist.
Tags: Advanced Memory Forensics in Incident Response, Audit Viewer, Black Hat, memory forensics, Memoryze, MIR 1.4, Training
Audit Viewer: Malware Rating Index Undocumented Features and Caveats
Written by Peter Silberman
Hopefully everyone has had a few weeks to recover from the M-Trends kickoff party in St. Louis and everyone has also had a chance to read the M-Trends report! I hope everyone enjoyed the talk I gave at DOD Cyber Crime Conference. I certainly had fun giving it, sorry to those that got hit with the squishy balls. I wanted to take a second to address some caveats and undocumented features of MRI that couldn’t be discussed in the talk.
A caveat within MRI I that I want to talk about is Process Path Verification. This rule set is very powerful but there are two ways to define to paths. Neither is documented because currently there is no documentation on MRI.. The first method of specifying a process path is to specify an absolute path such as this:
calc.exe:\windows\system32
MRI interprets this as the only valid path for calc.exe is \windows\system32\calc.exe. However, if I wrote the rule like:
calc.exe:\windows\system32\
MRI would interpret this as calc.exe can be run from any sub directory as long it’s a sub directory within \windows\system32\*
The reason this is important is it gives you flexibility in writing definitions. If I don’t want to specify the exact location of iexplore.exe I can say it needs to be launched from \program files\. This may prove to be too loose, and I may change this behavior going forward. For now you have the flexibility to specify absolute paths or sub paths.
The next “undocumented” tidbit that I want to discuss is within two behaviors. These behaviors actually have the ability to use regex when trying to match up their values. I did not build the regex option into the UI so it has to be manually added to the AuditViewerConfig.xml. The two XML lists that can take regex expressions are IgnoreFilesList, and ProcessSuspiciousHandleList. The regex elements are, IgnoreFileRegex, and HandleRegex. An example IgnoreFileRegex looks like:
<IgnoreFileRegex>mshist.*\\index.dat</IgnoreFileRegex>
This rule specifies that any file matching this regular expression should be ignored when doing process scoring. You can get creative just be careful.
An example HandleRegex looks like:
<HandleRegex>*:.*-7$:mutant:known conficker mutant</HandleRegex>
It breaks down like this:
Process: Regular Expressions : handle type: description
It breaks down like this:
Process: Regular Expressions : handle type: description
This allows you to get more out of your suspicious handles definitions.
Finally, I’d like to take a second to reiterate something I stated at DC3. The “Verify Digital Signatures” option in Memoryze and Audit Viewer wizard can ONLY be run when doing live memory. It is not possible to enable it when doing dead memory analysis. Which means the address scoring is not possible on dead memory, behavioral analysis still works on dead memory. If you are going to acquire memory, please run live analysis jobs as well as acquisition. This way you get the most information possible off the machine. The second thing I wanted to reiterate is that verify digital signatures is great, it really helps potentially speed up an analyst’s job. However, we are only verifying the digital signatures exist and are valid on disk. We are not verifying the module in memory hasn’t been modified. If a userland rootkit exists (again shame on the authors) then we won’t report that. It’s important to remember this. Verifying modules in memory short of doing rootkit detection is not a trivial task. The windows loader is a beast, a behemoth it does a lot to make verification in memory to disk is very hard (not impossible). Thanks again for all the interest in M-Trends, Audit Viewer and Memoryze. As always feedback is always appreciated.
Tags: Audit Viewer, DC3, DOD Cyber Crime Conference, M-Trends, Malware Rating Index, Memoryze, MRI, MTrends
Highlighter v1.1.1 Released
Written by Jed Mitten
MANDIANT is proud to announce a new version of Highlighter (version 1.1.1). There are big changes between our previous release and this one, so grab it while it’s hot! The biggest enhancements are bolded in the change log below. Download the new version at http://www.mandiant.com/software/highlighter.htm.
Don’t forget that we’re relying on the user community to suggest improvements. Check out http://forums.mandiant.com and head to the Highlighter section to give us your input. Feedback, feature requests, bugs, and use-cases are all very welcome.
Change Log (since v1.0.1):
- Fix: Tabs were mistakenly removed by input sanitization. This has been corrected.
- Fix: The highlight hit count was incorrect – an additional hit per line was mistakenly being added. This has been corrected.
- Fix: The events over time histogram was not properly displaying highlights. This has been corrected.
- Fix: If text was selected in the textbox, and the user clicked on the highlight button, the selection would not be highlighted. This has been corrected.
- Enhancement: The graphic overview now draws much faster.
- New Feature: The textbox is now a 100% custom control. It is virtualized, and supports a wider range of visual display effects. When words are highlighted, the actual word on each line will be surrounded by a colored translucent bubble with a slightly darkened border. The textbox selection and scrolling behavior is now more like a traditional Windows textbox.
- New Feature: Highlighter will now open MUCH larger files. NOTE: Highlighter now keeps a file open while you are working with it.
- New Feature: Highlighter will now accept a list of terms, one on a line, as input to automatically highlight or remove lines. Look under the right click menu, Highlight -> Import Simple List and under Line Operations -> Remove Using Simple List.
- Enhancement: Files will now open somewhat more quickly due to optimization of calculating the MD5 sum of the file.
- Enhancement: The events over time histogram has sharper numbers on the X and Y axis.
- Fix: The events over time histogram scale now properly adjusts when when switching from linear to log mode.
- Fix: A number of State issues were resolved.
- Fix: Various other minor bugs.
- New Feature: Highlighter support opening a document from a Mandiant Intelligent Response (MIR) controller. Look for the new option from the File -> Open menu.
- New Feature: Highlighter will add a Windows Explorer shell extension by default.
- Fix: A number of State issues were resolved, including improper handling of when a selection included a comma.
- Fix: A race condition existed in the implementation of retrieving lines from the current file.
- Fix: Not all hotkeys were actually implemented in code.
- Fix: Highlight counts in the status bar were incorrect sometimes.
- Fix: Sometimes you could not scroll to the bottom of a file using the scrollbar.
- Fix: Events over time histogram had a very sparse appearance.
- Fix: After opening a file, you could not use hotkeys like CTRL-O to open files, nor could you do things like ALT-F4 or any other key sequence with modifiers.
- Fix: The remove feature would not remove lines with selections that contained a TAB.
- Fix: Various other minor bugs.
Tags: highlighter, log analysis, product, software, visualization
Mandiant Highlighter featured on CyberSpeak podcast
Written by Jed Mitten
Jason Luttgens and I were interviewed by Bret Padres and Ovie Carroll over at the CyberSpeak podcast regarding our log analysis tool, Highlighter. Take some time to listen — the interview begins at 18m 10s, though I recommend listening to the whole show because those guys are fun and their content relevant.
Tags: forensics, free, highlighter, incident response, log analysis, software
Memoryze is the 2008 Toolsmith Tool Of the Year
Written by Michael J. Graven
Russ McRee recently wrote that Memoryze is the 2008 Toolsmith Tool of the Year, and how it helped him find the full name of a malware author. He also wrote up a great description of using Memoryze to chase down a password stealing trojan in the February 2009 issue of the ISSA Journal.
One of the interesting things about Russ’s approach in both cases is his use of the strings option. It turned up some great investigative information. However, strings generates a lot of data, and in a large environment that could be a bit of a challenge (imagine running Memoryze on, say, 20,000 systems.) But on the third hand, what if one of those strings in memory is truly your best indicator of compromise?
The key to solving that problem – large-scale searching for very specific information – is prefiltering the results (and indexing them). Using an XPath expression to match only your desired indicator-of-evil lets the investigator focus on just the relevant data. It also lets you scale up the search to very large numbers of systems.
We’ve built our Intelligent Response product for exactly that need, including features from Memoryze as well as other IR tools. If you’d like to hear more about it, or see a demo, drop me a line.
Tags: holisticinfosec.org, Intelligent Response, ISSA Journal, Memoryze, Russ McRee, Toolsmith
. 05 Feb 09 | Products | Comment (1)
