Reverse-Engineering Malware: Malware Analysis Tools and Techniques

The Reverse-Engineering Malware course, which I teach at the SANS Institute, discusses tools and techniques for analyzing malicious software such as bots, worms, and trojans. The course incorporates many exercises, offering an opportunity to practice malware analysis in a hands-on capacity. Its full version lasts four days at a conference. Students often have the opportunity to take the two halves of the course independently. Periodically I also teach this course live over the Internet via the SANS@Home program.

You can participate in this course via the following upcoming venues:

• Web-based SEC610 class via the SANS@Home program in May 2008. We "meet" twice a week for a total of 8 live sessions. SEC601 alumni receive 50% discount for this course. Everyone qualifies for a 10% discount, when they specify the code COINS-LZ. (SEC601 alumni will receive 10% off the 50%-discounted fee when they specify this code.)
• Four-day SEC610 class at the SANSFIRE 2008 conference in July 2008 in Washington, DC. SEC 601 alumni receive a 50% discount for this course, to encourage them to explore the additional topics that the newly-expanded course covers.

By covering both behavioral and code analysis approaches, this unique course provides a rounded approach to reverse-engineering. As a result, the course makes malware analysis accessible even to individuals with a limited exposure to programming concepts. The course does materials do not assume that the students are familiar with reverse-engineering; however, the difficulty level of concepts and techniques increases quickly as the course progresses.

In the first half of the course, you will learn how to set up an inexpensive and flexible laboratory for understanding inner-workings of malware, and demonstrate the process by exploring capabilities of real-world specimens. You will learn to examine the program's behavioral patterns and assembly code, and study techniques for bypassing common code obfuscation mechanisms. The course also takes a look at analyzing browser-based malware.

In the second half of the course, you will review key assembly language concepts. You will learn to examine malicious code to understand its flow by identifying key logic structures, looking at examples of bots, rootkits, key loggers, and so on. You will understand how to work with PE headers and handle DLL interactions. You will also develop skills for analyzing self-defending malware through advanced unpacking techniques and bypassing code-protection mechanisms. Finally, you will discover how to bypass obfuscation techniques employed by browser-based malicious scripts.

Hands-on workshop exercises are an essential aspect of this course, and allow you to apply reverse-engineering techniques by examining malicious code in a carefully-controlled environment. When performing the analysis, you will study the supplied specimen's behavioral patterns, and examine key portions of its assembly code.

Topics covered by the full course include:

• Configuring the laboratory environment
• Assembling the analysis toolkit
• Performing behavioral and code analysis
• Bypassing authentication mechanisms
• Reverse-engineering protected executables
• Intercepting network connections
• Patching compiled executables
• Examining shellcode
• Malware analysis shortcuts
• Core code reversing concepts
• Assembly language primer
• Identifying assembly logic structures
• Reversing seen in common malware categories
• Working with PE headers
• Handling DLL interactions and API hooking
• Packer identification
• Manual and automated unpacking
• Bypassing code-defense mechanisms
• Analyzing advanced browser malware

You will learn to analyze malware using tools such as:

• System Monitor, Process Explorer, Regshot
• BinText, LordPE, FireBug, VMware
• IDA Pro, OllyDbg, OllyDump, OllyScript
• Rhino, Malzilla, SpiderMonkey
• Microsoft Script Editor
• Snort, NetCat, Honeyd, fakeDNS

Who should attend:

• You will benefit from this course if your job ever requires you to understand key aspects of malicious programs.
• Individuals who found this course particularly useful often had responsibilities in the areas of incident handling, forensic analysis, Windows security, and system administration.
• Attendees of this course often focus on supporting their organizations' internal security needs. The class also frequently includes engineers from security product and service companies who are looking to deepen their malware analysis expertise.

Prerequisites:

• Students should have a computer system that matches the stated laptop requirements: some software needs to be installed before you come to class.
• Students should be familiar with using Windows and Linux operating environments and be able to troubleshoot general connectivity and setup issues.