Sniper Forensics - Part 4: Finding Evil

I was abroad last week working on a case. What makes this case unique and ultimately inspired me to write this blog post, was that there was no compromise to speak of.  We were called in to perform, "proactive forensics" of a newly deployed processing system. While there was no indication of a breach within the organization's current environment, a regulatory body requested a deep inspection of the newly deployed systems due to a previous incident within the organization.

If this does not strike you as odd, let me explain it in another way.  Say you are a police officer, and you are called to what you expect to be a crime scene.  However, when you arrive on the scene, no crime has been committed.  The individual that placed the call just wanted you to look around to see if you can find any evidence of a crime. So, you have to figure out some way to fulfill your obligation in the event that a crime actually did take place, but where do you even begin?

The first, and what I think was the most substantial challenge in this case was what I was going to do with myself.  As an incident responder I am used to...well...responding to incidents.  The same holds true in my role an as a forensic investigator (yes...there is difference). I am used to investigating some sort of crime involving some sort technology.  However in this case, I had neither.  So again, I was faced with the question of what am I going to do with myself...for a week...with the paying customer watching me.

So I went  back to my customer and asked them a few questions:

1. What are your goals?
2. What are you hoping that I don't find (because who wants me to actually find evidence of a breach, right)?
3. What data do you have that would be worth something to an attacker?

After getting some clarification from the customer, and thinking about my situation for a bit, I made my decision...I was going to pretend.  Not pretend to work, but pretend that there HAD been a breach. 

So let's break this down logically so you can see where I am going with it:

1. Bad guys are after stuff that is of some sort of value to them (or else why commit the crime in the first place)?
2. Where is the data on the customer network that is worth something.  In this case, credit card data, but that really doesn't matter.  Valuable data is valuable data regardless of what form it takes.
3. The bad guys have to have a way in.  This is the first component of the Breach Triad, referred to as Infiltration.
4. The bad guys have to have something to steal.  This is the second component of the Breach Triad, referred to as Aggregation.
5. The bad guys have to have a way out.  This is the third component of the Breach Triad, referred to as Exfiltration.

To begin my investigation, I asked the customer where the Card Holder Data (CHD) was and how did it get there.  Did they have Point of Sale (POS) systems that aggregated to a central server?  Did they have an E-Commerce site that sent CHD to a database? What?  Now, I know that this sounds pretty basic, but I have been on investigations where this phase took the customer almost two weeks to figure out!  Knowing where the likely targeted data resided at least gave me some sort of direction rather than aimlessly wandering around simply looking for something that was "out of place".
 In this case, the CHD was restricted to only seven systems, which is really pretty small – thank you data reduction!  Thankfully, they had firewall logs for the subnet that these systems sat on, as well as local logging enabled (best case scenario, right).  So I grabbed RAM, volatile data, and images from those systems.  I also asked for all of the logs they had from the appropriate firewalls as well as all of the web logs from the web servers.

Once I had my data, I was able to start analysis, but looking for what?  Drawing from lesson learned on past cases, I know that CHD is stolen (usually) in one of four ways:

1. RAM Dumper
2. Key Logger
3. Network Sniffer
4. Smash-n-Grab

These types of malware (RAM dumpers, Key Loggers, and Network Sniffers) all have two things in common.  They have to run from somewhere, and they have to generate some sort of output (or else what's the point of them running?).  So I started by looking through my RAM dumps using Memoryze for processes names that I KNOW are malicious (My team keeps a neat little list of any malware we run across during our investigations along with their MD5 sums and Fuzzy Hash values...imagine that) as well as any processes that are running from non-standard directories (like C:\Windows\temp, or C:\Windows\system32\config) or named slightly different than a legitimate Windows binary (like exp1orer.exe, 1sass.exe, or msoftice.exe).  This is a common technique used in numerous malware packages to fool administrators, and can be very effective.

Additionally, I checked all of the loaded DLLs, sorted them by Least Frequency of Occurrence and looked for anything under seven (which is the highest number of injected processes I have seen in a real infection) that was out of place.  Again, I used the same basic technique...known malware, odd or misspelled names, and non-standard directories.  Finally, I checked my timeline for "birth" dates for any files within the last six months.  Most of what I found was log files and temp files.  Using grep, I was able to quickly eliminate those kinds of files from my view.

Searching the timeline also helped me to identify potential dump files.  I know most modern RAM dumpers obfuscate the output files by either using some basic encryption or even just encoding (like with an XOR).  It doesn't have to be fancy – just put the stolen data in some format that a regex won't identify.

Now, I had data, I have something to look for, and based on previous cases I had some form of a success indicator.

The next data sample I needed to take a look at were the web logs and the firewall logs.  For the third time I was faced with the question, what was I looking for?  On previous cases, I have used basic data reduction technique known to me as, "known goods".  What does "normal" look like, right?  Since this was obviously not my network, I was not the subject matter expert, but my customer was.  So, I sat down with my logs and one of the network admins (who ended up taking me out later for some freaking awesome sushi...but that's not important right now) to identify and remove "normal" from my view.  This is really very easy if you use the "-v" switch with grep.  Working for most of the afternoon we successfully reduced our data sample to just a few lines that we later identified as being, unusual but normal traffic patterns.  We repeated these steps with the web logs, and basically came to the same conclusions.

Now that my work is complete, how do I report this to the customer?  Since I had no real intelligence to work from, I had to work with the customer to establish the goals of the investigation so ensure I was meeting their desired point.I still needed to effectively communicate what I did and why to the people that paid me to fly out and work for an entire week.  This turned out to be easier that I thought.  I was simply honest with them.

In the out briefing meeting, I told them that since there was evidence of a breach, and no intelligence per se, that I pretended.  I treated the case like any other breach investigation trying to ignore the fact that I knew that no breach had actually occurred.  Turned that they thought that was really good idea (which was good for me).  I told them what data I gathered and why, as well as what I looked for and why.  Then I told them my findings, and what that meant to them.  Mission accomplished, and another satisfied customer.

So what are the lessons learned here?  First of all, as much as I harp on being asked to "find evil" is a bad way to work an investigation, the reality is that every once in a while one of those cases comes along.  Sometimes what the customer wants, and what is the "best case scenario" are two very different things.  So as professionals, sometimes we have to improvise.  That does not mean that when you are asked to find something as nebulous as "evil" you cannot build some sort of framework around what you are looking for and why.  Giving yourself success indicators is an essential component of any investigation, and I would argue it is even more important in cases like this, where your goals are not very clear.  So yet again, Sniper Forensics proves that comprehensive investigations are all about the methodology and not about the tools.

Comments

[Harlan Carvey]

I posted some comments and a question here...

http://windowsir.blogspot.com/2011/03/links-and-stuff.html