Endpoint Security Ideas for Advanced Malware

Following a previous discussion regarding anti-malware solutions to address commodity malware, I’d like to now turn our attention to “semi-commodity malware” and advanced, persistent threats (APT). Malware in these categories may be modified commodity malware or based on some zero-day exploit/vulnerability. Antivirus product coverage for these types of malware is usually pretty spotty or non-existent.

So, how do we prevent these types of malware? I think the first step is to simply accept that you will not completely prevent malware in these categories from affecting your organization. Take preventative precautions, but the key here is going to be rapid detection and response.

I’m also not going to dive into operating system security mechanisms such as code signing (similar to what is done in iOS), ASLR, PaX, SELinux, grsecurity, etc. but will instead focus on add-ons to operating system security.

Here are some potential solutions.

###Change Monitoring This solution is probably my favorite in this space, though it will require someone to become familiar with your organization’s environment in order notice anomalies.

The point of this type of solution is to simply monitor for all changes to endpoints (new software, configuration changes, kernel modules, new processes, etc.). An analyst then utilizes some sort of data analysis tool to separate the normal, everyday behavior from the suspicious or malicious. It will take effort and the tuning of whatever analysis tool is used (I prefer using Tableau Software or Splunk), but is very powerful at spotting problems.

There are change control/monitoring solutions out there, though they are not typically geared toward regular end-user endpoints, but most antivirus products have the ability to log all changes to a system, requiring one to simply enable the logging and begin analyzing the data. OSSEC is another great tool one can use for this.

There are also tools such as Tripwire that can help with this, though Tripwire-like tools are typically less effective at spotting registry changes, the installation of Browser Helper Objects (BHO), or the misuse of legitimate tools (net use, etc.).

Lastly, Facebook and Etsy released a tool for Mac intrusion detection called MIDAS that looks to be promising.

###Network Anomaly Detection Understanding your network traffic and searching for anomalies is absolutely important in combating advanced malware. Efforts in this area can include:

  • Tracking applications used on your network (I’m not speaking of port numbers here, but rather monitoring the actual applications)
  • Utilizing DNS sinkholes, whether using vendor-supplied technology or custom-built
  • Creating alerts for traffic that should not be present on one’s network
  • Tracking download/upload sizes and monitoring for deviations from the norm

You are limited only by your imagination (well, and perhaps budgets or manpower) but network detection must go beyond a basic firewall and intrusion prevention product and requires good analysts willing to invest the time required to learn an organization’s network.

Some great products/tools in this space are:

  • Next-generation firewalls (e.g., Palo Alto Networks)
  • The open source Bro Network Security Monitor
  • NetWitness
  • Utilizing one’s SIEM to do analysis

###Sandboxing Technology Probably the most well-known of these technologies would be FireEye, Palo Alto Networks’ WildFire, or NetWitness’ Spectrum. These types of solutions attempt to run executable code, documents, or other dangerous files within a virtual system, watching the behavior of the software and making a judgement as to whether the software is malicious or benign in nature.

Of course, clever attackers have been seeking to evade sandboxing technology for a very long time, starting with efforts to evade analysis and debugging. Tricks such as checking for the presence of debuggers, drivers indicating virtualization, sleeping for long periods, and other schemes are used to identity sandboxing technology. Once the sandboxing technology is identified, the malware behaves in a benign manner, hoping to be passed to a real system where it will then wreak havoc.

Most sandboxing technology will have means of detecting evasion behavior, but the cat-and-mouse game sure is interesting.

###Virtualized Containers This type of control is quite new in the endpoint space and therefore hasn’t been subject to the in-the-wild rigors of the other solutions, but should still be considered.

These solutions attempt to run code within small virtual containers, effectively creating a small sandbox for every bit of code.

While holding promise, I frankly remain skeptical of these solutions due to potential performance and complexity issues. I also do not feel they have been properly tested in real-world situations long enough for me to be comfortable with them.

###Application Whitelisting While most anti-malware solutions seek to enumerate the “bad” software amongst a large list of software available (this is known as “blacklisting”), whitelisting turns that on its head, instead enumerating only the software allowed to run on an endpoint, denying all the rest. Several vendors provide this functionality, with the best known being Bit9, McAfee (with their acquisition of SolidCore), or Lumension.

Whitelisting is easiest on a system that remains fairly static, such as a server or point-of-sale device (Target, what were you thinking not doing this?), but gets a bit more tricky when you move to more general purpose computing systems such as those used by most people within an organization.

Some organizations are better equiped to do this from a policy/control standpoint than others. Most organizations do not control what software runs or is installed on endpoints and thus have no idea what is running on their systems. Whitelisting will be difficult to implement in these organizations or those with very open cultures, where individuals are not prohibited from installing software themselves.

Of course, even if one doesn’t actually enforce whitelisting, using these tools in “learning mode”, where they simply inventory installed software and track changes, is incredibly useful in spotting semi-commodity malware without interrupting or inconveniencing users.

###Incident Response Earlier I mentioned the importance of rapid detection and response to incidents as the most effective way to combat advanced malware. While prevention is great, chances are prevention will fail and there will be a compromise at some point. The rapid detection of these incidents is critical, as is an effective response.

I think the tools mentioned above aid a trained security analyst to identify security incidents. I also want to stress the importance of having staff who are trained in incident response.

I should also mention a couple of other fantastic products/tools to aid one in incident response: