OS X's internal malware protection systems.

For quite some time there has been a common misconception that Mac’s are ‘safer’, or are not susceptible to malware. Unfortunately this is not strictly true. With OS X continuing to gain market share, it will become a more juicy target for cyber criminals.

On the plus side though, Apple has included a form of built-in malware protection Since Mac OS 10.6 “Snow Leopard” which provides a blacklist of known Mac malware. The overarching mechanism is known as “File Quarantine” (Which has been around since version 10.5) and includes several components geared towards protecting unsuspecting users from malware.

First, a look at File Quarantine

The idea of File Quarantine, as the name implies, is that any file a user downloads from an unknown source is placed into quarantine, where it will not be allowed to execute until confirmation is provided by the user.

Any file downloaded by a “File Quarantine aware application” – primarily web browsers and mail clients – is tagged with an HFS+ extended attribute – com.apple.quarantine.

This attribute contains information about which application the file was downloaded from, its current quarantine status and an identifier which also corresponds to the primary key of an entry in the File Quarantine database.

This attribute instructs Launch Services to perform some additional actions before launching the application or installer package:

• Gatekeeper will verify the code-signing certificate.
• XProtect will check if the file is known malware.
• The user will be asked if they really want to run the file.

An application’s quarantine-awareness is defined by the value of the LSFileQuarantineEnabled property in the app bundles Info.plist. However, OS X also maintains a list of application exceptions for which quarantine is forced, regardless of what is defined in the Info.plist.

Some of the applications for which quarantine is forced are:

• Mozilla Firefox
• Steam
• Microsoft Office

Information about a file downloaded and quarantined is also be stored in the File Quarantine database,  located at:

/Users/supreme_overlord/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV2

This database stores the following information about a file downloaded through a Quarantine Aware application:

• UUID, which corresponds to the UUID value in the HFS+ attribute.
• Download time (as a UNIX timestamp).
• Name of application or process which created the file.
• HTTP URL to the location where appropriate.

However, this database only stores information about files listed as the quarantine type kLSQuarantineTypeWebDownload.

Anyway, lets look at an example of File Quarantine in action. For this example I have downloaded a DMG containing Android File Transfer, via Safari. As we can see in the last line of the xattr output the file has been tagged with com.apple.quarantine:

This consists of:

• Quarantine type (0002).
• Timestamp (5469fa65 / Mon 17 Nov 2014 21:38:45 AWST).
• Bundle name (Safari).
• UUID which corresponds to the primary key in the database explained earlier.

If we go ahead and mount the DMG and running the application will yield the following warning from OS X:

Whether you will see this prompt or not seems to be determined by both the quarantine type, as well as a quarantine status bit which changes depending on the actions of the user.

From what I can see in LSQuarantine.h, Quarantine type can be one of either:

• kLSQuarantineTypeWebDownload
• kLSQuarantineTypeOtherDownload
• kLSQuarantineTypeEmailAttachment
• kLSQuarantineTypeInstantMessageAttachment
• kLSQuarantineTypeCalendarEventAttachment
• kLSQuarantineTypeOtherAttachment

However, one bit of the QuarantineType property will change depending on the input from the user. For example:

Before opening:

`com.apple.quarantine: 0001;547190a4;Firefox;

After opening:

com.apple.quarantine: 0061;547190a4;Firefox;

This tells Launch Services that the user has confirmed that they want to run the file, and no further prompts will be presented.

A closer look at XProtect

XProtect is the system which provides protection against known Mac malware, and is part of the File Quarantine system.

When attempting to open a bundle or package which matches a signature within the XProtect database, you will receive a warning like the following:

XProtect works off a plist small file (Well, two small plist files – we’ll get to that later) containing definitions for known Mac malware. Different match conditions can be defined depending on the malware, so that known malware can be identified by it’s unique attributes. It would appear that SHA-1 digests encoded in Base64 are used for the majority of definitions.

Here is part of the entry for our Flashback-2.pkg file above:

<key>Identity</key>  
 <data>  
 vV5UHuCuughPELEUlFnbeJhnfkA=  
 </data>

As we can see, this matches the file signature:

cat Downloads/Flashback-2.pkg | openssl dgst -binary -sha1 | base64 vV5UHuCuughPELEUlFnbeJhnfkA=

Since this check with XProtect is part of the File Quarantine system,  files transferred to the computer through an application that is not quarantine-aware will not be analysed by XProtect.

For example : the same file downloaded via cURL will not be placed into quarantine and will be allowed to run:

Out of interest, for definitions where XProtect only performs a check on the file signature, modifying the file in some way will prevent XProtect from denying the appliction when you try to open it. However, doing so, under the default OS X settings will present you with either one of these warnings.

If the file is not signed:

Both of these warnings can be overridden by the user.

In addition, XProtect is also used to prevent out-of-date Flash and Java plugins from running on the system. These plugin definitions can be found at:

/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/XProtect.meta.plist

Both XProtect definition files are updated silently by Apple when changes need to be made.

Gatekeeper

The role of Gatekeeper is to verify the certificate used by developers to sign their application. The default settings on OS X allow only either applications from the App store, or applications signed with a valid developer certificate to be installed.

This prevents a large amount of malware from being inadvertently run by the user, and in the event that a piece of malware uses a valid developer certificate, Apple can revoke the certificate preventing further spread.

There is plenty of information about Gatekeeper around on the Internet so I won’t delve too far into it. Here are some resources:

OS X: About Gatekeeper – http://support.apple.com/en-gb/HT202491

Conclusion:

Apple seem to be going for a balance between effectiveness and useability without impacting too much on performance, which works well for them.

The combination of the quarantine prompts to alert the user that they may not want to run a file, XProtect to deny known malware from running and Gatekeeper to only allow properly signed applications is quite effective at stopping the average user from executing a malicious file.

Of course, these protections are only in place when opening files downloaded through quarantine-aware applications, such as a web browser or your mail client. However, the majority of users will not be downloading suspicious files via the command line.

There is always the risk of malware brought in via removable storage, such as a USB flash disk. This should be investigated further.

This is only a brief summary of how OS X’s malware protection works, and there is much more to be written. For now though, we will leave it at that. With Mac’s becoming more popular with high profile targets, there could be interesting times ahead for Apple.

Final note: Have I made an error? If you have any feedback, please let me know! Use the ‘Contact’ link at the top of the page.

References & Resources:

Converting binary plists: http://www.forensicswiki.org/wiki/Converting_Binary_Plists

StackExchange – What causes OS X to mark a folder as quarantined: https://apple.stackexchange.com/questions/104712/what-causes-os-x-to-mark-a-folder-as-quarantined

About the “Are you sure you want to open it?” alert (File Quarantine / Known Malware Detection) in OS X: http://support.apple.com/en-au/HT201940

Launch Services Framework Release Notes for OS X v10.5: https://developer.apple.com/library/mac/releasenotes/Carbon/RN-LaunchServices/index.html

OS X: About Gatekeeper: http://support.apple.com/en-gb/HT202491