Cache Me Outside
› apple's 'quicklook' cache may leak encrypted data
06/15/2018
//patrick:
Apple states that: "we believe privacy is a fundamental human right....[and] every Apple product is designed from the ground up to protect that"
...unfortunately marketing claims and reality are sometimes at odds.
In this guest blog post, Wojciech Reguła (@_r3ggi) discusses (with some additional content from yours truly), how macOS will cache (in the clear!) the names and thumbnails of files contents, even if when such files are stored within a password-protected encrypted container.
Note:
The fact that QuickLook caches thumbnails of file contents is well known, especially in forensics circles.
However the fact that behavior is still present in the latest version of macOS, and (though potentially having serious privacy implications), is not widely known by Mac users, warrants additional discussion.
The original version of the blog post, "Your encrypted photos revealed in macOS cache" was posted on Wojciech's site: wojciechregula.blog.
Read on below, for Wojciech's research!
Apple states that: "we believe privacy is a fundamental human right....[and] every Apple product is designed from the ground up to protect that"
...unfortunately marketing claims and reality are sometimes at odds.
In this guest blog post, Wojciech Reguła (@_r3ggi) discusses (with some additional content from yours truly), how macOS will cache (in the clear!) the names and thumbnails of files contents, even if when such files are stored within a password-protected encrypted container.
Note:
The fact that QuickLook caches thumbnails of file contents is well known, especially in forensics circles.
- "Quicklook thumbnails.data parser" (Mari DeGrazia)
- "Determining User Actions in OS X Based on Quicklook Thumbnail Cache Database Entries" (Sara Newcomer)
- "Quick Look Cache Parsing" (Lodrina Cherne)
However the fact that behavior is still present in the latest version of macOS, and (though potentially having serious privacy implications), is not widely known by Mac users, warrants additional discussion.
The original version of the blog post, "Your encrypted photos revealed in macOS cache" was posted on Wojciech's site: wojciechregula.blog.
Read on below, for Wojciech's research!
Background
'QuickLook' is a super cool mechanism of macOS, that allows you to quick check file contents without opening it in specialized application. When you press the space bar on for instance on an *.xlsx file, you can see following preview without having MS Excel installed:
While reading *OS Internals Volume I (that I highly recommend btw) I stopped on QuickLook chapter. I found out that Quicklook registers the com.apple.quicklook.ThumbnailsAgent XPC service that is responsible for creating thumbnails database and storing it in the $TMPDIR/../C/com.apple.QuickLook.thumbnailcache/ directory.
This means that all photos that you have previewed using space (or QuickLook cached them independently) are stored in that directory as a miniature and its path. They stay there even if you delete these files or if you have previewed them in encrypted HDD or TrueCrypt/VeraCrypt container.
Note:
- Quicklook will also generate thumbnails of other 'default' file type such as documents...again, even if those files are stored within encrypted containers.
- Depending on Finder's view settings (e.g. icon mode, list mode, etc), file thumbnails may be created and cached by QuickLook automatically when a directory is viewed via the UI.
Proof Of Concept
Let's create a VeraCrypt container, mount it and save Luke Skywalker's photo (in my case /Volumes/Container/luke-skywalker.png). Also, press space on it to make QuickLook cache it (though if thumbnail or 'cover flow' view is enabled in Finder, this step may not be needed):
Now we do the same thing, but with placing Darth Vader in a macOS Encrypted HFS+/APFS drive (in my case /Volumes/EncryptedHDD/test/darth-vader.jpeg):
Now we should have both images cached. Using following command we can find necessary QuickLook files:
$ find $TMPDIR../C/com.apple.QuickLook.thumbnailcache/ -type f -name "index.sqlite" /var/folders//d5/5p9d59rs67d59ttncml57pqw0000gp/C/com.apple.QuickLook.thumbnailcache/ index.sqlite $ find $TMPDIR../C/com.apple.QuickLook.thumbnailcache/ -type f -name "thumbnails.data" /var/folders//d5/5p9d59rs67d59ttncml57pqw0000gp/C/com.apple.QuickLook.thumbnailcache/ thumbnails.data
Now, copy them these file to some other location:
$ mkdir ~/Desktop/ql_post $ cp $TMPDIR../C/com.apple.QuickLook.thumbnailcache/index.sqlite $ cp $TMPDIR../C/com.apple.QuickLook.thumbnailcache/thumbnails.data
Open index.sqlite to investigate the file contents:
We have information about the full paths and the file names. Now, let's exfil the thumbnails.data file to retrieve the miniatures. I used a python script from the OSX-QuickLook-Parser project with my little modifications to provide macOS compatibility.
$ python quicklook_parser_v_3_5.py -d /Users/r3/Downloads/OSX-QuickLook-Parser/
-o /Users/r3/Downloads/OSX-QuickLook-Parser/output
Processing Complete
Records in table: 41
Thumbnails available: 41
Thumbnails extracted: 41
The output directory contains few thumbnail versions of our previews:
Below you can see the original miniatures (the biggest cached by QuickLook):
Enough to determine the encrypted picture contents, isn't it? (The Darth Vader photo, in original, has resolution equal 1920x1080 while it's cached thumbnail is 336x182).
Note:
If we had not manually previewed the images (by selecting the image, then pressing the space bar to trigger a QuickLook preview), a smaller version of image may still have been automatically created.
For example, here's the automatically generated QuickLook preview image for Luke (dimension 128x128):
...clearly still big enough to ascertain it's content!
If we had not manually previewed the images (by selecting the image, then pressing the space bar to trigger a QuickLook preview), a smaller version of image may still have been automatically created.
For example, here's the automatically generated QuickLook preview image for Luke (dimension 128x128):
...clearly still big enough to ascertain it's content!
//patrick:
Note the this same behavior can be replicated in an password-protected encrypted AFPS container:
For example if we create a file in the (mounted) container, and the simple view the container in the UI (i.e. not previewing the file!), we can observe the thumbnail(s) of the file being automatically created and cached:
$ echo ALL YOUR BASE ARE BELONG TO US > /Volumes/test/secret.txt $ sudo fs_usage -w -f filesystem | grep QuickLook RdData[AT2] /private/var/folders/l8/.../C/com.apple.QuickLook.thumbnailcache/index.sqlite open /private/var/folders/l8/.../C/com.apple.QuickLook.thumbnailcache/dirt WrData[AT2] /private/var/folders/l8/.../C/com.apple.QuickLook.thumbnailcache/index.sqlite-wal
If we unmount the encrypted volume, the thumbnails of the file are (as previously mentioned) still stored in the user's temporary directory, and thus can be extracted.
$ diskutil unmount /Volumes/test/ Volume test on disk1s6 unmounted
...and depending on the size of the 'preview' images generated for Finder (and other variables, such as the size of the font used in the file), the contents of the even documents may be discernible from the thumbnail alone:
Again, it should be noted that the file, secret.txt/secret.rtf was never manually previewed (or 'QuickLooked'). We simply viewed the directory (/Volumes/test) in Finder. That's all that was needed to trigger the creation and caching of the thumbnail!
If we had manually trigged a QuickLook preview of the file (by selecting the file, then hitting the space bar), an even higher resolution thumbnail may have been generated, and thus cached.
Another interesting (unfortunate?) side effect of QuickLook caching deals with removable drives. In posting on apple.stackexchange.com a user asked, "Do files on USB drive leave any trace on iMac (macOS Sierra)?".
The answer correctly noted that if the user opened a folder on the removable drive, that files contained in that folder would be cached:
"If you open a folder with files residing on an external drive, thumbnails will be created on the boot drive depending on the file type and the installed QuickLook plugins.
The previews, metadata and file paths are stored in SQLite database files deep inside the var folder in the com.apple.QuickLook.thumbnailcache folder. The path to this folder contains arbitrary folder names.
With the proper commands the preview pics can be extracted from the database."
The previews, metadata and file paths are stored in SQLite database files deep inside the var folder in the com.apple.QuickLook.thumbnailcache folder. The path to this folder contains arbitrary folder names.
With the proper commands the preview pics can be extracted from the database."
We can confirm this by searching thhe files extracted from the QuickLook thumbnail cache (on my Mac) - and sure enough, file paths and thumbnails from USB devices that have been inserted into my Mac. For example the USB stick ('NO NAME') that I travel with, which contains my conference talks:
$ grep Volumes extractedFiles.txt 776 /Volumes/NO NAME RSAC_2017.key 782 /Volumes/NO NAME PHDays_2017.key 778 /Volumes/NO NAME RSAC_2016.key 780 /Volumes/NO NAME HITB_2017.key ...
For a forensics investigation or surveillance implant, this information could prove invaluable. Imagine having a historic record of the USB devices, files on the devices, and even thumbnails of the files...all stored persistently in an unencrypted database, long after the USB devices have been removed (and perhaps destroyed).
For users, the question is: "Do you really want your Mac recording the file paths and 'previews' thumbnails of the files on any/all USB sticks that you've ever inserted into your Mac?" Me thinks not... //end additional content (patrick)
Conclusion
This technique is known and helps a lot in forensics, but I honestly didn't know about this before.
It was the big surprise for me to see that even files stored in encrypted containers may be that cached. Have it on mind when you will be using space to preview photos. 😉
//patrick:
As noted in this blog post QuickLook can cache the various information about files (including documents!), even if they exists solely in a password-protected encrypted container or on a removable USB device. This information includes:
- file path/name
- file contents (in the form of thumbnail, 'preview' images)
- stored in the clear - outside the encrypted containers
- stored in the user's $TMPDIR - meaning it's accessible to any code running in the context of the user
- stored persistently - even when the encrypted container is unmounted and across reboots.
However, if an attacker (or law enforcement) has access to the running system, even if the password-protected encrypted containers are unmounted (as thus their contents 'safe'), this caching 'feature' can reveal their contents.
In other words, the increased security encrypted containers were thought to provide, may be completely undermined by QuickLook.
...clearly, this isn't doing users' privacy any favors!
So what can you do if you want to store data in an encrypted container/volume without leaking it's contents via QuickLook? It seems that the solution is to manually clear the QuickLook cache when you unmount the container. Luckily it seems there is an Apple utility just for this: qlmanage
$ man qlmanage
NAME
qlmanage -- Quick Look Server debug and management tool
SYNOPSIS
qlmanage -r
...
qlmanage -r resets Quick Look Server and all Quick Look client's generator cache.
But executing qlmanage -r doesn't actually appear to clear the cache!?
$ ls -lart $TMPDIR/../C/com.apple.QuickLook.thumbnailcache/ total 32096 -rw------- 1 user staff 0 Jan 17 08:49 exclusive drwx------@ 37 user staff 1184 Jun 15 18:48 .. -rw-r--r-- 1 user staff 880 Jun 16 11:14 thumbnails.fraghandler -rw------- 1 user staff 15935808 Jun 16 11:33 thumbnails.data -rw-r--r-- 1 user staff 450560 Jun 16 11:33 index.sqlite -rw-r--r-- 1 user staff 0 Jun 16 11:33 index.sqlite-wal -rw-r--r-- 1 user staff 32768 Jun 16 11:33 index.sqlite-shm drwxr-xr-x 8 user staff 256 Jun 16 11:33 . $ qlmanage -r qlmanage: resetting quicklookd $ ls -lart $TMPDIR/../C/com.apple.QuickLook.thumbnailcache/ total 32096 -rw------- 1 user staff 0 Jan 17 08:49 exclusive drwx------@ 37 user staff 1184 Jun 15 18:48 .. -rw-r--r-- 1 user staff 880 Jun 16 11:14 thumbnails.fraghandler -rw-r--r-- 1 user staff 450560 Jun 16 11:33 index.sqlite -rw-r--r-- 1 user staff 0 Jun 16 11:33 index.sqlite-wal -rw------- 1 user staff 15935808 Jun 16 11:56 thumbnails.data -rw-r--r-- 1 user staff 32768 Jun 16 11:56 index.sqlite-shm drwxr-xr-x 8 user staff 256 Jun 16 11:56 .
Ok, so qlmanage appears to do nothing?
...so let's just nuke the entire com.apple.QuickLook.thumbnailcache directory, then reboot:
$ rm -rf $TMPDIR/../C/com.apple.QuickLook.thumbnailcache $ sudo reboot
On reboot, the com.apple.QuickLook.thumbnailcache directory is re-created along with the (now 'reset') cache files. Success!
Update:
@binaryriot pinged me to point out that if one passes the -r cache command line flag to qlmanage this will purge the cache:
qlmanage -r cache
...so no reboot needed!
He also noted that there appear to be various 'undocumented' qlmanage commands!
We can uncover such commands by dumping strings embedded in the qlmanage binary (specifically looking for those that deal with modifying the cache):
$ strings -a /usr/bin/qlmanage | grep "qlmanage -r" qlmanage -r qlmanage -r whenidle qlmanage -r soon qlmanage -r cache qlmanage -r disablecache qlmanage -r enablecache
If we execute the qlmanage -r disablecache while running a file monitor (fs_usage) we see this persistently modifies the ~/Library/Preferences/com.apple.QuickLookDaemon.plist file:
# fs_usage -w -f filesystem open F=3 (RW____) /Users/user/Library/Preferences/com.apple.QuickLookDaemon.plist WrData[AT2] /Users/user/Library/Preferences/com.apple.QuickLookDaemon.plist
Examining this file after executing the qlmanage -r disablecache command, we can see the QLUseCache flag as been set to false:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" ... ">
<plist version="1.0">
<dict>
<key>QLMTCacheSizeLastCheckAbsoluteTime</key>
<real>551322217.42825603</real>
<key>QLUseCache</key>
<false/>
</dict>
</plist>
To confirm this does indeed disable the cache, create various files (say on the desktop), then manually select & press the space bar to generate a QuickLook preview. Then (re)run the quicklook_parser_v_3_5.py command and note that no new thumbnails were created!
We can also monitor the file system while creating such files and note that the com.apple.QuickLook.thumbnailcache directory and it's files are not longer accesses nor modified:
# fs_usage -w -f filesystem | grep com.apple.QuickLook.thumbnailcache
love these blog posts & tools? you can support them via patreon! Mahalo :)