The TrueTree Concept
Download TrueTree
The TrueTree Concept
The process tree is incredibly important when it comes to threat hunting. It doesn’t matter what platform you’re on. Every action that occurs on the operating system can be tied back to the process that caused it. Based on the combined actions that this process performed, a determination can be made as to whether it’s malicious or benign (or somewhere in between). The process in question will have its own process ancestry that led to its creation. This ancestry is one of the most definitive ways to determine if a set of actions was indeed malicious.
On macOS, the process trees that get created are incredibly vague and unhelpful to threat hunting and incident response analysts. This is primarily due to the unique design of XPC, Apple’s own implementation of Inter-Process Communication. Nearly every built-in way to open an application results in the system sending a message to the launchd process which then opens the requested program. As a result, launchd tends to be the direct parent of 90% or more of processes on the system at a given time.
To get an idea of what I’m talking about, try the following…
- Open Finder.app, browse to the “Applications” folder, and double-click Google Chrome press enter
- cmd+space, type in calculator.app, press enter
- Open an application from your dock (I chose Calendar.app)
- In the Safari.app address bar enter “x-man-page://whoami” and click “allow” which will open the man page for whoami inside of Terminal.app
What we’ve done is open four different applications, each using a different open method. If you look at the “ps” output or Activity Monitor, you’ll find a process tree that looks like the following…
As an analyst this tells us nothing. We might as well just get a list of process names and not even bother looking at process ids. What we want to see is something more like this…
If we were able to see the above tree, we would have a much better idea of what happened on the system. We would know exactly how the existence of the running applications came to be. For the most part, this is attainable by using the data provided within the output of the “launchctl procinfo <pid>” command. It’s just a matter of grabbing the right information.
The Extra Pids
The Responsible Pid
The first topic of discussion is the responsible pid. This is an actual pid name used inside of the launchctl procinfo data. Its value is often the same as its normal process id, but not always. If you want to follow along just pick any of the current running “MTLCompilerService” process pids. I have one running as pid 5846, so I’ll run the following command.
sudo launchctl procinfo 5846
In the above screenshot we see that the parent process id for this process is pid 1 (launchd). However, if we look lower we see that it holds a responsible pid of 1388. Not only that, but we also see the application path that’s running as this responsible pid. In my case I see that VMWare Fusion was responsible for this particular instance of MTLCompilerService. If you’re looking at “ps” output, you will not have access to this responsible pid. To my knowledge, the launchctl procinfo command is the only way to access it without writing your own tool. However, Apple does use this responsible pid when displaying parent child relationships inside of the Activity Monitor application.
Notice that Apple doesn’t provide a way to view the ppid value inside of Activity Monitor, likely because it would only confuse most users if the hierarchy shows a parent with the responsible pid but the ppid points to another. As an analyst, it certainly helps to know both.
The Path - Submitted By
If you think documentation on the responsible pid was already scarce enough, let’s take it one step further by looking at the “submitted by pid.” I’ve made this name up because technically this isn’t a real pid, rather it’s a field inside the procinfo output that displays both a process name and a process id that requested another process to be opened. I have still yet to find any places where this pid is actually used by the macOS operating system, but that’s not really of concern. We just care that it exists and that we can use it to our advantage. The submitted by pid consistently tells us when one application was opened on behalf of another application. Apple DOES NOT use this pid inside of Activity Monitor, so it will allow us to build some useful process trees that we can’t see through any other built-in tool. Take a look at this launchctl procinfo output for my Google Chrome Browser process…
If I scroll far enough, I eventually hit the field called “path.” This field can hold two different items, both of which are valuable. The first item it might hold is a plist. We will look at that in the next section. The other item it might hold is a reference to a process in the format of “(submitted by <name>.<pid> ).”
In my case, we see that Google Chrome was requested for creation by the loginwindow process (pid 195). LoginWindow is the process responsible for running through programs that were open when the user last shut down the computer and ensuring they start back up on boot. This tells me that my Google Chrome Browser was likely re-opened automatically by the operating system due to it being open when the computer was restarted. If the submitted by pid is still active then we can grab its parent process and continue up the tree to show its full heritage.
The Path - Plist
As mentioned in the previous section, a submitted by name/pid combo is not the only item that may be displayed in the path field. It’s also possible that this field may be displayed as a plist. The plist will be a launch daemon or launch agent responsible for a specified process being opened. For example, take a look at the procinfo output from the AirPlayXPCHelper service.
Here we can see that “/System/Library/LaunchDaemons/com.apple.AirPlayXPCHelper.plist” was responsible for kicking off this service. If you are newer to macOS security, launch daemons and agents are small xml files that tell launchd what executables to run at startup. They are probably the most widely used persistence mechanisms on macOS by both legitimate and malicious software. Running the “launchctl list” command is the easiest way to see what launch daemons have been loaded. However, the output of this command displays items by their labels rather than their file names. I find this incredibly annoying because if I’m looking for a malicious executable I don’t care about the label it’s using. I want to know what file its persistence is located in. The procinfo output is perfect for this since it tells us the responsible plist file.
Keep in mind that I am not saying plist files open processes. The launchd process goes through each launch daemon and agent that exists and runs the requested executables. However, for threat hunting purposes it would be very helpful to display these plists inside of a process tree sparing us from having to link the plist and process together manually.
Forensic Items
Within the procinfo output there are two notable process-based forensic items.
Forensic Item 1 - Dead Submitted By Pid
We’ve already discussed the importance of the submitted by pid, but it’s important to note that even if the submitted by pid has terminated, the information in the procinfo output will not be modified. This means we have a potential forensic item! Often the submitted by pid will be a long running service which makes life easy for us. However, it is possible that said pid could be short lived and has already terminated. If this occurred we can’t continue climbing the “true” process tree. We aren’t able to look up the details on a process that doesn’t exist. Still, in most situations this submitted by pid is more helpful than its standard ppid as it allows us to see the “true” parent even if that parent has terminated.
Take the “open” command, for example. The open command has been used in many different adware and malware campaigns. After successful social engineering the open command can be used to display a document to a user. This way the user ends up looking at a document while malicious code runs in the background. It makes an attack based on social engineering look much more convincing. When the open command is run, a child process is created under launchd. For example, here is the procinfo output of the Microsoft Excel process after I’ve used the open command to open an Excel document with “open myDocument.xlsx.”
In the above image we see that Microsoft Excel is created by launchd. We also see that responsible pid matches the Excel pid which means there is no useful information to be gathered here. However, if we look for the submitted by pid in the output we see the following…
The submitted by pid tells us that the open command (pid 34753) was called to open Excel, but when we look up more details about process 34753 we see that it has terminated and additional details cannot be found. Regardless, even if we can’t climb the tree, it’s better to know this Excel instance was created using “open” rather than seeing the parent as launchd. It’s fairly uncommon for a document to be opened this way versus something like Finder or Spotlight.
Forensic Item 2 - Program
This field is very basic. It holds the path to the program that was executed, but as you might have noticed in the output, there is already a field for this called “Program Path.”
Further down we see…
The majority of the time these fields will match. I cannot tell you the exact difference between these two paths other than the fact that the lower image only exists if the program is managed by launchd.
So what is the forensic Item? It’s the fact that if this program performs an exec to create a new program the “program=” output will remain the same.
Exec’ing a process is a way of running an executable. This is commonly done after calling the fork() function which causes the process to clone itself with a new pid before the program that was exec’ed takes over. However, if the program author execs an executable without first calling fork, the new program will take over the current process image. The developer gives up control of the process that was once running and the new program will take over its pid. Yes, this can result in confusing process trees. If “program path” and “program” do not match, it likely means that the path held in “program” is no longer running and exec’ed the executable held in “program path,”
TrueTree Tool
If you find some of this confusing, just know that the TrueTree tool will handle all of the above work for you. What is TrueTree, you ask? Check out this follow up blog post to see some use cases.
