Hurdling the Runningboards
Hurdling the Running boards
NOTE: This research was relevant on macOS 10.16 and prior. Changes to macOS 11 have broken this approach. The blog post remains for research but true tree has been rolled back to its previous state which does not attempt to get around the runningboardd “hurdles”.
Welcome back to the never ending adventure of trying to keep the TrueTree tool operational. For those that missed it last time, we discussed that macOS Big Sur has a new process called “runningboardd” which almost all user opened applications now seem to be managed by in some sense. Here in this blog post, we’ll talk about “looking through” this process to find the true process that opened each application. If you don’t care about the logistics, stop reading and just go grab the new version of TrueTree and it will handle these steps for you.
A brief Recap
Runningboardd existed on Catalina, but to my knowledge did not play a heavy role in the management of applications until Big Sur was officially released. The image below shows that according to TrueTree (before the latest updates)the majority of UI applications i’ve opened are children of runningboardd now.
In my last blog post we discussed how the lsmp command provides additional mach port tid bits that may lead us to the actual application that performed the opening of another application.
Digging in with LSMP
LSMP is a tool built into macOS that can be used to view a process’s Mach ports. It is open-source and has a bit of a history for being flakey.
For testing, rather than looking at ALL of the output for lsmp (lsmp -a) we will just take a look at the output of an individual process. In my case, I’ll take a look at the executable from my Mail.app process which I opened using Finder.app.
Mail is running as process id 1971. So I’ll take a look at its Mach ports snapshot using lsmp. Note that what you see below is just a snippet of the lsmp output. Even for a single process, the output of the lsmp command can be quite large.
sudo lsmp -p 1971 
Note that the output columns are as follows…
name, ipc-object, rights, flags, boost, reqs, recv, send, sonce, oref, qlimit, msgcount, context, identifier, type Amongst all the chatter returned by lsmp we see one very interesting set of entries. Since I know that I opened this program via Finder, I was happy to find that it might be possible to link this action via the lsmp output.
Following the same approach, let’s take a look at another example where we open calculator.app via Spotlight (cmd+spacebar). Looking at the lsmp output for calculator.app we see…
Here we see the exact same format as we saw with the Finder.app opening Mail.app combination. It looks to be relatively consistent.
So... What are We Actually Looking at Here?
It seems we can indeed use lsmp to find the which applications are being opened by one another, but this raises the question of what in the heck we’re actually looking at. Mach ports are a fairly complex feature on macOS originally designed as the sole inter-process communication feature for the Mach microkernel. These ports are the underlying technology in XPC which is more commonly used in app development. Due to their complexity, we will not be breaking them down completely in this blog post (I’d really like to do one in the future). In fact, as a threat hunter or incident response handler, it’s not even important that you have a full understanding of Mach ports. What’s important to understand for this blog post is the following…
- Mach ports allow a process to send and receive data to and from other processes.
- Mach ports use two different rights: send rights and receive rights.
- To send a message to another Mach port owned by a different process you must have first been granted the send right from said process.
So here are the items we care about aligned with the lsmp column headers when investigating the Mail.app pid.
name ipc-object rights flags boost reqs recv send sonce oref qlimit msgcount context identifier type
--------- ---------- ---------- -------- ----- ---- ----- ----- ----- ---- ------ -------- ------------------ ----------- ------------
0x00008e03 0x95a1c707 recv -----I-- 0 --- 1 Y 5 0 0x0000000000000000
+ send -------- --- 1 <- 0x0000b41b (366) appleeventsd
+ send -------- D-- 1 <- 0x0003dc37 (588) Finder Here we see an Mach port receive right that holds two send rights: one to appleeventsd and one to Finder. As you would expect, this exchange occurred when I clicked on the Mail.app application from within Finder. Many recognize appleeventsd as the primary service at play when it comes to Applescript. Applescript is a simple text-like language that allows users to automate endless combinations of things on the operating system via a feature called “Apple Events.” However, here there was no Applescript being used. I’m unsure exactly what is causing this exchange of Mach ports but it seems to be reliable enough to get us a true parent in most cases. Perhaps Apple events are always in use behind the scenes when a user selects a program to open or when the system opens one.
Fixing TrueTree
Since lsmp is open source we may be able to write our own version that is better suited for our needs, but Apple has placed two entitlements (com.apple.system-task-ports and task_for_pid-allow) required by SIP in order to get this useful information from each process. This leaves us with having to place a shell call to lsmp in order to get the info necessary to fix TrueTree on Big Sur. We simply have to get the output of lsmp and search for the times we see the patterns described above. We can do this by running “lsmp -a” to get a list of all processes’ Mach ports. It also seems that lsmp has a nice feature to write its output to json. However, you’ll likely notice that when opting to write output to json, much of the data you’re looking for might not align with what is in the non-json version of the output! This seems like a miss on Apple’s part. It means that we won’t be able to use the json data and we’ll have to manually sift through the lsmp output. I won’t bore you with the details of how I went about doing this but for those who care it can be seen on my github in the getLsmpData() function.
TrueTree has been updated to use the lsmp pid data accordingly but only if the following conditions are met:
- TrueTree is run on Big Sur (or later)
- A process’ “submitted by” pid is runningb
As long as both conditions listed above are true then TrueTree will defer to the process id found in lsmp and mark it as the true parent process. If either of the conditions above is false, then TrueTree will operate per usual as described in previous blog posts.
Here’s a screenshot of TrueTree before we’ve implemented any changes. Take a look at Zoom.app and Mail.app.
Now here are screenshots of Zoom and Mail.app after we’ve implemented the new lsmp changes.
With the updated TrueTree not only can we see that Zoom was actually opened via Chrome (very likely using a custom url scheme), but we also see that Chrome was opened via the Dock. Mail.app we see was actually opened by Finder.
New Feature - Sources
Since things are getting a little out of hand with all of the pids we’re using, I’ve added one more feature to TrueTree which is a –sources argument. Enabling this feature will print the source of where its true parent was found. The possibilities are…
- parent_process_id
- responsible_pid
- submitted_by_pid
- submitted_by_plist
- lsmp
Going Experimental
For the most part, the addition of the lsmp lookup has fixed our issue with runningboardd. However, in introducing this fix there is also one problem. Lsmp looks at the current Mach ports that are active with processes. If a true parent is closed while the child stays open, we won’t have a way to find the true parent via lsmp. Example: In the Zoom.app example used above, if the user had closed Chrome, we will not be able to see that it was responsible for opening Zoom. Instead, TrueTree would just place it under the runningboardd process. There are many long running services on macOS when it comes to those that are responsible for opening other applications. Therefore, our solution still works a lot of the time. Regardless, this issue has turned TrueTree into somewhat more of an experimental tool rather than one that will always reflect consistent information. On top of that it could certainly use a refactoring of code at this point, but, hey, it’s free! I’ve made the latest version available for download. Feel free to grab it. Happy hunting.
