Hidden Docs and Functionality in macOS

Hidden Docs and Functionality in macOS

Overview


There are a multitude of useful command-line utilities that come with macOS. After using them day in and day out, a few of you may even know some of these tool inside and out, have all the documentation memorized, and even have written scripts around them to compensate for a few short-comings. What if there were a way to get a little more from these tools?

I am going to go over some of the techniques I have used to find some extra functionality in macOS. I’ve outlined a few of the tools and the general process I have used over the years to dig around macOS command-line utilities looking for anything useful. There are some pretty cool things you can uncover, but the first steps are knowing “what, where, and how.”

The What – Discovering File Types


file is a tool that can be used to read the magic number of files to find out what type of file it is:

Using file, we can easily know what kind of file type we are looking at (regardless of its file extension), and knowing is half the battle. If there isn’t any magic number for a file, you’ll simply get the output data, which is not particularly useful.

The Where – Finding Binaries


It’s one thing to know what type of file you have, but you also have to know where they are located. If you want a large list of executable binaries from a system, this is one comprehensive method I would suggest:

To provide a breakdown, this one-liner essentially uses $(echo $PATH | tr ':' ' ') to split up your  $PATH environment variable into its separate paths. It uses  -perm -111 to only print items with the executable bit set for owner, group, and everyone. The  -type f flag is used to only print files (i.e. exclude directories), and finally, we redirect any errors the command may encounter along the way with  2> /dev/null.

I feel like I used to have a script that did this using the  which command, but it seems to have been lost to time, and probably for the best.  find is a very powerful tool and I felt it was worth demonstrating some of its versatility.

If you want to take it a step further, you could pipe the output of this command into xargs file to print out the type of each file encountered:

Which would effectively give you the complete path, as well as they type of each and every executable available for tab completion in Terminal.

However, if you’re only interested in a particular directory, you can replace $(echo $PATH | tr ':' ' ') with any path and will get a list of every executable file in that directory (assuming you have the appropriate permissions to traverse the directory).

Now that you’ve found them, it’s time to read them.

The How – Sifting through the Data


When it comes to binaries, cat is not an option, everything is just interpreted by Terminal including control sequences (alert, backspace, vertical tab, etc.) which effectively feels like a electronic war zone, but don’t be discouraged! There is always another way!

strings is an incredibly useful tool for getting the printable information from binaries while ignoring the rest. Although they are compiled, hard-coded strings are left intact, so you can get at a lot of error messages and (most importantly) any help documentation that is included. Granted, this is not a catch-all for all finding undocumented features, but it is a great place to start.

There CAN be a lot to sift through though, and not all of it is enlightening. The output from strings /usr/sbin/diskutil alone is 2625 lines (as of macOS 10.12).

Other tools for consideration are: hexdump, otool, nm, and od. However, it’s a little out of the scope of this post to outline the use of these tools here.

A Good Example – Apple Software Restore


The most useful item I have found using strings is with asr  (Apple Software Restore). See man (8) asr.

Typical Usage:

Hidden Usage:

This chunk of text is roughly 800 lines into the output of strings /usr/sbin/asr, but highlighted above, plain as day, we have extra functionality as well as usage.

Once found, hidden features still take a significant amount of testing and experimentation to utilize properly. Because of their nature, there typically isn’t a whole lot of documentation (go figure). What you see is what you get. Sometimes it’s pretty obvious, other times it isn’t at all. You’ll have to experiment and reverse engineer each executable to discover the purpose of the features for yourself.

According to Der Flounder’s ASR’s Hidden Documentationthere are other ways at getting to this specific chunk of documentation:

However, you can’t rely on the incorporation of a universal  FULL_USAGE environment variable for all compiled executables… Then again, it might be fun to find out if any other macOS binaries use this flag.

Other Stuff


Here are some other hidden features I have found using this method:

Conclusion


I wish I had a super awesome script to give that would just uncover all of the hidden flags of our most used utilities, but unfortunately it’s not that easy. I discovered this hidden usage of asr a long time ago using this method, and tried to find a few others for this post, but it was hard work. It took me about 4 hours of reading very abstract inline help strings as well as testing to find the few examples I provided here (I didn’t want to leave you empty handed).

It’s a process, and a pretty involved one at that, but hopefully it will lead to some cool discoveries. Happy hunting!

Links


Gary Kessler’s: File Signatures

 

Der Flounder’s: ASR’s Hidden Documentation

No Comments

Leave a Reply