Operating systems continue to evolve to become ever-more secure. However, sometimes the quest for security breaks compatibility. One of the open source applications that I maintain — Traveling Ruby — was affected by this: we used the DYLD_LIBRARY_PATH feature on macOS, but that stopped working ever since Apple introduced System Integrity Protection (SIP). With this post, I’d like to take you into a deep dive about how I solved this issue, as well as how some macOS internals work.
Traveling Ruby is a tool that allows Ruby developers to easily ship Ruby apps to end users. It lets developers create self-contained Ruby app packages that run on multiple versions of Windows, Linux and macOS — without requiring users to install Ruby.
It’s an open source project, so I’d like to democratize its development. I want anyone to be able to contribute to the project, as easily as possible.
However, SIP is a significant barrier for democratization. Traveling Ruby’s build process relies on DYLD_LIBRARY_PATH, which is blocked by SIP. This means that:
This is very painful, so something had to be done about it. After some research and experimentation, I found an alternative to DYLD_LIBRARY_PATH, meaning that it’s no longer necessary to disable SIP.
Before we get to the fix, let’s revisit how the old solution worked.
How does macOS locate library dependencies for a given executable?
Answer: an executable contains a specification of library dependencies. Each entry is a path to that library, e.g. “/Users/hongli/example/libyaml.dylib”.
Here’s an example which compiles a C program that does nothing, but is linked to libyaml:
echo 'int main() { return 0; }' > foo.c
cc foo.c -o foo -L/Users/hongli/example -lyamlWe can use otool -L foo to inspect the list of libraries that this executable requires:
foo:
/Users/hongli/example/libyaml.dylib (compatibility version 3.0.0, current version 3.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)So here you go, foo contains information that says “I depend on /Users/hongli/example/libyaml.dylib”. Note that his is a full path. That’s different from Linux, where executables say “I depend on libyaml.so” (not a full path).
In many cases, it’s useful to have the OS locate dependencies relative to the executable. For example, suppose we want to distribute the above program foo to another user. We’ll need to package all its dependencies. Hypothetically we’ll want to package it like this:
foo.tar.gz
|
+- bin/
| |
| +- foo
|
+- lib/
|
+- libyaml.dylibSuppose our friend extracts foo.tar.gz into “/Users/xiangling/foo”, then runs “/Users/xiangling/foo/bin/foo”. We’ll want the OS to locate libyaml.dylib in “/Users/xiangling/lib”, not in “/Users/hongli/example”.
One way to achieve this is by ensuring that the executable’s dependency list specifies @executable_path/../lib/libyaml.dylib, instead of an absolute path to libyaml.dylib. macOS recognizes @executable_path as a special directive that means “the directory in which the executable is located”.
An executable’s dependency list can be modified even after it’s built, using install_name_tool. This tool is so called because each “path” in the dependency list is technically called an “install name”.
So let’s go ahead and modify our foo executable’s dependency list:
install_name_tool -change @executable_path/../lib/libyaml.dylib fooNow, when we inspect the dependency list using otool -L foo, we see that it’s indeed modified:
foo:
@executable_path/../lib/libyaml.dylib (compatibility version 3.0.0, current version 3.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)The Traveling Ruby build process goes like this:
/Users/hongli/traveling-ruby/osx/runtime./tmp/ruby-XXX/ruby-XXX.The final package directory looks like this:
+- bin/
| |
| +- ruby
|
+- lib/
|
+- libyaml.dylib
|
+- ...and other dependency libraries...We ensure that all executables use @executable_path/../lib to reference dependencies. So once packaged, the Ruby executable can locate all its dependencies.
But there’s a problem during step 2. As part of building Ruby, we need to run the built Ruby executable, before it’s copied over to the package directory. During this step, the Ruby executable is located in /tmp/ruby-XXX/ruby-XXX/ruby. How will that Ruby executable locate its dependencies, which at that point are in the runtime directory /Users/hongli/traveling-ruby/osx/runtime/lib?
We used to solve this problem by setting the environment variable DYLD_LIBRARY_PATH to /Users/hongli/traveling-ruby/osx/runtime/lib. This tells macOS to look for libraries in the given directories.
Now that DYLD_LIBRARY_PATH stopped working on macOS systems with SIP enabled, it’s time to look for a new solution.
Every macOS executable can embed a list of library search paths, or “rpaths”. Whenever macOS encounters a dependency path that references “@rpath”, macOS will search for that dependency in the embedded list of paths.
So unlike DYLD_LIBRARY_PATH, which is set during runtime, the library search paths are embedded in the executable, which Apple seems secure enough to not block via SIP.
Some useful facts about rpaths:
Here’s an example. Let’s build a C program which does nothing but is linked to “/Users/hongli/example/libyaml.dylib”. We also ensure that we add an rpath to this executable.
echo 'int main() { return 0; }' > foo.c
cc foo.c -o foo -L/Users/hongli/example -lyaml
install_name_tool -add_rpath @executable_path/../lib fooWhen we inspect the executable’s list of rpaths with otool -l foo | grep LC_RPATH -A2, we see:
cmd LC_RPATH
cmdsize 40
path @executable_path/../lib (offset 12)However, just having this rpath entry is not enough. When we examine the dependency list with otool -L foo, we see that the reference to libyaml.dylib is an absolute path.
foo:
/Users/hongli/example/libyaml.dylib (compatibility version 3.0.0, current version 3.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)So when we start foo, macOS will ignore the rpath, and will still locate libyaml in /Users/hongli/example. We can verify this by moving libyaml.dylib, and observing that foo fails to start:
$ mkdir ../lib
$ mv /Users/hongli/example/libyaml.dylib ../lib/
$ ./foo
dyld: Library not loaded: /Users/hongli/example/libyaml.dylib
Referenced from: ./foo
Reason: image not found
Abort trap: 6So we change foo’s dependency list to reference “@rpath”:
install_name_tool -change /Users/hongli/example/libyaml.dylib @rpath/libyaml.dylib fooThis yields the following dependency list:
foo:
@rpath/libyaml.dylib (compatibility version 3.0.0, current version 3.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)Now it works as expected. Suppose you package up foo and libyaml.dylib according to the package structure described earlier. If your friend extracts the tarball to “/Users/xiangling/foo” and runs /Users/xiangling/foo/bin/foo, then here’s what happens:
1. macOS encounters a dependency named “@rpath/libyaml.dylib” and concludes that it needs to look for libyaml.dylib in the list of rpaths.
2. macOS sees that the list of rpaths is ["@executable_path/../lib"], and looks in there for libyaml.dylib.
3. macOS interprets “@executable_path” as the actual executable’s path, so it finds libyaml.dylib in “/Users/xiangling/foo/bin/../lib”.
So the final solution is as follows:
@executable_path/../lib, and,This way, if macOS can’t find a dependency in ../lib, it will find it in the runtime directory.
Now that it’s no longer necessary to disable SIP, developing Traveling Ruby is a lot less of a hassle, and it paves the way to building on hosted CI services.
Modern operating systems are highly complex and are still evolving. While some of their features may seem like black magic sometimes, how they work make sense once you the mechanics behind them.
Originally published on joyfulbikeshedding.com.