An Alternative to MacOS’s DYLD_LIBRARY_PATH

Operating systems continue to evolve to become ever-more secure. However, sometimes the quest for security breaks compatibility. One of the open source applications that I maintain — Traveling Ruby — was affected by this: we used the DYLD_LIBRARY_PATH feature on macOS, but that stopped working ever since Apple introduced System Integrity Protection (SIP). With this post, I’d like to take you into a deep dive about how I solved this issue, as well as how some macOS internals work.

How SIP broke Traveling Ruby

Traveling Ruby is a tool that allows Ruby developers to easily ship Ruby apps to end users. It lets developers create self-contained Ruby app packages that run on multiple versions of Windows, Linux and macOS — without requiring users to install Ruby.

It’s an open source project, so I’d like to democratize its development. I want anyone to be able to contribute to the project, as easily as possible.

However, SIP is a significant barrier for democratization. Traveling Ruby’s build process relies on DYLD_LIBRARY_PATH, which is blocked by SIP. This means that:

• Contributors that build Traveling Ruby on their own laptops, must disable SIP. This requires rebooting to Recovery Mode and running obscure commands in the terminal.
• Traveling Ruby cannot be built on many CI hosting services, such as Azure DevOps and Github Actions, because it’s not possible to disable SIP there.

This is very painful, so something had to be done about it. After some research and experimentation, I found an alternative to DYLD_LIBRARY_PATH, meaning that it’s no longer necessary to disable SIP.

What did we use DYLD_LIBRARY_PATH for?

Before we get to the fix, let’s revisit how the old solution worked.

How macOS library lookup works

How does macOS locate library dependencies for a given executable?

Answer: an executable contains a specification of library dependencies. Each entry is a path to that library, e.g. “/Users/hongli/example/libyaml.dylib”.

Here’s an example which compiles a C program that does nothing, but is linked to libyaml:

echo 'int main() { return 0; }' > foo.c
cc foo.c -o foo -L/Users/hongli/example -lyaml

We can use otool -L foo to inspect the list of libraries that this executable requires:

foo:
/Users/hongli/example/libyaml.dylib (compatibility version 3.0.0, current version 3.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)

So here you go, foo contains information that says “I depend on /Users/hongli/example/libyaml.dylib”. Note that his is a full path. That’s different from Linux, where executables say “I depend on libyaml.so” (not a full path).

Relative dependency paths

In many cases, it’s useful to have the OS locate dependencies relative to the executable. For example, suppose we want to distribute the above program foo to another user. We’ll need to package all its dependencies. Hypothetically we’ll want to package it like this:

foo.tar.gz
|
+- bin/
| |
| +- foo
|
+- lib/
|
+- libyaml.dylib

Suppose our friend extracts foo.tar.gz into “/Users/xiangling/foo”, then runs “/Users/xiangling/foo/bin/foo”. We’ll want the OS to locate libyaml.dylib in “/Users/xiangling/lib”, not in “/Users/hongli/example”.

One way to achieve this is by ensuring that the executable’s dependency list specifies @executable_path/../lib/libyaml.dylib, instead of an absolute path to libyaml.dylib. macOS recognizes @executable_path as a special directive that means “the directory in which the executable is located”.

An executable’s dependency list can be modified even after it’s built, using install_name_tool. This tool is so called because each “path” in the dependency list is technically called an “install name”.

So let’s go ahead and modify our foo executable’s dependency list:

install_name_tool -change @executable_path/../lib/libyaml.dylib foo

Now, when we inspect the dependency list using otool -L foo, we see that it’s indeed modified:

foo:
@executable_path/../lib/libyaml.dylib (compatibility version 3.0.0, current version 3.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)

Looking up dependencies when building Traveling Ruby

The Traveling Ruby build process goes like this:

1. Before building Ruby, we build dependencies such as libyaml. Dependencies are installed to a temporary location that we call the “runtime directory”. This is something like /Users/hongli/traveling-ruby/osx/runtime.
2. Then we build Ruby. This is done in a temporary directory such as /tmp/ruby-XXX/ruby-XXX.
3. Finally, we copy the Ruby executable, as well as all dependencies, into a single directory tree, which we can then package into a tarball.

The final package directory looks like this:

+- bin/
| |
| +- ruby
|
+- lib/
|
+- libyaml.dylib
|
+- ...and other dependency libraries...

We ensure that all executables use @executable_path/../lib to reference dependencies. So once packaged, the Ruby executable can locate all its dependencies.

But there’s a problem during step 2. As part of building Ruby, we need to run the built Ruby executable, before it’s copied over to the package directory. During this step, the Ruby executable is located in /tmp/ruby-XXX/ruby-XXX/ruby. How will that Ruby executable locate its dependencies, which at that point are in the runtime directory /Users/hongli/traveling-ruby/osx/runtime/lib?

We used to solve this problem by setting the environment variable DYLD_LIBRARY_PATH to /Users/hongli/traveling-ruby/osx/runtime/lib. This tells macOS to look for libraries in the given directories.

Now that DYLD_LIBRARY_PATH stopped working on macOS systems with SIP enabled, it’s time to look for a new solution.

New solution based on @rpath

Every macOS executable can embed a list of library search paths, or “rpaths”. Whenever macOS encounters a dependency path that references “@rpath”, macOS will search for that dependency in the embedded list of paths.

So unlike DYLD_LIBRARY_PATH, which is set during runtime, the library search paths are embedded in the executable, which Apple seems secure enough to not block via SIP.

Some useful facts about rpaths:

• They do not have to be absolute paths: they too can reference “@executable_path”!
• They can be added or removed from an executable after it’s built, not just during compile time.

Here’s an example. Let’s build a C program which does nothing but is linked to “/Users/hongli/example/libyaml.dylib”. We also ensure that we add an rpath to this executable.

echo 'int main() { return 0; }' > foo.c
cc foo.c -o foo -L/Users/hongli/example -lyaml
install_name_tool -add_rpath @executable_path/../lib foo

When we inspect the executable’s list of rpaths with otool -l foo | grep LC_RPATH -A2, we see:

cmd LC_RPATH
cmdsize 40
path @executable_path/../lib (offset 12)

However, just having this rpath entry is not enough. When we examine the dependency list with otool -L foo, we see that the reference to libyaml.dylib is an absolute path.

foo:
/Users/hongli/example/libyaml.dylib (compatibility version 3.0.0, current version 3.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)

So when we start foo, macOS will ignore the rpath, and will still locate libyaml in /Users/hongli/example. We can verify this by moving libyaml.dylib, and observing that foo fails to start:

$ mkdir ../lib
$ mv /Users/hongli/example/libyaml.dylib ../lib/
$ ./foo
dyld: Library not loaded: /Users/hongli/example/libyaml.dylib
Referenced from: ./foo
Reason: image not found
Abort trap: 6

So we change foo’s dependency list to reference “@rpath”:

install_name_tool -change /Users/hongli/example/libyaml.dylib @rpath/libyaml.dylib foo

This yields the following dependency list:

foo:
@rpath/libyaml.dylib (compatibility version 3.0.0, current version 3.3.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)

Now it works as expected. Suppose you package up foo and libyaml.dylib according to the package structure described earlier. If your friend extracts the tarball to “/Users/xiangling/foo” and runs /Users/xiangling/foo/bin/foo, then here’s what happens:

1. macOS encounters a dependency named “@rpath/libyaml.dylib” and concludes that it needs to look for libyaml.dylib in the list of rpaths.
2. macOS sees that the list of rpaths is ["@executable_path/../lib"], and looks in there for libyaml.dylib.
3. macOS interprets “@executable_path” as the actual executable’s path, so it finds libyaml.dylib in “/Users/xiangling/foo/bin/../lib”.

Conclusion

So the final solution is as follows:

• We ensure that all executables are compiled with two rpaths:
  • @executable_path/../lib, and,
  • the absolute path to the runtime directory.

  This way, if macOS can’t find a dependency in ../lib, it will find it in the runtime directory.

• At the end of step 2 of the Traveling Ruby build process, we remove the absolute rpath to the runtime directory.

Now that it’s no longer necessary to disable SIP, developing Traveling Ruby is a lot less of a hassle, and it paves the way to building on hosted CI services.

Modern operating systems are highly complex and are still evolving. While some of their features may seem like black magic sometimes, how they work make sense once you the mechanics behind them.

Originally published on joyfulbikeshedding.com.