Thursday, March 10, 2016

DYLD_LIBRARY_PATH and El Capitan


With the release of Firebird 3 RC2 it is time to concentrate properly on the port of Firebird on MacOSX, Firebird 3 was buildable but we don't have a proper installer, ICU is no longer directly included and there
are a number of other bits and pieces that need to be addressed.

The last time I built Firebird 3 was before I upgraded to El Capitan. I did the upgrade to address some issues with the Firebird installer.

Lets just say when I started ny first build I wasn't expecting too many problems....

When we build Firebird on OSX we arbitrarily point to the ICU location (build_environment/firebird/lib), after copying over the files from the initial ICU build, the Firebird build then finds the ICU libraries dynamically (when needed) because DYLD_LIBRARY_PATH is sey to point to the location via prefix.darwin (make.platform).

While the ICU libraries are being built we set the final actual framework install names and locations of the ICU libraries (using -install_name) and they then placed in the appropriate directory of the framework
when the installer is packaged and built.

Every time I tried to build Firebird it would fall over when trying to build msg.fdb with ISQL.
The error was:
 "Could not find acceptable ICU library"
which seemed very strange as I know it was placed exactly where it should be and where DYLD_LIBRARY_PATH should be picking it up.

Setting DYLD_LIBRARY_PATH in my default terminal worked successfully, checking DYLD_PRINT_ENV seeemed to show the path being set etc., after much head scratching I eventually put the following line of code into unicode_util.cpp at the function:
UnicodeUtil::ConversionICU& UnicodeUtil::getConversionICU()

printf("DYLD_LIBRARY_PATH=%s\n", getenv("DYLD_LIBRARY_PATH"));

When I next ran the build, this is what was returned..

DYLD_LIBRARY_PATH=(null)

Something very strange was definitely going on. Some further research eventually turned up this thread on the
PostgreSQL forums

http://www.postgresql.org/message-id/561E73AB.8060800@gmx.net

"Apparently the behaviour is that DYLD_LIBRARY_PATH is prevented from being inherited into any system binaries. E.g. a shell. But specifying it inside a shell script will allow it to be inherited to children of
that shell, unless the child is a protected process in turn."

My reaction was the same as Tom Lane's but expressed in much stronger terms.

The simplest solution to the problem is to turn off System Integrity Protection

via Recovery Mode (Command-R), and then set csrutil disable in a terminal and reboot.

Note that this is a development only issue and will not affect anybody using the built
binaries when the port is complete.

2 comments:

Zabrane Mikael said...

Hi Paul,

I’m having a new issue while compiling Firebird-2.5.6.27020-0 on OSX El-Capitain 10.11.6.
My clang version is:
$ clang -v
Apple LLVM version 8.0.0 (clang-800.0.38)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

And here’s the error:
$ make
[…]
/Applications/Xcode.app/Contents/Developer/usr/bin/make -C gen firebird
/Applications/Xcode.app/Contents/Developer/usr/bin/make CFLAGS="-arch x86_64 -O0 -ggdb -g3 -D_THREAD_SAFE " -j1 -f ../gen/Makefile.extern.editline
make[2]: Nothing to be done for `libeditline'.
(cd ..; ./src/misc/writeBuildNum.sh rebuildHeader)
files are identical
/Applications/Xcode.app/Contents/Developer/usr/bin/make boot_phase1_build
/Applications/Xcode.app/Contents/Developer/usr/bin/make -j1 -f ../gen/Makefile.boot.gpre gpre_boot
clang -DBOOT_BUILD -I../src/include/gen -I../src/include -I../src/vulcan -DNAMESPACE=Vulcan -O1 -DDARWIN -pipe -p -MMD -fPIC -fno-common -arch i386 -mmacosx-version-min=10.7 -D_THREAD_SAFE -fvisibility-inlines-hidden -fvisibility=hidden -c ../src/common/utils.cpp -o ../temp/boot/common/utils.o
clang: warning: argument unused during compilation: '-p'
In file included from ../src/common/utils.cpp:43:
In file included from ../src/common/../common/utils_proto.h:33:
In file included from ../src/include/../common/classes/fb_string.h:39:
In file included from ../src/common/../jrd/os/../../common/classes/alloc.h:43:
../src/include/../common/classes/fb_atomic.h:380:4: error: invalid operand for instruction
"lock; xaddq %0, %1"
^
:1:14: note: instantiated into assembly here
lock; xaddq %eax, (%ecx)
^~~~~
1 error generated.
make[3]: *** [../temp/boot/common/utils.o] Error 1
make[2]: *** [gpre_boot] Error 2
make[1]: *** [../gen/firebird/bin/gpre_static] Error 2
make: *** [firebird] Error 2


Help very appreciated.

N.B: I also tried to compile other releases, and even Firebird3, but all seem to have the same issue.

Regards
Zab

Paul Beach said...

My guess
make CFLAGS="-arch x86_64 -O0 -ggdb -g3 -D_THREAD_SAFE " -j1 -f ../gen/Makefile.extern.editline
is followed by
clang -DBOOT_BUILD -I../src/include/gen -I../src/include -I../src/vulcan -DNAMESPACE=Vulcan -O1 -DDARWIN -pipe -p -MMD -fPIC -fno-common -arch i386 -mmacosx-version-min=10.7 -D_THREAD_SAFE -fvisibility-inlines-hidden -fvisibility=hidden -c ../src/common/utils.cpp -o ../temp/boot/common/utils.o
Something is wrong with your build environment.
Make is passing -arch x86_64 (64bit default build) while clang thinks -arch i386 (32bit). Have you set CFLAGS/CXXFLAGS to -arch i386?, if so don't, you only need to set them for a 32bit build.