Some perplexities on Linux binary compatibility

[Nicola Mingotti]

Hi guys,

Yesterday I set up the Linux compatibility mode to try to run Android Studio and I found a few things which I do not understand.
Note. I am running FreeBSD 11.2

-] brandelf returns wrong values

$> brandelf /bin/cat
File '/bin/cat' is of brand 'FreeBSD' (9).     # OK
$> brandelf /compat/linux/bin/cat
File '/compat/linux/bin/cat' is of brand 'SVR4' (0).   # WRONG

-] I installed Oracle Java SDK : /usr/ports/java/linux-oracle-jdk18
But the corresponding javac has been put into: /usr/local/linux-oracle-jdk1.8.0/bin/javac .
I don't understand, since this is Linux software it should be put under /compa/linux/xxxx, shouldn't it ?

-] brandelf again gives wrong value for aforementioned javac

$> brandelf /usr/local/linux-oracle-jdk1.8.0/bin/javac
File '/usr/local/linux-oracle-jdk1.8.0/bin/javac' is of brand 'SVR4' (0).   # WRONG

Did I miss something fundamental ? Can you replicate what i see ?

bye
Nicola

 

[olli@]

Well, both SVR4 and Linux binaries have brand number 0, so the result of brandelf isn't really wrong. When you run file(1) on the binary, you will see that it is really a Linux binary:

$ file /compat/linux/bin/cat
/compat/linux/bin/cat: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.18, BuildID[sha1]=9fce89ce63c6633a80f86ad648b88d8f34f46f11, stripped

Software from ports is almost always installed under /usr/local, including Linux applications. For example, the Linux versions of Opera, FlashPlayer and various games are installed under /usr/local, too. /compat/linux is only used for the linux base system.

 

[Nicola Mingotti]

Thank you for your reply olli@

-] ok, for the file method instead of brandelf to check what
kind of binary it is.

-] On the other side, If SRV4 and Linux have the same brand number, then why this output ?

$> brandelf -l
known ELF types are: FreeBSD(9) Linux(3) Solaris(6) SVR4(0)

--- this is wrong, corrected in next post ---------------------------------
-] I am still a quite perplexed, yesterday i red the relevant chapter of "Absolute FreeBSD 3rd ed." to
have an idea of how the Linux mode is working and there are relevant discrepancies.
For example this:

$> /compat/linux/bin/bash
$> uname -a
Linux fbs3 2.6.32 FreeBSD 11.2-RELEASE #0 r335510: Fri Jun 22 04:32:14 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$> which cat
/bin/cat
$> file /bin/cat
/bin/cat: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 11.2, FreeBSD-style, stripped

So, running the Linux bash, i am still using FreeBSD cat and not Linux one.

Reading the book I got the idea the Linux binaries run in some kind of chroot, and should look for relevant software first into /compat/linux, then if they do not find it, look out of that branch.

I suspect i red too fast the book or it is heavily outdated already.
-------------------------------------------------------------------------------------------------------

bye
n.

 

[Nicola Mingotti]

Let's consider what is written in the Handbook, chapter 10.3 and let's let aside "Absolute FreeBSD" which
you may not have at hand.

----- [a] ------------------------------------
Linux® mode dynamically reroots lookups. This is, in effect, equivalent to the union option to file system mounts. First, an attempt is made to lookup the file in /compat/linux/original-path. If that fails, the lookup is done in /original-path. This makes sure that binaries that require other binaries can run. For example, the Linux® toolchain can all run under Linux® ABI support. It also means that the Linux® binaries can load and execute FreeBSD binaries, if there are no corresponding Linux® binaries present, and that a uname(1) command can be placed in the /compat/linux directory tree to ensure that the Linux® binaries cannot tell they are not running on Linux®.
-------------------------------------------

According to [a] , in my humble opinion,

1) It would be better that Linux binary would be installed under /compat/linux. Ineed, this way we could have "javac" for Linux (the Oracle one) and "javac" of OpenJDK for FreeBSD, they would coexist easily.... the same holds for other software. E.g. we could have "adb" of FreeBSD and "adb" of Linux etc. etc. etc.

2) In my previoud post example i made a mistake, I used the file command, which is not available under /compat/linux, so the FreeBSD file was used and this messed things up. Lets use od which is available in /compat/linux. We can see "linux bash" correctly looks for "linux binaries" first.

# let's enter Linux bash shell
$> /compat/linux/bin/bash

# an extra check
$> uname -a
Linux fbs3 2.6.32 FreeBSD 11.2-RELEASE #0 r335510: Fri Jun 22 04:32:14 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

# /bin/bash is correctly resolved to Linux bash, since it is
# found under /compat/linux  (code 03)
#
$> od -N 8 -a -x /bin/bash
0000000 del   E   L   F stx soh soh etx
           457f    464c    0102    0301


# /bin/csh is resolved to FreeBSD "csh", since csh is not found
# under /compat/linux tree.  (code 09)
#
$> od -N 8 -a -x /bin/csh
0000000 del   E   L   F stx soh soh  ht
           457f    464c    0102    0901


# there are also not "03" codes under /compat/linux
# (code 00)
$> od -N 8 -a -x /bin/wc 
0000000 del   E   L   F stx soh soh nul
           457f    464c    0102    0001

$> od -N 8 -a -x /bin/ls
0000000 del   E   L   F stx soh soh nul
           457f    464c    0102    0001

$> od -N 8 -a -x /bin/cp
0000000 del   E   L   F stx soh soh nul
           457f    464c    0102    0001

3) The mistery of why some binaries in /compat/linux are marked with ABI 00 instead of ABI 03
remains. According to this wikipedia page this should not be the case. And the handbookt, same
chapter as before, states that:
---------
For Linux® binaries to function, they must be branded as type Linux using brandelf(1):
---------

bye
n.

 

[olli@]

-] On the other side, If SRV4 and Linux have the same brand number, then why this output ?

$> brandelf -l
known ELF types are: FreeBSD(9) Linux(3) Solaris(6) SVR4(0)

Ok, so the official brand number for Linux seems to be different, but the Linux binaries are still branded with ID 0. I don't know why, to be honest. Maybe the Centos people (from whom FreeBSD's Linux base is derived) just don't care because Linux doesn't use ABI emulation, so the brand ID doesn't matter there anyway.

By the way, only dynamically linked binaries have ID 0. The statically linked ones have ID 3. This probably has to do with the fact that dynamically linked Linux binaries use ld-linux-x86-64.so as the run time linker, which in turn is branded with ID 3.
Anyway, you usually don't have to care about branding at all. Normally it just works.

1) It would be better that Linux binary would be installed under /compat/linux. Ineed, this way we could have "javac" for Linux (the Oracle one) and "javac" of OpenJDK for FreeBSD, they would coexist easily....

They can already coexist esily. You can install as many JDKs as you like and switch between them with javavmwrapper and/or by setting the JAVA_HOME variable.

 

[Nicola Mingotti]

Hi olli@ ,

first of all that thank you for watching into the thing.

Ok, so the official brand number for Linux seems to be different, but the Linux binaries are still branded with ID 0. I don't know why, to be honest. Maybe the Centos people (from whom FreeBSD's Linux base is derived) just don't care because Linux doesn't use ABI emulation, so the brand ID doesn't matter there anyway.

True, but I guess when stuff enter FreeBSD it should meet our higher standards of complience with the documentation. It would not be difficult to rebrand all ELF when e.g. "linux_base-c7-7.4.1708_6" gets installed. I may propose that in Bugzilla. I think it is a good thing to do.

By the way, only dynamically linked binaries have ID 0. The statically linked ones have ID 3. This probably has to do with the fact that dynamically linked Linux binaries use ld-linux-x86-64.so as the run time linker, which in turn is branded with ID 3.

Good observation, i missed it.

Anyway, you usually don't have to care about branding at all. Normally it just works.

On this I disagree. Even if things work they must respect the documentation. Or people (as me in this case) would read doc, shoot some commands and find things do not match => think the stuff is broken / code abandoned.

They can already coexist esily. You can install as many JDKs as you like and switch between them with javavmwrapper and/or by setting the JAVA_HOME variable.

True, this is an option, which i already used

bye
n.

 

[olli@]

Even if things work they must respect the documentation. Or people (as me in this case) would read doc, shoot some commands and find things do not match => think the stuff is broken / code abandoned.

Well … If documentation and code differ, it is usually the documentation that is not up to date (or not precise enough), but the code is alright.
There are also cases where the documentation (e.g. the Handbook) simplifies things intentionally. Normally this is ok, because most people don't care. Only sometimes, people like you try to dig deeper.