Building mosquitto-auth-plug on FreeBSD

H

holala

Hello. I'm struggling with mosquitto-auth-plug on FreeBSD. I'm following the (pretty basic) instruction provided by the authors. I do successfully build mosquitto-1.6.9.
I do successfully build also mosquitto-auth-plug, but later I'm getting undefined symbols, when mosquitto starts and loads the auth-plug.so


Code: 
1610714616: mosquitto version 1.6.9 starting
1610714616: Config loaded from /etc/mosquitto/mosquitto.conf.
1610714616: Loading plugin: /home/mzk/mosquitto-auth-plug/auth-plug.so
1610714616: Error: Unable to load auth plugin "/home/mzk/mosquitto-auth-plug/auth-plug.so".
1610714616: Load error: /home/mzk/mosquitto-auth-plug/auth-plug.so: Undefined symbol "mosquitto_client_id"

What worries me is that when I do ldd on a linux machine, there are a lot more dependent libraries than on my FreeBSD machine. Here is the output from Ubuntu:
Code: 
ldd auth-plugin.so 
    linux-vdso.so.1 =>  (0x00007ffc017c8000)
    libcdb.so.1 => /usr/lib/x86_64-linux-gnu/libcdb.so.1 (0x00007f0cbba17000)
    libmysqlclient.so.20 => /usr/lib/x86_64-linux-gnu/libmysqlclient.so.20 (0x00007f0cbb460000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0cbb243000)
    libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f0cbaf6e000)
    libhiredis.so.0.13 => /usr/lib/x86_64-linux-gnu/libhiredis.so.0.13 (0x00007f0cbad61000)
    libpq.so.5 => /usr/lib/x86_64-linux-gnu/libpq.so.5 (0x00007f0cbab31000)
    libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f0cba8e0000)
    libcurl-gnutls.so.4 => /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4 (0x00007f0cba673000)
    libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f0cba22e000)
    libmosquitto.so.1 => /usr/lib/x86_64-linux-gnu/libmosquitto.so.1 (0x00007f0cba01e000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0cb9c54000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0cb9a50000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f0cb9836000)
    libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f0cb95ce000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f0cb924c000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0cb8f43000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f0cb8d2d000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f0cbbe28000)
    libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f0cb8ae3000)
    liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f0cb88d4000)
    libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f0cb86b9000)
    libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f0cb849e000)
    libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007f0cb825d000)
    libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f0cb7f2d000)
    libidn.so.11 => /usr/lib/x86_64-linux-gnu/libidn.so.11 (0x00007f0cb7cfa000)
    librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f0cb7ade000)
    libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007f0cb78a8000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f0cb76a0000)
    libcares.so.2 => /usr/lib/x86_64-linux-gnu/libcares.so.2 (0x00007f0cb748f000)
    libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f0cb71bd000)
    libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f0cb6f8e000)
    libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f0cb6d8a000)
    libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f0cb6b7f000)
    libheimntlm.so.0 => /usr/lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007f0cb6976000)
    libkrb5.so.26 => /usr/lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007f0cb66ec000)
    libasn1.so.8 => /usr/lib/x86_64-linux-gnu/libasn1.so.8 (0x00007f0cb644a000)
    libhcrypto.so.4 => /usr/lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007f0cb6217000)
    libroken.so.18 => /usr/lib/x86_64-linux-gnu/libroken.so.18 (0x00007f0cb6001000)
    libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f0cb5d9d000)
    libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f0cb5b8a000)
    libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007f0cb5957000)
    libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f0cb56d7000)
    libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f0cb54d3000)
    libwind.so.0 => /usr/lib/x86_64-linux-gnu/libwind.so.0 (0x00007f0cb52aa000)
    libheimbase.so.1 => /usr/lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007f0cb509b000)
    libhx509.so.5 => /usr/lib/x86_64-linux-gnu/libhx509.so.5 (0x00007f0cb4e50000)
    libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f0cb4c18000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f0cb4a10000)

And here is the output from my FreeBSD machine:
Code: 
ldd ./auth-plug.so 
./auth-plug.so:
    libmariadb.so.3 => /usr/local/lib/mysql/libmariadb.so.3 (0x800683000)
    libcrypto.so.111 => /lib/libcrypto.so.111 (0x800e00000)
    libc.so.7 => /lib/libc.so.7 (0x80024e000)
    libm.so.5 => /lib/libm.so.5 (0x8006e4000)
    libssl.so.111 => /usr/lib/libssl.so.111 (0x800716000)
    libz.so.6 => /lib/libz.so.6 (0x8007ae000)
    libthr.so.3 => /lib/libthr.so.3 (0x8007ca000)

I know I shouldn't compare the linux package as it might have been built with all the mosquitto-auth-plug backends, but it is a hint for me that probably on FreeBSD the library dependencies are not properly resolved.

Since I'm very far from porting (and building) software, I'll be very helpful if someone can give me some hints on this.

Thanks!
 
Can you post a link to the instructions so other people can have a look too? It's often a case of making sure dependencies are installed prior to the build and setting the correct options for ./configure (if it uses configure).
 
Sure.

I'm using this fork of mosquitto-auth-plug
----
github.com

GitHub - coldfire84/mosquitto-auth-plug: Authentication plugin for Mosquitto with multiple back-ends (MySQL, Redis, CDB, SQLite3)

Authentication plugin for Mosquitto with multiple back-ends (MySQL, Redis, CDB, SQLite3) - coldfire84/mosquitto-auth-plug
github.com github.com
----

The building instructions are untouched:
In order to compile the plugin you'll require:


  • a copy of the Mosquitto source code together with the libraries required for the back-end you want to use in the plugin, and
  • a recent version of OpenSSL (if the version with your OS, e.g., OS X, is too old, you may need to use one supplied by home brew or build your own).

Copy config.mk.in to config.mk and modify config.mk to suit your building environment. In particular, you have to configure which back-ends you want to provide as well as the path to the Mosquitto source and its library, and possibly the path to OpenSSL (OPENSSLDIR).


After a make you should have a shared object called auth-plug.so which you will reference in your mosquitto.conf.
Click to expand...

BTW I found unresolved, but closed issue posted on the original author's github page:
github.com

mosquitto_client_id: symbol not found · Issue #397 · jpmens/mosquitto-auth-plug

Here is my mosquitto.conf: persistence true persistence_location /mosquitto/data/ log_dest file /mosquitto/log/mosquitto.log ... auth_plugin /mosquitto/src/mosquitto-auth-plug/auth-plug.so auth_opt...
github.com github.com

The mosquitto source code can be downloaded from https://mosquitto.org/files/

The install instructions are:
The following packages can be used to add features to mosquitto. All of them
are optional.

* openssl
* c-ares (for DNS-SRV support, disabled by default)
* tcp-wrappers (optional, package name libwrap0-dev)
* libwebsockets (optional, disabled by default, version 1.3 and above)
* On Windows, a pthreads library is required if threading support is to be
included.

To compile, run "make", but also see the file config.mk for more details on the
various options that can be compiled in.

Where possible use the Makefiles to compile. This is particularly relevant for
the client libraries as symbol information will be included. Use cmake to
compile on Windows or Mac.

If you have any questions, problems or suggestions (particularly related to
installing on a more unusual device like a plug-computer) then please get in
touch using the details in readme.txt.
Click to expand...

There is no readme.txt file inside the mosquitto archive. The text in readme.md is no more informative, just saying that install instructions can be found at https://mosquitto.org/download/ which is untrue.

This isn't a big deal since I'm not sure that the problem is in mosquitto source. Neither I do believe it is in mosquitto-auth-plug, but I suspect something with the library linking...
 
holala said:
Code: 
Config loaded from /etc/mosquitto/mosquitto.conf.
Click to expand...
That needs patching. Third-party software has to be installed with a /usr/local prefix, the config should therefor be in /usr/local/etc/mosquitto/mosquitto.conf.

What does make printconfig output?
 
SirDice said:
That needs patching.
Click to expand...
mosquitto loads the config file with the -c argument, so I'm fine with it now. I agree that once ported it should utilize /usr/local/etc/

Do you believe that the problem is related to mosquitto and not to mosquitto-auth-plug?

This is printconfig for mosquitto-auth-plug
Code: 
 gmake printconfig
Selected backends:         MySQL
Using mosquitto source dir: ../mosquitto-1.6.9
OpenSSL install dir:        /usr

If you changed the backend selection, you might need to 'make clean' first

CFLAGS:  -I/usr/local/include  -I../mosquitto-1.6.9/src/ -I../mosquitto-1.6.9/lib/ -fPIC -Wall -Werror -DBE_MYSQL -I/usr/local/include/mysql -I/usr/local/include/mysql/mysql -I/src -DDEBUG=1 -I/usr/include
LDFLAGS: -L/usr/local/lib   -L../mosquitto-1.6.9/lib/
LDADD:   -L/usr/local/lib/mysql/ -lmariadb -L/usr/lib -lcrypto -lmosquitto
 
holala said:
Do you believe that the problem is related to mosquitto and not to mosquitto-auth-plug?
Click to expand...
Looking back at the error it looks like you're running the mosquitto application from your home directory, I suspect that the auth-plugin needs a library from the application and it can't find that. The Linux ldd(1) output seems to confirm that:
Code: 
 libmosquitto.so.1 => /usr/lib/x86_64-linux-gnu/libmosquitto.so.1 (0x00007f0cba01e000)

You might need to use LD_PRELOAD to temporarily include the application's lib directory in your home directory if you want to test it from there.
 
Well, the mosquitto application is systemwide installed (gmake install), and I can invoke it from anywhere.
The mosquitto-auth-plug is sitting in my home directory and I'm pointing it in mosquitto.conf.

Do I properly export LD_PRELOAD?
export LD_PRELOAD=/home/mzk/mosquitto-1.6.9/lib/libmosquitto.so

Also, would ldd return also missing libraries? Or it reports only the existing (and found) libraries? I believe it wold tell if can't find some library and makes me curious why libmosquitto.so is missing in my FreeBSD's ldd report...
 
holala said:
Also, would ldd return also missing libraries? Or it reports only the existing (and found) libraries?
Click to expand...
It shows a list of libraries that are linked to the executable or library you're examining. It would show 'missing' if they can't be found on the system.

For some reason your auth-plug.so hasn't been linked with libmosquitto.so.1, that's probably the reason for the "undefined Symbol" error.
 
SirDice said:
For some reason your auth-plug.so hasn't been linked with libmosquitto.so.1, that's probably the reason for the "undefined Symbol" error.
Click to expand...
Click to expand...
That's exactly what I think happens. But I don't understand why the compiler or the linker stay calm.

On macOS they mention the use of additional flags:
CFG_LDFLAGS = -undefined dynamic_lookup

But I'm not sure this is gcc compatible. Also no idea what it does.
 
holala said:
But I don't understand why the compiler or the linker stay calm.
Click to expand...
It does appear to get the right directories for CFLAGS and LDFLAGS:
Code: 
CFLAGS:  -I/usr/local/include  -I../mosquitto-1.6.9/src/ -I../mosquitto-1.6.9/lib/ -fPIC -Wall -Werror -DBE_MYSQL -I/usr/local/include/mysql -I/usr/local/include/mysql/mysql -I/src -DDEBUG=1 -I/usr/include
LDFLAGS: -L/usr/local/lib   -L../mosquitto-1.6.9/lib/
Both include ../mosquitto-1.6.9/lib/
And LDADD includes libmosquitto
Code: 
LDADD:   -L/usr/local/lib/mysql/ -lmariadb -L/usr/lib -lcrypto -lmosquitto
So everything indicates it should be linked. One thing you might try:
Code: 
LDADD:   -L/usr/local/lib/mysql/ -lmariadb -L/usr/lib -lcrypto -L/usr/local/lib -lmosquitto
I assume there is a libmosquitto.so.1 in /usr/local/lib.
 
SirDice, it seems that somehow I managed to compile it without the mosquitto lib. My current ldd output shows the usage of libmosquitto.so:


Code: 
/auth-plug.so:
    libmariadb.so.3 => /usr/local/lib/mysql/libmariadb.so.3 (0x800683000)
    libcrypto.so.111 => /lib/libcrypto.so.111 (0x800e00000)
    libmosquitto.so.1 => /usr/local/lib/libmosquitto.so.1 (0x8006e4000)
    libc.so.7 => /lib/libc.so.7 (0x80024e000)
    libm.so.5 => /lib/libm.so.5 (0x8006ff000)
    libssl.so.111 => /usr/lib/libssl.so.111 (0x800731000)
    libz.so.6 => /lib/libz.so.6 (0x8007c9000)
    libthr.so.3 => /lib/libthr.so.3 (0x8010f2000)

However mosquitto still fails to start with the same error:
Code: 
1610729680: Loading plugin: /home/mzk/mosquitto-auth-plug/auth-plug.so
1610729680: Error: Unable to load auth plugin "/home/mzk/mosquitto-auth-plug/auth-plug.so".
1610729680: Load error: /home/mzk/mosquitto-auth-plug/auth-plug.so: Undefined symbol "mosquitto_client_id"
 
Is /usr/local/lib/libmosquitto.so.1 from the same version as the one in ../mosquitto-1.6.9/lib/? The issue may be version dependent, I can imagine a newer (or older) version of a library not having a certain symbol.
 

$ md5 /usr/local/lib/libmosquitto.so.1
MD5 (/usr/local/lib/libmosquitto.so.1) = c5ac5da880fd6299ec74de8c037f80bb
$ md5 /home/mzk/mosquitto-1.6.9/lib/libmosquitto.so.1
MD5 (/home/mzk/mosquitto-1.6.9/lib/libmosquitto.so.1) = c5ac5da880fd6299ec74de8c037f80bb


I'll try the same version on my linux box to verify whether the problem is mosquitto related or not (some users mention that earlier version works OK). But I'll need some time to get back home, so I'll be away for a while.
 
On Linux it builds and starts as expected (not tested if actually works, but it doesn't quit with error)...
 
It went deeper than I thought it would...

1. I forgot that I installed the mosquitto binary package, so I removed it. No change (I removed everything and built and installed again).
2. I switched the compiler from gcc9 to gcc 7.5 (as on my linux box). No change.
3. I did ldd and nm on auth_plug.so and interestingly on both OSes nm shows "mosquitto_client_id" (and few others) as "U" (the symbol is undefined). This doesn't stop it from working on Linux, however, despite on both OSes the same symbols are shown as undefined.
Some light on the horizon from this post:

239128 – audio/swhplugins: fix shared library errors

bugs.freebsd.org bugs.freebsd.org

There some libs failed to load due unresolved symbols.

Log:
Three plugins used C89 inline semantics, which resulted in unresolved
symbols with C99 compilers.

PR: 239128
Reported by: Marcel Bonnet
Click to expand...
Any suggestions are welcome.

#######

Edit: just checked the compiler and I totally forgot that FreeBSD now uses clang by default. Added CC=gcc7 in both mosquitto and mosq-auth-plug, rebuilt everything. Again no luck.
Also I addred -export-dynamic in the LDFLAGS for mosq-auth-plug. From the man page:
Pass the flag -export-dynamic to the ELF linker, on targets that
support it. This instructs the linker to add all symbols, not only
used ones, to the dynamic symbol table. This option is needed for
some uses of "dlopen" or to allow obtaining backtraces from within
a program.
Click to expand...
###########

Edit 2:
I found that doing `cmake` inside the mosquitto dir did the job! Idk why and I'll be glad if someone explains, but here is the quick reference:
Download mosquitto 1.6.12
tar xvf...
from inside the dir you could do gmake uninstall (as root)
then cmake . (the current directory is the source directory)
gmake
gmake install (as root)
ldconfig

then clone https://github.com/coldfire84/mosquitto-auth-plug.git
cp config.mk.in config.mk
edit MOSQUITTO_SRC = ../mosquitto-1.6.12 (according to your path)
CFG_LDFLAGS =-L/usr/local/lib
gmake

Set auth_plugin in mosquitto.conf to point to your newly built auth-plug.so
and it should work now...
Whew, another day spent in software. Btw if the mosquitto readme didn't insist so much that cmake should be used on windows or Mac and make on other OSes, it would have spared me some 12 hours :/ As SirDice states, experience...
 
You must log in or register to reply here.
Back
Top