Solved Sendmail start fails after pkg upgrade

FKEinternet

FKEinternet

In the process of updating MySQL last week, I did a pkg upgrade and after my server rebooted, sendmail(8) won't start:
Code: 
/etc/mail # make start
Starting: sendmailShared object "libdb-6.1.so" not found, required by "sendmail"
 sendmail-clientmqueueShared object "libdb-6.1.so" not found, required by "sendmail"
I tried creating a symlink from libdb-6.2.so (which exists) as libdb-6.1.so and now sendmail(8) fails differently:
Code: 
# make start
Starting: sendmail554 5.0.0 Berkeley DB version mismatch: compiled against 6.1.26, run-time linked against 6.2.23
 sendmail-clientmqueueBerkeley DB version mismatch: compiled against 6.1.26, run-time linked against 6.2.23
I tried
Code: 
# pkg upgrade sendmail
Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
All repositories are up-to-date.
pkg: sendmail is not installed, therefore upgrade is impossible
Checking integrity... done (0 conflicting)
Your packages are up to date.
and
Code: 
/usr/ports/mail/sendmail # make
===>  License Sendmail accepted by the user
===>  Found saved configuration for sendmail+tls+sasl2+db6-8.15.2
===>  sendmail+tls+sasl2+db5-8.15.2_3 depends on file: /usr/local/sbin/pkg - found
=> sendmail.8.15.2.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch ftp://ftp7.freebsd.org/pub/FreeBSD/ports/distfiles/sendmail.8.15.2.tar.gz
fetch: ftp://ftp7.freebsd.org/pub/FreeBSD/ports/distfiles/sendmail.8.15.2.tar.gz: No route to host
=> Attempting to fetch ftp://ftp.sendmail.org/pub/sendmail/sendmail.8.15.2.tar.gz
fetch: ftp://ftp.sendmail.org/pub/sendmail/sendmail.8.15.2.tar.gz: Permission denied
=> Attempting to fetch ftp://ftp7.freebsd.org/pub/FreeBSD/ports/distfiles/sendmail.8.15.2.tar.gz
fetch: ftp://ftp7.freebsd.org/pub/FreeBSD/ports/distfiles/sendmail.8.15.2.tar.gz: No route to host
=> Couldn't fetch it - please try to retrieve this
=> port manually into /usr/ports/distfiles/ and try again.
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/mail/sendmail
*** Error code 1

Stop.
make: stopped in /usr/ports/mail/sendmail
Why is it checking dependencies for sendmail+tls+sasl2+db5-8.15.2_3 where there's a saved configuration for sendmail+tls+sasl2+db6-8.15.2?

What do I have to do to get mail/sendmail working again?

This is rather urgent, because this is a PRODUCTION server that blew up!

Code: 
# uname -a
FreeBSD Dreamer.FKEinternet.net 10.2-RELEASE FreeBSD 10.2-RELEASE #0: Mon Oct  5 23:53:36 EDT 2015  root@Dreamer.FKEinternet.com.:/usr/obj/usr/src/sys/GENERIC  amd64
 
Last edited by a moderator:
FKEinternet said:
I tried creating a symlink from libdb-6.2.so (which exists) as libdb-6.1.so and now sendmail fails differently:
Click to expand...

In general, you can't do that with versioned shared objects/libraries.

FKEinternet said:
=> Attempting to fetch ftp://ftp7.freebsd.org/pub/FreeBSD/ports/distfiles/sendmail.8.15.2.tar.gz
fetch: ftp://ftp7.freebsd.org/pub/FreeBSD/ports/distfiles/sendmail.8.15.2.tar.gz: No route to host
Click to expand...

Solve this problem (lack of network connectivity, to download sources), and you should be able to rebuild sendmail.

FKEinternet said:
Why is it checking dependencies for sendmail+tls+sasl2+db5-8.15.2_3 where there's a saved configuration for sendmail+tls+sasl2+db6-8.15.2?
Click to expand...

BDB 5 is the default version for FreeBSD, and it is quite possible that BDB 5 will end up being the final default version. Oracle changed BDB 6 to use a toxic license, so it is now unusable for many purposes where it was previously ok. Sendmail is possibly one of the only places where it is legally safe to use BDB 6 now; it exposes you to significant liability (Oracle charge a ridiculous per-CPU fee for it) if it gets anywhere near web content or app backend code. In general, avoid BDB 6 due to potentially large legal liability, and always use BDB 5 in preference. In the longer future, if BDB 5 starts to become too old/obsolete, migrate away from BDB to code with a good license (and from someone other than Oracle). For smaller installations, stick with the (relatively ancient, but it still works nicely for the stuff it always did) BDB 1.x (BSD licensed) which is included in the FreeBSD base.

To me, the correct question here is why there is a saved config using db6? That indicates something was deliberately changed from defaults in the past. Your rebuild attempt is simply using the normal FreeBSD default version for it (db5), and the build failure is due to lack of network connectivity rather than BDB versions.

My recommendation is to fix the problem with fetching sources, and rebuild Sendmail against db5. Install the latest databases/db5 if necessary, but don't remove db6 until you are certain that you don't have anything else depending on it.
 
FKEinternet said:
I tried creating a symlink from libdb-6.2.so (which exists) as libdb-6.1.so and now sendmail fails differently:
Click to expand...
Please undo that. Linking libraries like that is mostly just a way to take a clear error and make it stranger and more difficult to fix.
 
You learn something new every day. I have never started sendmail using make start and never heard of it till now. I always use service sendmail onestart
 
wblock@ said:
Please undo that. Linking libraries like that is mostly just a way to take a clear error and make it stranger and more difficult to fix.
Click to expand...
I didn't really expect linking the wrong library version to work, it was just an experiment I tried in the process of attempting to get my server working again. I had forgotten to remove the symlinks, though, so thanks for the reminder!
 
Murph said:
Solve this problem (lack of network connectivity, to download sources), and you should be able to rebuild sendmail.
Click to expand...
I found it really odd there wasn't a route to ftp://ftp7.freebsd.org/pub/FreeBSD/ports/distfiles/sendmail.8.15.2.tar.gz. I tried using traceroute(8) but it seems the UDP packets are being blocked by IPFW which is set up to allow ICMP packets for traceroute(8). I found and installed sysutils/tcptraceroute and checked the server address:
Code: 
# tcptraceroute ftp7.freebsd.org
Selected device re0, address 100.0.193.99, port 47109 for outgoing packets
Tracing the path to ftp7.freebsd.org (212.219.56.184) on TCP port 80 (http), 30 hops max
1  lo0-100.BSTNMA-VFTTP-304.verizon-gni.net (100.0.193.1)  0.873 ms  4.060 ms  1.295 ms
2  B3304.BSTNMA-LCR-21.verizon-gni.net (100.41.201.13)  4.982 ms  2.465 ms  3.963 ms
3  * * *
4  * * *
5  0.ae7.GW10.EWR6.ALTER.NET (140.222.231.129)  13.909 ms  13.042 ms  16.170 ms
6  customer.customer.alter.net (157.130.91.86)  12.101 ms  13.999 ms  14.463 ms
7  nyk-bb2-link.telia.net (62.115.134.109)  12.736 ms  12.905 ms  10.899 ms
8  ldn-bb2-link.telia.net (213.155.133.6)  87.360 ms  90.585 ms  88.851 ms
9  ldn-b3-link.telia.net (62.115.137.197)  87.316 ms  86.608 ms  85.195 ms
10  jisc-ic-318433-ldn-b3.c.telia.net (62.115.148.159)  80.234 ms  81.564 ms  81.709 ms
11  ae29.londpg-sbr2.ja.net (146.97.33.2)  82.335 ms  80.600 ms  81.831 ms
12  ae30.londtw-sbr2.ja.net (146.97.33.6)  81.793 ms  80.232 ms  82.312 ms
13  kpsn.londtw-sbr2.ja.net (146.97.41.86)  82.862 ms  82.107 ms  82.119 ms
14  212.219.171.222  83.104 ms  85.585 ms  88.442 ms
15  www.mirrorservice.org (212.219.56.184) [open]  85.161 ms  83.739 ms  85.358 ms
I double-checked to be sure something hadn't changed while I wasn't looking, but make failed with the same "No route to host" error. I checked to see if the firewall was blocking the FTP connection:
Code: 
# less /var/log/security
Aug 21 11:00:00 Dreamer newsyslog[20945]: logfile turned over due to size>100K
Aug 21 11:00:09 Dreamer kernel: ipfw: 56599 Deny TCP 123.59.55.92:48049 100.0.193.99:3128 in via re0
Aug 21 11:00:44 Dreamer kernel: ipfw: 56599 Deny TCP 93.174.95.87:36924 100.0.193.102:110 in via re0
Aug 21 11:00:47 Dreamer last message repeated 2 times
Aug 21 11:00:47 Dreamer kernel: ipfw: 299 Deny TCP 100.0.193.99:23489 212.219.56.184:21 out via re0
Aug 21 11:00:54 Dreamer kernel: ipfw: 299 Deny TCP 100.0.193.99:59597 212.219.56.184:21 out via re0
Aug 21 11:01:45 Dreamer kernel: ipfw: 56599 Deny TCP 182.191.88.195:45481 100.0.193.102:23 in via re0
Aug 21 11:01:48 Dreamer kernel: ipfw: 56599 Deny TCP 182.191.88.195:45481 100.0.193.102:23 in via re0
Aug 21 11:02:13 Dreamer kernel: ipfw: 299 Deny TCP 100.0.193.99:28015 212.219.56.184:21 out via re0
Aug 21 11:02:13 Dreamer kernel: ipfw: 299 Deny TCP 100.0.193.99:23314 209.246.26.16:21 out via re0
Aug 21 11:02:13 Dreamer kernel: ipfw: 299 Deny TCP 100.0.193.99:26752 212.219.56.184:21 out via re0
Aug 21 11:04:10 Dreamer kernel: ipfw: 299 Deny TCP 100.0.193.99:25337 212.219.56.184:21 out via re0
Aug 21 11:04:10 Dreamer kernel: ipfw: 299 Deny TCP 100.0.193.99:37933 209.246.26.16:21 out via re0
Aug 21 11:04:10 Dreamer kernel: ipfw: 299 Deny TCP 100.0.193.99:65061 212.219.56.184:21 out via re0
I then tried
Code: 
# wget ftp://ftp7.freebsd.org/pub/FreeBSD/ports/distfiles/sendmail.8.15.2.tar.gz
--2016-08-21 11:05:45--  ftp://ftp7.freebsd.org/pub/FreeBSD/ports/distfiles/sendmail.8.15.2.tar.gz
  => 'sendmail.8.15.2.tar.gz'
Resolving ftp7.freebsd.org (ftp7.freebsd.org)... 212.219.56.184, 2001:630:341:12::184
Connecting to ftp7.freebsd.org (ftp7.freebsd.org)|212.219.56.184|:21... failed: Permission denied.
Connecting to ftp7.freebsd.org (ftp7.freebsd.org)|2001:630:341:12::184|:21... failed: No route to host.
It seems the "No route to host" message is a red herring - the real problem is "Permission denied" on the IPV4 connection. Is there something that happened in my port upgrade that would have caused this? (cleaned log file attached)
 

Attachments

When I connect to ftp7.freebsd.org using WinSCP and navigate to /pub/FreeBSD/ports/distfiles/ I find a README.txt that says
Code: 
Our distfiles cache has been moved to:

http://distcache.FreeBSD.org/ports-distfiles/

distfiles can be fetched by name from there, as specified
in the corresponding port that uses it.
I just re-ran portsnap fetch update (again, for the nth time this week), then tried make again in the mail/sendmail ports directory - and got the same FTP failure.

If the distfiles cache has been moved, why is make in a port directory still using the old path when attempting to fetch the distfiles for the port?

Also, when I try to navigate to http://distcache.freebsd.org/ports-distfiles/ with my browser, I get a 403 Forbidden error page from nginx.

What do I have to do to retrieve the mail/sendmail distribution files???
 
I do not have a working mail/sendmail installation since my server rebooted itself last night. I need to fix this because it's a production server and none of the domains on the server are able to send or receive mail!

Because of the problems described above, I am unable to retrieve the mail/sendmail distribution files, so I tried
Code: 
# pkg install sendmail
Updating FreeBSD repository catalogue...
Fetching meta.txz: 100%  944 B  0.9kB/s  00:01
Fetching packagesite.txz: 100%  6 MiB  5.8MB/s  00:01
Processing entries: 100%
FreeBSD repository update completed. 25584 packages processed.
Updating database digests format: 100%
pkg: No packages available to install matching 'sendmail' have been found in the repositories
I can't get the mail/sendmail port's distribution files, and there isn't a package available, so at the moment I'm in a really difficult position.

Have I missed something? Am I not asking the right question? Did I overlook something? What do I have to do to fix this?
 
Hi,

More info -- Try pkg install mail/sendmail or do pkg search sendmail to get the available pkg name for sendmail. With pkg search mail/sendmail I get version sendmail+tls+sasl2-8.15.2_3 and pkg install mail/sendmail will install with that version.
Also, Have /etc/pkg/FreeBSD.conf or /usr/local/etc/pkg.conf been changed?
 
good-beastie said:
Hi,

More info -- Try pkg install mail/sendmail or do pkg search sendmail to get the available pkg name for sendmail. With pkg search mail/sendmail I get version sendmail+tls+sasl2-8.15.2_3 and pkg install mail/sendmail will install with that version.
Also, Have /etc/pkg/FreeBSD.conf or /usr/local/etc/pkg.conf been changed?
Click to expand...
I see what I did wrong - I didn't qualify the sendmail package properly: pkg install mail/sendmail got me back up and running.

Thanks again!
 
FKEinternet said:
I see what I did wrong - I didn't qualify the sendmail package properly: pkg install mail/sendmail got me back up and running.
Click to expand...
N.B. that packaged version is built without BDB. That might not actually be a problem for you. One of the questions I was going to ask you (had the issue not been resolved), is whether you actually need BDB for it. In essence, it's an optional feature, and only needed for larger instances of Sendmail or if you need specific functionality offered by BDB. Larger instances is pretty much the product (mail_volume * largest_table_size). I can't offer specific advice on a threshold for when BDB becomes essential, it's just something which may be useful for scaling up a Sendmail instance.

Historically, I would have generally encouraged the use of BDB with Sendmail, for a real MTA (i.e. a real mail server, and not just a random host). After Oracle assimilated BDB and did what they did to the license, I now discourage the use of it unless actually necessary.

If all is good running the non-BDB build from a package, stick with that to make your ongoing life easier (i.e. to hopefully avoid a repeat of this type of issue). Please do make sure that you have rebuilt all of your maps/tables/aliases/etc.
 
You must log in or register to reply here.
Back
Top