HOWTO: Bareos Backup Server on FreeBSD

Bareos is a good open source alternative forked from Bacula. But:

  • Their RDMBS design sucks. They are trying to fix that in the new version. Still the new version has a lot of bugs and I don't think it is production ready yet.
  • Using the WEB UI for restoring files on large environments simply does not work. You have to use the bconsole.

We are currently using the director on a CentOS box. The database sits on a 96GB RAM SSD 2X Processors CentOS box. And we have 10 FreeBSD ZFS Storages running the SD. We backup around 170 servers.
 
@ gkontos
Thanks for sharing your thoughts. I have several questions if I may.


gkontos said:
  • Their RDMBS design sucks. They are trying to fix that in the new version. Still the new version has a lot of bugs and I don't think it is production ready yet.
Click to expand...

Sucks in what way?

You mean that 17.x is bugged currently? Even in 17.2.5 maintenance version? What problems do you hit?

gkontos said:
  • Using the WEB UI for restoring files on large environments simply does not work. You have to use the bconsole.
Click to expand...

You say about 16.x or 17.x version here? Does not work you mean hangs in the process or something else?
 
vermaden said:
@ gkontos
Thanks for sharing your thoughts. I have several questions if I may.


Sucks in what way?
Click to expand...

The database has a size of 240GB. 80% of that is being used by one table. File. We are using Percona and not Postgresql but I think that does not change much.

vermaden said:
You mean that 17.x is bugged currently? Even in 17.2.5 maintenance version? What problems do you hit?
Click to expand...

I am still using the 16.X version. The upgrade path is complicated and there is no roll back. However, I am following their mailing lists and I see that there are many problems.

vermaden said:
You say about 16.x or 17.x version here? Does not work you mean hangs in the process or something else?
Click to expand...

Yes, the 16.X version always, however they say that the same problem occurs in the 17.X version. My environment is very demanding and we are backing up managed clients. Some of them are running very large shared hosting WHM servers with really a lot of data.
When you try to restore from the Web Ui and you choose "Merge all related jobs to last full backup of selected backup job" the application just hangs there. This is a know problem and it has been discussed a lot in their lists.

A temporary solution that I have found since my database server has enough horsepower, is to run the following cron job in the director:

*/2 * * * echo ".bvfs_update"|/usr/sbin/bconsole

Still, there are some cases that I need to use the bconsole. Which is fine by me but the purpose is to have other people to be able to use the restore option when I am not around!
 
When you disable the sendmail in rc.conf you may also consider to disable submission and queue runner.

sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

Then when there's no sendmail running you don't need the periodic to clear the hoststat so you can disable it with
daily_clean_hoststat_enable="NO"
 
@ VladiBG

Its not that simple :)

Generally when You set sendmail_enable to NONE it will make all other to NO (including sendmail_enable), check /etc/rc.d/sendmail for details.

This is on a 11.1-RELEASE-p9 system.
Bash: 
% grep -A 6 'case ${sendmail_enable} in' /etc/rc.d/sendmail
case ${sendmail_enable} in
[Nn][Oo][Nn][Ee])
        sendmail_enable="NO"
        sendmail_submit_enable="NO"
        sendmail_outbound_enable="NO"
        sendmail_msp_queue_enable="NO"
        ;;

... and this sendmail_enable=NONE was set by FreeBSD Installer (bsdinstall) ... and yet in man rc.sendmail page it is said that NONE option is deprecated and should not be used. It will be removed in a future release ...

Code: 
sendmail_enable
     (str) If set to ``YES'', run the sendmail(8) daemon at system
     boot time. If setto ``NO'', do not run asendmail(8) daemon to
     listen forincoming network mail.This does not preclude a
     sendmail(8) daemonlistening on the SMTP port of the loopback
     interface. The ``NONE'' option is deprecated and should not be
     used.  It will be removed in a future release.

About /etc/periodic.conf there are a lot of things that can be disabled, but as asticle was too big already I did not wanted to cover another 'case' for FreeBSD setup as it was Bareos focused, but Yes, that also can be added.
 
gkontos said:
Bareos is a good open source alternative forked from Bacula. But:

  • Their RDMBS design sucks. They are trying to fix that in the new version. Still the new version has a lot of bugs and I don't think it is production ready yet.
  • Using the WEB UI for restoring files on large environments simply does not work. You have to use the bconsole.

We are currently using the director on a CentOS box. The database sits on a 96GB RAM SSD 2X Processors CentOS box. And we have 10 FreeBSD ZFS Storages running the SD. We backup around 170 servers.
Click to expand...

thanks for sharing your experience on bareos, do you have any experience on bareos vs bacula / pros and cons ?
 
vermaden said:
Generally when You set sendmail_enable to NONE it will make all other to NO (including sendmail_enable), check /etc/rc.d/sendmail for details.

This is on a 11.1-RELEASE-p9 system.
Bash: 
% grep -A 6 'case ${sendmail_enable} in' /etc/rc.d/sendmail
case ${sendmail_enable} in
[Nn][Oo][Nn][Ee])
        sendmail_enable="NO"
        sendmail_submit_enable="NO"
        sendmail_outbound_enable="NO"
        sendmail_msp_queue_enable="NO"
        ;;

... and this sendmail_enable=NONE was set by FreeBSD Installer (bsdinstall) ... and yet in man rc.sendmail page it is said that NONE option is deprecated and should not be used. It will be removed in a future release ...

Code: 
sendmail_enable
     (str) If set to ``YES'', run the sendmail(8) daemon at system
     boot time. If setto ``NO'', do not run asendmail(8) daemon to
     listen forincoming network mail.This does not preclude a
     sendmail(8) daemonlistening on the SMTP port of the loopback
     interface. The ``NONE'' option is deprecated and should not be
     used.  It will be removed in a future release.
Click to expand...
I thought sendmail as a whole was being removed from a future release.
I'm using 11.2, and the NONE option no longer removes all sendmail named startups on reboot. VladiBG and vermaden may have answered that.
 
sidetone said:
I thought sendmail as a whole was being removed from a future release.
I'm using 11.2, and the NONE option no longer removes all sendmail named startups on reboot. VladiBG and vermaden may have answered that.
Click to expand...
Disabling sendmail is one thing, taking care of growing /var/spool/clientmqueue is another.

As I disabled sendmail in /etc/rc.conf file.

Code: 
# grep sendmail /etc/rc.conf
  sendmail_enable=NONE
  sendmail_submit_enable=NO
  sendmail_outbound_enable=NO
  sendmail_msp_queue_enable=NO
# ps auxwww | grep sendmail | grep -v grep
#

I also took care of that growing problem with separate crontab job.

Code: 
# cat /etc/cron.d/sendmail-clean-clientmqueue 
# CLEAN SENDMAIL
*/3     *       *       *       *       root    /bin/rm -r -f /var/spool/clientmqueue/*
 
gkontos said:
The database has a size of 240GB. 80% of that is being used by one table. File. We are using Percona and not Postgresql but I think that does not change much.
Click to expand...

And in what way would that suck? I'd suppose thats what a database is built for. BTW: it's 95% in R.17/postgres10.

I am still using the 16.X version. The upgrade path is complicated and there is no roll back.
Click to expand...

That may depend on the features used, but I didn't recognize it as complicated, only as time and disk consuming. It will basically need 3 times the diskspace (or 5 times when keeping redolog backup), and there was no more task than executing the conversion SQL commands.
Rollback would basically mean to run a point-in-time recovery on the database to the point before the conversion.

However, I am following their mailing lists and I see that there are many problems.
Click to expand...

I tried to read these things back in 2008 when I tried to fix some of the bugs. I was not really comfortable with the working style there. I dont know if or how that may have changed with the bareos fork.
 
VladiBG said:
When you disable the queue there's no need to remove the clientmqueue.
Click to expand...

Not quite, there will always happen someone/something will just write mail/notification to local account:

Code: 
# mail root
Subject: asd
asd
EOT

# ls -l /var/spool/clientmqueue
total 2
-rw-rw----  1 vermaden  smmsp    4 Aug  4 17:58 dfw74FwWGb015949
-rw-rw----  1 vermaden  smmsp  526 Aug  4 17:58 qfw74FwWGb015949

That is why I put this crontab job to not have disk filled with them ...
 
gkontos said:
PMc Are you using Bareos in a production environment or are you just trolling me here?
Click to expand...

Hä?? For production environment you probably should use Microsoft Windows Server and Tivoli Spectrum Protect, and make sure for good MCSE, not to forget ITIL as well. We are hackers.
 
vermaden said:
Not quite, there will always happen someone/something will just write mail/notification to local account:

Code: 
# mail root
Subject: asd
asd
EOT

# ls -l /var/spool/clientmqueue
total 2
-rw-rw----  1 vermaden  smmsp    4 Aug  4 17:58 dfw74FwWGb015949
-rw-rw----  1 vermaden  smmsp  526 Aug  4 17:58 qfw74FwWGb015949

That is why I put this crontab job to not have disk filled with them ...
Click to expand...
change the directory to readonly and disable the queue.
 
VladiBG said:
change the directory to readonly and disable the queue.
Click to expand...
or Soft link it to /tmp, or mount it as a RAM file system. I'm not using a crontab to remove it.

In my opinion, it's better to leave it, so I can see what happened. As long as it doesn't take up over 100MB, all sendmail services are off (not other applications that call it), and doesn't give me warning messages on boot-up, that accomplishes the task well enough for me.

I don't know why they complicated it. Before, sendmail related applications only had to be unset with NONE. Above solutions that removed all related sendmail instances from rc.conf will do.
 
You must log in or register to reply here.
Back
Top