Solved protection of /usr/local/etc/ files (or lack thereof)

in Gentoo if I change any configuration file (in /etc/) it will be protected from overwrite during package upgrades no matter how hard I emerge (or unmerge) pkgs that own those files.
how is this translated in FreeBSD with regards to files in /usr/local/etc/?

I just did the pkg-2 upgrade, and thus got msp430-debug-stack and mspdebug removed (among other 15 pkgs).

I had to manually install them back with `pkg install /var/cache/pkg/mspFOO.pkg` and surprise, my customized /usr/local/etc/devd/mspfet.conf got replaced with the file in the pkg. I recall wasting a full day with devd to reach a working setup and now it's lost. I can't even.

it looks like
pkg remove msp430-debug-stack
removes /usr/local/etc/devd/mspfet.conf even if it is has been customized by the user. personally I find this in no way acceptable.

So how do I make configuration files immutable as far as pkg interactions are concerned?
 
ports(7) (from which packages are build from) have an mechanism to protect customized application configuration files, but it must be configured to do so in their Makefile. For details see porters handbook "@sample file".

In short, if configured, a *.conf.sample file is installed besides a *.conf file.

For example print/cups
Code:
/usr/local/etc/devd/cups.conf
/usr/local/etc/devd/cups.conf.sample
Upon package removal/upgrade the *.conf file will be preserved, and only the *.conf.sample will be removed/replaced.

devel/msp430-debug-stack port isn't configuration to preserve /usr/local/etc/devd/mspfet.conf.

So how do I make configuration files immutable as far as pkg interactions are concerned?
I suspect the port maintainer hasn't considered that someone would customize the .conf file. You could ask the ports maintainer to implement it (open a PR).
 
Upon package removal/upgrade the *.conf file will be preserved, and only the *.conf.sample will be removed/replaced.
I think that .conf is preserved only if it differs from .conf.sample. But that should be good enough.

But yeah, in the past I encountered some bad packages that took configuration files with them, no matter if the files had been edited or not.
 
I'm not too sure that this is an actual problem, though I can fully understand (and agree!) that this behavior can be annoying to deal with.

Thing is... when dealing with the base system then the general idea is to avoid editing system config files and instead focus on 'local' files. For example /etc/rc.conf vs. /etc/rc.conf.local. It also ties into the general behavior of default config files defining current standards (referring to /etc/defaults) while configuration basically consists of overriding those default values when needed.

I know this isn't fully comparable, but ... it's considered bad practice to manually edit files in, say, /usr/local/etc/rc.d, instead you'd apply any needed changes by (re)setting the proper values in previously mentioned local rc file (rc.conf.local).

My point is simple: the main motivation behind all this is to avoid having to deal with config files getting overwritten during an upgrade. Of course that isn't always a valid argument, also considering the existence of mergemaster(8) (eventually to be replaced with etcupdate(8)), but it still seems reasonable to me to try and apply the same logic to other system-related config files. Such as those in /usr/local/etc/devd.

Wouldn't it make more sense to override any values in /etc/devd.conf instead? Also considering the fact that this config file is the only reason why /usr/local/etc/devd gets included?

At the very least it would avoid a few issues here.
 
At the very least it would avoid a few issues here.

I like the .conf.sample approach as described by T-Daemon in comment #2. it makes running a diff between the current config and the original one very easy. also it helps sometimes when you want to reconfigure an application and you forgot the name of the variable that needs to be customized. in Gentoo I sometimes needed to manually stash away the config, merge the package again just to obtain the original config and this is very messy in comparison.

however I would have liked if there would be a pkg setting that activates a check during package removal. if file is in /usr/local/etc AND it's checksum differs compared to install time then file would be kept in place.
package installs that would replace that file would redirect the contents to a special hidden .__000${AUTO_INCREMENT}_${pkg_config_filename} in the same dir and the user is expected to optionally run etcupdate to merge the differences.

regarding devd in particular, I don't think I can have multiple rule files that would match the same device being discovered (using the same match properties). I would instead have to somehow make sure my rules come in first and I don't trust that in a few versions (or when it gets rewritten) the ordering will remain the same.
 
I'd like to re-raise or maybe generalize the original question: are there any recommended practices or tools for managing locally-created and/or locally-customized files under /usr/local/etc? (And especially for personal or "lab" systems or where one might prefer pet-scale approaches?)

I'm not even sure how to identify which files under /usr/local/etc have been created or customized locally. It would seem as if the pkg database could be useful here, but I can't discern what relationship is meant to hold between the database and the file system. For instance, on my daily driver machine, I've got 538 pkgs installed, and there are 281 non-directory files reachable at paths starting with /usr/local/etc. Of those 281 paths, 86 don't exist in the pkg database, and 47 of those 86 have no corresponding .sample. As far as I can tell, I've created or edited 13 of those 86. A distinct 31 of the 281 have sha256 sums different than what's in the database: all of those happen to be symlinks, but I don't know whether that matters; I didn't create a single one of these myself.

Surely there must be some best practice or tooling to help out here, right?
 
/etc and /usr/local/etc are very small (well, not under Linux but anyway). You should have some form of automated backup from a cronjob, and one with versioning so that you don't overwrite a good backup with a bad primary.

Or you could check the files in question into rcs, which would create /usr/local/etc/RCS with the version control files.

You could also set file attributes to no change allowed. I don't think pkg overwrites that.
 
I'd like to re-raise or maybe generalize the original question: are there any recommended practices or tools for managing locally-created and/or locally-customized files under /usr/local/etc?
[...]
Surely there must be some best practice or tooling to help out here, right?
Keep in mind that on Unix everything is like elastic. There are no hard rules on doing things and with a bit of scripting you can set up a ton of flexible behavior. Still...

The following is probably a bit unorthodox here and there but I can assure you that it saved my skin several times...

#1 Include the structure(s) into a local (?) Git repository.
This can be done in multiple ways, either you set up several local repositories (/etc, /usr/local/etc and maybe keep things like /usr/local/etc/apache24 separate) but it's also not too hard to push all those repositories into a central one, now referring to git-worktree(1). I usually keep the main repository in /root/etc which then provides access to all these separate ones. Something like this:
Code:
$ git branch -lv
+ apache    9a3a6ff Added Breve's VPN address to allow Git repository access.
+ bind      9f2403e Added vps and removed old name 'unicron'.
+ etc       a003a7e Changes due to Git upgrade.
+ etc.local 8a2cbbc Various changes for VPN usage:      - Allow older Windows clients access to Samba.  - Disable the FreeBSD repository.
+ kernel    33ff944 Added extra sound module
* master    27be3b0 Fixed some incorrect & outdated information.

$ git worktree list
/usr/local/etc/apache24            9a3a6ff [apache] prunable
/usr/local/etc/namedb              9f2403e [bind] prunable
/etc                               a003a7e [etc] prunable
/usr/local/etc                     8a2cbbc [etc.local] prunable
/root/kernels                      33ff944 [kernel] prunable
/home/peter/etc                    27be3b0 [master]
This setup allows me to keep tap on everyting from one location (as shown above) and better yet... I can even push this 'main' repository into yet another one as a submodule (see git-submodule(1)) so that I can even maintain a "network repo" for a full server overview:

Code:
$ git submodule status
+13d85acc1fb7e32fefacc1cdbd08d7f4dc8ce932 breve (heads/etc.local)
 a6b8d1f46227ee6a549120ea5415ff5b3c357696 feliner (heads/master)
 4a1f97e7632041b2645748ec4a1198d50ed2e438 vps (start-2-g4a1f97e)
^ That's back from the good old days when I managed 3 servers at once, where some also included jails.

See, there is a major advantage to keep in mind here: retention. A normal backup probably has a retention of a few weeks, maybe a few months but eventually there's only so much you can go back. So including a whole Git repository into your backup will ensure that you always have a full history of everything you did (assuming you're careful with all your commits). Oh... the reason I prefer using worktrees is because I don't get "ugly" .git folders everywhere, but merely a few repo.git files.

#2 Try going for modular setups
Should be self explaining I think but if possible then I prefer using several config files vs. one huge one, this also makes it easier to maintain the whole lot. For example... I keep a local Portmaster logfile, I have logfiles from RKHunter (rootkit / security checks), my IDS (Tripwire), etc. So I use newsyslog(8) to mainain all those. Now, for my local server which only has a few logfiles it's sufficient to use /usr/local/etc/newsyslog.conf, but back when I maintained 3 servers at once I kept everything in separate configfiles that got stored in /usr/local/etc/newsyslog.conf.d and pushed onto relevant servers (using that Git structure btw.).

#3 Try to keep away from 'system files'
This is what I hinted at above, but truth be told.. I don't think it's possible to be fully consistent with this. Some services provide support for 'includes' or "*.local" files, whereas others don't. When it comes to my systems startup configuration then I simply maintain /etc/rc.conf and/or /boot/loader.conf, easy.

In other cases I always copy an 'example' file to the main one and use that for my configuration (portmaster.rc.sample vs. portmaster.rc, or pksd.conf.sample vs. pksd.conf).

But sometimes you simply don't have much of a choice. Like with, say, /usr/local/etc/smb4.conf (though this is probably a bad example considering that it usually gets generated, after which you customize it). Or... how about ~postgres/data16/postgresql.conf?

Still, having said that I would never directly mess around with files in "d directories" (like /etc/cron.d, /usr/local/etc/man.d and of course /usr/local/etc/rc.d).

#4 Always use whatever works best for you
IMO this is the most important rule. I know of people who probably shudder at the thought of those Git repositories, and that's fair enough (!). Fact remains that now, 5 years after date, I still have full access to the full custom configuration of all my previous 3 servers, and all their services. Without any issue. From important stuff like Apache, my firewall(s) and of course profiles. But also less serious things like my old IRC server, its services and I even managed to keep a hold on my old Irssi config.

Your milage may vary, as the saying goes.

Hope this can still give you some ideas!

(edit => note to self: I should use 'preview' some more ;))
 
/etc and /usr/local/etc are very small (well, not under Linux but anyway). You should have some form of automated backup from a cronjob, and one with versioning so that you don't overwrite a good backup with a bad primary.

Yeah, I've had automated backups for 15+ years (in my case, built on rsync, formerly with masses of hard links, now zfs snapshots). I'd just prefer not to back up files that would be recreated by reinstalling a package. (My experience has been that carrying forward old configurations can be dicey: a new version might choke on old configurations, or run in an unwanted backward-compatibility mode, or fail to play nicely with evolving ecosystem conventions, etc.)

So I guess I'm asking whether the pkg system does/can/could model the file hierarchy under /usr/local/etc enough to identify which things I've customized. That list would be for use as an input to backup or revision control systems.

You could also set file attributes to no change allowed. I don't think pkg overwrites that.

That's an interesting idea; I'll have a think about that. Thanks.
 
Back
Top