How to reuse /var/db/cache for upgrading multiple jails?

crypt47

crypt47

Hello, I've noticed that freebsd-update upgrade always starts fresh and it's a bit annoying on slow channel. Maybe someone has a tip how to download update once and then reinstall it multiple times?
 
One option to use a caching proxy as www/squid and to configure it to cache the update related stuff only. More than two years ago there has been a thread where related things have been discussed. Unfortunately I do not have details in mind.
 
chrbr said:
One option to use a caching proxy as www/squid and to configure it to cache the update related stuff only. More than two years ago there has been a thread where related things have been discussed. Unfortunately I do not have details in mind.
Click to expand...
Well, with https everywhere it's no fun. Patches are downloaded with some simple tool called httpget and stored on fs. I thought maybe someone did something to it, so it would be able to recognize cache on the disk.
 
This can be done with nginx , I did it 2 days ago and since it seems to work perfectly.
Well instead I should say I lamentably borrow the config files from SirDice and DtxdF .

For paches:
Create a jail which runs nginx configured with this config file, then once you fire up VMs or jails in the file /etc/freebsd-update point the ServerName directive to the jail's IP that caches the freebsd patches.

For pkg:
Same procedure than before but this time using this config file.
In your new jails create the following dir if it does'nt exist already:
mkdir -p /usr/local/etc/pkg/repos/
disable the offical repo:
echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
create yours, let's call it mycache.conf:
Bash: 
# cat /usr/local/etc/pkg/repos/mycache.conf
mycache: {
url: "http://10.10.10.10/${ABI}/quarterly",
mirror_type: "none",
signature_type: "fingerprints",
fingerprints: "/usr/share/keys/pkg",
enabled: yes
}

You may be need to change few things in the config files, but I would say it is 99% okay.
Then if you need to tinker with the options there is the nginx documentation.

Big thank you to SirDice and DtxdF , this has been really helpfull.
 
gotnull said:
This can be done with nginx , I did it 2 days ago and since it seems to work perfectly.
Well instead I should say I lamentably borrow the config files from SirDice and DtxdF .

For paches:
Create a jail which runs nginx configured with this config file, then once you fire up VMs or jails in the file /etc/freebsd-update point the ServerName directive to the jail's IP that caches the freebsd patches.

For pkg:
Same procedure than before but this time using this config file.
In your new jails create the following dir if it does'nt exist already:
mkdir -p /usr/local/etc/pkg/repos/
disable the offical repo:
echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
create yours, let's call it mycache.conf:
Bash: 
# cat /usr/local/etc/pkg/repos/mycache.conf
mycache: {
url: "http://10.10.10.10/${ABI}/quarterly",
mirror_type: "none",
signature_type: "fingerprints",
fingerprints: "/usr/share/keys/pkg",
enabled: yes
}

You may be need to change few things in the config files, but I would say it is 99% okay.
Then if you need to tinker with the options there is the nginx documentation.

Big thank you to SirDice and DtxdF , this has been really helpfull.
Click to expand...
No need to install web server if you don't need it for other reasons.

I have a configuration file, /usr/local/etc/pkg/repos/custom.conf as below.
custom: { url: "file:///poudriere/data/packages/14amd64-default", enabled: yes, priority: 100 }

You can change url: line anywhere pkg can access.
And with priority: 100 line, it can override default and pkg fetches pkgs from official repo only when latest version does not exist in the location url: line points to.

Note that if local pkgs and official ones are at the same version, sameoptions and official one are newer, pkg seems to attempt to download from official repo.
 
T-Aoki said:
No need to install web server if you don't need it for other reasons.
Click to expand...
I suppose you right if there is only one machine, for me it's different because I wanted the cache to be available for the LAN so yeah it depends on the use case.

T-Aoki said:
And with priority: 100 line, it can override default and pkg fetches pkgs from official repo only when latest version does not exist in the location url: line points to.

Note that if local pkgs and official ones are at the same version, sameoptions and official one are newer, pkg seems to attempt to download from official repo.
Click to expand...
Nice! that's good to known I wasn't aware of it, thank you.
 
T-Aoki said:
No need to install web server if you don't need it for other reasons.

I have a configuration file, /usr/local/etc/pkg/repos/custom.conf as below.
custom: { url: "file:///poudriere/data/packages/14amd64-default", enabled: yes, priority: 100 }
Click to expand...
Thanks, for me the question is about base system, not the packages.
 
crypt47 said:
Well, with https everywhere it's no fun. Patches are downloaded with some simple tool called httpget and stored on fs.
Click to expand...

httpget was written for portsnap to download a large number of small files efficiently using pipelining. If it's used through a cache without good pipelining support, caching files could be very slow.

Funnily enough, portsnap can update multiple trees, even though it's not very useful.
 
You must log in or register to reply here.
Back
Top