Can someone explain to me whats going on with pkg?

D

damian2002

Hey, i am new to FreeBSD and so far i really liked it. It feels a lot more robust and simple and complete. But today i got hit by pkg. I ran pkg upgrade and it automatically removed a lot of packages, which i didnt expect.
From what i could gather it seems like there is a bug / build error on the build servers. But how can it be that these packages just get removed? I dont really understand the pipeline.
I am using latest already and i dont really have time to figure out ports and compile for days.
Can someone explain whats really happening? Is this a regular thing i have to be aware of? What can i do to prevent accidentally purging important software from my system? I intended to use FreeBSD as a production server as well.

Thanks
 
  • Like
Reactions: mro
damian2002 said:
But how can it be that these packages just get removed?
Click to expand...
Because you can't keep old packages if everything else is updated. Then those old packages are going to depend on old versions of dependencies that don't exist anymore.

damian2002 said:
Is this a regular thing i have to be aware of?
Click to expand...
It most certainly isn't a 'regular' thing. Although there's sometimes a bit of fallout, those issues are typically quickly fixed. The current issue however is little more involved as it's caused by a bug in the OS itself.

damian2002 said:
What can i do to prevent accidentally purging important software from my system?
Click to expand...
Pay attention to whatever pkg(8) wants to do. It asks "Are you sure" for a reason.

damian2002 said:
I intended to use FreeBSD as a production server as well.
Click to expand...
I highly recommend setting up your own repository. That will put you in control. You get to decide which "mysql" to use as default (MySQL 8.0/8.4, MariaDB, etc), what version of PHP, what version of Python etc. Then you can decide when to upgrade those, or not. You can also build everything with specific options set or unset.
 
I have been using FreeBSD for about 28 years.

We never used to have these sorts of problems when the distribution mechanism was a Wallnut Creek CD!

However, I have never seen the package system as unstable and problematic as it is now.

I do expect that there's a fair bit of effort going into fixing that instability...

Do look at what pkg upgrade is proposing. Answer "No" if it looks problematic, and try again a day or three later.
 
gpw928 said:
I have been using FreeBSD for about 28 years.

We never used to have these sorts of problems when the distribution mechanism was a Wallnut Creek CD!

However, I have never seen the package system as unstable and problematic as it is now.

I do expect that there's a fair bit of effort going into fixing that instability...

Do look at what pkg upgrade is proposing. Answer "No" if it looks problematic, and try again a day or three later.
Click to expand...

If something happens once every 28 years, that seems reasonable to me hehe
I come from Debian and wanted to try freeBSD. Not finding the KDE (or any DE) package was a bit of a surprise.
 
gpw928 said:
I have been using FreeBSD for about 28 years.

We never used to have these sorts of problems when the distribution mechanism was a Wallnut Creek CD!

However, I have never seen the package system as unstable and problematic as it is now.
Click to expand...
Its going to get much worse if PkgBase ever makes it in.
  • pkg was better than pkg-ng for a variety of reasons
  • Xorg should be a set, not a spray of little packages
  • LinuxKPI drivers should at least have an initial version provided in base (or the Xorg set)
  • FreeBSD install images shouldn't be stupid sizes
OpenBSD is doing better in these key areas by... simply not changing things for a decade.
 
A notification in news would be nice to have when bad things like this happens, something that tells users the current state of pkg so they don't brick their system.
That could avoid the high number of threads where people ask "why my desktop is uninstalled" or "why xxx has been removed?".

kpedersen said:
Its going to get much worse if PkgBase ever makes it in.
Click to expand...
To be honest I am a bit worried about this too, I even wonder if what happens to pkg during last months is not related to PkgBase, that could explain a certain instability, may be devs are preparing the transition? After all December is not so far, I only speculate I don't know anything about it, it's just me thinking out loud.
 
We're having a couple of presently bricked desktops. Its a PITA.
In desperation, we're limping along having used dd to clone drives from a computer that fortunately was NOT updated.
How will we know when this KDE/plasma problem has been thoroughly fixed, so we can then invest some serious time to reinstall fresh systems ?
Or must we simply rely upon trial-and-error, and be left wondering where any problems are being caused ? Package problem, or PEBKAC ?
Being forced to compile lots of ports is a time killer. We prefer to use packages if possible.
 
bgroper said:
We're having a couple of presently bricked desktops. Its a PITA.
In desperation, we're limping along having used dd to clone drives from a computer that fortunately was NOT updated.
How will we know when this KDE/plasma problem has been thoroughly fixed, so we can then invest some serious time to reinstall fresh systems ?
Or must we simply rely upon trial-and-error, and be left wondering where any problems are being caused ? Package problem, or PEBAC ?
Being forced to compile lots of ports is a time killer. We prefer to use packages if possible.
Click to expand...
Are you using ZFS or UFS for root filesystems? If ZFS, it's rather trivial to create a new Boot Environment before doing quarterly package upgrades, that way if anything is missing, you simply roll back to the good one.

There is a package fallout mailing list which shows logs of failed packages and why.
As implied above the overall load of package building is a good stress test and seems to be triggering a bug in I think the host OS which I think (not 100% sure) is -CURRENT with jails doing the actualy building.
This bug causes a package to fail and then other packages that depend on it don't get built so a little bit of a cascade going on.
 
bgroper said:
Being forced to compile lots of ports is a time killer. We prefer to use packages if possible.
Click to expand...
Set up your own repository, build once, install many. This is especially beneficial if you have to provide for a bunch of machines. Added benefit, you can set your own default versions for mysql, perl, php, python, ruby, java, etc. And you get to set/unset options for everything. Meaning you can really tailor made the packages for your environment. And, last but not least, you get to decide what to update and what not, and when this happens. With a local repository you get the best of both worlds, the flexibility of ports and the ease of management from packages.
 
Looks like two separate problems:
  1. The package builders were updated to a version of -CURRENT that causes -RELEASE jails to crash:
  2. A performance regression in latest pkg(8)
I'm a little startled that the production package builders run -CURRENT, but I guess that has to be so they can build packages for -CURRENT. It would be nice to have separate builders for the development and release versions, but I'm guessing there's no money or personnel to accomplish that.
 
I run one FreeBSD machine, my daily driver ThinkPad x250. Seeing the official (quarterly) pkg repos being considered unreliable and dispensable scares me.

Maintaining the supply chain for pkg by myself for a single machine isn't appealing.

I have no problem with slightly dated packages. But this here looks different.

Edit: is there anything one could help to improve on that?
 
-CURRENT is usually pretty reliable.

But in this case some backward compatibility for 14.x seems to have been broken, which I suppose is hard to spot even for a committer that tests -CURRENT for functionality.
 
There has been an issue with the go language build failing. The cause is a newer build server processor having the LA57 extension which increases the size of virtual addresses from 48 bits to 57 bits. LA57 either needs to be taken into account as part of the go build or disabled. https://github.com/golang/go/issues/49405

Because some packages rely on the go language and the big desktop environments rely on those they all got wiped out. I'm on the default quarterly repo and was impacted about a week ago with the Q1 to Q2 migration. Here's the bug showing the solution: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=285963

Now that LA57 has been disabled on the affected servers a new package build was kicked off on Sun, 13 Apr 2025 01:01:41 UTC and the previously failed/skipped packages are building successfully. Only 6 out of 18,000 packages remain - once those complete and the packages propagate to the repo servers everyone should be back in business. https://pkg-status.freebsd.org/beefy20/build.html?mastername=142amd64-quarterly&build=4764684db30f
 
I picked the wrong week to dive into learning and installing FreeBSD haha. Thanks so much for these threads - I thought I had done something very wrong with my system! Am I correct in assuming that, staying on "quarterly", the packages will eventually be built and the repos will update, or should I switch to "latest" to get the new builds?
 
For the quarterly repo there is still an issue with full fat gnome being wiped out because of gnome-chess. gnome-lite, kde, xfce, cinnamon, & mate all built fine. There is 1 package still building that will probably take another 6.5 hours to complete.
For the latest repo kde failed due to qt6-webengine-6.8.3_1. gnome, gnome-lite, xfce, cinnamon, & mate all built fine. A new build has been kicked off so there's hope for a quick fix on kde with latest now that maintainers can focus on port specific issues instead of all the go related failures. https://pkg-status.freebsd.org/beefy22/build.html?mastername=142amd64-default&build=e3d1564d6c13
 
gpw928 said:
I have been using FreeBSD for about 28 years.

We never used to have these sorts of problems when the distribution mechanism was a Wallnut Creek CD!

However, I have never seen the package system as unstable and problematic as it is now.

I do expect that there's a fair bit of effort going into fixing that instability...

Do look at what pkg upgrade is proposing. Answer "No" if it looks problematic, and try again a day or three later.
Click to expand...
DEar gpw928:
now this pkg upgrade destroy much more apps, that cause i can't work normal . why the department of pkg don't push all new apps at same time ? they just push part of app to news, part of app to disappear ...that was very bad . in the production system. we don't like that. thanks for your reply.. jedi of freebsd .
 
Is there an explanation of the pipeline from Poudriere to pkg.FreeBSD.org somewhere? The bulk build has finished and `pkg update` still isn't picking up any updates on my side. Same with `pkg update -f` - I'm getting zero results for `pkg search gopls`, despite it succeeding in the previous bulk build. I'd just like to understand how all this is connected; it's not intuitive as a newcomer.
 
SirDice said:
I highly recommend setting up your own repository.
Click to expand...
aslyle wrote that this is a rather delicate and time-consuming event. Especially for beginners. For people like me, it is easier to have a rollback on the boot environment or not update until the situation returns to normal.
 
USerID said:
aslyle wrote that this is a rather delicate and time-consuming event.
Click to expand...
Let it churn at night, when you come back in the morning you'll have everything set up. And the package build software doesn't rebuild everything from scratch every time you start a new bulk build, it only rebuilds what is needed. On my build server a "full" rebuild takes ~4 hours to complete (I don't need all 40.000+ packages, just a selection), updates often less than 1.
 
SirDice said:
Let it churn at night, when you come back in the morning you'll have everything set up. And the package build software doesn't rebuild everything from scratch every time you start a new bulk build, it only rebuilds what is needed. On my build server a "full" rebuild takes ~4 hours to complete (I don't need all 40.000+ packages, just a selection), updates often less than 1.
Click to expand...
Can you share some insights on how many and which packages, as well as storage requirements?
 
It's going to heavily depend on how much you need obviously, and how big the individual ports are, but here are some figures for you:

14.2 "desktop" build (added x11/gnome, x11/mate and x11-wm/xfce4 for test, I normally don't build these, don't need them):
Code: 
root@chibacity:/usr/local/etc/poudriere.d # wc -l 142-release-desktop-package.lst
      55 142-release-desktop-package.lst
root@chibacity:/usr/local/etc/poudriere.d # du -h /usr/local/poudriere/data/packages/142-release-desktop/
5.4G    /usr/local/poudriere/data/packages/142-release-desktop/.real_1744309846/All
1.0K    /usr/local/poudriere/data/packages/142-release-desktop/.real_1744309846/Latest
5.4G    /usr/local/poudriere/data/packages/142-release-desktop/.real_1744309846
5.4G    /usr/local/poudriere/data/packages/142-release-desktop/
14.2 "server" build (webservers, PHP, RoR, mariadb, mail, various web applications, etc)
Code: 
root@chibacity:/usr/local/etc/poudriere.d # wc -l 142-release-server-package.lst
     125 142-release-server-package.lst
root@chibacity:/usr/local/etc/poudriere.d # du -h /usr/local/poudriere/data/packages/142-release-server/
4.9G    /usr/local/poudriere/data/packages/142-release-server/.real_1744140599/All
1.0K    /usr/local/poudriere/data/packages/142-release-server/.real_1744140599/Latest
4.9G    /usr/local/poudriere/data/packages/142-release-server/.real_1744140599
4.9G    /usr/local/poudriere/data/packages/142-release-server/

What's important to note is that you do not need to build everything (all 33272 ports), just what you need/use.

I've split my builds up into a "desktop" and "server" set, because I have different requirements/needs for each usage, I might enable a GUI for some application in the "desktop" set but don't want any X11/GUI related stuff on the "server" set.

You really don't need terrabytes of storage, a really super fast CPU or many gigabytes of RAM (although both would certainly help speed things up), I've build repositories on old Core i5 with 16 GB, it took a lot longer than my current dual Xeon (12 cores/24 threads) with 192GB obviously, but the old system built everything just fine.

Code: 
root@chibacity:/usr/local/etc/poudriere.d # zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
zroot   140G   106G  34.0G        -         -    64%    75%  1.00x    ONLINE  -
 
You must log in or register to reply here.
Back
Top