some gnome packages issues after recent quarterly pkg repo update

[astyle]

Why does this stuff get pushed out if it's broken / not built?

If a package is missing, it doesn't get published to the repo, and pkg(8) should be able to tell you that it can't find the package. This is very different from downloading a package only to discover that it's a corrupted file.

I suspect this has to do with the fact that pkg(8) tries to read the INDEX file of a repo first. I personally have no idea what playbook is used to create that INDEX file, and what point in the repo creation process the INDEX file gets built. I do suspect that even that detail was on the table when pkg(8) itself was getting upgraded from 1.21.x to 2.0.x. All this is educated speculation and guesses, and verification is left as an exercise for the reader.

 

[thedaemon]

If a package is missing, it doesn't get published to the repo, and pkg(8) should be able to tell you that it can't find the package. This is very different from downloading a package only to discover that it's a corrupted file.

I suspect this has to do with the fact that pkg(8) tries to read the INDEX file of a repo first. I personally have no idea what playbook is used to create that INDEX file, and what point in the repo creation process the INDEX file gets built. I do suspect that even that detail was on the table when pkg(8) itself was getting upgraded from 1.21.x to 2.0.x. All this is educated speculation and guesses, and verification is left as an exercise for the reader.

I believe it should be someone's job to look at the pkg releases being pushed and saying: I see things didn't build, let's not push this out to PRODUCTION servers... Instead it's automated? And it pushed out a bad set, it was better if it just didn't update imho. This looks really bad on the FreeBSD team.

 

[astyle]

I believe it should be someone's job to look at the pkg releases being pushed and saying: I see things didn't build, let's not push this out to PRODUCTION servers... Instead it's automated? And it pushed out a bad set, it was better if it just didn't update imho. This looks really bad on the FreeBSD team.

Are you saying that you actually got some corrupted .pkg files that pkg(8) could not handle on your end? What errors did you get? Can you please post a text dump of those errors?

 

[mariachi]

This is why I stick to ports - I don't have to rely on someone else to build the packages for me (and have building issues and be aware of schedules, what got done, for which package, which arch, and whether remote repos are having issues that prevent pkg from accessing them properly). Granted, you gotta have capable metal, and to do your own scheduling, and to figure out how to manage your own updates.

Yeah, one also needs to be comfortable running make && make install, but that normally takes less time than it takes to figure out that a remote service running the exact same thing is having issues building that exact package, and things won't shake out for a while. Even if it is a major package like editors/libreoffice.... I have compiled it on an i5-2467M. Consider this: If a package doesn't build for the official repos, it's gonna take at least a few days for things to shake out. But a compilation will probably take about 20 hours straight, your own machine will work nonstop until the compilation is over. User can take a break from research and go eat/sleep, computer is not gonna stop, you can set up an overnight job and go to sleep, and wake up to a shiny new package that your computer made for you while you slept.

As a new user I was surprised by this situation, because I never experienced it as a Linux user.
If I use an earlier version e.g., 13 will the problem be the same? I want to setup a few VPSs and the ephemeral use case I have (build other software, test instances, etc) means setup is "automatic".

If the potential for broken/missing pkgs is simply the nature of the current setup, are FreeBSD users in general relying more on Ports and building their own? I'm not trying to criticize the situation here, just want to understand how others commonly interact with pkgs/ports. I understand this is an open source effort and I'm grateful for it.

 

[astyle]

If I use an earlier version e.g., 13 will the problem be the same?

It actually would be worse. FreeBSD 13 is EoL, so there's no support for that, and definitely no officially maintained package repos. (Not out of question for a random somebody out there to maintain an unofficial repo or a mirror, but good luck finding THAT, esp. a reasonably complete mirror that has everything you want). If somebody wants to run an earlier version of FreeBSD, ports are the only viable option to get up-to-date software.

Even with up-to-date, supported releases, the FreeBSD packages still suffer from the same dependency hell that Linux packages do. One reason I like FreeBSD's Ports system is that it lets me set my own compile flags and turn on all the available features. In Linux, I just couldn't get to that point, and had to stick with precompiled packages that turned off random features for random reasons. If I wanted a feature turned on, I had to upgrade the package on the Linux distro, and that sent me straight into dependency hell. With FreeBSD's Ports, it was easy for me to learn how to make adjustments to get the features and results I wanted.

There has been talk on these Forums about mixing ports and packages. And that mixing does have its pitfalls. The important thing to keep in mind: FreeBSD's installer does offer a ports.txz set to install. There is something special about that set - it's THE snapshot of ports that is used to build package repos for a given release! If you use that particular snapshot, then yeah, it's OK to mix ports and packages. If not, then mixing of ports and packages is strongly discouraged, because the versions would be too far apart, and it's a one-way ticket to the train wreck of dependency hell. Not many people seem to understand that.

Frankly, even Linux repos do suffer from the same issues - official repos may not have all the up-to-date stuff, there's still flatpak/snap/.tgz/whatever hoopla going on, and recently, there was a blow-up over packaging of OBS Studio for Fedora repos... Some people on the Forums did a bit of research, it turned out to be material.

 

[drsnx60]

I spent last sunday rebuilding a workstation i7-13700K , using 14-stable from scratch doing Buildworld buildkernel and using ports to build Xorg, Gnome and KDE6/Plasma6 from a fresh Ports archive.
the only real issues i had was /usr/ports/science/netcdf which has a 8 missing html files in the staging area when its about to do the installation, and also
/usr/ports/math/openblas has not been building for me for a long time, which I installed with pkg.

Apart from this there were no Compilation failures during the build.

 

[astyle]

/usr/ports/science/netcdf which has a 8 missing html files in the staging area when its about to do the installation,

I just use /usr/bin/touch for the missing files in such situations, and then rerun make install. Yeah, the proper thing to do is to file a bugzilla report for that port, but using /usr/bin/touch is less work, and doesn't bite later.

 

[drsnx60]

I just use /usr/bin/touch for the missing files in such situations, and then rerun make install. Yeah, the proper thing to do is to file a bugzilla report for that port, but using /usr/bin/touch is less work, and doesn't bite later.

Yes I did that , which solves the issue. Havent managed to setup up a bugzilla account yet , but will eventually.

 

[joe81tx]

As a new user I was surprised by this situation, because I never experienced it as a Linux user.
If I use an earlier version e.g., 13 will the problem be the same? I want to setup a few VPSs and the ephemeral use case I have (build other software, test instances, etc) means setup is "automatic".

If the potential for broken/missing pkgs is simply the nature of the current setup, are FreeBSD users in general relying more on Ports and building their own? I'm not trying to criticize the situation here, just want to understand how others commonly interact with pkgs/ports. I understand this is an open source effort and I'm grateful for it.

This level of package failures isn't common, I've never seen gnome, xfce, and kde all get taken out like this in the last several years. If you can wait I think you'd be better off going with 14.2. I don't know which specific packages you need for your VPSs but gnome and xfce are available on 13. kde is also available under the latest repo but not the default quarterly repo.

It actually would be worse. FreeBSD 13 is EoL, so there's no support for that, and definitely no officially maintained package repos. (Not out of question for a random somebody out there to maintain an unofficial repo or a mirror, but good luck finding THAT, esp. a reasonably complete mirror that has everything you want). If somebody wants to run an earlier version of FreeBSD, ports are the only viable option to get up-to-date software.

Even with up-to-date, supported releases, the FreeBSD packages still suffer from the same dependency hell that Linux packages do. One reason I like FreeBSD's Ports system is that it lets me set my own compile flags and turn on all the available features. In Linux, I just couldn't get to that point, and had to stick with precompiled packages that turned off random features for random reasons. If I wanted a feature turned on, I had to upgrade the package on the Linux distro, and that sent me straight into dependency hell. With FreeBSD's Ports, it was easy for me to learn how to make adjustments to get the features and results I wanted.

There has been talk on these Forums about mixing ports and packages. And that mixing does have its pitfalls. The important thing to keep in mind: FreeBSD's installer does offer a ports.txz set to install. There is something special about that set - it's THE snapshot of ports that is used to build package repos for a given release! If you use that particular snapshot, then yeah, it's OK to mix ports and packages. If not, then mixing of ports and packages is strongly discouraged, because the versions would be too far apart, and it's a one-way ticket to the train wreck of dependency hell. Not many people seem to understand that.

Frankly, even Linux repos do suffer from the same issues - official repos may not have all the up-to-date stuff, there's still flatpak/snap/.tgz/whatever hoopla going on, and recently, there was a blow-up over packaging of OBS Studio for Fedora repos... Some people on the Forums did a bit of research, it turned out to be material.

FreeBSD Security Information

FreeBSD is an operating system used to power modern servers, desktops, and embedded platforms.

www.freebsd.org

Branch	Release	Release Date	Expected EoL
stable/14	n/a	n/a	November 30, 2028
releng/14.2	14.2-RELEASE	December 3, 2024	September 30, 2025
stable/13	n/a	n/a	April 30, 2026
releng/13.5	13.5-RELEASE	March 11, 2025	April 30, 2026
releng/13.4	13.4-RELEASE	September 17, 2024	June 30, 2025

 

[astyle]

This level of package failures isn't common, I've never seen gnome, xfce, and kde all get taken out like this in the last several years. If you can wait I think you'd be better off going with 14.2. I don't know which specific packages you need for your VPSs but gnome and xfce are available on 13. kde is also available under the latest repo but not the default quarterly repo.

FreeBSD Security Information

FreeBSD is an operating system used to power modern servers, desktops, and embedded platforms.

www.freebsd.org

Branch	Release	Release Date	Expected EoL
stable/14	n/a	n/a	November 30, 2028
releng/14.2	14.2-RELEASE	December 3, 2024	September 30, 2025
stable/13	n/a	n/a	April 30, 2026
releng/13.5	13.5-RELEASE	March 11, 2025	April 30, 2026
releng/13.4	13.4-RELEASE	September 17, 2024	June 30, 2025

I was talking about -RELEASE stuff, not -STABLE. It's probably not impossible to switch to -STABLE after installing a -RELEASE, but I never bothered. So I don't know whether there are repo tags to pay attention to in such situations. I think there might be (judging by the amount of discussion on these Forums). Well, I use a fresh snapshot of ports every time with a fresh install, and I don't bother with pkg. It does take me at least a couple days to compile my way into a functional KDE desktop, but at least I don't have to put up with a messy repo, missing features on pre-compiled stuff, and missing packages

 

[joe81tx]

I was talking about -RELEASE stuff, not -STABLE. It's probably not impossible to switch to -STABLE after installing a -RELEASE, but I never bothered. So I don't know whether there are repo tags to pay attention to in such situations. I think there might be (judging by the amount of discussion on these Forums). Well, I use a fresh snapshot of ports every time with a fresh install, and I don't bother with pkg. It does take me at least a couple days to compile my way into a functional KDE desktop, but at least I don't have to put up with a messy repo, missing features on pre-compiled stuff, and missing packages

You said FreeBSD 13 is EoL but both 13.4-RELEASE and 13.5-RELEASE are currently supported. I use binary packages so I can't speak to the port side of things but FreeBSD does officially maintain package repos for version 13.

 

[mariachi]

I was talking about -RELEASE stuff, not -STABLE. It's probably not impossible to switch to -STABLE after installing a -RELEASE, but I never bothered. So I don't know whether there are repo tags to pay attention to in such situations. I think there might be (judging by the amount of discussion on these Forums). Well, I use a fresh snapshot of ports every time with a fresh install, and I don't bother with pkg. It does take me at least a couple days to compile my way into a functional KDE desktop, but at least I don't have to put up with a messy repo, missing features on pre-compiled stuff, and missing packages

Nothing against this, especially since my needs are "compile caddy plus a rust app plus serve some clojure app as a java jar" so not looking at days. However, compilation of go fails using ports, meaning I can't compile caddy either. This is the same situation online.
What I'm wondering is: how come you can compile stuff locally if the CI is failing? Isn't the system compiling the pkg binaries using freebsd just like if I install poudiere?

 

[astyle]

What I'm wondering is: how come you can compile stuff locally if the CI is failing? Isn't the system compiling the pkg binaries using freebsd just like if I install poudiere?

Well, I just grab the most recent available ports snapshot and use that to compile my stuff locally. The CI merely uses a snapshot with a different timestamp. Also - I compile a subset of ports - one that I personally need. I do get compilation failures, too. I do have to decide if I resolve those failures by trying a different port or different Makefile flags, or even by carefully dragging in a precompiled package.

BTW, I tried Poudriere. It was a fun project, one that I never finished because of things IRL demanding my attention. It probably would have gone faster if I just tried to replicate what CI is doing - but no, I had my own ideas. They made for a fun project, it just ate up a lot of time and brainpower.

Well, lang/go is a dependency I compile early on. I do have my own preferred order of compilation: First, compilers like GCC and LLVM and rust, then Graphviz, then FFMPEG... Once everything is said and done on those, Golang has been pulled in as a dependency at some point during compilation of that stuff, and compiles just fine. I'm sure that CI does things differently - I just do things on my end, in a manner that works for me.