doxygen = pkg bomb

H

hruodr

Code: 
% uname -a
FreeBSD fbsd.local 11.3-RELEASE-p3 FreeBSD 11.3-RELEASE-p3 #0: Mon Aug 19 21:08:43 UTC 2019     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
% pkg install doxygen
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 42 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
        firefox-69.0.1,1
        poppler-utils-0.78.0

New packages to be INSTALLED:
        doxygen: 1.8.15_3,2
        texlive-base: 20150521_43
        teckit: 2.5.7
        python37: 3.7.6
        wpebackend-fdo: 1.4.0
        libwpe: 1.4.0.1
        xpdfopen: 0.86
        texlive-tlmgr: 20150523_2
        tex-kpathsea: 6.2.1_2
        tex-ptexenc: 1.3.3_2
        libpotrace: 1.12
        zziplib: 0.13.69_1
        tex-web2c: 20150521_3
        openjpeg15: 1.5.2_1
        t1lib: 5.1.2_5,1
        tex-formats: 20150521_2
        texlive-texmf: 20150523_4
        tex-basic-engines: 20150521
        tex-dvipsk: 5.995_2

Installed packages to be UPGRADED:
        libreoffice: 6.2.4_2 -> 6.3.4
        libcdr01: 0.1.5_2 -> 0.1.5_6
        webkit2-gtk3: 2.24.4 -> 2.26.4
        raptor2: 2.0.15_11 -> 2.0.15_12
        spidermonkey52: 52.9.0_3 -> 52.9.0_5
        libzmf: 0.0.2_15 -> 0.0.2_19
        icu: 64.2,1 -> 65.1,1
        libmspub01: 0.1.4_10 -> 0.1.4_14
        libe-book: 0.1.3_12 -> 0.1.3_16
        libqxp: 0.0.0_10 -> 0.0.0_14
        libvisio01: 0.1.6_12 -> 0.1.7
        boost-libs: 1.70.0_2 -> 1.72.0
        liborcus: 0.14.1_1 -> 0.14.1_4
        harfbuzz-icu: 2.5.3 -> 2.6.4
        qt5-core: 5.12.2_1 -> 5.13.2
        evolution-data-server: 3.28.5_2 -> 3.28.5_4
        poppler-glib: 0.78.0 -> 0.82.0_2
        poppler: 0.78.0 -> 0.82.0_1
        tracker: 2.0.4_3 -> 2.0.4_5
        libical: 3.0.4_1 -> 3.0.4_3
        libcmis: 0.5.1_12 -> 0.5.2_1

Number of packages to be removed: 2
Number of packages to be installed: 19
Number of packages to be upgraded: 21

The process will require 1 GiB more space.
775 MiB to be downloaded.

Proceed with this action? [y/N]:
 
Assuming, since you didn't actually ask a question, that you are perplexed by the removal of firefox and poppler. According to other threads on this subject, firefox failed to build in one of the FreeBSD repositories so will work itself out next build.

Thread 74413
 
I was also a little bit frustrated to see doxygen pull in so many packages, including texlive base / texmf.

Last time I checked, doxygen generated the .tex files but didn't require you to compile them (and was unable to by itself).

It is a bit like adding Codeblocks as a dependency for CMake because it *can* generate .cpproj files.
 
To reduce the dependencies, one could use Plugins, or addons.

Something as simple as:
Code: 
printf("please execute pkg install x to use this feature!/n");
return 0;
 
To reduce the dependencies??!! I repeat: doxygen is not dependent of texlive bomb.

Packages made continuously dependencies of non-dependencies.
 
  • Thanks
Reactions: a6h
hruodr said:
To reduce the dependencies??!! I repeat: doxygen is not dependent of texlive bomb.

Packages made continuously dependencies of non-dependencies.
Click to expand...

Well it "can be". If you look at the compile flags of a number of bits of software, you will see that there are lots of --enable-* options. There is no possible way the packager will know which ones you need. They also don't have the manpower to hack on the code to make them loosely bound plugins.

In this case, because doxygen provides a GENERATE_LATEX option and creates a Makefile to build it; I am assuming it needed information about your installed latex compiler to create a suitable Makefile. This is perhaps why it is set as a runtime dependency.

Ports do well to solve this. Otherwise we are into the realm of duplicating a number of packages with subtle build / dependency differences.

I agree with you in this case, just like the nvidia driver dragging in linux compat, this seems a less common requirement and should perhaps just be provided to the user as a note during install.
 
In my make.conf, I removed doxygen as a dependency.

One port labeled doxygen as standard documentation, so I had to use an additional entry to remove that from building. I told and explained to the maintainers about that port twice, to label it as doxygen instead of typical documentation in that makefile.

make.conf:
Code: 
OPTIONS_UNSET=DOXYGEN
textproc_libe-book_UNSET=DOCS

You can use this example to remove tex, texmf, texlive or other redundant dependencies for doxygen, and tell the port maintainers of the minimum requirements needed to prevent it pulling in so many dependencies. They can make those optional and not a requirement. Some ports make things dependent that they'll override make.conf, so you may have to grep their makefiles and calls to /usr/ports/Mk.

I know doxygen is important, but for my needs, I simply removed it as a dependency. Improving its dependency needs further down will be another needed improvement. It is an improvement I would also like to see implemented in ports. There are a lot of places dependency needs can be improved for ports, as I found a few in the past and informed maintainers, and I bet there are many like this left as the one you found.

This topic is really my gripe about Linuxism dependency hell, and why I complain about it so much. FreeBSD ports provides a way to clean it up, until you get to some upstream software suites, where they complicate things as soon as you point out inefficiencies in it. Like for a Gnome application, such as a game, when I don't use Gnome, it's as if they are making a chess move when they reintroduce a new complication.
 
I do not use ports, but packages. I do not have the time to continuously compile all ports on a computer, that is more than enough powerful for my needs, but not powerful enough for compiling everything at every time. And if I need doxygen, then I just download and compile it like I did with TeX. I do need TeX, but do not like the TeX life bloat. Those who build packages seem not to have a little of empathy for those that do not like bombs like TeXlive: it continuously appear as dependency.
 
I did notice another fun one the other day.

vlc from packages installs (directly and indirectly):
gtk+2
gtk+3
qt4

How many GUI libraries does one video player need? ;)
 
Please talk to the port maintainers if it make sense to create port-options instead of making everything possible. While a friendly email might be enough to point to the problem a PR might be appropriate in some cases also.

It is useful to discuss it here, but if you want to stop some annoyances you need to talk to port maintainers or even to upstream devels.
 
I have in the past but to be fair, the port system is potentially already able to deal against this. The issue is that packages are built with the default (99% sane) options that need to drag in more packages than necessary because other use-cases do require the additional functionality.

Before I make noise to the maintainers, I would like to have a potential solution but in this case, I don't think there is. So there is nothing really to fix. In the few ports that I maintain I cannot think of a better system and they are trivial in comparison to other ports.

<ramble>
For me, I install "big" software in a jail and then just redirect the /usr/local folder. Sure it wastes space potentially having duplicate libraries installed but I also like the way Windows does basic software (providing the libs it needs with the .exe in a single folder). Not very UNIX like but modern GUI software is anything but UNIX-like anyway.

The biggest issue in my system is that software is hardcoded to look in '/usr/local/' for plugins, data files, etc. I have various solutions for this but they are all currently a bit naff. Bring me back /opt/sfw, /usr/sfw, /opt/csw any day! ;)
</ramble>
 
It can be fixed. It's been pointed out in the past, that certain ports weren't required, and it cleaned up a whole lot of bloat. BSD like replacements also have reduced the need for such bloat.
 
I'm no expert on the freebsd ports system (still learning it and setting up a local mirror,) but is there an option to make the dependencies subject to the configured options at the port level? That seems like a more appropriate way - that way if someone picks an option like "--with-documentation" it goes "oh, I need doxygen" whereas it's not an actual dependency otherwise?
 
CoryG said:
but is there an option to make the dependencies subject to the configured options at the port level?
Click to expand...
That exactly what a lot of options do. Besides enabling (or disabling) features, they can also pull in extra dependencies. Sometimes options/features require additional dependencies.
 
It turned out to be java/openjdk7 in my case, I ended up having to install things via the package manger for it to resolve the dependencies enough for doxygen and apache24. Hoping the openjdk7 port will get patched soon so I can swap it out for a locally-built copy.
 
CoryG said:
It turned out to be java/openjdk7 in my case
Click to expand...
It's not openjdk7 that depends on doxygen, not directly at least. Perhaps you enabled an option on some of its dependencies that pulled in doxygen? OpenJDK7 does pull in a truckload of dependencies. You can remove quite a lot of them if you build it with X11 turned off.
 
Compiling is a total package bomb, as well. I've been through a few port compilation cycles myself, and picked up a few things from my experiences. For example, due to enabling just about every available option in make config, I ran into a circular dependency of graphics/graphviz->lang/ruby27->deve/doxygen->back to graphics/graphviz! A solution I found by googling around is to first compile graphics/graphviz without ruby or doxygen or java dependencies, and finish that. After that, compile lang/ruby27. It's OK to enable the graphviz dependency, but don't do doxygen in there. After that is done, compile deve/doxygen with EVERYTHING enabled. And, this whole enchilada is best done first, on a freshly installed system, BEFORE installing anything else. And it took me about 5 hours from start to finish, because a really clean install will also pull in deps like devel/llvm10, devel/llvm90 and lang/gcc10.
 
SirDice said:
Enabling options has consequences. This is one of them
Click to expand...
I did find a solution, though. And, I figured out how to dig my way out. And I'm happy with results. One example of why I enable everything I can: on FreeBSD, multimedia/vlc precompiled package does an awful job with subtitles and codecs, and volume control was not working. Exactly the same version on Windows handles them like a champ. Re-compiling VLC with all available options enabled, and chasing down deps in an organized fashion has resulted in a FreeBSD install that works exactly the same as on Windows. It takes some effort to learn, but I'm happy I did it. That's what it takes to claim that the software is truly cross-platform.
 
SirDice said:
Just because you can, doesn't mean you should.

Enabling options has consequences. This is one of them.
Click to expand...
I don't think this is a consequence, rather that it is a bug: if there really is a circular dependency, it is not useable by automation.

For instance, I do point /var/db/ports back into /usr/ports/.private/options so that they are fully versioned, then put the git HEAD objname into the annotations of the created pkgs (so it can be retrieved via pkg-rquery()), then determine what can be downloaded and what needs to be rebuilt with git diff <rquery-head> HEAD -- <portdir>/<portname> <portdir>/<masterportname> .private/options/<portdir>_<portname>. The dependency tree for this is computed from make -V BUILD_DEPENDS etc.

There is no way fiddling with the options on-the-fly. (OTOH such a process can run unattended at any place where there is spare compute.)

BTW: we are entirely lacking an automation interface for these options settings: there is no way initializing or modifying them except going thru dialog4ports. :(
 
PMc said:
I don't think this is a consequence, rather that it is a bug: if there really is a circular dependency, it is not useable by automation.
Click to expand...
Circular dependencies are going to exist. Especially if you turn on incompatible options by enabling everything everywhere. There's a reason why some options are off by default. The system isn't "smart" enough to detect you enabled an incompatible option two, three levels deep in the dependency chain that causes it to loop back. It'll detect it's looping by presenting you with an error; "circular dependency detected" and refuse to build, but it's not going to stop you from selecting those options that caused it to loop in the first place.
 
SirDice said:
Circular dependencies are going to exist.
Click to expand...
Then how should we work them out?

SirDice said:
Especially if you turn on incompatible options by enabling everything everywhere.
Click to expand...
I don't think I do that. And just today I run into the very same issue. :(

I have rewritten my code due to the git happening. Then, careful as usual, I have run a full build on 2021Q1, and that did work. But now with 2021Q2 it does not work anymore.

In my case it comes down to this:
Code: 
trying audio/jack : 
    missing devel/doxygen
trying devel/doxygen : 
    missing print/tex-dvipsk
trying print/tex-dvipsk : 
    missing devel/tex-web2c
trying devel/tex-web2c : 
    missing devel/zziplib
trying devel/zziplib : 
    missing devel/sdl20
trying devel/sdl20 : 
    missing audio/jack

1. What are the offending options here:
audio/jack -> DOXYGEN (nondefault)
devel/doxygen -> LATEX (default)
print/tex-dvipsk -> USE_TEX (mandatory)
devel/tex_web2c -> LIB_DEPENDS (mandatory)
devel/zziplib -> SDL (nondefault)
devel/sdl20 -> JACK (nondefault)

I must enable JACK for sdl20, because my audio runs with jack and everything depends on that.
I did enable DOXYGEN for jack, because it's a bit obscure and any doc one might get is certainly useful.
I do not remember why SDL was enabled - I have an old paper (10+ years back) where SDL is considered generally useful to enable.

2. why did this problem appear now out of nothing:
Because devel/zziplib used to require devel/sdl12, but now requires devel/sdl20 (sdl12 did not have a JACK option).

3. which of the options is a wrong decision?
JACK is absolutely needed.
DOXYGEN for jack is considered useful to necessary.
So SDL for zziplib is the only remaining - and from what I get, that does something with media for something that reads zip-files. Not that I would like to use that (so I can get rid of it), but this does not look useless and might be desired.
 
You must log in or register to reply here.
Back
Top