Slow package fetching

blind0ne said:
Pacific is which one one in terms of urls? Please, give me the contents of your /etc/pkg/FreeBSD.conf or thous which you edited with line like this
url: "pkg+http://pkg0.pkt.freebsd.org/${ABI}/quarterly",

Thank you
Click to expand...
I can not give you a line I have changed in FreeBSD.conf as I have not touched and changed any config file so far.
As said in my post I needed a quick fix and just changed my Geo location via vpn. This was good enough as a quick fix.
Of course I like to make the server I found permanent but have to figure out first how to do that and also at this
stage I do not even know the servers url.
I have a bit of time over the weekend and have a closer look at it. I will let you now my findings soon.

At this stage: I'm in New Zealand and without vpn my box connects to a server in Malaysia with this super low speed.
Malaysia is around 10.000 km away from me but I think that's not the point. The server gives me this speed as my share of bandwidth.
Connect the vpn to Sydney Australia speeds it up to 5MB/s and this reliable and all the time. But as said so far no config file touched.

Here my untouched FreeBSD.conf pkg file:
Code: 
# $FreeBSD$
#
# To disable this repository, instead of modifying or removing this file,
# create a /usr/local/etc/pkg/repos/FreeBSD.conf file:
#
#   mkdir -p /usr/local/etc/pkg/repos
#   echo "FreeBSD: { enabled: no }" > /usr/local/etc/pkg/repos/FreeBSD.conf
#

FreeBSD: {
  url: "pkg+[URL]http://pkg.FreeBSD.org/$[/URL]{ABI}/quarterly",
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}
 
meaw229a said:
I can not give you a line I have changed in FreeBSD.conf as I have not touched
Click to expand...
Good. You should not edit that file anyway.

At the bottom of the page you will find a list of mirrors: http://pkg.freebsd.org

Create a /usr/local/etc/pkg/repos/FreeBSD.conf file, with this content:
Code: 
FreeBSD: {
  url: "pkg+http://some.mirror.from.list/${ABI}/quarterly"
}
Replace some.mirror.from.list with one you found at the bottom of that page. You can try to use one of the other mirrors, maybe you can find one that works better for you. But keep in mind that there's no guarantee that mirror will always be available.
 
SirDice said:
Good. You should not edit that file anyway.

At the bottom of the page you will find a list of mirrors: http://pkg.freebsd.org

Create a /usr/local/etc/pkg/repos/FreeBSD.conf file, with this content:
Code: 
FreeBSD: {
  url: "pkg+http://some.mirror.from.list/${ABI}/quarterly"
}
Replace some.mirror.from.list with one you found at the bottom of that page. You can try to use one of the other mirrors, maybe you can find one that works better for you. But keep in mind that there's no guarantee that mirror will always be available.
Click to expand...
Don't have such file. Instead edited
etc/pkg/FreeBSD.conf
 
SirDice said:
I said, create that file. I did not say edit.


Don't. The next time you run updates this will get overwritten and your changes will be lost.
Click to expand...
Yep, thanks, I've done this. But the speed is still quite low, max is around 900 kb/s, but it's the highest bound, the average is still around 300
 
the systems I am responsible for reside in Europe, with a bandwidth from 40 to 1000MBit, and I quite often have problems with the speed of FreeBSDs pkg servers, especially within the last 3 years. sad, but kind of normal.
 
The speed is very bad for now, not only on pkg but also on composer install and git clone. Every single cli tool is so slow

Tried with xterm, the same story.
 
diizzy said:
If you have issues, try https://www.freshports.org/ports-mgmt/fastest_pkg/
Click to expand...
Thank you for this info. Just tried the script and looks promising. Shows me a server with 1500 KB/s
That would be good enough for day to day stuff. I will try this in real tomorrow.

However for anyone who has no knowledge about this matter. At the end of the scripts output it pretty much gives a "copy and paste"
instruction how to change to the fastest mirror. In my case it looks like this:

Fastest:
pkg0.tuk.freebsd.org: 1513.729 KB/s


Write configuration:
mkdir -p /usr/local/etc/pkg/repos/
echo 'FreeBSD: { url: "http://pkg0.tuk.freebsd.org/${ABI}/latest" }' \
> /usr/local/etc/pkg/repos/FreeBSD.conf

But I'm on release. Switching to latest I guess would end up in a "Franken-BSD" at the best.
For anyone on release replace the word latest with quarterly.
 
meaw229a said:
Switching to latest I guess would end up in a "Franken-BSD" at the best.
Click to expand...
Not really. You would have newer packages, that's all. Those package versions are more in line with what the ports tree is doing. The ports tree is in constant 'flux', the quarterly branches are called 'stable' because they don't have a lot of changes (at least during that 3 month period). It has nothing to do with the perceived 'stability' of the code.

That said, if you want to stay on quarterly then by all means change latest to quarterly.
 
SirDice said:
Not really. You would have newer packages, that's all. Those package versions are more in line with what the ports tree is doing. The ports tree is in constant 'flux', the quarterly branches are called 'stable' because they don't have a lot of changes (at least during that 3 month period). It has nothing to do with the perceived 'stability' of the code.

That said, if you want to stay on quarterly then by all means change latest to quarterly.
Click to expand...

Thank you for explaining this. I really did not know. I thought mixing repos is not a good idea. Learning something new here all the time.

Means if someone does not have the cpu power or is to lazy to compile a stable kernel just stay with the release kernel
but change the repo and use the newest packages anyway. That's cool.

I have 2 FreeBSD setups here. One is my work computer setup with release and release packages and I will not change that as I need
stability. But there is another one that's just for fun and to try new things out. If I mess it up no problem. Cant wait to feed it the
newest bleeding edge packages soon. Plasma 23.4 is coming my way. Very nice.
 
blind0ne said:
The speed is very bad for now, not only on pkg but also on composer install and git clone. Every single cli tool is so slow

Tried with xterm, the same story.
Click to expand...
Just looked at your profile. Your profile mentions that you're in Kiev. I'm sorry, but with the geopolitical mess in that part of the world, I wouldn't be surprised if ISP's end up lowering bandwidth priority for traffic in and out of there. I do feel for you, though, it is annoying when politics affect an important aspect of our lives.
 
astyle said:
Just looked at your profile. Your profile mentions that you're in Kiev. I'm sorry, but with the geopolitical mess in that part of the world, I wouldn't be surprised if ISP's end up lowering bandwidth priority for traffic in and out of there. I do feel for you, though, it is annoying when politics affect an important aspect of our lives.
Click to expand...
Not looking at this, the internet is quite good. The machine on linux has speed x8 of mine, with the same link on libre office - check out the posts on the top. So the problem is not in Bandwidth rather in the my machine. See only one way - reinstall the system completely.
 
grahamperrin said:
For me – in the UK – <{link removed}> results in Amsterdam. …
Click to expand...

This morning, <{link removed}> results in UK pkg0.bme.freebsd.org ☑ – a match for the currently winning server:

Code: 
% time /usr/local/bin/fastest_pkg
pkg0.tuk.freebsd.org: 452.405 KB/s
pkg0.nyi.freebsd.org: 567.505 KB/s
pkg0.bme.freebsd.org: 8003.113 KB/s
pkg0.pkt.freebsd.org: 7727.833 KB/s

Fastest:
pkg0.bme.freebsd.org: 8003.113 KB/s


Write configuration:
mkdir -p /usr/local/etc/pkg/repos/
echo 'FreeBSD: { url: "http://pkg0.bme.freebsd.org/${ABI}/latest" }' \
        > /usr/local/etc/pkg/repos/FreeBSD.conf


0.257u 0.250s 5:30.02 0.1%      218+16331k 716+0io 310pf+0w
% uclcmd get --file /usr/local/etc/pkg/repos/FreeBSD.conf FreeBSD.url
"pkg+http://pkg0.bme.freebsd.org/${ABI}/latest"
%

blind0ne said:
… See only one way - reinstall the system completely.
Click to expand...

Gut feeling: overkill, with no guarantee of improvement.

What did you get from fastest_pkg?
 
Last edited:
I know this isn't an answer but I also have an issue.
My download speed is the same (almost) as the file size.
Is there any reason for that?
 

Attachments

  • file size.PNG
    file size.PNG
    40 KB · Views: 154
If a file with size 33KB downloads within a second the speed would be calculated as 33 KB/s. If the file is 24 KB and downloads within a second the speed is 24 KB/s. The measurements aren't that granular it can tell the difference between 400ms or 900ms, it's still counted as 1 second.
 
Zay said:
I know this isn't an answer but I also have an issue.
My download speed is the same (almost) as the file size.
Is there any reason for that?
Click to expand...
Yeah, you gotta pay attention to your max connection speed. For example, mine is 430 KB/sec. So, a 50 KB file will get downloaded in under a second in most cases. So will a 25 KB file. Because of that, my download speeds will show to be the same as file size. Once your files get big enough, the download speed will max out at a rate of N KB/sec.
 
SirDice said:
If a file with size 33KB downloads within a second the speed would be calculated as 33 KB/s. If the file is 24 KB and downloads within a second the speed is 24 KB/s. The measurements aren't that granular it can tell the difference between 400ms or 900ms, it's still counted as 1 second.
Click to expand...
Okay fair enough, but why is it taking ages to do that then? :/
 
For a few weeks now I experience pretty much everything hosted on the FreeBSD.org domain(s) as very slow compared to "how it used to be". To the point where I'm considering reaching out to provide some server infrastructure for mirroring or similar efforts - just didn't yet look into whom to contact (cue creepy look at SirDice).

But then also: I think slow is fine. I'd choose stability/reliability over speed any day (which is something FreeBSD does well in general, as far as I can tell). So if this is an effort by the Foundation to minimize servers costs I'm all pro that. I'd still like to help out where I can (in terms of providing infrastructure) tho.
 
jbodenmann said:
To the point where I'm considering reaching out to provide some server infrastructure for mirroring or similar efforts - just didn't yet look into whom to contact (cue creepy look at @SirDice ).
Click to expand...
They're all administered and maintained by the cluster administration team.

www.freebsd.org

FreeBSD Quarterly Status Report 3rd Quarter 2021

FreeBSD is an operating system used to power modern servers, desktops, and embedded platforms.
www.freebsd.org www.freebsd.org
 
Zay said:
Okay fair enough, but why is it taking ages to do that then? :/
Click to expand...
Slow endpoints. You may have a fast connection, the Internet may have the bandwidth to carry the file, but the machine hosting the file is slow to upload.
 
You must log in or register to reply here.
Back
Top