Pkg download speed is ultra slow i could compile these and still finish earlier. I live in Greece

christhegeek · Jul 23, 2023

Pkg download speed is ultra slow i could compile these and still finish earlier. I live in Greece.
Making a new installation is a nightmare because downloading packages could take many many hours .
You need to upgrade the servers dont do anything else.

yuripv79 · Jul 23, 2023

christhegeek said:
You need to upgrade the servers

Or you need to upgrade your ISP (I'm getting 20MBps download speeds).

Seriously though, try using the mirror other than selected for you by your location, try using ports-mgmt/fastest_pkg.

christhegeek · Jul 23, 2023

yuripv79 said:
Or you need to upgrade your ISP (I'm getting 20MBps download speeds).

Seriously though, try using the mirror other than selected for you by your location, try using ports-mgmt/fastest_pkg.

Yes it helped a lot, I remember I tried to use it before but it didn't work. Probably I should reboot my PC or something cause now it works.
Thank you.
Still I would like to be much faster.

bapt@ · Aug 1, 2023

which mirror by default do you hit ? (going to https://pkg.freebsd.org website should give you the name of the mirror) and which one was picked by fasttest_pkg?

Cath O'Deray · Apr 26, 2024

christhegeek said:
… slow pkg download speeds …

FreeBSD or NetBSD?

christhegeek · May 4, 2024

Cath O'Deray said:
FreeBSD or NetBSD?

FreeBSD in Greece !
One of the reasons i stopped a little FreeBSD project i had for some (~3-4?) months (cultbsd) was that pkg downloading speed is painfully slow.

cabriofahrer · May 5, 2024

Well, what generally helps to speed up your Internet connection is to use 8.8.8.8 as DNS rather than the DNS given by your ISP which is preconfigured in your router. Maybe give that a try?

Cath O'Deray · May 6, 2024

cabriofahrer said:
DNS

Unlikely to help with speeds of downloads from the pkg mirror infrastructure, but thanks.

Fastest for me, today, at home, is Frankfurt at 5.8 MB/s:

Code:

root@mowa219-gjp4-zbook-freebsd:~ # date ; fastest_pkg -v
Mon May  6 10:07:10 BST 2024
pkg0.nyi.freebsd.org: 100%
pkg0.nyi.freebsd.org: 4.0 MB/s
pkg0.fmt.freebsd.org: 100%
pkg0.fmt.freebsd.org: 2.7 MB/s
pkg0.bbt.freebsd.org: 82%
(28, 'Operation timed out after 5001 milliseconds with 6113313 out of 7453912 bytes received')

pkg0.bbt.freebsd.org: 1.2 MB/s
pkg0.fra.freebsd.org: 100%
pkg0.fra.freebsd.org: 5.8 MB/s
pkg0.bme.freebsd.org: 0%
(7, "Failed to connect to pkg0.bme.freebsd.org port 80 after 38 ms: Couldn't connect to server")
pkg0.bme.freebsd.org: 0.0 B/s
pkg0.bra.freebsd.org: 82%
(28, 'Operation timed out after 5002 milliseconds with 6144329 out of 7453912 bytes received')

pkg0.bra.freebsd.org: 1.2 MB/s
pkg0.sjb.freebsd.org: 100%
pkg0.sjb.freebsd.org: 5.2 MB/s
pkg0.kul.freebsd.org: 100%
pkg0.kul.freebsd.org: 1.7 MB/s
pkg0.kwc.freebsd.org: 100%
pkg0.kwc.freebsd.org: 1.5 MB/s
pkg0.jinx.freebsd.org: 100%
pkg0.jinx.freebsd.org: 2.2 MB/s
pkg0.tuk.freebsd.org: 100%
pkg0.tuk.freebsd.org: 2.6 MB/s
pkg0.twn.freebsd.org: 0%
(28, 'Connection timed out after 5002 milliseconds')
pkg0.twn.freebsd.org: 0.0 B/s
pkg0.syd.freebsd.org: 84%
(28, 'Operation timed out after 5004 milliseconds with 6320513 out of 7453912 bytes received')

pkg0.syd.freebsd.org: 1.3 MB/s

Fastest:
pkg0.fra.freebsd.org: 5.8 MB/s


Write configuration:
mkdir -p /usr/local/etc/pkg/repos/
echo 'FreeBSD: { url: "http://pkg0.fra.freebsd.org/${ABI}/latest", mirror_type: "NONE" }' \
        > /usr/local/etc/pkg/repos/FreeBSD.conf


root@mowa219-gjp4-zbook-freebsd:~ #

Actual speeds were far lower on Tuesday 30th April, I didn't keep a note of where I was at the time:

Code:

% pwd
/usr/home/grahamperrin/Documents/IT/BSD/FreeBSD/poudriere/pkgbase
% grep -i mb\/s 2024-04-30\ jail\ update\ mysteries.txt
[1/166] Fetching FreeBSD-libsqlite3-dev-15.snap20240429151916.pkg: 100%    2 MiB   2.2MB/s    00:01   
[5/166] Fetching FreeBSD-kerberos-lib-dev-15.snap20240429151916.pkg: 100%    5 MiB   1.6MB/s    00:03   
[9/166] Fetching FreeBSD-utilities-dev-lib32-15.snap20240429151916.pkg: 100%    4 MiB   1.3MB/s    00:03   
[19/166] Fetching FreeBSD-src-sys-15.snap20240429220940.pkg: 100%   93 MiB   1.7MB/s    00:59   
[26/166] Fetching FreeBSD-zfs-lib32-15.snap20240429151916.pkg: 100%    1 MiB   1.5MB/s    00:01   
[27/166] Fetching FreeBSD-dtrace-dev-lib32-15.snap20240429151916.pkg: 100%    1 MiB   1.2MB/s    00:01   
[30/166] Fetching FreeBSD-utilities-dev-15.snap20240429151916.pkg: 100%    6 MiB   2.0MB/s    00:03   
[32/166] Fetching FreeBSD-openssl-lib-dev-lib32-15.snap20240429151916.pkg: 100%    7 MiB   1.8MB/s    00:04   
[37/166] Fetching FreeBSD-rescue-15.snap20240429151916.pkg: 100%    6 MiB   1.6MB/s    00:04   
[47/166] Fetching FreeBSD-zfs-dev-15.snap20240429151916.pkg: 100%    7 MiB   1.6MB/s    00:05   
[56/166] Fetching FreeBSD-clang-15.snap20240429151916.pkg: 100%   46 MiB   1.6MB/s    00:30   
[60/166] Fetching FreeBSD-clibs-dev-lib32-15.snap20240429151916.pkg: 100%   13 MiB   2.2MB/s    00:06   
[70/166] Fetching FreeBSD-tests-15.snap20240429151916.pkg: 100%   18 MiB   1.5MB/s    00:13   
[71/166] Fetching FreeBSD-libarchive-dev-15.snap20240429151916.pkg: 100%    1 MiB   1.6MB/s    00:01   
[75/166] Fetching FreeBSD-utilities-lib32-15.snap20240429151916.pkg: 100%    2 MiB   1.1MB/s    00:02   
[78/166] Fetching FreeBSD-zfs-dev-lib32-15.snap20240429151916.pkg: 100%    7 MiB   1.8MB/s    00:04   
[79/166] Fetching FreeBSD-ssh-dev-15.snap20240429151916.pkg: 100%    1 MiB   1.4MB/s    00:01   
[89/166] Fetching FreeBSD-runtime-15.snap20240429151916.pkg: 100%    2 MiB   2.6MB/s    00:01   
[97/166] Fetching FreeBSD-ssh-dev-lib32-15.snap20240429151916.pkg: 100%    1 MiB   1.3MB/s    00:01   
[101/166] Fetching FreeBSD-openssl-lib-lib32-15.snap20240429151916.pkg: 100%    2 MiB   1.7MB/s    00:01   
[103/166] Fetching FreeBSD-dtrace-dev-15.snap20240429151916.pkg: 100%    1 MiB   1.3MB/s    00:01   
[113/166] Fetching FreeBSD-clang-dev-15.snap20240429151916.pkg: 100%   15 MiB   1.6MB/s    00:10   
[120/166] Fetching FreeBSD-utilities-15.snap20240429151916.pkg: 100%   11 MiB   1.9MB/s    00:06   
[121/166] Fetching FreeBSD-runtime-dev-lib32-15.snap20240429151916.pkg: 100%    3 MiB   1.2MB/s    00:03   
[132/166] Fetching FreeBSD-unbound-dev-15.snap20240429151916.pkg: 100%    2 MiB   1.1MB/s    00:02   
[133/166] Fetching FreeBSD-openssl-lib-dev-15.snap20240429151916.pkg: 100%    8 MiB   1.3MB/s    00:07   
[136/166] Fetching FreeBSD-unbound-dev-lib32-15.snap20240429151916.pkg: 100%    2 MiB   2.0MB/s    00:01   
[137/166] Fetching FreeBSD-runtime-dev-15.snap20240429151916.pkg: 100%    4 MiB   1.9MB/s    00:02   
[145/166] Fetching FreeBSD-libarchive-dev-lib32-15.snap20240429151916.pkg: 100%    1 MiB   1.5MB/s    00:01   
[150/166] Fetching FreeBSD-clibs-lib32-15.snap20240429151916.pkg: 100%    1 MiB   1.6MB/s    00:01   
[163/166] Fetching FreeBSD-clibs-dev-15.snap20240429151916.pkg: 100%   16 MiB   1.7MB/s    00:10   
[166/166] Fetching FreeBSD-kerberos-lib-dev-lib32-15.snap20240429151916.pkg: 100%    4 MiB   1.1MB/s    00:04   
Fetching data.pkg: 100%    7 MiB   1.5MB/s    00:05   
%

meaw229a · May 6, 2024

christhegeek said:
FreeBSD in Greece !
One of the reasons i stopped a little FreeBSD project i had for some (~3-4?) months (cultbsd) was that pkg downloading speed is painfully slow.

I have been there before. You need to change your Geo location. Means use a VPN.
You can try around and find pkg servers with lightning fast speeds.

Cath O'Deray · May 6, 2024

meaw229a said:
… need to change your Geo location. Means use a VPN. …

Hopefully no need.

vocom said:
I've tried VPNs. No difference.

Rob D. said:
… MaxMind GeoLite based geo-dns …

christhegeek if you have not already done so, check that your /etc/pkg/FreeBSD.conf is set to use pkg+https for the url.

I don't imagine an improvement in speed, but you may as well bring the file up-to-date with the OS capabilities.

christhegeek · May 8, 2024

Cath O'Deray said:

Unlikely to help with speeds of downloads from the pkg mirror infrastructure, but thanks.

Fastest for me, today, at home, is Frankfurt at 5.8 MB/s:

Code:

root@mowa219-gjp4-zbook-freebsd:~ # date ; fastest_pkg -v
Mon May  6 10:07:10 BST 2024
pkg0.nyi.freebsd.org: 100%
pkg0.nyi.freebsd.org: 4.0 MB/s
pkg0.fmt.freebsd.org: 100%
pkg0.fmt.freebsd.org: 2.7 MB/s
pkg0.bbt.freebsd.org: 82%
(28, 'Operation timed out after 5001 milliseconds with 6113313 out of 7453912 bytes received')

pkg0.bbt.freebsd.org: 1.2 MB/s
pkg0.fra.freebsd.org: 100%
pkg0.fra.freebsd.org: 5.8 MB/s
pkg0.bme.freebsd.org: 0%
(7, "Failed to connect to pkg0.bme.freebsd.org port 80 after 38 ms: Couldn't connect to server")
pkg0.bme.freebsd.org: 0.0 B/s
pkg0.bra.freebsd.org: 82%
(28, 'Operation timed out after 5002 milliseconds with 6144329 out of 7453912 bytes received')

pkg0.bra.freebsd.org: 1.2 MB/s
pkg0.sjb.freebsd.org: 100%
pkg0.sjb.freebsd.org: 5.2 MB/s
pkg0.kul.freebsd.org: 100%
pkg0.kul.freebsd.org: 1.7 MB/s
pkg0.kwc.freebsd.org: 100%
pkg0.kwc.freebsd.org: 1.5 MB/s
pkg0.jinx.freebsd.org: 100%
pkg0.jinx.freebsd.org: 2.2 MB/s
pkg0.tuk.freebsd.org: 100%
pkg0.tuk.freebsd.org: 2.6 MB/s
pkg0.twn.freebsd.org: 0%
(28, 'Connection timed out after 5002 milliseconds')
pkg0.twn.freebsd.org: 0.0 B/s
pkg0.syd.freebsd.org: 84%
(28, 'Operation timed out after 5004 milliseconds with 6320513 out of 7453912 bytes received')

pkg0.syd.freebsd.org: 1.3 MB/s

Fastest:
pkg0.fra.freebsd.org: 5.8 MB/s


Write configuration:
mkdir -p /usr/local/etc/pkg/repos/
echo 'FreeBSD: { url: "http://pkg0.fra.freebsd.org/${ABI}/latest", mirror_type: "NONE" }' \
        > /usr/local/etc/pkg/repos/FreeBSD.conf


root@mowa219-gjp4-zbook-freebsd:~ #

Actual speeds were far lower on Tuesday 30th April, I didn't keep a note of where I was at the time:

Code:

% pwd
/usr/home/grahamperrin/Documents/IT/BSD/FreeBSD/poudriere/pkgbase
% grep -i mb\/s 2024-04-30\ jail\ update\ mysteries.txt
[1/166] Fetching FreeBSD-libsqlite3-dev-15.snap20240429151916.pkg: 100%    2 MiB   2.2MB/s    00:01  
[5/166] Fetching FreeBSD-kerberos-lib-dev-15.snap20240429151916.pkg: 100%    5 MiB   1.6MB/s    00:03  
[9/166] Fetching FreeBSD-utilities-dev-lib32-15.snap20240429151916.pkg: 100%    4 MiB   1.3MB/s    00:03  
[19/166] Fetching FreeBSD-src-sys-15.snap20240429220940.pkg: 100%   93 MiB   1.7MB/s    00:59  
[26/166] Fetching FreeBSD-zfs-lib32-15.snap20240429151916.pkg: 100%    1 MiB   1.5MB/s    00:01  
[27/166] Fetching FreeBSD-dtrace-dev-lib32-15.snap20240429151916.pkg: 100%    1 MiB   1.2MB/s    00:01  
[30/166] Fetching FreeBSD-utilities-dev-15.snap20240429151916.pkg: 100%    6 MiB   2.0MB/s    00:03  
[32/166] Fetching FreeBSD-openssl-lib-dev-lib32-15.snap20240429151916.pkg: 100%    7 MiB   1.8MB/s    00:04  
[37/166] Fetching FreeBSD-rescue-15.snap20240429151916.pkg: 100%    6 MiB   1.6MB/s    00:04  
[47/166] Fetching FreeBSD-zfs-dev-15.snap20240429151916.pkg: 100%    7 MiB   1.6MB/s    00:05  
[56/166] Fetching FreeBSD-clang-15.snap20240429151916.pkg: 100%   46 MiB   1.6MB/s    00:30  
[60/166] Fetching FreeBSD-clibs-dev-lib32-15.snap20240429151916.pkg: 100%   13 MiB   2.2MB/s    00:06  
[70/166] Fetching FreeBSD-tests-15.snap20240429151916.pkg: 100%   18 MiB   1.5MB/s    00:13  
[71/166] Fetching FreeBSD-libarchive-dev-15.snap20240429151916.pkg: 100%    1 MiB   1.6MB/s    00:01  
[75/166] Fetching FreeBSD-utilities-lib32-15.snap20240429151916.pkg: 100%    2 MiB   1.1MB/s    00:02  
[78/166] Fetching FreeBSD-zfs-dev-lib32-15.snap20240429151916.pkg: 100%    7 MiB   1.8MB/s    00:04  
[79/166] Fetching FreeBSD-ssh-dev-15.snap20240429151916.pkg: 100%    1 MiB   1.4MB/s    00:01  
[89/166] Fetching FreeBSD-runtime-15.snap20240429151916.pkg: 100%    2 MiB   2.6MB/s    00:01  
[97/166] Fetching FreeBSD-ssh-dev-lib32-15.snap20240429151916.pkg: 100%    1 MiB   1.3MB/s    00:01  
[101/166] Fetching FreeBSD-openssl-lib-lib32-15.snap20240429151916.pkg: 100%    2 MiB   1.7MB/s    00:01  
[103/166] Fetching FreeBSD-dtrace-dev-15.snap20240429151916.pkg: 100%    1 MiB   1.3MB/s    00:01  
[113/166] Fetching FreeBSD-clang-dev-15.snap20240429151916.pkg: 100%   15 MiB   1.6MB/s    00:10  
[120/166] Fetching FreeBSD-utilities-15.snap20240429151916.pkg: 100%   11 MiB   1.9MB/s    00:06  
[121/166] Fetching FreeBSD-runtime-dev-lib32-15.snap20240429151916.pkg: 100%    3 MiB   1.2MB/s    00:03  
[132/166] Fetching FreeBSD-unbound-dev-15.snap20240429151916.pkg: 100%    2 MiB   1.1MB/s    00:02  
[133/166] Fetching FreeBSD-openssl-lib-dev-15.snap20240429151916.pkg: 100%    8 MiB   1.3MB/s    00:07  
[136/166] Fetching FreeBSD-unbound-dev-lib32-15.snap20240429151916.pkg: 100%    2 MiB   2.0MB/s    00:01  
[137/166] Fetching FreeBSD-runtime-dev-15.snap20240429151916.pkg: 100%    4 MiB   1.9MB/s    00:02  
[145/166] Fetching FreeBSD-libarchive-dev-lib32-15.snap20240429151916.pkg: 100%    1 MiB   1.5MB/s    00:01  
[150/166] Fetching FreeBSD-clibs-lib32-15.snap20240429151916.pkg: 100%    1 MiB   1.6MB/s    00:01  
[163/166] Fetching FreeBSD-clibs-dev-15.snap20240429151916.pkg: 100%   16 MiB   1.7MB/s    00:10  
[166/166] Fetching FreeBSD-kerberos-lib-dev-lib32-15.snap20240429151916.pkg: 100%    4 MiB   1.1MB/s    00:04  
Fetching data.pkg: 100%    7 MiB   1.5MB/s    00:05  
%

I always do that and the best speed i got was from France i suppose to have 200kbytes per second but most of the time i got arround 120kbytes/sec
Nope ! This isn't even doable !

Cath O'Deray · May 8, 2024

christhegeek said:
the best speed i got was from France i suppose to have 200kbytes per second

Oh :-(

I might ask someone in the know whether that's to be expected in your area.

Do you get acceptable speeds from other sites that are served from the same region?

Cath O'Deray · May 8, 2024

christhegeek said:
France

I'm not aware of a server in France.

http://pkg0.fra.freebsd.org/ is in Frankfurt, if that helps to explain things.

gotnull · May 8, 2024

Following what you said in this topic:

Pkg download speed is ultra slow i could compile these and still finish earlier. I live in Greece

Pkg download speed is ultra slow i could compile these and still finish earlier. I live in Greece. Making a new installation is a nightmare because downloading packages could take many many hours . You need to upgrade the servers dont do anything else.

forums.freebsd.org

There are actually 2 mirrors in Europe: Germany and Sweden.
The logical thing to do would be to set the closest one (Germany) but if the speed isn't good enough nothing stops you from manually set another mirror instead, you can give a try to the one from Sweden to compare:

Code:

mkdir -p /usr/local/etc/pkg/repos/
echo 'FreeBSD: { url: "https://pkg0.sjb.freebsd.org/${ABI}/quarterly", mirror_type: "NONE" }'  > /usr/local/etc/pkg/repos/FreeBSD.conf

If you want to test others mirrors, the full list is at the bottom of that page https://pkg.freebsd.org, just change the url accordingly in the file /usr/local/etc/pkg/repos/FreeBSD.conf

christhegeek · May 8, 2024

Cath O'Deray said:
I'm not aware of a server in France.

http://pkg0.fra.freebsd.org/ is in Frankfurt, if that helps to explain things.

Ah frankfurt , that is the fastest mirror fastest-pkg gives me with a 200kb/s but most of the time i get arround 80-130kbs

christhegeek · May 8, 2024

richardtoohey2 said:
If you look at christhegreek’s posts many are about poor network performance at his location.

For example:

Packages download speed is ultra slow ! 00's slow ! I could actually compile from source these packages and still would be installed faster.

Packages download speed is ultra slow ! 00's slow ! I could actually compile from source these packages and still many of them would be installed faster. I live in Greece and pkg download speed is awful.

forums.freebsd.org

Well its slow what can i tell.

Cath O'Deray · May 9, 2024

gotnull said:
http

Instead, https

gotnull · May 9, 2024

Cath O'Deray said:
Instead, https

Thank you, I edited the message.