Tensorflow package/binary install

J

japoc

Hello everyone,

I needed to install Tensorflow 2.1 for a project. Tensorflow is not available in the packages and the port is a very old version of Tensorflow (1.4 if I remember well).
So I tried to compile it from source. And, actually, it seems to be possible to compile Tensorflow for FreeBSD just by downloading the release and issuing a single build command:

Code: 
fetch 'https://github.com/tensorflow/tensorflow/archive/v2.1.0.tar.gz'
tar zxvf v2.1.0.tar.gz
cd tensorflow-2.1.0
bazel build -c opt --linkopt=-lexecinfo --host_linkopt=-lexecinfo //tensorflow/tools/pip_package:build_pip_package

(I found this info on these two links: https://github.com/tensorflow/tensorflow/issues/37345 and https://github.com/tensorflow/tensorflow/issues/23613#issuecomment-440757155 )

I did that and it started compiling the thing but the only problem is that it's been more than 24 hours now and the compilation is still not over for me.
So, that's very unpractical. I'm not going to be able to use FreeBSD for this project since I need to be able to deploy quickly.

Any chance we'll have a package some day? I believe if a port were to be created, it would be the same problem since ports usually compile what they install, but I could be wrong, I'm not sure about the limitations of ports vs packages.
 
Last edited by a moderator:
I created the original port of 1.14 for tensorflow as I needed it for my masters. I haven't had time to upgrade it due to university work, but my semester finishes mid May and I'll upgrade it then as I need tensorflow for my work.
 
Thanks for the reply Amzo. Won't the port take exactly the same time to build as my compilation script, though?
 
japoc said:
Won't the port take exactly the same time to build as my compilation script, though?
Click to expand...
Yes. That's the nature of compiling from ports. But if a port exists a package can be created for it too (which is what the build clusters do to create our package repositories). Everything always begins with a port. No port, no package, it's as simple as that. In essence a package is nothing more than a pre-compiled port.
 
Thanks. I understand that packages have to be created from ports. So let's assume I manage to create a port that compiles the version of Tensorflow I want. How can I transform this port into a package, so that, later on, I'm able to install Tensorflow quickly on other machines, as a package?
 
japoc said:
How can I transform this port into a package, so that, later on, I'm able to install Tensorflow quickly on other machines, as a package?
Click to expand...
If the port has been correctly configured it will already be able to do this (it's an integral part of the whole port system).
Code: 
     package              Make a binary package for the port.  The port will
                          be installed if it has not already been.  The
                          package is a .tbz file that you can use to install
                          the port on other machines with pkg-add(8).  If the
                          directory specified by PACKAGES does not exist, the
                          package will be put into the current directory.  See
                          PKGREPOSITORY and PKGFILE.
See ports(7).
 
If you want to take a stab at this you're going to need this too: FreeBSD Porter's Handbook

Because this is an existing port you can use the 'old' version as a basis. Change the version numbers at the top of the Makefile and see how far this gets you. Sometimes it just works. But most of the time you're going to need to fix a bunch of rejected patches, compilation errors, changed dependencies and whatnot.
 
japoc said:
How can I transform this port into a package, so that, later on, I'm able to install Tensorflow quickly on other machines, as a package?
Click to expand...
With make package from ports(7), as SirDice pointed out, but this works only if the other machines have the same architecture and version. If the other machines differ, then ports-mgmt/poudriere is needed (Handbook chapter 4.6. Building Packages with Poudriere).

Alternative to make package, if the port is allready installed on the system, without the need to build again, you can use pkg-create(8).
 
If I fix the port, the build machine will create a package and it will make it's way into the repository. The current port is marked as broken at the minute, bazel detects how many cores are available and tries to use the maximum amount. On the server when loads of packages are being built in parallel it causes it to eat resources and ends up being killed. For that reason it needs fixing, but once fixed and added pack into the port tree, a package will end up in the repository. It was only marked as broken 7 days ago.

FreeBSD Tensorflow port
 
I was thinking of adding an option in the Makefile so by default it only uses 1 job, but a person then has the option to tick the option to build with multiple jobs.
 
I managed to build tensorflow package with port a while ago. But today, it seems to fail in a new way (at least to me):
Code: 
ERROR: /usr/ports/science/py-tensorflow/work-py37/tensorflow-1.14.0/tensorflow/lite/tools/optimize/BUILD:14:1: undeclared inclusion(s) in rule '//tensorflow/lite/tools/optimize:quantization_utils':
this rule is missing dependency declarations for the following files included by 'tensorflow/lite/tools/optimize/quantization_utils.cc':
  'external/com_google_absl/absl/strings/string_view.h'
  'external/com_google_absl/absl/base/internal/throw_delegate.h'
(portsnap done today, FreeBSD 12.1)
 
Last edited by a moderator:
You must log in or register to reply here.
Back
Top