Solved build cloud image using package

N

Niks

Hi,
I want to build a cloud image (qcow2) from pkg so many good tools available which can use for creating a cloud-ready image, I am using kiwi tools to build RedHat and Debian Linux, now looking similar utility to build the image.

Thanks,
Nk
 
I can give you two recommendations.
First is poudriere image. This tool lets you build your own images for deployment.

Poudriere image [BSD Router Project]

bsdrp.net bsdrp.net

The second is rudimentary. Build using FreeBSD release scripts found in our source code.
/usr/src/release/
Take note of the Makefile.vm build directive. It has a qcow option.
I used this method for making deployable bhyve images. I copied /usr/src/release/ to a custom directory for my build
forums.freebsd.org

Solved - Building with release.sh on -STABLE

I have a question. What do I end up with if I build with /usr/src/release/release.sh on a -FreeBSD 12 -STABLE box? Does it built a 12 -STABLE image or 12 -RELEASE?
forums.freebsd.org forums.freebsd.org
 
Phishfry said:
I can give you two recommendations.
First is poudriere image. This tool lets you build your own images for deployment.

Poudriere image [BSD Router Project]

bsdrp.net bsdrp.net

The second is rudimentary. Build using FreeBSD release scripts found in our source code.
/usr/src/release/
Take note of the Makefile.vm build directive. It has a qcow option.
I used this method for making deployable bhyve images. I copied /usr/src/release/ to a custom directory for my build
forums.freebsd.org

Solved - Building with release.sh on -STABLE

I have a question. What do I end up with if I build with /usr/src/release/release.sh on a -FreeBSD 12 -STABLE box? Does it built a 12 -STABLE image or 12 -RELEASE?
forums.freebsd.org forums.freebsd.org
Click to expand...
Hi ,

could you please share steps or link to document to create qcow2 image, I checked in the source from svn and got /usr/src/release/ and also found Makefile.vm but unable to create image .

Thanks in advance.
 
For a /usr/src/release/ build checkout the directives in release.conf.sample
## If WITH_VMIMAGES is set to a non-empty value, this is a list of disk
## image formats to create. Valid values are listed in the mkimg(1)
## manual page, as well as 'mkimg --formats' output.
#VMFORMATS="vhdf vmdk qcow2 raw"
Click to expand...
So you need to set VMFORMATS variable to qcow2 in your custom build file.
You see release.conf is used to build a release. In it you can use variables to customize your build.
I believe you can use make vm-image command from the directory to build the qcow2 image you desire.
The reason I break out /usr/src/release/ is that I like to maintain the integrity of /usr/src/release/ and create a custom directory with my mods. I cp -vipr /usr/src/release/ /usr/src/custom/.
From there I rename release.conf to custom.conf and edit+customize to my taste.
Then you would run make vm-image from that directory.
Be advised that make release uses -HEAD by default.
You need to specify what branch you want to use if not -HEAD in your conf file.
 
Hi ,
I tried to customized conf file but getting errors unable to fix the same
I did following..
1. copied /usr/src/release to /usr/src/build (my build dir)
2. enabled custom.conf from release.conf.sample , update the chrootdir path , VMFORMAT , VMSIZE with rest default config

when tried with make release, and make vm-image bot exited

make vm-image got following output
Code: 
[root@freebsd /usr/src/build]# make vm-image
`vm-image' is up to date.
while checking /usr/obj/usr/src/amd64.amd64  only vm-image file created (empty file)
[root@freebsd /usr/obj/usr/src/amd64.amd64/build]# ls
dist            vm-image
[root@freebsd /usr/obj/usr/src/amd
while make release has followign error out put

Code: 
[root@freebsd /usr/src/build]# make release
make -C /usr/src/build  obj
make -C /usr/src/build  ftp cdrom memstick.img mini-memstick.img
mkdir -p dist
cd /usr/src/build/.. && make TARGET_ARCH=amd64 TARGET=amd64 distributeworld DISTDIR=/usr/obj/usr/src/amd64.amd64/build/dist
make[3]: "/usr/src/Makefile.inc1" line 97: A build is required first.  You may have the wrong MAKEOBJDIRPREFIX set.
*** Error code 1

Stop.
make[2]: stopped in /usr/src
*** Error code 1

Stop.
make[1]: stopped in /usr/src/build
*** Error code 1

Stop.
make: stopped in /usr/src/build

I don't know what I am missing yet, I checked variables in Makefiles but could not find any clue here :(
 
You need to have completed a build before you can create the release images.

make -C /usr/src buildworld buildkernel
 
Hi,

Successfully completed with following stage


make -C /usr/src buildworld buildkernel
make release

it created the memstick.img and mini-memstick img in build dir.

but when I run make vm-image it only throwing following message , vm-image is empty file. so how can I get qcow2 image
root@freebsd12:/usr/src/build # make vm-image
`vm-image' is up to date.
root@freebsd12:/usr/src/build #

secondly, can I customize packages , config files like (cloud-init,dns,ntp and other configs) in image while build qcow2?

my ultimate goal is to build the image through Jenkins . as we would like to create qcow2 image monthly with updated pkgs/ports;
 
Phishfry said:
For a /usr/src/release/ build checkout the directives in release.conf.sample

So you need to set VMFORMATS variable to qcow2 in your custom build file.
You see release.conf is used to build a release. In it you can use variables to customize your build.
I believe you can use make vm-image command from the directory to build the qcow2 image you desire.
The reason I break out /usr/src/release/ is that I like to maintain the integrity of /usr/src/release/ and create a custom directory with my mods. I cp -vipr /usr/src/release/ /usr/src/custom/.
From there I rename release.conf to custom.conf and edit+customize to my taste.
Then you would run make vm-image from that directory.
Be advised that make release uses -HEAD by default.
You need to specify what branch you want to use if not -HEAD in your conf file.
Click to expand...
I got an empty vm-image file from make-vmimage , I have also customized release.conf but no help.. could you please share your release.conf(customize.conf) or sample file which works well .. if you can share.!
 
Have you read the man page for release.sh ?
You must use a WITH_VMIMAGES in addition to VM_FORMATS in your conf file.

release(7)

www.freebsd.org www.freebsd.org
I used a plain image file in my experiments so I cannot help much..
 
For packages you are going to need to read the source code.
Especially these files
/release/tools/vmimage.subr
/release/scripts/mk-vmimage.sh
I believe the function 'vm_extra_install_packages' does that and {VM_EXTRA_PACKAGES} goes in your conf file.
There is no helping you in this task. You just need to dig into the source and try things.
 
Phishfry said:
Have you read the man page for release.sh ?
You must use a WITH_VMIMAGES in addition to VM_FORMATS in your conf file.

release(7)

www.freebsd.org www.freebsd.org
I used a plain image file in my experiments so I cannot help much..
Click to expand...

Yes I was reading man page of release and tried commands from examples after customization of release.conf
 
Niks said:
I got an empty vm-image file from make-vmimage , I have also customized release.conf but no help.. could you please share your release.conf(customize.conf) or sample file which works well .. if you can share.!
Click to expand...
Thanks,

Here is my release.conf ..
Code: 
root@freebsd:/usr/src/build # vi release.conf
#!/bin/sh
#
# $FreeBSD: stable/12/release/release.conf.sample 339435 2018-10-19 00:24:23Z gjb $
#

## Redefine environment variables here to override prototypes
## defined in release.sh.
#load_chroot_env() { }
#load_target_env() { }
#buildenv_setup() { }

## Set the directory within which the release will be built.
CHROOTDIR="/osbuild"

## Set the svn host.
#SVNROOT="svn://svn.FreeBSD.org/"

## Set the src/, ports/, and doc/ branches or tags.
#SRCBRANCH="base/stable/12@rHEAD"
#DOCBRANCH="doc/head@rHEAD"
#PORTBRANCH="ports/head@rHEAD"

## Run svn co --force for src checkout.
#SRC_FORCE_CHECKOUT=yes

## Sample configuration for using git instead of svn.
VCSCMD="/usr/local/bin/git clone --branch master"
SVNROOT=""
SRCBRANCH="[URL]https://github.com/freebsd/freebsd[/URL]"
DOCBRANCH="[URL]https://github.com/freebsd/freebsd-doc[/URL]"
PORTBRANCH="[URL]https://github.com/freebsd/freebsd-ports[/URL]"

## Set to override the default target architecture.
TARGET="amd64"
TARGET_ARCH="amd64"
KERNEL="GENERIC"
## Multiple kernels may be set.
#KERNEL="GENERIC XENHVM"

## Set to specify a custom make.conf and/or src.conf
#MAKE_CONF="/etc/local/make.conf"
#SRC_CONF="/etc/local/src.conf"

## Set to use make(1) flags.
#MAKE_FLAGS="-s"

## Set to use world- and kernel-specific make(1) flags.
#WORLD_FLAGS="-j $(sysctl -n hw.ncpu)"
#KERNEL_FLAGS="-j $(( $(( $(sysctl -n hw.ncpu) + 1 )) / 2 ))"

## Set miscellaneous 'make release' settings.
#NODOC=
#NOPORTS=
#NOSRC=
#WITH_DVD=
#WITH_COMPRESSED_IMAGES=

## Set to '1' to disable multi-threaded xz(1) compression.
#XZ_THREADS=0

## Set when building embedded images.
#EMBEDDEDBUILD=

## Set to a list of ports required to build embedded system-on-chip
## images, such as sysutils/u-boot-rpi.
#EMBEDDEDPORTS=

## Set to the hardware platform of the target userland.  This value
## is passed to make(1) to set the TARGET (value of uname -m) to cross
## build.
#EMBEDDED_TARGET=

## Set to the machine processor architecture of the target userland.
## This value is passed to make(1) to set the TARGET_ARCH (value of uname -p)
## to cross build.
#EMBEDDED_TARGET_ARCH=

## Set to skip the chroot environment buildworld/installworld/distribution
## step if it is expected the build environment will exist via alternate
## means.
#CHROOTBUILD_SKIP=

## Set to a non-empty value skip checkout or update of /usr/src in
## the chroot.  This is intended for use when /usr/src already exists.
#SRC_UPDATE_SKIP=

## Set to a non-empty value skip checkout or update of /usr/doc in
## the chroot.  This is intended for use when /usr/doc already exists.
#DOC_UPDATE_SKIP=

## Set to a non-empty value skip checkout or update of /usr/ports in
## the chroot.  This is intended for use when /usr/ports already exists.
#PORTS_UPDATE_SKIP=

## Set to pass additional flags to make(1) for the build chroot setup, such
## as TARGET/TARGET_ARCH.
#CHROOT_MAKEENV=

## Set to a non-empty value to build virtual machine images as part of the
## release build.
WITH_VMIMAGES=

## Set to a non-empty value to compress virtual machine images with xz(1)
## as part of the release build.
#WITH_COMPRESSED_VMIMAGES=

## If WITH_VMIMAGES is set to a non-empty value, this is the name of the
## file to use for the installed userland/kernel.
#VMBASE="vm"

## If WITH_VMIMAGES is set to a non-empty value, this is the size of the
## virtual machine disk filesystem.  Valid size values are described in
## the truncate(1) manual page.
VMSIZE="6G"

## If WITH_VMIMAGES is set to a non-empty value, this is a list of disk
## image formats to create.  Valid values are listed in the mkimg(1)
## manual page, as well as 'mkimg --formats' output.
VMFORMATS="qcow2"

## Set to a non-empty value to build virtual machine images for various
## cloud providers as part of the release build.
release.conf: 127 lines, 3939 characters.
root@freebsd:/usr/src/build # cat release.conf
#!/bin/sh
#
# $FreeBSD: stable/12/release/release.conf.sample 339435 2018-10-19 00:24:23Z gjb $
#

## Redefine environment variables here to override prototypes
## defined in release.sh.
#load_chroot_env() { }
#load_target_env() { }
#buildenv_setup() { }

## Set the directory within which the release will be built.
CHROOTDIR="/osbuild"

## Set the svn host.
#SVNROOT="svn://svn.FreeBSD.org/"

## Set the src/, ports/, and doc/ branches or tags.
#SRCBRANCH="base/stable/12@rHEAD"
#DOCBRANCH="doc/head@rHEAD"
#PORTBRANCH="ports/head@rHEAD"

## Run svn co --force for src checkout.
#SRC_FORCE_CHECKOUT=yes

## Sample configuration for using git instead of svn.
VCSCMD="/usr/local/bin/git clone --branch master"
SVNROOT=""
SRCBRANCH="[URL]https://github.com/freebsd/freebsd[/URL]"
DOCBRANCH="[URL]https://github.com/freebsd/freebsd-doc[/URL]"
PORTBRANCH="[URL]https://github.com/freebsd/freebsd-ports[/URL]"

## Set to override the default target architecture.
TARGET="amd64"
TARGET_ARCH="amd64"
KERNEL="GENERIC"
## Multiple kernels may be set.
#KERNEL="GENERIC XENHVM"

## Set to specify a custom make.conf and/or src.conf
#MAKE_CONF="/etc/local/make.conf"
#SRC_CONF="/etc/local/src.conf"

## Set to use make(1) flags.
#MAKE_FLAGS="-s"

## Set to use world- and kernel-specific make(1) flags.
#WORLD_FLAGS="-j $(sysctl -n hw.ncpu)"
#KERNEL_FLAGS="-j $(( $(( $(sysctl -n hw.ncpu) + 1 )) / 2 ))"

## Set miscellaneous 'make release' settings.
#NODOC=
#NOPORTS=
#NOSRC=
#WITH_DVD=
#WITH_COMPRESSED_IMAGES=

## Set to '1' to disable multi-threaded xz(1) compression.
#XZ_THREADS=0

## Set when building embedded images.
#EMBEDDEDBUILD=

## Set to a list of ports required to build embedded system-on-chip
## images, such as sysutils/u-boot-rpi.
#EMBEDDEDPORTS=

## Set to the hardware platform of the target userland.  This value
## is passed to make(1) to set the TARGET (value of uname -m) to cross
## build.
#EMBEDDED_TARGET=

## Set to the machine processor architecture of the target userland.
## This value is passed to make(1) to set the TARGET_ARCH (value of uname -p)
## to cross build.
#EMBEDDED_TARGET_ARCH=

## Set to skip the chroot environment buildworld/installworld/distribution
## step if it is expected the build environment will exist via alternate
## means.
#CHROOTBUILD_SKIP=

## Set to a non-empty value skip checkout or update of /usr/src in
## the chroot.  This is intended for use when /usr/src already exists.
#SRC_UPDATE_SKIP=

## Set to a non-empty value skip checkout or update of /usr/doc in
## the chroot.  This is intended for use when /usr/doc already exists.
#DOC_UPDATE_SKIP=

## Set to a non-empty value skip checkout or update of /usr/ports in
## the chroot.  This is intended for use when /usr/ports already exists.
#PORTS_UPDATE_SKIP=

## Set to pass additional flags to make(1) for the build chroot setup, such
## as TARGET/TARGET_ARCH.
#CHROOT_MAKEENV=

## Set to a non-empty value to build virtual machine images as part of the
## release build.
WITH_VMIMAGES=

## Set to a non-empty value to compress virtual machine images with xz(1)
## as part of the release build.
#WITH_COMPRESSED_VMIMAGES=

## If WITH_VMIMAGES is set to a non-empty value, this is the name of the
## file to use for the installed userland/kernel.
#VMBASE="vm"

## If WITH_VMIMAGES is set to a non-empty value, this is the size of the
## virtual machine disk filesystem.  Valid size values are described in
## the truncate(1) manual page.
VMSIZE="6G"

## If WITH_VMIMAGES is set to a non-empty value, this is a list of disk
## image formats to create.  Valid values are listed in the mkimg(1)
## manual page, as well as 'mkimg --formats' output.
VMFORMATS="qcow2"

## Set to a non-empty value to build virtual machine images for various
## cloud providers as part of the release build.
#WITH_CLOUDWARE=

## If WITH_CLOUDWARE is set to a non-empty value, this is a list of providers
## to create disk images.
#CLOUDWARE="EC2 GCE VAGRANT-VIRTUALBOX VAGRANT-VMWARE"
root@freebsd:/usr/src/build # make -C /usr/src buildworld buildkernel
root@freebsd:/usr/src/build # cat release.conf
#!/bin/sh
#
# $FreeBSD: stable/12/release/release.conf.sample 339435 2018-10-19 00:24:23Z gjb $
#

## Redefine environment variables here to override prototypes
## defined in release.sh.
#load_chroot_env() { }
#load_target_env() { }
#buildenv_setup() { }

## Set the directory within which the release will be built.
CHROOTDIR="/osbuild"

## Set the svn host.
#SVNROOT="svn://svn.FreeBSD.org/"

## Set the src/, ports/, and doc/ branches or tags.
#SRCBRANCH="base/stable/12@rHEAD"
#DOCBRANCH="doc/head@rHEAD"
#PORTBRANCH="ports/head@rHEAD"

## Run svn co --force for src checkout.
#SRC_FORCE_CHECKOUT=yes

## Sample configuration for using git instead of svn.
VCSCMD="/usr/local/bin/git clone --branch master"
SVNROOT=""
SRCBRANCH="[URL]https://github.com/freebsd/freebsd[/URL]"
DOCBRANCH="[URL]https://github.com/freebsd/freebsd-doc[/URL]"
PORTBRANCH="[URL]https://github.com/freebsd/freebsd-ports[/URL]"

## Set to override the default target architecture.
TARGET="amd64"
TARGET_ARCH="amd64"
KERNEL="GENERIC"
## Multiple kernels may be set.
#KERNEL="GENERIC XENHVM"

## Set to specify a custom make.conf and/or src.conf
#MAKE_CONF="/etc/local/make.conf"
#SRC_CONF="/etc/local/src.conf"

## Set to use make(1) flags.
#MAKE_FLAGS="-s"

## Set to use world- and kernel-specific make(1) flags.
#WORLD_FLAGS="-j $(sysctl -n hw.ncpu)"
#KERNEL_FLAGS="-j $(( $(( $(sysctl -n hw.ncpu) + 1 )) / 2 ))"

## Set miscellaneous 'make release' settings.
#NODOC=
#NOPORTS=
#NOSRC=
#WITH_DVD=
#WITH_COMPRESSED_IMAGES=

## Set to '1' to disable multi-threaded xz(1) compression.
#XZ_THREADS=0

## Set when building embedded images.
#EMBEDDEDBUILD=

## Set to a list of ports required to build embedded system-on-chip
## images, such as sysutils/u-boot-rpi.
#EMBEDDEDPORTS=

## Set to the hardware platform of the target userland.  This value
## is passed to make(1) to set the TARGET (value of uname -m) to cross
## build.
#EMBEDDED_TARGET=

## Set to the machine processor architecture of the target userland.
## This value is passed to make(1) to set the TARGET_ARCH (value of uname -p)
## to cross build.
#EMBEDDED_TARGET_ARCH=

## Set to skip the chroot environment buildworld/installworld/distribution
## step if it is expected the build environment will exist via alternate
## means.
#CHROOTBUILD_SKIP=

## Set to a non-empty value skip checkout or update of /usr/src in
## the chroot.  This is intended for use when /usr/src already exists.
#SRC_UPDATE_SKIP=

## Set to a non-empty value skip checkout or update of /usr/doc in
## the chroot.  This is intended for use when /usr/doc already exists.
#DOC_UPDATE_SKIP=

## Set to a non-empty value skip checkout or update of /usr/ports in
## the chroot.  This is intended for use when /usr/ports already exists.
#PORTS_UPDATE_SKIP=

## Set to pass additional flags to make(1) for the build chroot setup, such
## as TARGET/TARGET_ARCH.
#CHROOT_MAKEENV=

## Set to a non-empty value to build virtual machine images as part of the
## release build.
WITH_VMIMAGES=

## Set to a non-empty value to compress virtual machine images with xz(1)
## as part of the release build.
#WITH_COMPRESSED_VMIMAGES=

## If WITH_VMIMAGES is set to a non-empty value, this is the name of the
## file to use for the installed userland/kernel.
#VMBASE="vm"

## If WITH_VMIMAGES is set to a non-empty value, this is the size of the
## virtual machine disk filesystem.  Valid size values are described in
## the truncate(1) manual page.
VMSIZE="6G"

## If WITH_VMIMAGES is set to a non-empty value, this is a list of disk
## image formats to create.  Valid values are listed in the mkimg(1)
## manual page, as well as 'mkimg --formats' output.
VMFORMATS="qcow2"

## Set to a non-empty value to build virtual machine images for various
## cloud providers as part of the release build.
#WITH_CLOUDWARE=

## If WITH_CLOUDWARE is set to a non-empty value, this is a list of providers
## to create disk images.
#CLOUDWARE="EC2 GCE VAGRANT-VIRTUALBOX VAGRANT-VMWARE"
 
Hi ,

I have managed to create an image with following command.

[cmd=root@freebsd:usr/src/build #]make WITH_VMIMAGES=yes VMFORMATS=qcow2 VMSIZE=6G vm-image[/cmd]

with above command it create vm-image directory and mounted on md0 for chrooting

I don't know why I have to pass these parameter every time, ideally it should read the release.conf file.

I tried to pass following command to read release.conf but still no luck
Code: 
root@freebsd:/usr/src/build # make -T release.conf vm-image
touch vm-image
with above command it simply creates vm-image file and nothing happens.

Looks like something is missing in release.conf or could be bug?

Thanks,
 
Niks said:
I don't know why I have to pass these parameter every time, ideally it should read the release.conf file.
Click to expand...
It's a configuration file for the release.sh script, not for the make process.
 
As with a lot of things, use the source. Read the release.sh script to see what it does. Then read the various Makefiles to see what options there are, what needs to be set, etc. That's what I did and I'm pretty sure Phishfry did too.
 
Hi ,

I tried following command to build an image with packages required for Openstack env. .

# make WITH_CLOUDWARE=yes CLOUDWARE=OPENSTACK VMSIZE=4G VMFORMATS=qcow2 vm-cloudware

Ps: CLOUDWARE has following options available in release conf file but I was able to use OPENSTACK (which is not listed in conf file) and seems it use the tools/openstack.conf file while building the image
Code: 
## If WITH_CLOUDWARE is set to a non-empty value, this is a list of providers
## to create disk images.
#CLOUDWARE="EC2 GCE VAGRANT-VIRTUALBOX VAGRANT-VMWARE"
I got img and qcow2 files ,
Code: 
-rw-r--r--   1 root  wheel   4.0G Apr 21 16:17 openstack.img
-rw-r--r--   1 root  wheel   3.6G Apr 21 16:22 openstack.qcow2

If you have any other suggestion /input then would be great to add it here , which may helpful to others who is looking qcow2 image building on FreeBSD else I will close this thread.

Thank you SirDice and Phishfry for your assistance here!
 
Niks said:
Hi ,

I tried following command to build an image with packages required for Openstack env. .

# make WITH_CLOUDWARE=yes CLOUDWARE=OPENSTACK VMSIZE=4G VMFORMATS=qcow2 vm-cloudware

Ps: CLOUDWARE has following options available in release conf file but I was able to use OPENSTACK (which is not listed in conf file) and seems it use the tools/openstack.conf file while building the image
Code: 
## If WITH_CLOUDWARE is set to a non-empty value, this is a list of providers
## to create disk images.
#CLOUDWARE="EC2 GCE VAGRANT-VIRTUALBOX VAGRANT-VMWARE"
I got img and qcow2 files ,
Code: 
-rw-r--r--   1 root  wheel   4.0G Apr 21 16:17 openstack.img
-rw-r--r--   1 root  wheel   3.6G Apr 21 16:22 openstack.qcow2

If you have any other suggestion /input then would be great to add it here , which may helpful to others who is looking qcow2 image building on FreeBSD else I will close this thread.

Thank you SirDice and Phishfry for your assistance here!
Click to expand...
Secondly , I was able to add few pkgs , define pkgs in tools/openstack.conf file .. Tested qcow2 image and successfully created VM from the image.
 
You must log in or register to reply here.
Back
Top