jails Question about ordering of 'exec.prepare' and 'mount.fstab' during jail creation

L

liborator

Hello!

I am unsure if my case is working as intended or is a bug, but consider the following jail config file:

Code: 
 $ cat /etc/jail.conf.d/test-ordering.conf
test-ordering {

    host.hostname = "${name}";
    path = "/usr/local/jails/containers/${name}";

    exec.prepare = "echo tmpfs ${path}/tmp tmpfs rw 0 0 > ${path}/etc/fstab";

    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    exec.clean;
    exec.consolelog = "/var/log/jail_${name}_console.log";

    mount.fstab = "${path}/etc/fstab";
    persist;

}

in /usr/local/jails/containers/test-ordering is a ZFS clone of 14.1-RELEASE@p3 snapshot.

Starting the jail for the first time, I get the following error message:

Code: 
 # service jail onestart test-ordering
Starting jails: cannot start jail  "test-ordering": 
jail: test-ordering: mount.fstab: /usr/local/jails/containers/test-ordering/etc/fstab: No such file or directory
.

Reading the relevant bits of documentation for jail(8)():

Code: 
     exec.prepare
             Command(s) to run in the system environment to prepare a jail for
             creation.  These commands are executed before assigning IP
             addresses and mounting filesystems, so they may be used to create
             a new jail filesystem if it does not already exist.

     mount.fstab
             An fstab(5) format file containing filesystems to mount before
             creating a jail.

I thought that 'exec.prepare' would run to its completion and only after it the 'mount.fstab' checks the existence of the file, however, the error message pasted above indicates otherwise.

Meanwhile, if I "out-of-band" touch the "${path}/etc/fstab":

# touch /usr/local/jails/containers/test-ordering/etc/fstab

and start the jail afterwards, it comes up without problems and reads the actual contents of the fstab file!

Code: 
 # mount -t tmpfs | grep test-ordering
tmpfs on /usr/local/jails/containers/test-ordering/tmp (tmpfs, local)

This leads me to believe I have encountered a bug within ordering of the jail pseudo-parameters, but I want to check here first.

Thank you for comments!
Libor
 
As you've noted, the jail() man page specifies that exec.prepare runs commands that are executed BEFORE assigning IP addresses and mounting filesystems. The command that you've asked exec.prepare to run is running before the filesystem is mounted, so I'm guessing that's why it's failing. Have you tried looking in the log file for any error messages from the creation of the jail? You could also try moving your fstab command to get run by the exec.prestart hook instead and see if that works correctly.
 
liborator said:
Hello!

I am unsure if my case is working as intended or is a bug, but consider the following jail config file:

Code: 
 $ cat /etc/jail.conf.d/test-ordering.conf
test-ordering {

    host.hostname = "${name}";
    path = "/usr/local/jails/containers/${name}";

    exec.prepare = "echo tmpfs ${path}/tmp tmpfs rw 0 0 > ${path}/etc/fstab";

    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    exec.clean;
    exec.consolelog = "/var/log/jail_${name}_console.log";

    mount.fstab = "${path}/etc/fstab";
    persist;

}

in /usr/local/jails/containers/test-ordering is a ZFS clone of 14.1-RELEASE@p3 snapshot.

Starting the jail for the first time, I get the following error message:

Code: 
 # service jail onestart test-ordering
Starting jails: cannot start jail  "test-ordering":
jail: test-ordering: mount.fstab: /usr/local/jails/containers/test-ordering/etc/fstab: No such file or directory
.

Reading the relevant bits of documentation for jail(8)():

Code: 
     exec.prepare
             Command(s) to run in the system environment to prepare a jail for
             creation.  These commands are executed before assigning IP
             addresses and mounting filesystems, so they may be used to create
             a new jail filesystem if it does not already exist.

     mount.fstab
             An fstab(5) format file containing filesystems to mount before
             creating a jail.

I thought that 'exec.prepare' would run to its completion and only after it the 'mount.fstab' checks the existence of the file, however, the error message pasted above indicates otherwise.

Meanwhile, if I "out-of-band" touch the "${path}/etc/fstab":

# touch /usr/local/jails/containers/test-ordering/etc/fstab

and start the jail afterwards, it comes up without problems and reads the actual contents of the fstab file!

Code: 
 # mount -t tmpfs | grep test-ordering
tmpfs on /usr/local/jails/containers/test-ordering/tmp (tmpfs, local)

This leads me to believe I have encountered a bug within ordering of the jail pseudo-parameters, but I want to check here first.

Thank you for comments!
Libor
Click to expand...
I had a look at the code out of curiosity and here's what I understand:
What I don't understand is how you managed to make it work with a simple touch.
If the file exists, it will proceed with starting the jail and executing your prepare command. But at that point the fstab contents will already have been parsed into IP__MOUNT_FROM_FSTAB parameters. So you should not be finding the mount in the jail.

I'd be glad to read what I'm missing.

Or is $path your jail's filesystem root directory? In which case it's just the jail reading its fstab as it boots?
 
Hello!

Sorry for my rather untimely response, I was away for some time and then forgot I do not have email notifications enabled for threads I participate in...

So, I tried everything again and results are mixed:

1. touch does solve the issue of the jail not starting at all on first start.
2. As you said, mount.fstab reads the empty file and on first run, does not mount anything - that's an error in my original post - but the exec.prepare writes to that file nonetheless, meaning on subsequent starts of the jail the ${path}/etc/fstab serves as an fstab file.


Here is the procedure - every command is prefixed with either "host" or "jail" to clarify in which context is the command run.

1. I create 14.1-p4 ZFS snapshot. Snapshot is of the base system located in /usr/local/jails/templates/14.1 and it does not contain /etc/fstab:

host # chroot /usr/local/jails/templates/14.1-RELEASE cat /etc/fstab
cat: /etc/fstab: No such file or directory


2. I clone the snapshot into /usr/local/jails/containers/test-ordering:

host # zfs clone zstorage/jails/templates/14.1-RELEASE@p4 zstorage/jails/containers/test-ordering

3. I start the jail test-ordering, copy-pasting the contents of its config file here:

Code: 
host # cat /etc/jail.conf.d/test-ordering.conf
test-ordering {

    host.hostname = "${name}";
    path = "/usr/local/jails/containers/${name}";

    exec.prepare = "echo tmpfs ${path}/tmp tmpfs rw 0 0 > ${path}/etc/fstab";

    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    exec.clean;
    exec.consolelog = "/var/log/jail_${name}_console.log";

    allow.mount;
    allow.mount.tmpfs;

    mount.fstab = "${path}/etc/fstab";
    persist;

}

host # service jail onestart test-ordering
Starting jails: cannot start jail "test-ordering":
jail: test-ordering: mount.fstab: /usr/local/jails/containers/test-ordering/etc/fstab: No such file or directory
.


4. After I get the error, I touch the "${path}/etc/fstab" = /usr/local/jails/containers/test-ordering/etc/fstab file:

host # touch /usr/local/jails/containers/test-ordering/etc/fstab

5. I start the jail again and check mount command, therefore verify that it started correctly by the nature of "&&" process chaining.

host # service onestart test-ordering && mount -t tmpfs
Starting jails: test-ordering.
tmpfs on /tmp (tmpfs, local)


So, only the host's tmpfs is picked up. However, if I check the contents of the /usr/local/jails/containers/test-ordering/etc/fstab and also restart the jail, we can verify the file was written to and it gets picked up on restart:

host # cat /usr/local/jails/containers/test-ordering/etc/fstab && service jail onerestart test-ordering && mount -t tmpfs
tmpfs /usr/local/jails/containers/test-ordering/tmp tmpfs rw 0 0
Stopping jails: test-ordering.
Starting jails: test-ordering.
tmpfs on /tmp (tmpfs, local)
tmpfs on /usr/local/jails/containers/test-ordering/tmp (tmpfs, local)



Does the procedure make sense?


To be honest, I am still convinced this behaviour is either unintuitive or outright incorrect, because to me "exec.prepare" should come before "mount.fstab" - that's how I read the manpage, especially this part:

... These commands are executed before assigning IP addresses and mounting filesystems ...
Click to expand...

And fraxamo - thank you for your response! - I was aware of the exec.prestart hook, however, you always need to pair it with poststop otherwise you have a dangling mount, which prompted me to experiment with this exec.prepare...

Thank you!
 
I agree it's counterintuitive. I would expect like you that exec.prepare happens before the fstab file is parsed.

That might be worth filing a problem report on https://bugs.freebsd.org. It does not seem to be reported yet.

Please share the link to the PR here if you do, I would be interesting in the discussion / changes that could come from it.
 
You must log in or register to reply here.
Back
Top