I recently built a new file server and thought I'd share the in's and outs. For this KB I'm assuming the following.
4 drives
32GB of ram
10GB intel network card..
I'm also assuming this is a multi purpose server, meaning the host lives on the main zpool.. it may run other jails, bhyve instances or other services in addition to plex. Hence raid1+0 vs raidz1
Enough jabber, lets roll.
Step I: Install FreeBSD 12+
NOTE: I'm installing directly to the zpool NOT a usb key/drive strictly for encryption convenience and performance. With the zfs file system, snapshots and various other tools there is no reason to install to a boot device.
install as normal until you get to disk lay out
select ZFS file system
select all 4 drives and choose raid 10
check off encrypt drives
encrypt swap
zpool
yes/next to everything / install to taste
NOTE: if your installing to a USB key/drive .. just before reboot drop to command prompt
add:
this will add a 1 second delay at boot, giving the system enough time to properly poll the usb drive and boot without issues.
Step II: Update and optimize the HOST
vi /etc/rc.conf
/boot/loader.conf
NOTE: I purposely disabled hyper-treading for tcip performance reasons ..
the sysctl file has a LOT of changes, do NOT copy and paste without looking at the included reference links .
/etc/sysctl.conf
Reboot.
Part III: Installing Iocage and creating the plex jail
next on the host we need to create a new data set/s
for this example we will use movies and tv .. but you can add as many as you like
zfs list should show 3 data sets. plex .. with movies and tv under it..
let's create the jail
# change plex.network.local to the name of the plex server , update router and ip address as needed --thick-jail is also optional, I prefer full jails to use zfs send/receive
once the jail is done.. you should see it attached to your bridge device
should show the jail running
now we will mount the host datasets aand give plex access to read them
Part III : Installing Iocage and creating the plex jail
You should now have a hardened host with encrypted zpool and a single jail with access to your movies and tv data-sets
Step IV: Install Plex
from another computer.. load up chrome and go to
Presto ... Plex in a jail on it's own IP and read access to the dataset. scan your media folder and bust out the popcorn
4 drives
32GB of ram
10GB intel network card..
I'm also assuming this is a multi purpose server, meaning the host lives on the main zpool.. it may run other jails, bhyve instances or other services in addition to plex. Hence raid1+0 vs raidz1
Enough jabber, lets roll.
Step I: Install FreeBSD 12+
NOTE: I'm installing directly to the zpool NOT a usb key/drive strictly for encryption convenience and performance. With the zfs file system, snapshots and various other tools there is no reason to install to a boot device.
install as normal until you get to disk lay out
select ZFS file system
select all 4 drives and choose raid 10
check off encrypt drives
encrypt swap
zpool
yes/next to everything / install to taste
NOTE: if your installing to a USB key/drive .. just before reboot drop to command prompt
vi /boot/loader.confadd:
Code:
kern.cam.boot_delay="10000" :wq!this will add a 1 second delay at boot, giving the system enough time to properly poll the usb drive and boot without issues.
Step II: Update and optimize the HOST
vi /etc/rc.conf
Code:
clear_tmp_enable="YES" # optional, clears tmp folder
ifconfig_ix0="DHCP" # assumes you use dhcp, set it to whatever you like man ifconfig
syslogd_flags="-ss"
sendmail_enable="NONE"
hostname="hostname.domain.local" ## changeme
sshd_enable="YES" ## change if needed "NO"
ntpdate_enable="YES"
ntpd_enable="YES"
powerd_enable="YES" ## change if you see errors or your processor does not support powerd
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
zfs_enable="YES"
keyrate="250.34"
# IOCage with VNET jails
iocage_enable="YES"
cloned_interfaces="bridge0"
ifconfig_bridge0="addm ix0 up" ### change the ix0 to your nic .. mine just happens to be an intel 10gb/boot/loader.conf
Code:
aesni_load="YES"
geom_eli_load="YES"
security.bsd.allow_destructive_dtrace=0
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
opensolaris_load="YES"
zfs_load="YES"
cc_htcp_load="YES"
vfs.zfs.dirty_data_max_max="17179869184" # change me to your memory.. this writes arc out at 16gb
net.inet.tcp.hostcache.cachelimit="0"
kern.geom.label.disk_ident.enable="0" # (default 1) diskid/DISK-ABC0123...
kern.geom.label.gptid.enable="0" # (default 1) gptid/123abc-abc123..
machdep.hyperthreading_allowed="0" # (default 1, allow Hyper Threading (HT))
net.tcp.soreceive_stream="1" # (default 0)NOTE: I purposely disabled hyper-treading for tcip performance reasons ..
the sysctl file has a LOT of changes, do NOT copy and paste without looking at the included reference links .
/etc/sysctl.conf
Code:
security.bsd.see_other_gids=0
security.bsd.see_jail_proc=0
security.bsd.unprivileged_read_msgbuf=0
security.bsd.unprivileged_proc_debug=0
kern.randompid=1
vfs.zfs.min_auto_ashift=12------------------------------------------------------
vfs.zfs.arc_max=16G # change me to the 100% arc size
## iocage https://www.ccammack.com/create-vnet-jails-in-freebsd-12-using-iocage/
## additonal information on Iocage
net.inet.ip.forwarding=1
net.link.bridge.pfil_onlyip=0
net.link.bridge.pfil_bridge=0
net.link.bridge.pfil_member=0
## calomel https://calomel.org/freebsd_network_tuning.html
## make sure you read this article
net.inet.tcp.recvbuf_inc=65536 # (default 16384)
net.inet.tcp.recvbuf_max=4194304 # (default 2097152)
net.inet.tcp.recvspace=65536 # (default 65536)
net.inet.tcp.sendbuf_inc=65536 # (default 8192)
net.inet.tcp.sendbuf_max=4194304 # (default 2097152)
net.inet.tcp.sendspace=65536
net.inet.tcp.mssdflt=1460 # Option 1 (default 536)
net.inet.tcp.minmss=536 # (default 216)
net.inet.tcp.abc_l_var=44 # (default 2) if net.inet.tcp.mssdflt = 1460
net.inet.tcp.cc.algorithm=htcp # (default newreno)
net.inet.tcp.cc.htcp.adaptive_backoff=1 # (default 0 ; disabled)
net.inet.tcp.cc.htcp.rtt_scaling=1 # (default 0 ; disabled)
net.inet.tcp.cc.abe=1 # (default 0, disabled)
net.inet.tcp.rfc6675_pipe=1 # (default 0)
net.inet.tcp.syncache.rexmtlimit=0 # (default 3)
net.inet.ip.maxfragpackets=0 # (default 63474)
net.inet.ip.maxfragsperpacket=0 # (default 16)
net.inet6.ip6.maxfragpackets=0 # (default 507715)
net.inet6.ip6.maxfrags=0
net.inet.tcp.syncookies=0 # (default 1)
net.inet.tcp.isn_reseed_interval=4500 # (default 0, disabled)
net.inet.tcp.tso=0 # (default 1)
kern.random.fortuna.minpoolsize=128 # (default 64)
kern.random.harvest.mask=65887 # (default 66047, FreeBSD 12 with Intel Secure Key RNG)
##system hardening
hw.kbd.keymap_restrict_change=4 # disallow keymap changes for non-privileged users (default 0)
kern.ipc.shm_use_phys=1 # lock shared memory into RAM and prevent it from being paged out to swap (default 0, disabled)
kern.msgbuf_show_timestamp=1 # display timestamp in msgbuf (default 0)
kern.randompid=1 # calculate PIDs by the modulus of an integer, set to one(1) to auto random (default 0)
net.bpf.optimize_writers=1 # bpf is write-only unless program explicitly specifies the read filter (default 0)
net.inet.icmp.drop_redirect=1 # no redirected ICMP packets (default 0)
net.inet.ip.check_interface=1 # verify packet arrives on correct interface (default 0)
net.inet.ip.portrange.first=32768 # use ports 32768 to portrange.last for outgoing connections (default 10000)
net.inet.ip.portrange.randomcps=9999 # use random port allocation if less than this many ports per second are allocated (default 10)
net.inet.ip.portrange.randomtime=1 # seconds to use sequental port allocation before switching back to random (default 45 secs)
net.inet.ip.random_id=1 # assign a random IP id to each packet leaving the system (default 0)
net.inet.ip.redirect=0 # do not send IP redirects (default 1)
net.inet6.ip6.redirect=0 # do not sendevops@coppertree.netd IPv6 redirects (default 1)
net.inet.sctp.blackhole=2 # drop stcp packets destined for closed ports (default 0)
net.inet.tcp.blackhole=2 # drop tcp packets destined for closed ports (default 0)
net.inet.tcp.drop_synfin=1 # SYN/FIN packets get dropped on initial connection (default 0)
net.inet.tcp.fast_finwait2_recycle=1 # recycle FIN/WAIT states quickly, helps against DoS, but may cause false RST (default 0)
net.inet.tcp.fastopen.client_enable=0 # disable TCP Fast Open client side, enforce three way TCP handshake (default 1, enabled)
net.inet.tcp.fastopen.server_enable=0 # disable TCP Fast Open server side, enforce three way TCP handshake (default 0)
net.inet.tcp.finwait2_timeout=1000 # TCP FIN_WAIT_2 timeout waiting for client FIN packet before state close (default 60000, 60 sec)
net.inet.tcp.icmp_may_rst=0 # icmp may not send RST to avoid spoofed icmp/udp floods (default 1)
net.inet.tcp.keepcnt=2 # amount of tcp keep alive probe failures before socket is forced closed (default 8)
net.inet.tcp.keepidle=62000 # time before starting tcp keep alive probes on an idle, TCP connection (default 7200000, 7200 secs)
net.inet.tcp.keepinit=5000 # tcp keep alive client reply timeout (default 75000, 75 secs)
net.inet.tcp.msl=2500 # Maximum Segment Lifetime, time the connection spends in TIME_WAIT state (default 30000, 2*MSL = 60 sec)
net.inet.tcp.path_mtu_discovery=0 # disable for mtu=1500 as most paths drop ICMP type 3 packets, but keep enabled for mtu=9000 (default 1)
net.inet.udp.blackhole=1 # drop udp packets destined for closed sockets (default 0)
net.inet.udp.recvspace=1048576 # UDP receive space, HTTP/3 webserver, "netstat -sn -p udp" and increase if full socket buffers (default 42080)
security.bsd.hardlink_check_gid=1 # unprivileged processes may not create hard links to files owned by other groups, DISABLE for mailman (default 0)
security.bsd.hardlink_check_uid=1 # unprivileged processes may not create hard links to files owned by other users, DISABLE for mailman (default 0)
security.bsd.see_other_gids=0 # groups only see their own processes. root can see all (default 1)
security.bsd.see_other_uids=0 # users only see their own processes. root can see all (default 1)
security.bsd.stack_guard_page=1 # insert a stack guard page ahead of growable segments, stack smashing protection (SSP) (default 0)
security.bsd.unprivileged_proc_debug=0 # unprivileged processes may not use process debugging (default 1)
security.bsd.unprivileged_read_msgbuf=0 # unprivileged processes may not read the kernel message buffer (default 1)Reboot.
Part III: Installing Iocage and creating the plex jail
freebsd-update fetch install pkg install py36-iocage iocage activate zroot (or what ever you called your ZFS pool) iocage fetch -r LATESTnext on the host we need to create a new data set/s
for this example we will use movies and tv .. but you can add as many as you like
zfs set compression=lz4 zroot #optional, but tbh compression should be on your pool. zfs create -o mountpoint=/plex zroot/plex zfs create -o mountpoint=/plex/movies zroot/plex/movies zfs create -o mountpoint=/plex/tv zroot/plex/tvzfs list should show 3 data sets. plex .. with movies and tv under it..
let's create the jail
iocage create -n "plex.network.local" -r latest vnet="on" allow_raw_sockets="1" boot="on" defaultrouter="10.100.10.1" ip4_addr="10.100.10.200/24" --thick-jail# change plex.network.local to the name of the plex server , update router and ip address as needed --thick-jail is also optional, I prefer full jails to use zfs send/receive
once the jail is done.. you should see it attached to your bridge device
iocage listshould show the jail running
iocage exec plex mkdir /media/movies iocage exec plex mkdir /media/tvnow we will mount the host datasets aand give plex access to read them
Part III : Installing Iocage and creating the plex jail
iocage fstab -a plex.network.local zroot/plex/movie /media/movie nullfs rw 0 0 iocage fstab -a plex.network.local zroot/plex/tv /media/tv nullfs rw 0 0You should now have a hardened host with encrypted zpool and a single jail with access to your movies and tv data-sets
Step IV: Install Plex
iocage console plex pkg install plexmediaserver sysrc plexmediaserver_enable=YES service plexmediaserver startfrom another computer.. load up chrome and go to
Presto ... Plex in a jail on it's own IP and read access to the dataset. scan your media folder and bust out the popcorn
