Other Adventures in remote attached storage

I

inkfish

Hi everyone,

I just recently joined the freeBSD club and I really appreciate many features and the structure of the OS itself.
But this post isn't supposed to be a diary entry of why I switched to it on one of my servers or any ramblings of the sort.
Who would want to read that anyway, right?

I do however come to you today because I'm in the need of some guidance. If you read my post and you feel like hey this was solved in this thread already please do point me toward it. I sadly couldn't find any that matched my problem.

To get started though I will write down what I'm doing, what I tried to do, my band aid fix for now and why I need what I need. I'll mark the following part as background information for those who don't actually care for the extra bits and just want to get to the question at hand.

BACKGROUND INFORMATION STARTS HERE (skip if you don't care)

So I set up a media serving service (for myself mostly and family and what not) and there was some performance issues with the management part of it so I decided to split services onto different nodes.
I decided to try freeBSD for the media serving part of it and the efficiency and resource handling I'm really happy with so I went with it.
For a media server I'm using Jellyfin which is a FOSS alternative to plex. The way the setup works is that I have a tiny freeBSD node doing the actual serving of the content but the content itself lives on a remote storage device.
That remote storage device is offering several ways to access it. SSH, SMB/CIFS, WebDAV and a couple of others. On the previous system (which was ubuntu) I chose for no particular reason CIFS to mount it in my OS and little did I know at the time that decision saved me a lot of time.

When I switched to freeBSD I saw that there's no good native way to get SMB/CIFS to work above version 1. I wasn't really set on the SMB/CIFS protocol and I didn't wanna use v1 (which is still there to use) so I looked for the next best thing. Which appeared to me to be SSHFS.
I set it up, ported the metadata from the previous install and was happy how snappy it was. Fixed the issue where the storage would just detach after some inactivity.

So far so good! Thing is I have a lot of automations set up and I won't go into it here because this is already too long. But essentially files may be added to the storage pool dynamically.
Jellyfin has a way to detect when new files and directories are added to the library paths. Works like a charm too! .... on supported file systems.
Turns out SMB/CIFS is such a supported filesystem, my guess is that most local filesystems as well, give notifications when something has been added.
Mounting with SSHFS however offers no such functionality, from what I've read about it generally the FUSE doesn't offer that functionality on either freeBSD or linux. It just doesn't have any implementations for these api calls.
The SMB/CIFS implementation on linux as I figured out later makes calls to inotify api but from what I've gathered there's no equivalent on freeBSD.

Bummer. So I started looking how to mount CIFS v2 and up on freeBSD, unfortunately I haven't found a good way to do that and even the other threads I read about it say it's all kinda janky.
I even gave the good smbfs_mount (which works for v1) a try but it also didn't notify Jellyfin of changes in the filesystem. It was also on the verge of being too slow for my usecase.

At this point you may be asking (if you actually read all this LOL) why not just let jellyfin scan your library every 15 minutes and let it detect newly added files that way?
Well as it turns out full scanning almost 10 TB of data takes a while, so long in fact that the scan isn't even finished before the next one is triggered. The machine also gets slowed down and becomes more sluggish to respond.
But hats off to freeBSD here, because when I had full library scans on my linux box and it was working really hard I couldn't even connect with SSH sometimes.

Here's my very jank and not at all perfect solution for the problem. I knew jellyfin has an API I can call to initiate scans. So I wrote a script which gets triggered everytime a new item gets added to a library but as I said before a full scan is probably bogging down the system for quite some time I opted to do a just scan for new files call. Issue is if I do that it just essentially detects new paths and files and doesn't tag it with any metadata. So no images and not even the duration is listed.
I need to trigger a full scan which then just scans the entire library.
Unfortunately there is no endpoint to just grab the newest entries and just do a scan on those. It really just seems that Jellyfin relies on the filesystem to report new paths to selectively scan them and not just the entire thing.

BACKGROUND INFORMATION ENDS HERE

Now all of this left me wondering.

Is there a good way to mount CIFS shares in the OS with the necessary notify functionality for new files being added? If so what is it called?
I also looked what kind of remote filesystems are generally available. Of the usual suspects SMB/CIFS, NFS, SSHFS only SMB/CIFS supports notify functionality what I could see.

I do wonder now, on freeBSD what would be the preferred remote filesystem protocol to use? What is the best supported one with the most features? Am I missing one that would tick those boxes?

This is my first post on this forum and for the poor admin who has to review this. I'm sorry and I hope I put it in the right category. But if I didn't let me know?

Anyways for whoever read it all. Thank you! I hope anyone here has a good idea because I've sadly can't think of anything else I could do.
 
I admit, I skipped the background information. That being said:

The smbfs mount in base is very outdated. There's multiple groups attempting to get this to a more recent version, though that will take a while to deliver results.

You might be better off using something like NFS - if all your clients support it.

Also, FreeBSD does not have anything out of the box that triggers when changes occur in a file system - at least not that I know of. You might be able to build your own - depending on your capabilities you could go with a port that builds on inotify or similar, look into kqueue (eventually that will likely support directories as well) or use mtree to detect changes to a directory structure.
 
cmoerz said:
I admit, I skipped the background information. That being said:

The smbfs mount in base is very outdated. There's multiple groups attempting to get this to a more recent version, though that will take a while to deliver results.

You might be better off using something like NFS - if all your clients support it.

Also, FreeBSD does not have anything out of the box that triggers when changes occur in a file system - at least not that I know of. You might be able to build your own - depending on your capabilities you could go with a port that builds on inotify or similar, look into kqueue (eventually that will likely support directories as well) or use mtree to detect changes to a directory structure.
Click to expand...
Thank you for the response. Though you confirmed what I already suspected, unfortunately. NFS is not really an option and it doesn't support notify on changes to the file system either, from what I could tell.

What is very interesting though is what you said about FreeBSD not having any triggers at all for changes in the filesystem. Not even for native filesystems like ZFS? I thought such functionality would be standard at least in some way for at least local filesystems.
Interesting non the less.
Unfortunately I don't think I have the capability to write something myself. Truth be told, I wouldn't even know where to start. Truth be told, I know very little about filesystems. I'd have to figure out how to poll the attached storage get all the Information I need and what the client (Jellyfin) needs and how to properly pass it along. At that point unfortunately, it would be easier to just use Linux for the media serving part.

But appreciate the information :)
 
inkfish said:
The way the setup works is that I have a tiny freeBSD node doing the actual serving of the content but the content itself lives on a remote storage device.
Click to expand...
Just to get a more complete picture of your set up: what is the nature/type of that remote storage device (connected to your FreeBSD node that services all your media clients)? Is that perhaps a (network attached) ZFS pool with datasets (without a separate OS)?

You mentioned looking for an OS/API hook that you could use in order to getting more efficiently notified of changes in the library as opposed to a multimedia/jellyfin scan. However, from:
inkfish said:
why not just let jellyfin scan your library every 15 minutes and let it detect newly added files that way?
Click to expand...
I take it that it's not very critical for every client to be up to date of changes in the library at every moment; a delay of 15 mins or so would be quite acceptable?
 
As others have already mentioned, you can't expect monitoring w/ network shares. But I'll give a few options:
The first two options are probably the easiest. Personally, I use fscrawler since I'm spotlight indexing for Samba and have it notify plex/jellyfin.
 
Erichans said:
Just to get a more complete picture of your set up: what is the nature/type of that remote storage device (connected to your FreeBSD node that services all your media clients)? Is that perhaps a (network attached) ZFS pool with datasets (without a separate OS)?
Click to expand...
I'm not quite sure what storage solution is under the hood. It's not local so I'm not running it. It's a hetzner storage box.
(I wanted to post a link but not sure if that's allowed here)

Erichans said:
I take it that it's not very critical for every client to be up to date of changes in the library at every moment; a delay of 15 mins or so would be quite acceptable?
Click to expand...
A delay of 15 minutes would be acceptable only issue is that in order for it to be even 15 minutes I'd have to run a full scan every 15 minutes and a full scan takes probably 30 to 40 minutes. Meaning the entire system would sluggish basically all the time.

Basically, currently this happens Request made (through jellyseerr a different program) -> passed on to the responsible *arr program -> fetches the content and renames and moves it to the storage box -> Jellyfin (could previously) detect a change in the filesystem and trigger a scan just on the added directories (ignoring all of the unchanged library) and tag them and download all the metadata. That process would take 5 minutes from request to content ready to be served.
 
nocsi said:
As others have already mentioned, you can't expect monitoring w/ network shares.
Click to expand...
It does very much look like it. I'm a bit confused though why the monitoring worked when I mounted the exact same storage device with CIFS on my linux box. There does seem to be some capability to monitor network shares, at least with newer CIFS.
nocsi said:
*arr clients webhook call to jellyfin on import
Click to expand...
That's actually my current solution. Unfortunately all API calls to jellyfin pretty much trigger a full scan of at least a library. There is a way to just trigger a scan which just looks for new directories, which is faster but it completely avoid all metadata.

nocsi said:
NFS
Click to expand...
I do not have NFS available. But from what I read briefly about it, it also doesn't notify if the file system has changed. I could be wrong though, I just read that in passing.
nocsi said:
Click to expand...

This looks very interesting and might be exactly what I'm looking for! Thank you! Now I just have to see if it also works with an SSHFS. But this is great.
 
You must log in or register to reply here.
Back
Top