ZFS Nextcloud, ZFS and Redis

[jbo@]

Let there be a FreeBSD host running ZFS with Nextcloud (a PHP application). The Nextcloud documentation states:

You can significantly improve your Nextcloud server performance with memory caching, where frequently-requested objects are stored in memory for faster retrieval. There are two types of caches to use: a PHP opcode cache, which is commonly called opcache, and data caching for your Web server.

It then proceeds by suggesting using Redis as a data cache.
For this discussion, let's ignore the PHP opcode cache and solely focus on the "data cache" (i.e. files hosted by Nextcloud).

My question is this: ZFS already does in-memory caching of frequently used data in ARC (and optionally L2ARC). Is there any real-world benefit of still slapping Redis on top of this?
I don't doubt that this can make a difference on large scale, high-performance installations but for your every-day five-people-use-it type scenario running on your own server - Would running Redis in this scenario yield any feel-able performance gains?

 

[mer]

My opinion only:

"For faster retrieval" this is the key I would think.
Redis is interesting simplified I look at it as "in memory database"
So the Nextcloud application gets a request, needs to get data from somewhere to satisfy.
"get data from somewhere" then becomes "look up in Redis, if not there, get from disk"
How does data get into Redis? Likely request that misses, goes to disk, saves in Redis.
ARC can have things fall out of ARC, so request would ask Redis, not there, ask ZFS, ZFS goes to ARC, not there, goes to device, gets data, data winds up in ARC, winds up in Redis, request satisfied.

Your final question:
In a typical home user/small office home office, I don't think Redis adds anything performance-wise (perhaps theoretical but not reality). Lots of requests for lots of different data would thrash ARC but fill Redis to maybe steady state. So depends on workload, requests of data (same data, different data), but again my opinion for your last question, no I don't think Redis would add any performance.

 

[Jose]

Well this gives me the heebie-jeebies "Redis is an excellent modern memcache to use for distributed caching, and as a key-value store for Transactional File Locking..." Redis distributed locking is notoriously unreliable. I guess it'll be fine as long as you have a single-node Redis, but what happens when that node goes down? I'd rather continue to rely on "slow" database locking.

BTW, I don't think Redis is suitable for caching large files anyway. I suspect they're only using it to cache metadata. It may provide advantages on top of ZFS caching in that case.

 

[bakul]

My question is this: ZFS already does in-memory caching of frequently used data in ARC (and optionally L2ARC). Is there any real-world benefit of still slapping Redis on top of this?

It will be faster but the question you should ask yourself is, is it worth doing? I would keep things simple until there is a clear need for better performance. Presumably it is not hard to add a redis based datacache later. But I would measure and keep tabs on performance metrics that I care about as the use increases.

 

[SKull]

In a typical home user/small office home office, I don't think Redis adds anything performance-wise

Redis is definitely faster. However:

I suspect they're only using it to cache metadata.

That is exactly what nextcloud is doing

It IS faster. But only when one uses for example the search function and certain apps.
However it does nothing for file operations.

Setting up redis is a piece of cake
But it's definitely not necessary in smaller setups.

 

[gpw928]

My instincts are that you would need to benchmark on a case by case basis. But if your tank is on spinning disks, and access to metadata is a bottleneck, then adding a special vdev on reliable SSDs would be worth testing.

 

[Lamia]

Redis adds a performance boost even for a small workload be it in metadata, specific app or file access. NC automatically requires it as its storage and/or app grows.

 

[Alain De Vos]

You can use postgresql to store "json".
Or you can use mongodb.