FreeBSD supports of EXT4 ?

Spartrekus

Spartrekus

Hello,

What would you think if FreeBSD had a support for EXT4 filesystem.
Why not by default allowing EXT4 during installation also.

Wouldn't it be good for the development and future of FreeBSD?

Have nice easter holidays!
 
I think the cost/benefit would not be worth it. What has EXT4 to offer that is direly needed? That is worth dealing with the GPL in kernel space (because of being root FS)?
 
Crivens said:
I think the cost/benefit would not be worth it. What has EXT4 to offer that is direly needed? That is worth dealing with the GPL in kernel space (because of being root FS)?
Click to expand...

There is not much directly needed, but it would allows servers running FreeBSD to be closer to LINUX operating system. Maybe some interests for some companies... and wider use of *BSD.
 
FreeBSD supports of EXT4?
Click to expand...

Yes with fusefs-ext4fuse package.

% ext4fuse
Version: ext4fuse-0.1.3
Usage: ext4fuse <disk> <mountpoint>

% pkg which $( which ext4fuse )
/usr/local/bin/ext4fuse was installed by package fusefs-ext4fuse-0.1.3_1,1
 
Many people run FreeBSD just because it isn't Linux, so adding Linux-based features into it could make the OS less appealing to that inner base of fans. Then what vermaden said: ext4 is already supported but not through the base system.

Which suits most people just fine I think, because this makes the OS sit a whole lot closer to Unix: doing a specific (not too extensive) task and be good at it. By default FreeBSD supports at least four major filesystems (UFS, ZFS, CDFS and FAT) out of the box and that's just fine for FreeBSD based operations. You really don't need more, and if you do there's always # pkg install.

Quite frankly I don't see any added value here, only more overhead and risk of adding bloat. I also seriously doubt that this will ever come to pass considering how FreeBSD tries to move away from the GPL3+ restrictions (which is one of the main reasons they dropped GCC).
 
I don't know if it's directly related to the stricter requirements of GPLv3, but FreeBSD have been working to remove GPL code from base for years, including having to fully rewrite many utilities. It's highly unlikely that any Linux file system could be included and supported for the root file system.

ZFS is currently the best choice for a file system that can be used by both systems. For most Linux file systems (other than the basic ext2/3 support) the only real option is the fuse modules that allow them to be used without needing in-kernel support.
 
Ext4 is supported under FreeBSD via fuse only

One must must take care with ext4. Ext4 is a journalised filesystem (not EXT2 so more recommendable for portable data, and there is a native kernel module support in FreeBSD), as such it can raise issues when writing such volume with a non native OS. Journalisation is not supported and so this can corrupt the journal with a potential loss of data.

In fact journalisation can be deactivated on ext4. So for a mobile drive formatted in EXT4 it is recommended to deactivate the journal via the linux tunefs command. This is exactly the same problem with NTFS, and many people ignore that.

As far as NTFS is used only on non native OS, journal is not activated, so there is apparently no problem, but if the volume is connected further to a Windows machine, this machine could activate the journal, and there is always a little risk when returning to a nix machine. If the file system is dirty, writing on it with a nix machine and it may invalidate the journal, and so returning to a windows machine it will be impossible to recover the data with the journal.

sysutils/fusefs-ext4fuse : in read only mode (very interesting to prevent any risk of data corruption if the filesystem is journalised)

sysutils/fusefs-ext2 : looking to the description, this port seems to support ext2, ext3, ext4 in read and write mode

sysutills/fusfefs-lkl : via linux kernel as a library. This port supports in read and write mode BTFRS, XFS, EXT3, EXT4

The last port has been marked as broken quite recently, as FreeBSD switched to GCC6 as the new default.
Anyway, uncommentting the IGNORE line in the Makefile and compiling with GCC 5 seems to work
The port build for me... it would just need some further check to verify if it is working properly

make USE_GCC=5 install

or

make USE_GCC=5 NO_IGNORE="yes" install (so no need to modify the Makefile)

There is absolutely no interest to port EXT4 to FreeBSD. In fact EXT4 is a very limited filesystem targeting workstation ONLY. BSD flavors generally target servers, big data... this is the reason why for many years BSD kept on using the Unix File System, while Linux was creating ten thousand of vaporware filesystems, and so later FreeBSD decided to integrate ZFS, which is also as UFS a big data filesystem.

You know what ? Ext4 is just a little toy with no interest for OS targeting primarily servers.
Just look at the Wikipedia... and you will see the very big difference between UFS, ZFS... engineered for big data, and... EXT4

XFS would be far more interesting (there was a kernel module in read only support, but since FreeBSD 11, XFS has been dropped, only fuse ports are available)

Hammer from Dragonfly seems to be promising if I remember some famous words of a member of this forum Oko

At this moment BTFRS is still... very experimental compared to ZFS... offered only by some Linux flavor as OpenSuse, but.... dropped by Red Hat. So BTFRS is quite controversial at this moment.

And finally.... Linux is Linux, Windows is Windows, FreeBSD is FreeBSD... there is from my point of view no interest to try to make them the same.

And remember that many people choose Linux because they hate Windows.... some people choose BSD because they hate Linux.

First, the BSD license and the GPL are totally different. This is just not only a little difference, and this does explain the "war" between the Linux (GPL) world and the BSD World.

The real and the only real FREE license is BSD, MIT.... not GPL
FreeBSD is built according to its own philosophy and we all love this philosophy, so we don't want that FreeBSD becomes one day a kind of Linux. If this day happens, I would migrate to OpenBSD, DragonFly or NetBSD
 
They will never implement it for you despite you've spend your own writing an ext2.ko, ext3.ko, ext4.ko, jfs.ko, xfs.ko, btrfs.ko... with a BSD license. Everything they could throw to you now is: ZFS is superior, why use that Linux sh*t :mad:
 
It's not going to be added, ever, due to the GPL. Nothing more, nothing less. Any other motive you think they have is pure fantasy.
 
giahung1997 said:
They will never implement it for you despite you've spend your own writing an ext2.ko, ext3.ko, ext4.ko, jfs.ko, xfs.ko, btrfs.ko... with a BSD license.
Click to expand...
Correct. No one in their right mind would add a Linux filesystem library which got released under any other license but the GPL. For the simple reason that doing so would violate said GPL which would open up an enormous can of worms.

It would appear that you don't realize this small fact, but this is why many people such as myself have grown a serious dislike for the GPL. It doesn't promote freedom but only enforces dictatorship. The GPL mandates that anything released under the GPL, including any forked or combined projects, may not be re-licensed and can only be distributed under said GPL.

And it gets even more bizarre... If my BSD licensed open source project uses a GPL licensed library then I am not allowed to distribute this library together with my software. Not even if I include all the relevant licenses and make it obviously clear that my BSD licensed program relies on a GPL licensed library. Nope.

b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License.
Click to expand...
Also important:
These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it.
Click to expand...

So yeah... I have no problem at all with people criticizing FreeBSD and/or the way it works or is being run, but it becomes annoying when they include plain out lies or raise impossible expectations and / or scenarios. What you're proposing is simply impossible due to GPL license restrictions. So trying to blame FreeBSD for that isn't really a fair argument to make.
 
ShelLuser: I don't know GPL that evil. So the best solution to access Linux FS from BSD is create a trimmed down Linux VM and export the block device to the Linux VM to mount/read/write to it (VBox can, not know about Bhyve) then using NFS to share with the host so we can read/write a Linux FS formated partition o_O
 
No. The GPL is much worse than a Windows license.

The MS EULA (for end users, not developers) is tough but fair. It is very one-sided, but that reflects the fact that MS is in a dominant market position, and can afford to enforce a tough license: most customers don't have an alternative to Windows and Office. Most importantly, it is pretty clear: you don't need huge teams of lawyers to understand the fine details, and it seldom leads to lawsuits.

Microsoft allows source code access, both to specific SDKs (for example for 3rd parties writing file systems for Windows), and for the Windows source code itself. The licenses for those are relatively concise, very clear, and easy to understand. MS does not grant source code access to very many 3rd parties (full source access must be extremely rare, and I've never heard of anyone who is not a MS employee having commit privileges to it), but for those that qualify, MS is pretty easy to work with. BTDT, got the T-shirt (not exactly me personally, I haven't actually seen the full Windows source code, but I've helped develop file systems that ship for Windows).

In contrast, the GPL is a disaster on several levels. To begin with, it was written by activists who are actually fanatics with a nearly religiously held dogma, with relatively little oversight by competent lawyers. It is unclear and subject to interpretation. If you want to do serious work (for hire) in the GPL arena, you will need an army of lawyers, and you can expect lawsuits to fly. The cause of the lawsuits is that the GPL is subjective, and there will always be someone (be it a troll or a fanatic or a highway robber) who will try to exploit that subjectivity. The early versions of the GPL were the worst in that respect, and subsequent revisions have helped a little bit. The next thing is that the GPL is terribly restrictive, making a lot of for-profit software development difficult or unviable. This got so bad that at some point they had to split off the LGPL, because applying the GPL to libraries became unworkable, and would have been self-defeating; the nasty side effect of this is that with both GPL and LGPL in play, the legal complexity just doubled. Then there is the infectious nature of the GPL, which scares anyone who tries to make money by selling software.

Fundamentally, the GPL is an attempt by a set of activists to undermine the system of copyright (and other forms of IP protection) for software. I understand what they are doing, and I even find their end goal to be laudable. But they picked completely the wrong tool. You don't eliminate copyrights and software licenses by creating a new copyright and license that try to blow up the system by using the systems internal tools. To a autistic technologist, this seems like a great hack; what they forgot in the process is that society is made of humans, who have goals and aspirations, and making money by writing software (we all want to get paid, starvation is not fun) is one of those. The FSF would have been much better served to take their fight to the US congress and similar legislatures, but they knew that it would be hopeless. So instead, they created a big mess, and now we need to deal with it.

Compared to that, I take the very clear and logical "supplier <-> customer" relationship with Microsoft any day, thank you.

Footnote: My criticism of the (L-) GPL above does not extend to other FOSS licenses, which have their own goals and problems. It only applies to Linux and its licensing system.
 
Spartrekus said:
Hello,

What would you think if FreeBSD had a support for EXT4 filesystem.
Why not by default allowing EXT4 during installation also.

Wouldn't it be good for the development and future of FreeBSD?

Have nice easter holidays!
Click to expand...
The first thing I thought when I saw your post was: "Oh finally somebody (different than me) is trying to pull April fools' day prank". It just sounded a bit lame comparing to my April first bombastic announcement of port of HAMMER to FreeBSD or my fake news of the Oracle law suite against FreeBSD foundation for the patent infringement. Than I checked the date and sure enough you were serious but at the same time that is what makes your post really funny. Linux is not using EXT4 file system. Last time I checked here in old good U.S. of A. Linux (which means Red Hat and Ubuntu) were using XFS (Red Hat) which is old Silicon Graphic file system from Irix UNIX days ported to Linux by the SGI out of desperation for working file system and God knows what on Ubuntu as the typical Ubuntu installation looks as follows

Code: 
root@observium:~# uname -a

Linux observium 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

root@observium:~# df -h

Filesystem                      Size  Used Avail Use% Mounted on

udev                             16G     0   16G   0% /dev

tmpfs                           3.2G   41M  3.2G   2% /run

/dev/mapper/observium--vg-root  203G  7.0G  185G   4% /

tmpfs                            16G     0   16G   0% /dev/shm

tmpfs                           5.0M     0  5.0M   0% /run/lock

tmpfs                            16G     0   16G   0% /sys/fs/cgroup

/dev/xvda1                      472M  155M  294M  35% /boot

tmpfs                           3.2G     0  3.2G   0% /run/user/1000

EXT4 is a joke and until recently was not supporting file system larger than 16TB. That alone makes it non production file system. Ubuntu is pushing all their resources to ZFS.

FreeBSD doesn't have much to show for in the past 10 years but one thing that came out of the ZFS port to FreeBSD and untimely death of Solaris is that FreeBSD is more or less the only choice for storage servers.
 
SirDice said:
It's not going to be added, ever, due to the GPL. Nothing more, nothing less. Any other motive you think they have is pure fantasy.
Click to expand...
If somewhere, someone creates a implementation of a linux filesystem licensed under BSD or any license that FreeBSD accept, would that technically acceptable to be added?
 
Gray Jack said:
If somewhere, someone creates a implementation of a linux filesystem licensed under BSD or any license that FreeBSD accept, would that technically acceptable to be added?
Click to expand...
How are you going to do that? Your goal is to implement a Linux file system (such as ext4), for which the "master" source code (namely the one in Linux which natively reads and writes ext4) is under GPL. There are three ways of doing this (with gray zones in between them). All require an expert file system implementor (and to get to production-worthy quality in a reasonable amount of time, probably a team of several experts):
  1. Read the GPL'ed source code carefully, understand it completely, take notes (for example with pencil and paper) that describe the on-disk data structures. Then start creating a new implementation from scratch, writing all new source files, for example designed for inclusion in the FreeBSD kernel. The problem is: Independent of what you (as the creator of the new source) say the copyright and license of the new code should be, it will be also subject to the GPL. Why? Because by reading the GPL Linux source code you have polluted your brain with Linux knowledge. In a lawsuit, people can claim that you are copying knowledge from the GPL implementation to your implementation, therefore your new implementation is a "derived work", and under the GPL's infectious clause will be subject to the GPL. Therefore your new implementation is GPL, and not acceptable to FreeBSD. You failed.
  2. Never ever look at the Linux source code. Make sure you don't get infected or contaminated in any way with GPL'ed source code. Instead, create an ext4 file system on a disk using Linux, and create/destroy/modify files using a variety of test cases. Start with an empty file system, then use "hexdump" to example the on-disk data structures. Create one directory and one file, and look at them again. Delete things, move things around, keep checking on-disk data structures. Start testing crazy corner cases: what if you crash the system exactly in the middle of a (supposedly atomic) rename operation? What if you create one file from directory A while simultaneously having a million threads trying to create a billion files simultaneously in directory B? Study how the on-disk data structure react to all possible test scenarios, and use that knowledge to create a complete and correct "black-box" description of what ext4 does to the on-dis data structures. Then use that description of the on-disk format, and write a file system implementation that happens to work with with the data structures you found.

    Is your implementation contaminated by the GPL? I don't think so. You never looked at a single line of GPL'ed source code. If you ship your implementation and it competes with Linux, you are certain to be sued. You will have to spend a few million $ on a team of lawyers to defend yourself. But you might win in court: your implementation might be completely "clean-room", and not GPLed. You might also lose in court. I'm not a lawyer, but I have a few good friends who are IP lawyers, and the GPL scares them. However, there is one really big problem with this scheme: It is only as compatible with ext4 as your tests were. If your tests forgot a crazy corner case, there will be examples where your implementation breaks (crashes, loses data, ...). And there is simply no way to test all crazy corner cases, there are way too many of them. Also, whenever ext4 changes (which it does frequently, perhaps just to fix bugs), your implementation will become obsolete, meaning wrong. So most likely your implementation (which may or may not be infected by the GPL) is probably garbage and useless.
  3. Find a friend (or hire a person). Ask that person to read the GPL'ed source code of ext4 in Linux very carefully, and make a detailed description of the on-disk data structures, and their semantics. Make it very clear to the person that their job is solely to describe the on-disk data structures, not describe how the GPL'ed source code in Linux works. Have them give you that description, and then never to talk to them again. Now implement a new file system (in FreeBSD using a license of your choice) that happens to read and write exactly those data structures. This approach is a hybrid of approach 1 and 2: approach 1 was complete "white box", where you were deliberately reading GPL'ed source code, while approach 2 was complete "black box", where you were only looking at on-disk data structures; in this approach you have someone else derive the knowledge about the data structure from GPL'ed source code, but then throw just the data structure specification over the fence. Will your new file system be subject to the GPL? I have no idea. With a large enough team of lawyers, I might risk doing this, but it is risky. There have been examples of people trying to reverse-engineer data formats and protocols, and all sorts of crazy things happen in these cases. One example is Andy Tridgell being thrown out of the Linux world because he reverse-engineered the on-wire messages used by Bitkeeper (without ever looking at Bitkeeper source code), which caused Linus Torvalds to write "git" as a revenge.
As you see, this is a difficult task, and that's a massive understatement. In my opinion, creating a non-GPL'ed version of ext4 is unreasonably difficult, and should not be attempted.
 
ralphbsz said:
How are you going to do that? Your goal is to implement a Linux file system (such as ext4), for which the "master" source code (namely the one in Linux which natively reads and writes ext4) is under GPL. There are three ways of doing this (with gray zones in between them). All require an expert file system implementor (and to get to production-worthy quality in a reasonable amount of time, probably a team of several experts):
  1. Read the GPL'ed source code carefully, understand it completely, take notes (for example with pencil and paper) that describe the on-disk data structures. Then start creating a new implementation from scratch, writing all new source files, for example designed for inclusion in the FreeBSD kernel. The problem is: Independent of what you (as the creator of the new source) say the copyright and license of the new code should be, it will be also subject to the GPL. Why? Because by reading the GPL Linux source code you have polluted your brain with Linux knowledge. In a lawsuit, people can claim that you are copying knowledge from the GPL implementation to your implementation, therefore your new implementation is a "derived work", and under the GPL's infectious clause will be subject to the GPL. Therefore your new implementation is GPL, and not acceptable to FreeBSD. You failed.
  2. Never ever look at the Linux source code. Make sure you don't get infected or contaminated in any way with GPL'ed source code. Instead, create an ext4 file system on a disk using Linux, and create/destroy/modify files using a variety of test cases. Start with an empty file system, then use "hexdump" to example the on-disk data structures. Create one directory and one file, and look at them again. Delete things, move things around, keep checking on-disk data structures. Start testing crazy corner cases: what if you crash the system exactly in the middle of a (supposedly atomic) rename operation? What if you create one file from directory A while simultaneously having a million threads trying to create a billion files simultaneously in directory B? Study how the on-disk data structure react to all possible test scenarios, and use that knowledge to create a complete and correct "black-box" description of what ext4 does to the on-dis data structures. Then use that description of the on-disk format, and write a file system implementation that happens to work with with the data structures you found.

    Is your implementation contaminated by the GPL? I don't think so. You never looked at a single line of GPL'ed source code. If you ship your implementation and it competes with Linux, you are certain to be sued. You will have to spend a few million $ on a team of lawyers to defend yourself. But you might win in court: your implementation might be completely "clean-room", and not GPLed. You might also lose in court. I'm not a lawyer, but I have a few good friends who are IP lawyers, and the GPL scares them. However, there is one really big problem with this scheme: It is only as compatible with ext4 as your tests were. If your tests forgot a crazy corner case, there will be examples where your implementation breaks (crashes, loses data, ...). And there is simply no way to test all crazy corner cases, there are way too many of them. Also, whenever ext4 changes (which it does frequently, perhaps just to fix bugs), your implementation will become obsolete, meaning wrong. So most likely your implementation (which may or may not be infected by the GPL) is probably garbage and useless.
  3. Find a friend (or hire a person). Ask that person to read the GPL'ed source code of ext4 in Linux very carefully, and make a detailed description of the on-disk data structures, and their semantics. Make it very clear to the person that their job is solely to describe the on-disk data structures, not describe how the GPL'ed source code in Linux works. Have them give you that description, and then never to talk to them again. Now implement a new file system (in FreeBSD using a license of your choice) that happens to read and write exactly those data structures. This approach is a hybrid of approach 1 and 2: approach 1 was complete "white box", where you were deliberately reading GPL'ed source code, while approach 2 was complete "black box", where you were only looking at on-disk data structures; in this approach you have someone else derive the knowledge about the data structure from GPL'ed source code, but then throw just the data structure specification over the fence. Will your new file system be subject to the GPL? I have no idea. With a large enough team of lawyers, I might risk doing this, but it is risky. There have been examples of people trying to reverse-engineer data formats and protocols, and all sorts of crazy things happen in these cases. One example is Andy Tridgell being thrown out of the Linux world because he reverse-engineered the on-wire messages used by Bitkeeper (without ever looking at Bitkeeper source code), which caused Linus Torvalds to write "git" as a revenge.
As you see, this is a difficult task, and that's a massive understatement. In my opinion, creating a non-GPL'ed version of ext4 is unreasonably difficult, and should not be attempted.
Click to expand...
I don't plan to do any of these, mate, I'm just interested in the subject.
But now, since it's all gray area, I have more to ask.
What if, independent of the source license, since it's a potential hazard to FreeBSD legality, the code made would only be available in ports or package to be installed as external kernel module, like ZFS is available on Linux due to legal incompatibility, would that be legal?
I mean, we have many GPL'ed available on the repository and ports, so as long the driver stays away from the base system, we're legally safe, right?
 
ralphbsz said:
Ports/packages? Don't know. I was wondering about GPL'ed code in the FreeBSD kernel, which is clearly a no-go zone.
Click to expand...
Mate, I admire your knowledge. Please answer me if my solution works. I will create a virtual machine with a trimmed down version of Linux, the most lightweight configurations so it not consume very much the resources. I export the block device/raw partition of the real hard disk to it for read/write since that's it native fs. Then I use NFS/SMB/or sth else... to share with the host running FreeBSD, so I could read/write the to FS natively in FreeBSD. Will it work? No GPL'ed code in kernel. And I wonder too if bhyve could map raw partition to the VMs? VBox could do it :confused:
 
And I wonder too if bhyve could map raw partition to the VMs?
Click to expand...

You can provide a raw block device to a bhyve guest so I see no reason why that couldn't be a disk or partition containing a Linux file system. As you say, you'd then need to use NFS to get access to it from FreeBSD.
 
ralphbsz since that old zombie of oracle ./. Google still lurches around in some court room, could it be that using the on-disk format alone would be enough to sumon-monster(lawyer) on your doorstep? APIs seem to be cancerous after all.
 
ralphbsz said:
There have been examples of people trying to reverse-engineer data formats and protocols, and all sorts of crazy things happen in these cases.
Click to expand...

Big corporations sure can put the big chill effect on activities that would/should be perfectly lawful. It's a detriment to society that this chill slows creative thought and diversity. One of my peevs is camera raw footage formats. These are fairly easily reverse-engineered (since no real compression, etc) - but nobody will do it thanks to the "big chill." So, for some formats, you need to have a Windows or Apple machine stuffed in the corner someplace to convert your silly video.
 
giahung1997 said:
I will create a virtual machine with a trimmed down version of Linux, the most lightweight configurations so it not consume very much the resources. I export the block device/raw partition of the real hard disk to it for read/write since that's it native fs. Then I use NFS/SMB/or sth else... to share with the host running FreeBSD, so I could read/write the to FS natively in FreeBSD. Will it work?
Click to expand...
Technically will it work? Pretty well. It is a bit of a waste of resources: You'll use dozens of MB of RAM to run an operating system (Linux) in a virtual machine which you really didn't need or want. Your data will have to make the trip through memory and in/out of the kernel multiple times: real disk to virtual disk, through the VM interface, into Linux and through the file system and NFS... server in there, then through the virtual network interface, back out of the VM, into another virtual network interface on the FreeBSD side, and one more time through the FreeBSD kernel's NFS... client file system. That may seem wasteful, but with today's CPU and memory speed it will work fast enough. I bet you'll get the whole performance of a disk drive (100-200 MByte/s) through it with reasonable CPU and memory bandwidth consumption.

From an administrative standpoint, it will also work, but it is a pain. You now have one more operating system flavor to maintain, you need to install upgrades, worry about security vulnerabilities, learn how to administer Linux. All of this is extra work and extra pain. But for an experienced admin, who already has a whole slew of different OSes and machines to work on, one more won't make a huge difference.

Legally, this is completely clean and approved, with no risk.
 
ronaldlees said:
Big corporations sure can put the big chill effect on activities that would/should be perfectly lawful.
Click to expand...
It's not just big corporations. It's also small corporations, individuals (many patent trolls are a single person), and trade organizations and interest groups.

Whether reverse engineering should be lawful is a matter of much debate. You think it should. A lot of politicians and the lobbying groups that feed them think it should not be. Matter-of-fact, the DMCA can be used to punish reverse engineering. It's all very complicated.
 
You must log in or register to reply here.
Back
Top