Well, it took a long time but here it is in all of its glory:
Unreal Engine 4.23.1. Like mentioned earlier,
4.23.0 won't be released for FreeBSD because Lightmass is broken in that version.
Chaos Destruction (the 4.23 showcase beta plugin) isn't supported since I couldn't get it to work at all, not even on a native Windows. I'm guessing it may have some issues still. On top of that I discovered that the Windows version warns that an Unreal Engine 4 Editor with Chaos built-in can't open any project that doesn't set the Unique build type.
That's why I haven't enabled it in the FreeBSD port, even though this message isn't displayed on FreeBSD. I'm pretty sure it should be.
On top of that there were quite a lot of issues linking to the system's OpenSSL 1.1 shared object, while later the OpenSSL 1.0 static library was linked into the final binary/lib. What a blast to track those down ... I might've missed a few.
Another thing that was changed in 4.23 is the way platforms are defined. What used to be in code is now contained in configuration files, so I had to re-do some of the porting.
Note that I've only tested this on
FreeBSD 12.0 (amd64). I haven't updated to FreeBSD 12.1 yet, though I intend to reinstall my BSD on a new M.2 disk sometime this or next week. I'll test UE4 out when everything is up and running.
The highlights
Audio Spatialization
The feature that I really wanted to get running eventually made it, cfr. the demo below. It wasn't easy since I couldn't just rely on the built-in audio spatialization features. Libraries that aren't open-source and available for FreeBSD like Steam Audio weren't an option.
The only possibility was Resonance Audio (see below), but in the end I'm really happy about this one. It does its job perfectly and it even sounds better than Steam Audio under Windows.
Resonance Audio
Resonance Audio is an open-source Google library for audio spatialization. It can use Intel Embree, which is also open-source. Of course neither of them openly support FreeBSD, but the changes were minimal.
Intel Embree uses the SIMD instructions of modern processors to off-load raycasting from the CPU.
Note that this is the first time that I've included something in the FreeBSD port that isn't available under Linux. You read it: Resonance Audio with Intel Embree isn't configured for Linux. It could be, but it isn't.
Without Embree the audio spatialization just doesn't work, so go brag against your Linux-using friends about that
On top of that I've upgraded the version of Intel Embree from 2.14.0 (dated 2017/02/09) to 2.17.7, which is the latest 2.X version available right now. This includes quite a lot of bug fixes. That's something even Mac and Windows users don't have!
Alembic Importer
Another change I really wanted to make was to provide some way to import meshes, since there is no FBX import. The Alembic importer does just that.
Note that internally it's built pretty ugly, since it relies on the system's OpenEXR version. On other platforms (thus by Epic's or the contributor's design), it uses a hacked way of linking to OpenEXR 2 but at the same time the engine links to OpenEXR 1.
Another reason to keep third party libraries up to date I guess ...
Breakpad dump_syms segmentation faults
If you've been building before, you may have experienced segmentation faults on certain modules. That's the dump_syms process crashing, which runs after linking and extract debugging symbols into a separate file.
Since I was really fed up with this - it slows down porting considerably - I decided to track down the issue. Apparently the LibCxx that FreeBSD 12.0 has is either an older one or contains a bug (most likely both).
That's why I've included LLVM's LibCxx and LibCxxabi sources and it's also built along the third party libraries. Breakpad now links against this one.
On Linux this external LibCxx is always used, even for the engine itself but unless I find out that there's a good reason to do so I won't be changing this. The system's c++ lib is generally the way to go.
When I can find the time to look into this I'll create a PR for an update in the FreeBSD sources if it still hasn't changed in 13.0.
Vulkan
Vulkan is becoming a recurring theme in my updates. Epic is deprecating OpenGL in favour of Vulkan. I'm not sure when OpenGL will be removed from the codebase entirely, but it'll get less attention for sure now.
So we really need to start begging NVidia to update their drivers
I'll probably create another post for this on the forum.
Of course, I hear AMD is going to release new GPU's soon so I might just switch brands for the first time in decades.
Note that thanks to the solution
shkhln has built there's always another path. Native support would still be nice to have though.
Epic has also added a
pop-up to development builds stating that
OpenGL has been deprecated. It's an annoying thing, but I've left it in place since it's important to know.
GPU brand is now recognized
It might look like a small one - and it kind of is - but it's one more thing that Linux doesn't have. The Linux port will always show the GPU as being a GenericGPU. FreeBSD no longer does.
For NVidia I use the official driver's sysctls and for all others - or if the sysctl doesn't exist - I query the pci bus like
pciconf does.
LogInit: OS: FreeBSD 12.0-RELEASE-p8 GENERIC (), CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz, GPU: NVIDIA GeForce GTX 770 (yes, I
was behind on security updates)
LogInit: OS: Windows 7 (Service Pack 1), CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz, GPU: NVIDIA GeForce GTX 770
PhysX compilation issue fix and update to 3.4.2
Apparently the sources (version 3.4.0, dated early 2017) didn't compile on Clang 6+. I hadn't noticed that the PhysX compilation no longer worked because the cleaning method that I used for third party libraries wasn't adequate.
All
4.22 releases may have been affected by this, at least when building on FreeBSD 12.
From now on I'll be deleting my entire Engine/Source/Thirdparty before building third party libs to find out about issues before release.
Version
3.4.2 has the necessary fixes and I have upgraded the sources to that version. It means there's one more library that isn't exactly the same as it is on other platforms. Especially one hack of a "fix" has been removed since the original code now asserts that everything is working correctly. I'm pretty sure that at least the FreeBSD part is fine (after blowing up massively earlier), but it's always possible that I've missed an Epic modification to the sources.
Updated the internal version numbers to the released versions
This is the first time the engine pretends to be the exact same version as the Windows release. Even though it's not exactly true, it means that it's now possible to compile games and have them communicate with each other effortlessly.
It means that replays can be shared and probably some other goodies as well. It also means that it's easier to share the same build directories under different OS's.
I intend to keep doing this from now on for every release.
WebM support
I haven't tested this yet, but the necessary third party libraries have been compiled. I'll test it out somewhere with the next release.
It's meant to play WebM videos from within the engine, next to the other supported formats.
LTO support and dynamic linker optimization
LTO (Link-Time Optimization) is now supported for the FreeBSD version. To enable it, add this to the BuildConfiguration.xml:
XML:
<Configuration xmlns="https://www.unrealengine.com/BuildConfiguration">
<BuildConfiguration>
<!--possibly other options here -->
<bAllowLTCG>true</bAllowLTCG>
<!--possibly other options here -->
</BuildConfiguration>
</Configuration>
I've built my current editor build with LTO so I'm pretty confident it works well. Expect the gains to be minimal unless it's a monolithic build. Also expect higher memory usage from the linker. You need to have at least one of the llvm ports installed, the base system (at least on 12.0) doesn't have the required llvm-ar binary.
The dynamic linker will now also have to search considerably less libraries due to having less dependencies in the linked shared objects. This will probably speed up the editor's and other modular projects' startup.
One more demo to rule 'em all
SpatialAudioTemple (640M) SHA512: 02e16b37466e81c5339fa6dd18a51f55bbacd0a38dfa06d29b90968c4d8bdf430528a81ba06887969a7274206dc9188b3e79f1076841431e03ca1461b0771eca
An already existing demo that Epic modified to showcase the audio spatialization features of UE4. This build is configured to use Google's open-source Resonance Audio library (see above).
Note that this demo also marks the first time something works on FreeBSD which isn't supported on Linux!
Changes to the build process
Additional requirements
There are now additional requirements in order to use the editor and compile the third party libraries.
pkg install openexr tbb ispc yasm
New third party build system (PortNext)
The current way of building third party libraries is extremely cumbersome to maintain and over the years I've been wanting to improve on it with much simpler scripts for each library.
I've decided to build something better from scratch and the first fruits of that labour can be found in this version of the port. It's very premature since I've lost too much time tracking down issues with 4.23.0.
The scripts can be found in
<UE4ROOT>/Engine/Build/BatchFiles/Linux/PortNext.
As you might assume judging from the name, this system has been loosely inspired by the FreeBSD ports system. I could have called it PortLite or something, but as far as UE4 goes this really is an improvement thus the 'next'.
I might try to get this upstreamed somewhere in the future, although I think Epic is currently exploring the avenue of building third party libraries using the Unreal Build system directly. That's probably better for UE4 anyway.
Future route
* I intend to move most, if not all third party builds to PortNext. I'll start with new libraries first.
* I might also include new versions of more libraries, but I'm hesitant about that since many of these may have negative effects on actual builds.
* Master integration has been delayed quite a lot with all the time lost, so that's also going to happen.
Epilogue
This has definitely been the most involving release I've done. I've been able to set things right everywhere and several changes are now beyond the Linux version.
It's been a long time I've been able to work on something with at least a modicum of focus and I'm really happy to have done this in almost one straight go.
Special thanks to Frédéric Chopin, Pyotr Ilyich Tchaikovsky, Franz Liszt, Claude Debussy, Antonín Dvořák and all others who helped me through this.
Oh, one more thing. If your project hangs and prints messages like 'Waiting 10 seconds for IO' or something like it, disable "event driven loader" in the project. For some reason this isn't working well.
e.g. PlatformerGame has this issue.
