Using OpenCL on FreeBSD

astyle

astyle

I'm trying to get OpenCL going on FreeBSD. The GPU that I have is an Asus Radeon RX 550 4 GB. Per these instructions, I installed lang/clover, benchmarks/clpeak, devel/ocl-icd, deve/clinfo, and graphics/drm-kmod. I successfully got a Plasma Wayland session going, so I know graphics/drm-kmod is working. And I have everything compiled from ports, with as many options enabled as possible. The ports snapshot is from early July.

My problem is really the fact that benchmarks/clpeak keeps dumping core and crashing. If I don't start Plasma Wayland, I can actually get my benchmarking numbers on the TTY before clpeak announces 'core dumped'. Trying to run clpeak inside a Plasma Wayland session (e.g., Konsole) crashes the GUI session so hard, I can't even SSH in and reboot.

Today, I tried to see if Blender can take advantage of OpenCL. There's a config option that I tried:
1627097133999.png

Trouble is, selecting OpenCL made Blender crash and dump core.

This is making me conclude that even though I do have properly compiled codecs on VLC, and it will play stuff reasonably smooth, even the HD stuff, I can't use GPU acceleration properly at this point. Not for playing videos, not for speeding up renders on Blender, not for transcoding/subtitling videos. My Ryzen 5 1400 (3.4 GHz) is still stuck doing the heavy lifting for all that.

I strongly suspect this has something to do with devel/ocl-icd not working properly... both clpeak and clinfo crash. I've seen suggestions to use valgrind, and I discovered oclgrind recently to troubleshoot this.
 
Did some research, only to discover that my hardware may be unsupported:
  1. According to ROCm's github page, my GPU is actually supported, but my CPU is not on the list. Closest thing to my CPU is a Ryzen 5 1600 (Mine is 1400), and Amazon shows it for ~$200 USD. The difference is something called PCI-E Atomics. According to ROCm docs, it is important to have that. Haven't explored the issue further, but I suspect that PCI-E Atomics could be the reason behind the core dumps I mentioned in the previous post.
    1. It may be a viable option to look at sysctl(8) or even the BIOS to see if PCI-E Atomics can be set using those methods - needs to be explored further.
  2. My mobo (Asus B350 Prime): Yeah, it will support the 1600 (and even one more GPU for CrossFire, per GPUmag.com and pcpartpicker.com). Doesn't look like a B-chipset vs X-chipset mobo is really the issue... At least that's the conclusion I drew from a quick glance at official documentation.
  3. It does sound like X570 mobos with Zen 3 CPU's will propertly support PCI-E Atomics. Looks like Zen 3 first, then an X570 mobo for best results.
  4. I did briefly consider trying a Windows install on my hardware just to see if that makes a difference for Blender being able to use OpenCL. After all, it's just a matter of swapping in a different SSD. But even then, that's time consuming, and I'm reluctant to blow $30 on an SSD just to find that out.
 
grahamperrin said:
Recall: <https://forums.freebsd.org/threads/segmentation-fault-gpu.78825/>
Click to expand...
Yeah, my research did confirm that Vull had compatible hardware, unlike me. That's most likely why he had no segfaults, and could go much further than I could.

Seems like that mesa-libs bug is still open... As I suspected, it does have something to do with graceful exits, as in, not properly handling lack of support for PCI-E Atomics.

240761 – graphics/mesa-dri: Segmentation fault occurs while executing atexit handlers (affects lang/clover: Segmentation fault in OpenCL programs)

bugs.freebsd.org bugs.freebsd.org

Looks like my options are: 1. Buy a compatible CPU ($200), or 2. Save up and build another rig with compatible parts (Cezanne Zen 3 CPU on an X570 mobo with a Big Navi GPU). I'm frankly leaning towards the latter, even though it will come much later.

Marking this SOLVED, because I did enough research to track the problem down to incompatible hardware...
 
astyle said:
Yeah, my research did confirm that Vull had compatible hardware, unlike me. That's most likely why he had no segfaults, and could go much further than I could.

Seems like that mesa-libs bug is still open... As I suspected, it does have something to do with graceful exits, as in, not properly handling lack of support for PCI-E Atomics.

240761 – graphics/mesa-dri: Segmentation fault occurs while executing atexit handlers (affects lang/clover: Segmentation fault in OpenCL programs)

bugs.freebsd.org bugs.freebsd.org

Looks like my options are: 1. Buy a compatible CPU ($200), or 2. Save up and build another rig with compatible parts (Cezanne Zen 3 CPU on an X570 mobo with a Big Navi GPU). I'm frankly leaning towards the latter, even though it will come much later.

Marking this SOLVED, because I did enough research to track the problem down to incompatible hardware...
Click to expand...
Might check pawn shops; I just got lucky finding this Lenovo G50-45 laptop w/1TB hdd for $200 at a nearby pawnshop and it's proven to be very compatible with FreeBSD-13.0-Release and the newer drm-kmod stuff.
 
astyle said:
According to ROCm's github page, my GPU is actually supported, but my CPU is not on the list. Closest thing to my CPU is a Ryzen 5 1600 (Mine is 1400), and Amazon shows it for ~$200 USD. The difference is something called PCI-E Atomics. According to ROCm docs, it is important to have that. Haven't explored the issue further, but I suspect that PCI-E Atomics could be the reason behind the core dumps I mentioned in the previous post.
Click to expand...
What's driving me nuts is that ROCm (Debuted in 2019) only supports a limited subset of AMD's own processors - but Intel - no problem since Haswell (Released in 2013)! From get-go Intel's CPU had no problems, but AMD's own got no love. To top it off, NVidia has no problems running OpenCL kernels - clpeak's project page features clpeak being run on an NVidia card!
 
shkhln said:
1. Blender always crashes with Clover (due to bugs in the latter).
Click to expand...
Not Clover, but mesa-dri. The bug report suspects problematic exit handling in that Mesa lib, and I did see which file. I also hypothesized the exit crash might be be due to mesa-dri not handling lack of CPU support for PCI-E Atomics.
shkhln said:
2. Clover doesn't have anything to do with ROCm's hardware requirements.
Click to expand...
Yes and no. Clover kernels can run on NVidia no problem. lang/intel-compute-runtime is 'Clover for Intel CPUs with iGPU' and lang/pocl is 'Clover compiler for ignoring a GPU, and running on a CPU'. All that doesn't change the fact that if you want to run a Clover kernel on an AMD GPU, you have to have ROCm-compatible hardware.
shkhln said:
3. Future Blender versions will not support OpenCL: https://code.blender.org/2021/04/cycles-x/.
Click to expand...
Yeah, I saw this, too. Until Blender releases 3.0, it is actually able to use OpenCL. For time being, I'm treating that as a quick-and-dirty way to check that OpenCL actually works on AMD GPU.
 
astyle said:
Not Clover, but mesa-dri. The bug report suspects problematic exit handling in that Mesa lib, and I did see which file.
Click to expand...
Clover is the name of the OpenCL driver bundled with Mesa: https://www.phoronix.com/scan.php?page=news_item&px=Mesa-20.3-OpenCL-1.2-Clover. (Mesa's components have their own names because there are often multiple alternative ones.)

astyle said:
I also hypothesized the exit crash might be be due to mesa-dri not handling lack of CPU support for PCI-E Atomics.
Click to expand...
That's a completely baseless assumption.

astyle said:
Yes and no. Clover kernels can run on NVidia no problem.
Click to expand...
"Kernel" is not a part of the driver, that's a fancy term for the program executed on GPU ("shader" is another one). So, they are just OpenCL kernels. And, of course, Clover doesn't work with Nvidia [proprietary driver], Nvidia has their own OpenCL implementation.

To be clear, a GPU driver is the code that runs on CPU that is tasked with compiling and submitting shaders for GPU execution as well as submitting/receiving data for/from those shaders. GPU drivers don't run on GPUs!

astyle said:
lang/intel-compute-runtime is 'Clover for Intel CPUs with iGPU' and lang/pocl is 'Clover compiler for ignoring a GPU, and running on a CPU'.
Click to expand...
No, just no.

astyle said:
All that doesn't change the fact that if you want to run a Clover kernel on an AMD GPU, you have to have ROCm-compatible hardware.
Click to expand...
Incorrect.
 
Last edited:
shkhln said:
To be clear, a GPU driver is the code that runs on CPU that is tasked with compiling and submitting shaders for GPU execution as well as submitting/receiving data for/from those shaders. GPU drivers don't run on GPUs!
Click to expand...

Interesting. You know any books that discusses about making GPU drivers from scratch, for instance like how to make my own OpenCL language? Particularly for AMD GPUs.
 
Ahhh... the FreeBSD project does have a lot of documentation at https://docs.freebsd.org/en/ ... ? From there, type driver into the search box. The first hit is most likely what you're looking for - with some caveats.

I don't think you're gonna find anything GPU-specific. There's nothing stopping you from investing time and effort into looking at pieces of code and making them work together - that's what Open Source is about. But, writing your own OpenCL language??? now that is, I'm sorry, just un-informed. For more context, read the rest of this thread...

I'd suggest that you start small, and figure out how device drivers are even implemented in FreeBSD.
 
The FreeBSD documentations are rudimentary and is just a guide and need more information for my taste (I rather read a full blown book).

So I have found a book which does delve into FreeBSD device driver writing:

Amazon.com: FreeBSD Device Drivers: A Guide for the Intrepid eBook : Kong, Joseph: Kindle Store

Amazon.com: FreeBSD Device Drivers: A Guide for the Intrepid eBook : Kong, Joseph: Kindle Store
www.amazon.com

And seems there are tons for Linux:

I wonder writing drivers for Linux is almost identical for FreeBSD....

OpenCL have overhead to provide standard portability, it's the overhead which bugs me. Hence would rather program for GPU using it's ISA directly than using OpenCL. I think AMD GPUs rely heavily on OpenCL (its their way of providing CUDA programming, but this has changed since AMD implementing ROCm standards which is their official way for programming their GPUs) and might not have great latency as compared to NVIDIA GPUs (CUDA).

I guess I start with FreeBSD driver writing and then read about OpenCL.
 
A book in print usually contains pretty outdated information. That book was published in 2012, and FreeBSD was at v. 9.0 in 2012 - which is no longer supported. The content of the book can't cover anything later than 8.1-RELEASE... ? This is why everybody on the forums point people to the latest online documentation.

For context, OpenCL is just about as old as the book - debuted in 2009. ROCm was open-sourced in 2019 (As I pointed out). HIP only started making waves in 2022... Read this for a good summary of what's happened in the last 15 years in the GPU programming arena: https://www.anandtech.com/show/15746/opencl-30-announced-hitting-reset-on-compute-frameworks.

Writing drivers - great idea overall, go for it. It simply helps to be informed about what you're getting yourself into.
 
astyle said:
A book in print usually contains pretty outdated information. That book was published in 2012, and FreeBSD was at v. 9.0 in 2012 - which is no longer supported. The content of the book can't cover anything later than 8.1-RELEASE... ? This is why everybody on the forums point people to the latest online documentation.
Click to expand...

Honestly, I use FreeBSD because it's old school and not the latest and greatest OS in the block as compared to Linux. Also I use FreeBSD 13 and not the latest v14.

If someone (happens to be a Senior Software Engineer) took the immense time to write 352 pages on writing drivers the proper way, specifically for FreeBSD 9 and it turns out it's outdated, best believe I'll download a copy of FreeBSD 9 and get things rolling. Its better that way in my opinion, I can also learn why things have changed internally down to FreeBSD 14. I would go nowhere and learn nothing reading rudimentary online documents for the latest FreeBSD OS.

This is the very reason why I had to read the super old book "The Design of the UNIX Operating System" from 1986 to actually learn what exactly is a Unix OS and how it works, it's easier to learn advanced modern FreeBSD OS when reading super outdated stuff: Amazon

By the time I finish reading 352 pages on writing drivers for FreeBSD, the versions for FreeBSD wouldn't matter anymore since I can just simply write drivers ;).
 
You must log in or register to reply here.
Back
Top