I made some progress with implementing amdgpudrmIoctl. I also improved drmIoctl decoder that allows to log interaction between Vulkan userland driver and amdgpu. Now it crash because shader buffer is NULL.
I also found a way to avoid mmap call. There is drm_mmap macro in libdrm, I made it redirect to real function drmMmap that internally calls create_area.
I made my own libdrm.so that have no major/minor fd tricks, KMS and most other parts. It is enough for Radeon Vulkan driver. libdrm_amdgpu.so is left unchanged, it provides quite nice abstractions.
For some unknown reason after recompiling Radeon Vulkan driver, shader creation started working and it now attempts to send commands to GPU ring buffer (that is currently not implemented). It currently fails at unimplemented waiting for finished GPU ring buffer commands.
I have pile of Radeon cards ranging from Fiji all the way up to Navi RDNA2 if you need someone to help test or debug. I also know some stuff about Vulkan.
You don’t need to repeatedly press it (at least, I don’t). I find it easier to turn on computer, wait for the hard drive to light up the first time, and then just hold the spacebar down until the menu appears.
That’s for real hardware. For a VM, it gets a bit trickier.