My Haiku RISC-V port progress

X512 · May 14, 2021, 7:40am

I implemented timer interrupt support. Now ProcessController live updating is working. Before that thread preemption was not working and system had cooperative multitasking. Also it required some hacks (disabling snooze() etc.) to avoid freeze than can be removed now.

Timer handling is tricky. Timer interrupt can be only handled in machine mode, but kernel run in supervisor mode. Machine mode don’t support MMU and page table so it is not suitable for running kernel. So machine mode timer interrupt handler that will forward interrupt to supervisor mode is needed. I already have tiny machine mode “kernel” inside Haiku kernel that have its own syscalls (test, switchToSmode, setTimer), I adapted it so it become usable in multithreaded environment and implemented status_t setTimer(bool enable, uint64 time) syscall that enable/disable timer, set when timer should trigger and deliver sTimerInt interrupt to supervisor mode. Similar thing is implemented in OpenSBI (lib/sbi/sbi_timer.c).

Timer interrupt delivery looks like this:

MTrap(exception sEcall)
  mstatus: (ie: {}, pie: {s}, spp: s, mpp: s)
  mepc: <kernel_riscv64> msyscall + 0
  mie: {sTimer}
  mip: {sTimer, mTimer}
  sip: {sTimer}
Stack:
FP: 0x80c5dcd0, PC: <kernel_riscv64> MTrap + 263
FP: 0x80c5ddd0, PC: <kernel_riscv64> MVec + 77
FP: 0x80c5dec0, PC: <kernel_riscv64> 0x8013365f // STrap, msyscall
FP: 0x80c5dfc0, PC: <kernel_riscv64> SVec + 77
FP: 0x80c5dfe0, PC: <kernel_riscv64> arch_int_init + 341
FP: 0x80c5e000, PC: <kernel_riscv64> _start + 369
FP: 0x80145f00, PC: 0x8000019f
FP: 0x80145f60, PC: 0x800266af
FP: 0x80146000, PC: 0x800002dd
FP: 0x0, PC: 0x8000000d
setTimerMmodeSyscall()

MTrap(interrupt mTimer)
  mstatus: (ie: {s}, pie: {s, m}, spp: u, mpp: s)
  mepc: <kernel_riscv64> arch_int_init + 342
  mie: {sTimer, mTimer}
  mip: {mTimer}
  sip: {}
Stack:
FP: 0x80c5dec0, PC: <kernel_riscv64> MTrap + 263
FP: 0x80c5dfc0, PC: <kernel_riscv64> MVec + 77
FP: 0x80c5dfe0, PC: <kernel_riscv64> arch_int_init + 341
FP: 0x80c5e000, PC: <kernel_riscv64> _start + 369
FP: 0x80145f00, PC: 0x8000019f
FP: 0x80145f60, PC: 0x800266af
FP: 0x80146000, PC: 0x800002dd
FP: 0x0, PC: 0x8000000d

STrap(interrupt sTimer)
  sstatus: (ie: {}, pie: {s}, spp: s, fs: 1, sum: 0)
  sepc: <kernel_riscv64> arch_int_init + 342
  sie: {sTimer}
  sip: {sTimer}
  sscratch: 0x0, 0x803eb470
  tp: 0x803eb0f0()
  trap level: 0x1

screenshot52

X512 · May 14, 2021, 9:23am

Finally I managed to run full desktop environment with Tracker and Deskbar. The problem was in plaintext locale add-on that call std::basic_ios<char, std::char_traits<char> >::init(std::basic_streambuf<char, std::char_traits<char> >*) that caused crash in __gnu_cxx::__exchange_and_add(int volatile*, int). I suspect missing ELF TLS relocation support.

Stack dump (<address> <raw value> <decoded value>):

vm_page_fault: vm_soft_fault returned error 'Bad address' on fault at 0x0, ip 0xffffffd8612f12d8, write 0, user 1, thread 0x15d
Stack:
FP: 0x807f0e20, PC: <kernel_riscv64> STrap + 1341
FP: 0x807f0f20, PC: <kernel_riscv64> SVecU + 109
FP: 0x0, PC: <libstdc++.so.6> _ZN9__gnu_cxx18__exchange_and_addEPVii + 69
Stack dump
  fffffff0281e7e60  fffffff0281e80f0  0xfffffff0281e80f0
  fffffff0281e7e68  0000000000000000  <commpage> 0x0
  fffffff0281e7e70  ffffffdeaf057358  <libroot.so> pthread_cancel + 0
  fffffff0281e7e78  fffffff0281e7ec8  0xfffffff0281e7ec8
  fffffff0281e7e80  fffffff0281e8fa8  0xfffffff0281e8fa8
  fffffff0281e7e88  ffffffd8612e9262  <libstdc++.so.6> _ZNSt6localeaSERKS_ + 52
  fffffff0281e7e90  fffffff0281e90b8  0xfffffff0281e90b8
  fffffff0281e7e98  fffffff0281e8db8  0xfffffff0281e8db8
  fffffff0281e7ea0  ffffffd8613a2ea8  </boot/system/lib/libstdc++.so.6.0.25> 0x166ea8
  fffffff0281e7ea8  fffffff0281e7ec8  0xfffffff0281e7ec8
  fffffff0281e7eb0  fffffff0281e8ed8  0xfffffff0281e8ed8
  fffffff0281e7eb8  ffffffd8612e8a18  <libstdc++.so.6> _ZNSt8ios_base7_M_initEv + 42
  fffffff0281e7ec0  ffffffdeaf057358  <libroot.so> pthread_cancel + 0
  fffffff0281e7ec8  ffffffd8613aecb0  </boot/system/lib/libstdc++.so.6.0.25> 0x172cb0
  fffffff0281e7ed0  ffffffd8613aecb0  </boot/system/lib/libstdc++.so.6.0.25> 0x172cb0
  fffffff0281e7ed8  0000000000000000  <commpage> 0x0
  fffffff0281e7ee0  fffffff0281e8ed8  0xfffffff0281e8ed8
  fffffff0281e7ee8  ffffffd86132aa2e  <libstdc++.so.6> _ZNSt9basic_iosIcSt11char_traitsIcEE4initEPSt15basic_streambufIcS1_E + 16
  fffffff0281e7ef0  fffffff0281e9048  0xfffffff0281e9048
  fffffff0281e7ef8  ffffffd8613a29c0  </boot/system/lib/libstdc++.so.6.0.25> 0x1669c0
  fffffff0281e7f00  fffffff0281e9040  0xfffffff0281e9040
  fffffff0281e7f08  ffffffd0e434367c  <plaintext> _ZN8BPrivate16PlainTextCatalog12ReadFromFileEPKc + 182

screenshot53

extrowerk · May 14, 2021, 10:23am

Thanks for the full-blown screenshot and for your great work!

extrowerk · May 14, 2021, 10:25am

The system doesnt yet uses hpkgs, right?

Begasus · May 14, 2021, 10:38am

Can only say “wow”! Great work there @X512 (and team)!

X512 · May 14, 2021, 10:52am

Yes. It can use HPKG, but it makes test cycle much longer because I need to fully rebuild haiku.hpkg on each change. Currently I compile only selected targets and upload it with a script that use bfs_shell.

X512 · May 14, 2021, 10:58am

Bash is currently not compiled and it is not present in Haiku source. HaikuPorts cross-compiling is needed.

It is funny that GUI is implemented before console. When making/porting OSes console is usually implemented first. But my first thing to do is setup and output something to framebuffer

Zenja · May 14, 2021, 11:25am

Thats so sweet

X512 · May 14, 2021, 5:07pm

Most things are working, system is quite stable. Now applications crash show error dialog instead of KDL. Also after removing no more needed workarounds (fully allocate stack memory instead of allocating on demand by page fault handler) memory usage was reduced.

Maybe I release revision 2 soon that will be much more useable than revision 1.

What is still not working:

~~Terminal. Crashes because of “Failed to init pseudo tty”.~~ Done.
~~Write to file system and shutdown because of unimplemented page table API (RISCV64VMTranslationMap::ClearFlags).~~ Done. Shutdown and writing to file system is working.
plaintext locale add-on. Probably needs libstdc++.so rebuild.
~~Userland signal frames.~~ Done.
Userland debugger.
~~Second launch_daemon is not starting. Probably problem with fork() call. It cause need of special hacks to start Tracker and Deskbar.~~ Done.

screenshot54

nephele · May 14, 2021, 5:50pm

I think you broke discuss :D

X512 · May 14, 2021, 5:51pm

Even VirtIO network adapter is recognized. But network is not working because of missing API in host Haiku system required by TinyEMU. Network may work if compile and run TinyEMU on Linux.

screenshot55

leavengood · May 15, 2021, 1:31am

I guess another question related to this port is whether this bare-metal approach would also be a faster way to get the ARM port working, but using QEMU rather than TinyEMU. I feel like a lot of the issues in this port might also apply to ARM.

I don’t know if all our apparent difficultly with the ARM port has been just taking the wrong approach and fighting with bootloaders and UEFI, or if X512 just happened to prepare himself better for this task (with the mini OS) and was more persistent in fixing issues, which obviously there were a lot.

I really like RISC-V, like many computer enthusiasts, but practically speaking right now Haiku would be more useful on ARM, especially on something like the Raspberry Pi 4.

But this work is still really exciting, and it shows that Haiku can indeed work on non-x86 platforms, and also sets the stage for Haiku to be a first class option on real RISC-V computers (with some more driver and bootloader work, of course.)

win8linux · May 15, 2021, 7:18am

Not sure about RISC-V boards, but ARM boards can get quite weird with the booting process. For example, Raspberry Pi hardware is initialized by the GPU. Knowing that, perhaps it may indeed be better to get ARM ports working in emulators before moving on to bare metal; that is where adaptations regarding all of the different booting quirks are needed and where the ARM ports are seemingly stuck in.

Something else to consider is that there are people in the community capable of doing porting and those who have the hardware targeted for ports. Unfortunately, these groups are not always one and the same. Perhaps something can be done to get these folks working together? It’s far from ideal compared to a porter having hardware themselves, but it’s at least something.

tqh · May 15, 2021, 8:47am

I don’t think ARM is that far off. The UEFI ARM code has the following problems:

UEFI for 32 bit ARM doesn’t allow FPU, but all boot packages are compiled with hard float.
32 bit UEFI needs the same memory handling for loading ELF binaries, as we can’t map virtual addresses.
This is what 64 bit does. There are some more similar changes needed I think.
Setup MMU
Drivers

For ARM 64, I’ve been working on building the boot packages, which I am still a bit stuck on. I don’t have any up to date info on what the failure was, so need to redo setup.

PulkoMandy · May 15, 2021, 8:47am

You have no choice if you want to run on real hardware. This approach of skipping the low level bootloader is reasonable on emulators, but not so much on real hardware because you end up having to do all the work of the bootloader yourself, super low level things such as configuring RAM timings, etc.

We are not fighting bootloaders, we are using them to get one part of the job done. In the past we have tried u-boot, we found out that it is not so great at doing this job, which is why, now we are going with EFI, which is the recommended way for 64bit ARM and getting quite popular even on 32bit ARM now.

The same work will have to be done for RISC-V as well eventually to run on any kind of real hardware.

X512 · May 15, 2021, 9:02am

I see no major problems to add more boot loader platforms when system is already working on virtual machine. “Bare metal” boot loader has a huge advantage that it is starting fast (less than second from starting emulator to boot splash), have no 3rd party dependencies (it may be hard to configure and build on Haiku) and provide big freedom to control hardware.

RISC-V seems to use OpenSBI, that is yet another platform. I don’t know how UEFI can manage RISC-V machine mode stuff (timers, SMP etc.).

apgreimann · May 15, 2021, 12:49pm

Guys I’d say stick with RISC-V! It’s totally open, SiFive and BeagleBoard have them, and unlike arm there aren’t a ton of boards to try to support — it’d be the best path for getting R1 to something outside x86/64! And what @X512 has done is super crazy amazing!

SamuraiCrow · May 16, 2021, 5:18am

I hate to rain on your parade but with GPU support being up soon on the agenda, do the RISC-V boards you want to target have open-source drivers without binary blobs? (I would like to hope so but without checking, you never know…)

win8linux · May 16, 2021, 6:44am

SiFive HiFive Unmatched can use PCI-E 3.0 devices, so that would mean prolly AMD discrete GPUs being the best possible options in the long term unless the upcoming Xe cards from Intel prove to be easier to support.

The other boards don’t have GPUs, can’t support GPUs, or have video encoders/decoders at most.

The main issues with RISC-V SBCs will prolly be supporting the various specialized chips in them, such as AI or deep learning chips. The only RISC-V board that does support proper GPUs (AFAICT) is the HiFive Unmatched via PCI-E 3.0, so it shouldn’t hopefully be too difficult to bring over support from x86/x86_64 which also uses that standard.

X512 · May 16, 2021, 6:58am

It have, but DRM port is still needed as in x86. Personally I don’t play games with heavy 3D graphics so I don’t get trouble a lot without hardware 3D acceleration. I hope that setting framebuffer on real hardware will be not so difficult.