My Haiku RISC-V port progress

X512 · June 11, 2021, 12:54pm

I will need real board or accurate emulator to run Haiku on real hardware. I am not sure if @kallisti5 can do real hardware port work himself. Feel free to ask me if needed.

kallisti5 · June 11, 2021, 12:56pm

Technically, as long as we keep the EFI loader in a functional state that still works in qemu, running on the HiFive is just a small step more.

X512 · June 11, 2021, 1:04pm

It would be nice to have fully working Qemu 6.0 on Haiku. Current 6.0 patch version often crash on 64 bits and always crash and require to disable ASLR on 32 bits.

kallisti5 · June 12, 2021, 4:31pm

I’ve added the u-boot binaries for the SiFive Unmatched, along with the needed build infrastructure, and some documentation…

Our current MMC images are MBR based (like arm)… I’ll have to adjust that on riscv64

X512 · June 12, 2021, 4:43pm

I prefer to separate low-level board boot loader and OS image. On real hardware OS can be installed to USB or NVMe disk. USB and NVMe seems to be enabled in u-boot config: configs/sifive_unmatched_defconfig · master · U-Boot / U-Boot · GitLab.

u-boot stuff is not Haiku business.

kallisti5 · June 12, 2021, 10:19pm

I strongly disagree here. We used to build “per-board” images for ARM. It was completely and utterly unmaintainable from an infrastructure and development viewpoint. (instead of 1 arm build, you get 45 arm builds in parallel)

Everyone (Debian, Fedora, CentOS, Ubuntu, etc) is going EFI + u-boot or just u-boot on these platforms since it allows you to:

Focus on a single image for “xxx” architecture
Focus on shipping a single bootloader (efi) for said architecture
Use bios services, so one platform to support getting serial, framebuffer, etc for in the bootloader
Allow end users to “customize” the images for “whatever device” they want. (aka rune) without breaking our MIT licensing by shipping blobs with questionable licenses.
u-boot / tianocore provides the device trees (or acpi) through EFI.

everything gets simpler on everyone. Per-board builds gets you 1000’s of lines of #ifdef RASPBERRY_PI_2 (I should know, I erased most of it)

Supporting a new board is as easy as:

Adding a u-boot build for that board (which includes a built-in FDT)
Adding details on what that board expects in a json document.

As long as upstream u-boot supports the board, the board offers spec enough to run Haiku, and our kernel has drivers for it, we can run on it with minimal work.

We don’t have the man-power to write (and maintain) a large number of platform bootloader implementations.

X512 · June 13, 2021, 3:49am

You seems to misunderstood me. I agree that per-board images is bad approach, UEFI images without built-in u-boot/FDT should be used instead. Thats why I say “u-boot stuff is not Haiku business”, it is domain of responsibility of board manufacturer and maintainer. Haiku will use UEFI interface provided by board manufacturer instead of custom u-boot builds and images.

This is the same as PC BIOS firmware, Haiku distributions don’t come with BIOS firmware images, existing ROM image preinstalled at factory is used. Haiku only use BIOS/UEFI interfaces.

Why not use 2 disks instead? One for u-boot/FDT and another for Haiku image with EFI boot loader. I think that u-boot stuff and OS should be “galvanically isolated”. Also it should be possible to boot OS from USB and install it to internal disk keeping u-boot/FDT stuff untouched.

X512 · June 13, 2021, 1:28pm

I managed to run Mini OS inside Qemu. That required fixing FDT handling, adding PMP (physical memory protection) support (see Fix boot on newer qemu by lf- · Pull Request #62 · mit-pdos/xv6-riscv · GitHub) and implementing ramfb video driver over Qemu “fwcfg” bus that it more complex than TinyEMU “simple-framebuffer” driver. Virtio keyboard and tablet devices are recognized, but currently not working (can’t move mouse pointer etc.). On Qemu side I added HTIF support with the same address as in TinyEMU (0x40008000) (on Qemu by default HTIF is only avalible on Spike machine and ELF kernel with exported symbols tohost, fromhost).

> Run.sh
virt_machine_init
AddPath: /boot/home/config/non-packaged/add-ons/x86/opengl
Addon: /boot/home/config/non-packaged/add-ons/x86/opengl/libswpipe.so
Addon registered: /boot/home/config/non-packaged/add-ons/x86/opengl/libswpipe.so
AddPath: /boot/system/non-packaged/add-ons/x86/opengl
AddPath: /boot/system/add-ons/x86/opengl
+SoftwareRenderer
GalliumContext: CreateDisplay: Using llvmpipe (LLVM 9.0.1, 128 bits) driver.
double buffer enabled
Do(0, 0x87000000)
Memory size: 134217728
fdt tree:
0, 0: node('')
  prop('#address-cells'): 2 (len 4)
  prop('#size-cells'): 2 (len 4)
  prop('compatible'): 'riscv-virtio' (len 13)
  prop('model'): 'riscv-virtio,qemu' (len 18)
  100, 1: node('fw-cfg@10100000')
    prop('dma-coherent'): ? (len 0)
    prop('reg'): 0x10100000, 0x18 (len 16)
    prop('compatible'): 'qemu,fw-cfg-mmio' (len 17)
  196, 1: node('chosen')
    prop('bootargs'): '' (len 1)
    prop('stdout-path'): '/soc/uart@10000000' (len 19)
  260, 1: node('memory@80000000')
    prop('device_type'): 'memory' (len 7)
    prop('reg'): 0x80000000, 0x8000000 (len 16)
  332, 1: node('cpus')
    prop('#address-cells'): 1 (len 4)
    prop('#size-cells'): 0 (len 4)
    prop('timebase-frequency'): 10000000 (len 4)
    392, 2: node('cpu@0')
      prop('phandle'): 1 (len 4)
      prop('device_type'): 'cpu' (len 4)
      prop('reg'): 0x0 (len 4)
      prop('status'): 'okay' (len 5)
      prop('compatible'): 'riscv' (len 6)
      prop('riscv,isa'): 'rv64imafdcsu' (len 13)
      prop('mmu-type'): 'riscv,sv48' (len 11)
      544, 3: node('interrupt-controller')
        prop('#interrupt-cells'): 1 (len 4)
        prop('interrupt-controller'): ? (len 0)
        prop('compatible'): 'riscv,cpu-intc' (len 15)
        prop('phandle'): 2 (len 4)
    652, 2: node('cpu-map')
      664, 3: node('cluster0')
        680, 4: node('core0')
          prop('cpu'): ? (len 4)
  724, 1: node('soc')
    prop('#address-cells'): 2 (len 4)
    prop('#size-cells'): 2 (len 4)
    prop('compatible'): 'simple-bus' (len 11)
    prop('ranges'): ? (len 0)
    800, 2: node('flash@20000000')
      prop('bank-width'): ? (len 4)
      prop('reg'):  (len 32)
      prop('compatible'): 'cfi-flash' (len 10)
    908, 2: node('rtc@101000')
      prop('interrupts'): 11 (len 4)
      prop('interrupt-parent'): 3 (len 4)
      prop('reg'): 0x101000, 0x1000 (len 16)
      prop('compatible'): 'google,goldfish-rtc' (len 20)
    1020, 2: node('uart@10000000')
      prop('interrupts'): 10 (len 4)
      prop('interrupt-parent'): 3 (len 4)
      prop('clock-frequency'): 3686400 (len 4)
      prop('reg'): 0x10000000, 0x100 (len 16)
      prop('compatible'): 'ns16550a' (len 9)
    1144, 2: node('poweroff')
      prop('value'): ? (len 4)
      prop('offset'): ? (len 4)
      prop('regmap'): ? (len 4)
      prop('compatible'): 'syscon-poweroff' (len 16)
    1240, 2: node('reboot')
      prop('value'): ? (len 4)
      prop('offset'): ? (len 4)
      prop('regmap'): ? (len 4)
      prop('compatible'): 'syscon-reboot' (len 14)
    1332, 2: node('test@100000')
      prop('phandle'): 4 (len 4)
      prop('reg'): 0x100000, 0x1000 (len 16)
      prop('compatible'): 'sifive,test1' (len 33)
    1444, 2: node('pci@30000000')
      prop('interrupt-map-mask'): ? (len 16)
      prop('interrupt-map'): ? (len 384)
      prop('ranges'): ? (len 84)
      prop('reg'): 0x30000000, 0x10000000 (len 16)
      prop('dma-coherent'): ? (len 0)
      prop('bus-range'): ? (len 8)
      prop('linux,pci-domain'): ? (len 4)
      prop('device_type'): 'pci' (len 4)
      prop('compatible'): 'pci-host-ecam-generic' (len 22)
      prop('#size-cells'): 2 (len 4)
      prop('#interrupt-cells'): 1 (len 4)
      prop('#address-cells'): 3 (len 4)
    2164, 2: node('virtio_mmio@10008000')
      prop('interrupts'): 8 (len 4)
      prop('interrupt-parent'): 3 (len 4)
      prop('reg'): 0x10008000, 0x1000 (len 16)
      prop('compatible'): 'virtio,mmio' (len 12)
    2280, 2: node('virtio_mmio@10007000')
      prop('interrupts'): 7 (len 4)
      prop('interrupt-parent'): 3 (len 4)
      prop('reg'): 0x10007000, 0x1000 (len 16)
      prop('compatible'): 'virtio,mmio' (len 12)
    2396, 2: node('virtio_mmio@10006000')
      prop('interrupts'): 6 (len 4)
      prop('interrupt-parent'): 3 (len 4)
      prop('reg'): 0x10006000, 0x1000 (len 16)
      prop('compatible'): 'virtio,mmio' (len 12)
    2512, 2: node('virtio_mmio@10005000')
      prop('interrupts'): 5 (len 4)
      prop('interrupt-parent'): 3 (len 4)
      prop('reg'): 0x10005000, 0x1000 (len 16)
      prop('compatible'): 'virtio,mmio' (len 12)
    2628, 2: node('virtio_mmio@10004000')
      prop('interrupts'): 4 (len 4)
      prop('interrupt-parent'): 3 (len 4)
      prop('reg'): 0x10004000, 0x1000 (len 16)
      prop('compatible'): 'virtio,mmio' (len 12)
    2744, 2: node('virtio_mmio@10003000')
      prop('interrupts'): 3 (len 4)
      prop('interrupt-parent'): 3 (len 4)
      prop('reg'): 0x10003000, 0x1000 (len 16)
      prop('compatible'): 'virtio,mmio' (len 12)
    2860, 2: node('virtio_mmio@10002000')
      prop('interrupts'): 2 (len 4)
      prop('interrupt-parent'): 3 (len 4)
      prop('reg'): 0x10002000, 0x1000 (len 16)
      prop('compatible'): 'virtio,mmio' (len 12)
    2976, 2: node('virtio_mmio@10001000')
      prop('interrupts'): 1 (len 4)
      prop('interrupt-parent'): 3 (len 4)
      prop('reg'): 0x10001000, 0x1000 (len 16)
      prop('compatible'): 'virtio,mmio' (len 12)
    3092, 2: node('plic@c000000')
      prop('phandle'): 3 (len 4)
      prop('riscv,ndev'): 53 (len 4)
      prop('reg'): 0xC000000, 0x210000 (len 16)
      prop('interrupts-extended'): (2, 11), (2, 9) (len 16)
      prop('interrupt-controller'): ? (len 0)
      prop('compatible'): 'riscv,plic0' (len 12)
      prop('#interrupt-cells'): 1 (len 4)
      prop('#address-cells'): 0 (len 4)
    3272, 2: node('clint@2000000')
      prop('interrupts-extended'): (2, 3), (2, 7) (len 16)
      prop('reg'): 0x2000000, 0x10000 (len 16)
      prop('compatible'): 'riscv,clint0' (len 13)
  3384, 1: node('htif')
    prop('compatible'): 'ucb,htif0' (len 10)
gFwCfgRegs: 0x10100000
fwCfgSelectSignature: 0x554D4551
fwCfgSelectFileDir: 0x00000003
count: 5

size: 0
select: 32
reserved: 0
name: bios-geometry

size: 0
select: 33
reserved: 0
name: bootorder

size: 4
select: 34
reserved: 0
name: etc/boot-fail-wait

size: 28
select: 35
reserved: 0
name: etc/ramfb

size: 28672
select: 36
reserved: 0
name: vgaroms/vgabios-ramfb.bin
file.select: 35
mtvec: MVec + 0
stvec: SVec + 0
        test1Syscall
        test2Syscall(10, -15, 12345)
        switchToSModeSyscall
        mstatus (2): (ie: {s}, pie: {m}, spp: u, mpp: s)
gPreemptTimer: 80371318
VirtIO devices:
        0: (0x10008000, 0x1000, 8): (0)
        1: (0x10007000, 0x1000, 7): (0)
        2: (0x10006000, 0x1000, 6): (0)
        3: (0x10005000, 0x1000, 5): (0)
        4: (0x10004000, 0x1000, 4): (0)
        5: (0x10003000, 0x1000, 3): (0)
        6: (0x10002000, 0x1000, 2): input(name: "QEMU Virtio Tablet")
        7: (0x10001000, 0x1000, 1): input(name: "QEMU Virtio Keyboard")
0x10002000.features: 39000000
0x10001000.features: 39000000
virtio console not found

screenshot76

X512 · June 13, 2021, 2:33pm

Keyboard and mouse is working. Qemu have more accurate PLIC implementation and require enabling IRQs and setting priorities.

screenshot77

kallisti5 · June 13, 2021, 4:08pm

Technically it is as it stands today… u-boot isn’t in any images we ship. haiku/firmware is purely to provide a workflow for end users to provision the generic Haiku image for your device. Expecting users to cross-compile u-boot for their board is asking a little much.

What is Mini OS? It definitely doesn’t look like Haiku

X512 · June 13, 2021, 4:25pm

This is simple OS written by me from scratch for experimenting with runtime mechanisms, emulators and RISC-V architecture. It currently consists only from raw firmware image without file system and dynamic module loading. It have one common address space and no userland, most of code including graphics are running in supervisor mode. All memory is managed by one global malloc/free heap that is initialized on all available RAM. It has optional virtual memory support that currently identity maps RAM and MMIO devices.

It it much easier to test basic architecture and platform features (trap handing, interrupts, Virtio and framebuffer drivers etc.) on Mini OS than on Haiku kernel because it is much simpler, there are less places to fail and it loads much faster, system boots in less than second.

Some code of my Haiku RISC-V port is based on Mini OS code (arch_cpu_defs.h, Virtio declarations and drivers, trap handling code, virtual memory (LookupPte)).

kallisti5 · June 13, 2021, 4:33pm

ah. That makes sense. Thanks!

After https://review.haiku-os.org/c/haiku/+/4088 , the UEFI bootloader is back to where is stalls out on ARM on the SiFive Unmatched

So something in the kernel handoff / mmu / etc.

X512 · June 13, 2021, 4:45pm

I reproduced that too, this is expected behavior for now. There are many reasons why it currently fail:

HTIF is used for kernel serial output, it is virtual machine only feature and not present on real hardware. Current remote/master don’t have serial output code at all. ns16550a should be used instead on real hardware, it can be emulated by both TinyEMU and Qemu.
RISC-V port is currently expecting kernel to be started in machine mode, it will immediately crash if started in supervisor mode.
Machine mode code (MVec, MTrap, MSyscall) should be replaced with SBI.
Page table should be initialized by haiku_loader and arch/vm code should be adapted. Basically some code in vm_translition_map.cpp should be copied or shared with haiku_loader mmu.cpp.

kallisti5 · June 13, 2021, 4:48pm

Just a reminder I had this WIP here to form the sv39 kernel page tables:
https://review.haiku-os.org/c/haiku/+/3733

I slowed down on risc-v since you had such a big backlog of patches to merge. Feel free to make adjustments if needed

X512 · June 14, 2021, 7:07am

Haiku with “riscv” platform is running on Qemu. The same “haiku_loader.riscv” and Haiku disk image is working with both TinyEMU and Qemu. “efi” platform is not ready yet.

screenshot78

Diver · June 14, 2021, 8:07am

Is there any difference in boot time/overall speed between TinyEMU and Qemu?

X512 · June 14, 2021, 8:08am

TinyEMU seems faster in general. I don’t know why because Qemu is supposed to have more advanced CPU emulation with JIT compilation.

Null · June 14, 2021, 10:09am

Is there a binary difference between RISC-V 32/64 (and the future 128) that requires different builds? If it is, shouldn’t the naming reflect the binary format? Ie “riscv32/riscv64/riscv128” instead of the general “riscv”?

X512 · June 14, 2021, 10:11am

Yes. It is something similar to x86 and x86-64, separate builds will be required. But most modern CPUs are 64 bits so there are little benefits in support 32 bit RISC-V.

“riscv” in “haiku_loader.riscv” is a platform name of bare metal RISC-V machine without any 3rd-party boot loaders and firmwares. Other platform names are “efi” and “u-boot”. This platform can be used for both 32 and 64 bit RISC-V, but will need separate builds. The same is with “efi”/“u-boot” platforms.

Null · June 14, 2021, 10:17am

Then riscv is the wrong name for a 64 bit RISC-V. It should be riscv64. If not, this will cause problems if Haiku will support 32 or 128 bits in the future.