My progress on real RISC-V hardware

X512 · August 7, 2021, 3:18am

Then it will require ati-vga to get graphics on QEMU with EFI loader. And no boot splash.

kallisti5 · August 7, 2021, 3:19am

Pretty sure you’re right. Though, couldn’t we use the EFI frame buffer like on x86?

X512 · August 7, 2021, 3:22am

Only if EFI implementation provides GOP interface. u-boot supports only simple-framebuffer virtual machine graphics that is not implemented in QEMU, it is only implemented in TinyEMU. EDK2 supports more graphics drivers, but it is currently not fully ported to RISC-V yet.

kallisti5 · August 7, 2021, 3:29am

ah ok. That makes sense. The u-boot folks have virtio drivers… it might be a small matter of time until someone gets virtio + qemu working upstream at u-boot.

X512 · August 7, 2021, 3:32am

Do you mean VirtIO video drivers? There are no Haiku driver for it as I remember. Adding QEMU fwcfg ramfb driver to u-boot would be not so difficult, it is already present in Mini OS, haiku_loader.riscv and EDK2.

kallisti5 · August 7, 2021, 3:48am

u-boot has a full EFI bios implementation and offers a framebuffer on several platforms (x86, arm). Haiku on x86 can leverage the EFI framebuffer directly for drawing via the vesa driver.

If u-boot’s EFI bios implementation has a virtio-gpu driver, then u-boot’s EFI bios can provide framebuffer hooks to our vesa driver

virtio-gpu → u-boot (virtio driver) → u-boot EFI framebuffer → haiku_loader → vesa (efi framebuffer)

Anyway… i’m going to focus on getting the code + patches we have today merged first

X512 · August 7, 2021, 3:49am

OK, that will work.

X512 · August 7, 2021, 11:02am

Lets continue work on RISC-V port.

I implemented syscall restart and managed to successfully build Perl on real hardware.

screenshot3
screenshot4

X512 · August 7, 2021, 11:08am

Attempting to compile autoconf that use perl currently cause reproduceple KDL:

STrap(exception storePageFault)
  sstatus: (ie: {}, pie: {}, spp: s, sum: 0)
  sie: {sTimer, sExtern}
  sip: {sTimer}
  stval: 0xffffffc12b806000
  tp: 0xffffffc12c2f8a40(perl)
PANIC: page fault with interrupts disabled
Welcome to Kernel Debugging Land...
Thread 1768 "perl" running on CPU 0
Stack:
FP: 0xffffffc12b805bf0
FP: 0xffffffc12b805c10, PC: 0xffffffc00215ab5f <kernel_riscv64> arch_debug_call_with_fault_handler + 31
FP: 0xffffffc12b805c60, PC: 0xffffffc0020d25a3 <kernel_riscv64> debug_call_with_fault_handler.localalias.7 + 129
FP: 0xffffffc12b805cf0, PC: 0xffffffc0020d3bcd <kernel_riscv64> _ZL20kernel_debugger_loopPKcS0_Pvi + 299
FP: 0xffffffc12b805d60, PC: 0xffffffc0020d3eb9 <kernel_riscv64> _ZL24kernel_debugger_internalPKcS0_Pvi + 135
FP: 0xffffffc12b805da0, PC: 0xffffffc0020d4201 <kernel_riscv64> panic + 101
FP: 0xffffffc12b805ee0, PC: 0xffffffc00215c0fb <kernel_riscv64> STrap + 1345
FP: 0xffffffc12b805fe0, PC: 0xffffffc002159c7d <kernel_riscv64> SVec + 77
FP: 0xffffffc12b807070, PC: 0xffffffc002159c37 <kernel_riscv64> SVec + 7
FP: 0xffffffc12b807170, PC: 0xffffffc00215bec3 <kernel_riscv64> STrap + 777
FP: 0xffffffc12b807270, PC: 0xffffffc002159c7d <kernel_riscv64> SVec + 77
FP: 0xffffffc12b807280, PC: 0xffffffc0020f866b <kernel_riscv64> _ZN11IOOperation16SetOriginalRangeElm + 5
FP: 0xffffffc12b807350, PC: 0xffffffc12b80734f <make_1768_kstack> 0x134f
FP: 0xffffffc12b8074b0, PC: 0xffffffc002325085 <nvme_disk> _ZL12nvme_disk_ioPvP9IORequest + 273
FP: 0xffffffc12b807500, PC: 0xffffffc0020ef9e5 <kernel_riscv64> _ZL8devfs_ioP9fs_volumeP8fs_vnodePvP9IORequest + 129
FP: 0xffffffc12b807550, PC: 0xffffffc002124b43 <kernel_riscv64> vfs_vnode_io.localalias.0 + 37
FP: 0xffffffc12b807650, PC: 0xffffffc002124ca5 <kernel_riscv64> _ZL26do_iterative_fd_io_iteratePvP9IORequestPb + 271
FP: 0xffffffc12b8076a0, PC: 0xffffffc0021250e5 <kernel_riscv64> do_iterative_fd_io + 193
FP: 0xffffffc12b8076f0, PC: 0xffffffc002124b43 <kernel_riscv64> vfs_vnode_io.localalias.0 + 37
FP: 0xffffffc12b807870, PC: 0xffffffc002125209 <kernel_riscv64> vfs_read_pages + 133
FP: 0xffffffc12b8078b0, PC: 0xffffffc0020857ff <kernel_riscv64> file_cache_read + 115
FP: 0xffffffc12b807910, PC: 0xffffffc00210bc29 <kernel_riscv64> _kern_read + 131
FP: 0xffffffc12b807930, PC: 0xffffffc002167089 <kernel_riscv64> pread + 29
FP: 0xffffffc12b807940, PC: 0xffffffc00239e6df <packagefs> _ZN5BFdIO6ReadAtElPvm.localalias.6 + 25
FP: 0xffffffc12b807980, PC: 0xffffffc00216977b <kernel_riscv64> _ZN11BPositionIO13ReadAtExactlyElPvmPm + 61
FP: 0xffffffc12b8079c0, PC: 0xffffffc00239b131 <packagefs> _ZN11BPackageKit5BHPKG8BPrivate27PackageFileHeapAccessorBase12ReadFileDataEmPvm.localalias.0 + 41
FP: 0xffffffc12b807a00, PC: 0xffffffc00239b19b <packagefs> _ZN11BPackageKit5BHPKG8BPrivate27PackageFileHeapAccessorBase26ReadAndDecompressChunkDataEmmmPvS3_ + 43
FP: 0xffffffc12b807a60, PC: 0xffffffc00239ac97 <packagefs> _ZN11BPackageKit5BHPKG8BPrivate27PackageFileHeapAccessorBase16ReadDataToOutputElmP7BDataIO.localalias.4 + 143
FP: 0xffffffc12b807ad0, PC: 0xffffffc0023827d5 <packagefs> _ZN16CachedDataReader14_ReadIntoPagesEPP7vm_pagemm.localalias.2 + 107
FP: 0xffffffc12b807c90, PC: 0xffffffc002383073 <packagefs> _ZN16CachedDataReader14_ReadCacheLineElmlmP7BDataIO.localalias.6 + 677
FP: 0xffffffc12b807ce0, PC: 0xffffffc002383379 <packagefs> _ZN16CachedDataReader16ReadDataToOutputElmP7BDataIO.localalias.9 + 107
FP: 0xffffffc12b807d50, PC: 0xffffffc00238b13f <packagefs> _ZN11PackageFile4ReadEP9IORequest.localalias.7 + 177
FP: 0xffffffc12b807d70, PC: 0xffffffc002384b29 <packagefs> _ZL12packagefs_ioP9fs_volumeP8fs_vnodePvP9IORequest + 41
FP: 0xffffffc12b807dc0, PC: 0xffffffc002124b43 <kernel_riscv64> vfs_vnode_io.localalias.0 + 37
FP: 0xffffffc12b807f40, PC: 0xffffffc002125209 <kernel_riscv64> vfs_read_pages + 133
FP: 0xffffffc12b8082e0, PC: 0xffffffc002084a77 <kernel_riscv64> _ZL15read_into_cacheP14file_cache_refPvlimmbP19vm_page_reservationm + 317
FP: 0xffffffc12b8083d0, PC: 0xffffffc002084701 <kernel_riscv64> _ZL8cache_ioPvS_lmPmb + 1139
FP: 0xffffffc12b808410, PC: 0xffffffc0020857c7 <kernel_riscv64> file_cache_read + 59
FP: 0xffffffc12b808470, PC: 0xffffffc00210b305 <kernel_riscv64> _ZL14common_user_ioilPvmb + 357
FP: 0xffffffc12b8084b0, PC: 0xffffffc0020a6d31 <kernel_riscv64> syscall_dispatcher + 3191
FP: 0xffffffc12b8085b0, PC: 0xffffffc00215bd9d <kernel_riscv64> STrap + 483
FP: 0xffffffc12b8086b0, PC: 0xffffffc002159d3d <kernel_riscv64> SVecU + 109
FP: 0x3bfb6b3450, PC: 0x2ade71ee27 <libroot.so> _kern_read + 7
FP: 0x0, PC: 0x30248289eb <libperl.so> PerlIOUnix_read + 79
initial commands: dump_virt_page 0xffffffc12b806000
satp: 0x800000000008eca1
  0xffffffc12b806000 - 0xffffffc12b806fff: 0x00000000 - 0x00000fff, 0x1000, {}
kdebug>

It can be kernel stack overflow.

kdebug> area contains 0xffffffc12b806000
AREA: 0xffffffc00681ea80
name:           'make_1768_kstack'
owner:          0x1
id:             0x7aac
base:           0xffffffc12b806000
size:           0x5000
protection:     0xb0
page_protection:0x0000000000000000
wiring:         0x2
memory_type:    0x0
cache:          0xffffffc12c3817e8
cache_type:     RAM
cache_offset:   0x0
cache_next:     0x0000000000000000
cache_prev:     0x0000000000000000
page mappings:  0
kdebug> dump_virt_page 0xffffffc12b806000
satp: 0x800000000008eca1
  0xffffffc12b806000 - 0xffffffc12b806fff: 0x00000000 - 0x00000fff, 0x1000, {}
kdebug> dump_virt_page 0xffffffc12b807000
satp: 0x800000000008eca1
  0xffffffc12b807000 - 0xffffffc12b807fff: 0x8ed5e000 - 0x8ed5efff, 0x1000, {valid, read, write, accessed, dirty}
kdebug>

X512 · August 7, 2021, 11:24am

This actually looks like stack overflow with hitting guard page ( 0xffffffc12b806000) and jumping over it. Kernel stack overflow handling should be improved to not overwrite memory before stack.

SCollins · August 7, 2021, 1:17pm

i hope this is the kernel bug I think it is, and if it is, it’s on x86 too.

be awesome if this got fixed.

X512 · August 7, 2021, 11:24pm

It seems some kernel stack leak when exec syscall is used. I reproduced problem in emulator by Bash script recursively calling exec. Problem seems to be riscv64 specific.

memsom · August 8, 2021, 3:16pm

My point was purely about wasting time with changing working code because some external developer was unhappy and waddlesplash messing about with Gerber voting. And alienation of the developer working on the port. It looks like you are in the process of addressing that, so I don’t really see the point of your comment.

I’d love to do more, but I am already busy on other projects. Maybe if and when I can get something booting 64bit.

X512 · August 10, 2021, 7:33am

Video playback is working with ffmpeg.

I managed to build many packages including libpng, cmake, ffmpeg, openssl, wget, various archivers. gettext package is also compiled so Debugger now can be compiled. It starts, but doesn’t actually work currently. Required kernel and debugging library parts seems to be not implemented.

I fixed following things:

Implemented syscall restart.
Fixed kernel stack leak on exec by freeing all kernel stack when enter userspace. x86 does the same.
Fixed userland stack frame chain termination. Now userland stack terminates with zero FP register. This allows go get correct stack after previous change.
Limit kernel stack trace by 1000 frames. Some packages build use stack overflow test that cause kernel stack trace run for several minutes.
Implemented thread exit by commpage_thread_exit(). Patch #4064 is no longer needed.
Fixed GCC C++ threads. CMake now compiles fine. It was caused by wrong header being used. I copied bits/gthr-posix.h to gthr.h and repacked GCC package. Issue will be probably fixed by natively compiling GCC package, but this is not done yet.

screenshot2

Zenja · August 10, 2021, 8:39am

Fantastic progress Ilya.

X512 · August 10, 2021, 12:57pm

Wonderbrush is working.

screenshot5

CodeforEvolution · August 10, 2021, 7:13pm

Amazing work @X512! @stippi, your paint program from the BeOS days has come a long way!

SCollins · August 11, 2021, 2:49am

gonna have to build me a riscv system soon

X512 · August 11, 2021, 9:19am

I managed to start other CPU cores by SBI. Now it is possible to start implementing SMP for real hardware.

The problem was uninitalized stack and infinite crashing when accessing stack. I found problem by using QEMU monitor register dump command (info registers) and OpenSBI image with symbol information to lookup where other CPU cores stucked.

arch_smp_register_cpu()
cpu
  id: 0
arch_smp_register_cpu()
cpu
  id: 1
arch_smp_register_cpu()
cpu
  id: 2
arch_smp_register_cpu()
cpu
  id: 3
arch_smp_boot_other_cpus()
  status:
    hart 0: started
    hart 1: stopped
    hart 2: stopped
    hart 3: stopped
starting CPU 1
[PRE] sbi_hart_get_status() -> (0, stopped)
sbi_hart_start() -> (0, 0)
CpuEntry(1)
[POST] sbi_hart_get_status() -> (0, started)
starting CPU 2
[PRE] sbi_hart_get_status() -> (0, stopped)
sbi_hart_start() -> (0, 0)
CpuEntry(2)
[POST] sbi_hart_get_status() -> (0, started)
starting CPU 3
[PRE] sbi_hart_get_status() -> (0, stopped)
sbi_hart_start() -> (0, 0)
CpuEntry(3)
[POST] sbi_hart_get_status() -> (0, started)
  status:
    hart 0: started
    hart 1: started
    hart 2: started
    hart 3: started

X512 · August 11, 2021, 10:07am

Calling kernel entry point from other CPU cores is working:

arch_smp_boot_other_cpus(0x800000000008c0c6, 0xffffffc00209353c)
  status:
    hart 0: started
    hart 1: stopped
    hart 2: stopped
    hart 3: stopped
  starting CPU 1
  stack: 0xffffffc0025df000 - 0xffffffc0025e3fff
Kernel entry point
  starting CPU 2
  stack: 0xffffffc0025e4000 - 0xffffffc0025e8fff
Kernel entry point
  starting CPU 3
  stack: 0xffffffc0025e9000 - 0xffffffc0025edfff
Kernel entry point
  status:
    hart 0: started
    hart 1: started
    hart 2: started
    hart 3: started