EDK2 would be nice, but as I remember it supports only 64-bit.
So in that case we’d need an aarch64 EL2 to arm32 SVC mode transition in the bootloader.
It’s the latter: raspberry pi 3 refuses to boot from esp partition. The green activity led won’t even light up.
so far i built u-boot manually but i’ll take a look at the build script for sure.
1 Like
Oof. That’s a pain. I guess the SoC on the Raspberry Pi 3 has some built-in loader that is explicitly looking for 0x0c partitions. We moved from 0x0C to 0xef pretty recently when I introduced mbrtool:
https://cgit.haiku-os.org/haiku/tree/build/jam/images/MMCImage#n135
We need the ESP on pretty much every board to ensure EFI boots properly from u-boot (or edk2). I’ll do some digging. Another victim of the severe lack of arm common practice.
btw, i’m focused on u-boot atm because it compiles and runs on a huge range of boards. The EDK2 bios is only available for a few devices. tldr; if the EDK2 bios runs on a board, haiku EFI should boot just fine with it and rune can be adjusted for that individual board.
I opened Raspberry Pi 3 - 0x0b vs 0xef · Issue #1711 · raspberrypi/firmware · GitHub with this issue. It seems limited to just the Raspberry Pi 3 and doesn’t exist on any other tested boards.
I also opened this enhancement to rune to track it:
https://dev.haiku-os.org/ticket/17679
1 Like
some more experiments on the rpi3… i was able to get some debug output from the kernel with some really ugly hacks 
- we need to go back to an old u-boot version, someting like v2021.01. actually the one from haiku firmware repo seems to be OK. newer u-boot versions enable caches in hypervisor mode and I don’t have the required cache maintenance in the hyp>svc mode switching yet…
- atomics don’t work, this includes smp_cpu_rendezvous as well as spinlocks and mutexes. if I comment out the calls to smp_cpu_rendezvous in _start() and the mutexes for dprint, the
Welcome to kernel debugger output!message appears on the serial console.
maybe it’s only the S bit though… as currently the kernel pages are mapped with flags B=1, C=1, S=0 i.e. cacheable but non-shareable. i’ll try adding the S bit next. (see e.g. here - so probably we need C=1, S=1)
3 Likes
Regarding point 2, do’t work is a very vague description. Do they lock up or what happens? Perhaps it is related to the EFI call to stall() for spin(bigtime_t microseconds)
2 Likes
the kernel initialization locks up in atomic_add (called from smp_cpu_rendezvous)
atomic_add seems to be using the __atomic_fetch_add intrinsic which relies on ldrex/strex
there’s a similar lockup in mutex_acquire which uses atomic_get_and_set / __atomic_exchange_n
2 Likes
once I enable the caches, the atomics work much better.
the kernel goes further now, but then panics because the interrupt controller is not detected.
time for some code cleanup and submitting my patches.
18 Likes
the bootup sequence on rpi3 is slowly taking shape…
- the hyp>svc transition works fine on u-boot versions up to v2021.01. in newer u-boot we need one more d-cache cleaning before changing to svc mode as in v2021.04 onwards the MMU and caches are enabled in hyp mode (bits 0x1005 are asserted in HSCTLR). We might also need to invalidate i-cache, I’m not entirely sure.
- enabling cache in svc mode is enough to get the atomics working (set bits 0x1005 in SCTLR). once the caches are enabled, we need to adjust TTBR0 accordingly (set bits 0x59 to mark the page table walks as cacheable) and set B and C bits in the page mapper (it’s already there on the initial pages mapped by the bootloader, has to be added on pages mapped by the kernel)
- u-boot did not detect the amount of RAM, this was fixed by adding fixup.dat and device tree binaries to the SD card image
- a few more bits are still needed for fdt parsing e.g. handle register ranges not starting on page boundary, address remapping on /soc simple-bus etc.
with these, now at least the bootloader is getting in a good shape for rpi3/rpi4
edit:
i can’t post more than 3 consecutive updates to i’ll add it here at the end of my last post
rpi4 bootup reaches 4th icon. then we get “no boot partitions”
if there’s anyone interested in joining in with the arm port, the SD card driver could be a good target.
31 Likes
Nice work! Most changes should already be merged for anyone interested in keeping up with development. Mostly posting to breakup the three posts in a row rule though 
6 Likes
resuming my ARM contributions after some break…
unfortunately I get a build error in ffs.c, it seems that features.h is missing.
update 1: the error message
Cc objects/haiku/arm/release/system/kernel/lib/ffs.o
In file included from ../src/system/libroot/posix/musl/arch/arm/atomic_arch.h:1,
from ../src/system/libroot/posix/musl/internal/atomic.h:6,
from ../src/system/libroot/posix/musl/misc/ffs.c:1:
../src/system/libroot/posix/musl/internal/libc.h:8:10: fatal error: features.h: No such file or directory
8 | #include <features.h>
| ^~~~~~~~~~~~
compilation terminated.
update 2: we’re looking for src/system/libroot/posix/musl/include/features.h
I can workaround by modifying src/system/kernel/lib/Jamfile:
--- a/src/system/kernel/lib/Jamfile
+++ b/src/system/kernel/lib/Jamfile
@@ -52,6 +52,7 @@ local muslSources =
;
SourceHdrs $(muslSources) :
+ [ FDirName $(posixSources) musl include ]
[ FDirName $(posixSources) musl internal ]
[ FDirName $(posixSources) musl arch $(TARGET_KERNEL_ARCH_DIR) ] ;
@waddlesplash I see in the commit history that you modified Jamfile for ffs recently. Can you take a look at the diff above? Do you think it’s a good idea? Or any other way to get around this cross-compile issue?
7 Likes
Yeah, that’s fine, but the real question is why exactly this works on all other architectures but not on ARM. Perhaps there is a problem with the headers configuration elsewhere (e.g. in the compiler?)
2 Likes
It is a bit annoying and potentially confusing that we currently have 3 files named features.h:
src/system/libroot/posix/musl/include/features.h
src/system/libroot/posix/glibc/include/features.h
headers/compatibility/bsd/features.h
Anyway, the problem here seems to be simply that the ARM atomic_arch.h includes libc.h, while the other architectures don’t:
https://cgit.haiku-os.org/haiku/tree/src/system/libroot/posix/musl/arch/arm/atomic_arch.h
https://cgit.haiku-os.org/haiku/tree/src/system/libroot/posix/musl/arch/x86_64/atomic_arch.h
That explains why ARM doesn’t build while the other architectures do. I don’t know which approach is correct or if there is a reason for this difference.
7 Likes
Welcome back. 
3 Likes
Probably it isn’t needed? I guess we can adjust it and find out.
2 Likes
I took one more look at arch/arm/atomic_arch.h and it uses at least the “hidden” define from features.h
It’s possible to copy it and then we don’t need to include libc.h or features.h, like this:
@@ -6,7 +6,7 @@
#define BLX "blx"
#endif
-extern hidden uintptr_t __a_cas_ptr, __a_barrier_ptr;
+extern __attribute__((__visibility__("hidden"))) uintptr_t __a_cas_ptr, __a_barrier_ptr;
but maybe this is not a very nice solution… perhaps it’s better to adjust the includes in Jamfile as above?
we also need a new (un-)bootstrap for the ARM port as the current packages are broken (icu crashes, glue code missing)
can I upload the hpkg files from unbootstrap script to the “limerick” server like last time?
@PulkoMandy @waddlesplash @kallisti5
Sorry for the delay. We moved to kubernetes recently, while I fixed the sftp upload system, we’re running into a few scaling issues given the grouping of applications around haikuports buildmaster sharing multiple filesystems. (haikuporter buildmaster alone consumes 4)
We can’t have more than 7 persistent volume mounts on a single k8s node with Digital Ocean. We’re right at the limit with the complex set of dependencies around buildmaster.
In the mean time, we’re going to have to figure out an alternative way to get these to me. Do you have anywhere to upload them and send me a link directly?
1 Like
You could probably use a service like WeTransfer…