Idea: userland virtual machine to allow running binaries on any platform

Development OS
21

This can be a way to get x86_32 on the x64 too?

22

I would hope that is the case. Like Rosetta on the Mac or similar.

2 Likes
23

This actually could unlock the potential for my earlier, half-hearted attempt at making a WebAssembly WASM32 package format for Haiku. I didnā€™t understand how to make an ELF loader altenative using a different code format. Now I ā€œjustā€ have to figure out how to make a WebAssembly loader compatible with TinyEmu or LibRISCV.

My confidence in my abilities is at an all-time low. Maybe this could be the jump-start I need.

2 Likes
24

Yes, UserlandVM approach can be used to run WebAssembly binaries. It should be also possible to statically translate WebAssembly bytecode to host CPU machine code in .text sections.

Do you mean some non-ELF executable format? Or code section incompatible with host CPU? UserlandVM use simple ELF loader to load guest runtime_loader, the rest loading is done by guest runtime_loader.

4 Likes
25

I meant that the WASI standard ABI is transitioning to interface types but donā€™t use a standard ELF object format as far as I know. Once the interface format changes from witx to wit, it should be possible to create reusable modules to wrap most shared objects in WebAssembly bytecode.

26

Couldnā€™t WASM binaries just be associated in FileTypes preferences with some sort of interpreter that then supports WASI or similar? Why would you necessarily need to use this special runtime_loader approach?

27

If native Haiku API WASM applications are planned, it should be handled like yet another CPU architecture like x86 or riscv64. It need system API libraries (runtime_loader, libroot.so, libbe.so, etc.) to be compiled for WASM target. UserlanVM can handle WASM code execution and host->guest syscall forwarding.

Or maybe WASM binaries can be converted to host architecture so it will use host OS libraries. But in this case it will be probably a lot of ABI matching problems.

3 Likes
28

Re:WebAssembly

There are two approaches to follow:

  1. Making a WebAssembly processor architecture for the existing package manager that uses root.so as its runtime engine and the standard kits. This would statically compile to the native executable format at install time.

  2. Following the WASI standard to use WebAssembly modules and use the WASI Runtimeā€™s version of package management.

The First Approach: Package Manager

As for the first approach, I think this is the best way to implement for Haiku but doesnā€™t accommodate the WASI standard directly and therefore, is not a total solution for WebAssembly outside the browser. It would share a lot of infrastructure with the second approach which is why Iā€™d suggest using both approaches.

The Second Approach: WASI Runtime Engine

Wasmer has a package manager called WAPM that allows OS neutral binaries to be created and installed either as JIT, AOT or Universal (static) execution specified at install time. Wasmer even supports multiple compiler backends: LLVM (C++), CraneLift (Rust) or Singlepass (a simpler approach with minimal optimization). Wasmer is run by a small startup company. (Disclaimer: Iā€™m trying to get a job working at Wasmer but my chances are waning with each passing day that I donā€™t know what to do.)

Another WASI runtime called WasmTime exists but only supports one backend written in C++ using LLVM, I think. The development time of WasmTime lags behind Wasmer a bit but is run by a non-profit organization called the Bytecode Alliance. The Bytecode Alliance organization exists as a support mechanism for the mainstream operating system and processor manufacturers to use as a tax write-off. (IMO, of course. This is why Iā€™m trying for the other one first. Iā€™d rather use Haiku as a tax-writeoff.
:slight_smile: )

Closing Notes

I hope this doesnā€™t go too far off-topic but as anyone who has been following my posts knows, I favor WebAssembly as a bytecode mainly as a means of sharing infrastructure with the LLVM project whose well-paid Apple and Google (and Microsoft?) employees have released open-source infrastructure for the rest of the industry to use freely. It could be a way for Haiku, Inc. to piggy-back on the latest trends and eventually, to deliver better performance than the mainstream OS providers can match.

Sorry about the long post. If this veers too far off-topic Iā€™ll apologize in advance for any clean-up the moderators have to put up wtih.

4 Likes
29

I managed to get 64-32 bit switching for x86. 32 bit code segment was already provided so no kernel changes were needed. But setting FSBASE should be added to set TLS register used for x86 Haiku ABI.

Far jump instruction is used for mode switching. Some code should be available from 32 bit mode, so it is copied to separate area with 32 bit address. Data segments (DS, ES) should be initialized when operating in 32 bits.

19 Likes
30

Any chance of this work being added to Haikuā€™s main codebase?

31

It is planned to be added as runtime_loader add-on, but later. Currently it is not ready.

4 Likes
32

I managed to run UserlandVM in chroot environment and compile programs with riscv64 GCC. I added syscall overrides for fork, load_image and exec (prepend UserlandVM as first argument).

In theory it should be possible to run haikuporter and compile HaikuPorts packages for any architecture without haikuports.cross.

~/Tests/UserlandVM/build.x86_64> uname -a
Haiku pc-1 1 hrev55969+49 Mar 31 2022 04:50: x86_64 x86_64 Haiku
~/Tests/UserlandVM/build.x86_64> chroot ../root /boot/system/runtime/UserlandVM /boot/system/bin/bash
/> cd /boot/home/Tests/MinApp
~/Tests/MinApp> make clean
rm -rf "objects.x86_64-cc11-release"
~/Tests/MinApp> make
mkdir -p objects.x86_64-cc11-release; \
mkdepend  -I./ -p .cpp:objects.x86_64-cc11-release/%n.o -m -f "objects.x86_64-cc11-release/MinApp.d" MinApp.cpp
g++ -c MinApp.cpp -iquote./  -iquote./  -O3    -o "objects.x86_64-cc11-release/MinApp.o"
cc -o "objects.x86_64-cc11-release/MinApp"      objects.x86_64-cc11-release/MinApp.o  -Xlinker -soname=_APP_  -L./    -lstdc++  -lsupc++  -lbe 
mimeset -f "objects.x86_64-cc11-release/MinApp"
~/Tests/MinApp> readelf -h objects.x86_64-cc11-release/MinApp
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           RISC-V
  Version:                           0x1
  Entry point address:               0x45d2
  Start of program headers:          64 (bytes into file)
  Start of section headers:          36016 (bytes into file)
  Flags:                             0x5, RVC, double-float ABI
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         3
  Size of section headers:           64 (bytes)
  Number of section headers:         24
  Section header string table index: 23
~/Tests/MinApp> objects.x86_64-cc11-release/MinApp
~/Tests/MinApp> ls /boot/system/runtime
UserlandVM  lib
~/Tests/MinApp> ls /boot/system/runtime/lib
libgcc_s.so.1  libroot.so  libstdc++.so  libstdc++.so.6  libstdc++.so.6.0.29
~/Tests/MinApp> exit
exit
~/Tests/UserlandVM/build.x86_64>
26 Likes