PXE booting from an Ubuntu server

[6foot3]

Hi all, it’s been a couple of years since I last tried a recent build of Haiku on real hardware. I’ve been messing around with some hackintosh stuff and noticed my test system was always trying to boot from the network. It occurred to me that booting from the network would be an excellent way of testing haiku daily builds, if all I have to do is replace an image file on a “server” with the most recent one and boot from the network.

I’ve managed to set up the DHCP server to point to the boot file and set up a tftp server following Marcus’ instructions from here:

[openbeos] Haiku PXE boot information - openbeos - FreeLists

I’m using isc-dhcp-server instead of the dhcp3-server in Marcus’ instructions but, I’m assuming that all the DHCP server is doing is assigning an IP address and starting the PXE boot proccess, which appears to be working. I downloaded haiku-netboot.tgz and pxehaiku-loader from links provided by Olivier Coursiere in the following post:

[openbeos] Setting up Windows to boot Haiku through PXE - haiku - FreeLists

I edited my dhcpd.conf to point to pxehaiku-loader instead of “pxehaiku” as stated in Marcus’ post since it doesn’t make sense to point to file that does not exist and I assumed that “pxehaiku” and :“pxehaiku-loader” are the same file.

Now when I boot the test system from the network, it does it’s DHCP request, finds the PXE boot file and starts to load Haiku but, gets stuck showing the boot screen with all the boot icons grayed out.

Both Marcus and Olivier refer to running remote_disk_server to serve the disk image to the boot-loader.Olivier provide a link to his source and an executable for Windows while Marcus seems to assume that readers have the pre-requisites specified at the following page:

https://www.haiku-os.org/guides/network_booting/

i think I need the remote_disk_server but, I am not a programmer and there appear to be no executables for linux that I can download. Can anyone provide instructions on how to build the remote_disk_server for Linux. I am running Ubuntu 16.04 LTS. An executable that can be downloaded would be great.

[PulkoMandy]

Yes, you need the remote_disk_server. This is because Haiku doesn’t yet know how to boot from the more usual NFS, and for the time being has its own protocol for doing something similar.

Following the instructions in the page you linked (https://www.haiku-os.org/guides/network_booting/) should get you started:

• You will need to download and configure the Haiku sources (the server is shipped with them)
• You will need a raw image of the Haiku filesystem (preferably the same version as the pxehaiku-loader you used)
• In Haiku sources directory after configuring: jam -q run ':remote_disk_server' /path/to/haiku.image

Note that last time I tried this, I cound not get it to boot to the Desktop, but I think I got a bit further (it was at least trying to mount the remote_disk_server filesystem). Good luck!

[6foot3]

Thanks PulkoMandy, I appreciate the quick response but, not being a real programmer even downloading and configuring the Haiku sources seems a bit over my head. From what I can discern, getting the build tools for Linux seems straightforward enough but, I would have to install and learn how to use git. I just watched “Learn Git in 20 Minutes” on Youtube so, even that doesn’t seem that much of a stretch but, it just feels like a lot of work to get one 321 line source code file to compile.

Ok. So being lazy, I cheated. I found some source code here:

https://searchcode.com/file/33300005/src/tools/remote_disk_server/remote_disk_server.cpp

and compared it to Oliver’s code. It appeared to be the same thing without the Windows specific bits so I copied and pasted it to gedit, saved it as remote_disk_server.cpp and attempted to compile it using gcc. On the first attempt I got a “fatal error: boot/net/RemoteDiskDefs.h: No such file or directory” message so, I searched the net for RemoteDiskDefs.h and found it here:

https://searchcode.com/codesearch/view/15188299/

I created a directory net in my /boot directory, cut and pasted the code into gedit, saved the file as RemoteDiskDefs.h and copied it to /boot/net. On my next attempt I got the same message but, noticed that there was no “/” in front of “boot/net” so I checked the source in remote_disk_server.cpp. Surely enough line 18 reads “#include <boot/net/RemoteDiskDefs.h>” so, I changed it to “#include <**/**boot/net/RemoteDiskDefs.h>” and tried again. This time I got six warnings, three each in “member function ‘void Server::_HandleReadRequest()” and “member function ‘void Server::_HandleWriteRequest()”

Am I getting anywhere or just being too clever for my own good?

[PulkoMandy]

If you don’t understand git, just use xkcd wiseness:

The “commands to sync up” are here:
https://www.haiku-os.org/guides/building/get-source-git

For your convenience:

git clone https://git.haiku-os.org/buildtools
git clone https://git.haiku-os.org/haiku

That wasn’t too hard, I think.

Let’s proceed with building:
https://www.haiku-os.org/guides/building/compiling-x86

cd haiku
./configure –build-cross-tools x86_gcc2 ../buildtools

And then you should be able to build and run the disk server:

jam -q run ':remote_disk_server' /path/to/haiku.image

I think this will be much simpler than trying to rebuild everything by hand and getting header files one by one.

[6foot3]

Hi PulkoMandy, I followed your instructions and had to give it a couple hours to download. I had to issue the commands as root (sudo) since Ubuntu linux wont allow ordinary users to create directories outside of their home directory. I went into the newly created haiku directory and tried to run configure but, got the following message:

“Invalid argument: `–build-cross-tools”

Is there something different for linux build environmnts?

[PulkoMandy]

Why are you working outside of the home directory? These commands can be run inside your home, you do not need to mess with other parts of the OS. The “/boot/home” paths you may have seen are just where the home directory is when running Haiku. And the “boot/net/…” path to include files does not start with a slash, because it is a relative path (the build system knows where to look for the file, don’t worry)

I see there was some website formatting playing tricks on me. --build-cross-tools takes two dashes, and somehow it got converted to a single long dash, which won’t work.

[6foot3]

Somehow my terminal was at root. I think I have moved everything to my home folder by issuing “cp -r /buildtools buildtools” and “cp -r /haiku haiku”. I then replaced the long dash with two short dashes and entered the configure command which ran for several minutes with lots of out put to the screen.

At this point I decided to follow Marcus’ intsructions from here:

[openbeos] Haiku PXE boot information - openbeos - FreeLists

but, when I issued “TARGET_BOOT_PLATFORM=pxe_ia32 jam pxehaiku-loader”, the response was, “You can only build Haiku with Haiku’s Jam, not Perforce Jam or some other variant (you are using 2.6).”. I had thought I was going to need jam at some point so I had done an “apt-get install jam”. To get rid of Ubuntu’s version of jam I issued “apt-get purge jam” after which the response to the jam command was “bash: /usr/bin/jam: No such file or directory”. It looks like I have messed up something!

I have created a “develop” folder in my home directory and I am in the process of using git to clone the buildtools to that directory after which I will be cloning haiku again and trying the process over again.

[PulkoMandy]

To get rid of Ubuntu’s version of jam I issued “apt-get purge jam” after which the response to the jam command was “bash: /usr/bin/jam: No such file or directory”. It looks like I have messed up something!

No, that is fine. You now need to build and install Haiku’s version of Jam. This is documented here (in this case it seems our build instructions are a bit out of order, we should probably rework this): https://www.haiku-os.org/guides/building/jam

cd buildtools/jam
make
sudo ./jam0 install

Then you should be able to run Jam again. To check it is the right version, jam -v should say something with “Haiku” in it.

[6foot3]

I’ve folowed your latest instructions but when I tried to run “jam -v” I get “bash: /usr/bin/jam: No such file or directory”. I noticed that the last three lines of out put from the jam install command were:

“Install /usr/local/bin/jam
Chmod1 /usr/local/bin/jam
…updated 28 target(s)…”

I issued “/usr/local/bin/jam -v” and the response was “Jam 2.5-haiku-20111222. OS=LINUX. Copyright 1993-2002 Christopher Seiwald.”. I suspect that somehow the path to jam has been set to “/usr/bin/jam” instead of “/usr/local/bin/jam”. What went wrong and how can I fix this?

[6foot3]

PulkoMandy, I just did some poking around in my /usr/local/bin directory and the only file visible was jam! Does this mean that there is some sort of error in the build instructions that is putting jam in /usr/local/bin instead of /usr/bin ?

I also just ran “/usr/local/bin/jam -q run ‘:remote_disk_server’ /tfptboot/haiku.image” and surprisingly enough some messages that look very similar to the messages I got when trying to compile remote_disk_server on it’s own. The messages follow:

src/tools/remote_disk_server/remote_disk_server.cpp: In member function ‘void Server::_HandleReadRequest()’:
src/tools/remote_disk_server/remote_disk_server.cpp:199:65: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘uint64_t {aka long unsigned int}’ [-Wformat=]
printf(“READ request: offset: %llu, %hd bytes\n”, offset, size);
^
src/tools/remote_disk_server/remote_disk_server.cpp:216:37: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘uint64_t {aka long unsigned int}’ [-Wformat=]
"%s.", offset, strerror(errno));
^
src/tools/remote_disk_server/remote_disk_server.cpp:221:29: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘uint64_t {aka long unsigned int}’ [-Wformat=]
offset, strerror(errno));
^
src/tools/remote_disk_server/remote_disk_server.cpp: In member function ‘void Server::_HandleWriteRequest()’:
src/tools/remote_disk_server/remote_disk_server.cpp:242:66: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘uint64_t {aka long unsigned int}’ [-Wformat=]
printf(“WRITE request: offset: %llu, %hd bytes\n”, offset, size);
^
src/tools/remote_disk_server/remote_disk_server.cpp:261:37: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘uint64_t {aka long unsigned int}’ [-Wformat=]
"%s.", offset, strerror(errno));
^
src/tools/remote_disk_server/remote_disk_server.cpp:266:29: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 3 has type ‘uint64_t {aka long unsigned int}’ [-Wformat=]
offset, strerror(errno));
^

[PulkoMandy]

Usually the “local” directory is for things you compile and install yourself, while the normal “bin” directory is reserved for things installed through a package manager. However, this may be different depending on which distribution you use. It’s fine to run Jam with the absolute path as you did.

The warnings you listed are apparently because the remote_disk_server was not completely adjusted to build on 64bit. If you only see these warnings and no build errors, it should still run (you may have problems with format of some messages). If there is an error later on in the build, please let us see it. I guess no one has tested this in a while so it may be currently broken.

[6foot3]

The latest on my attempts is that I built the pxehaiku-loader and haiku-netboot-archive using:

" TARGET_BOOT_PLATFORM=pxe_ia32 /usr/local/bin/jam -q pxehaiku-loader haiku-netboot-archive"

I then copied both to my /tftpboot folder by issuing the following:

“cp /home/alan/develop/haiku/generated/objects/haiku/x86_gcc2/release/system/boot/pxe_ia32/pxehaiku-loader ~/tftpboot/”

and

“cp /home/alan/develop/haiku/generated/haiku-netboot.tgz ~/tftpboot/”

I then ran

“/usr/local/bin/jam -q run ‘:remote_disk_server’ ~/tftpboot/haiku.image”

The test system is now getting an IP address from the DHCP server and starting the bootloader. The seven boot icons appear on the screen and the first four “light up”, the last three stay grayed out. The remote disk server on the host PC is disolaying output as shown on the page at the following link:

https://www.haiku-os.org/guides/network_booting

After a while the output from remote disk server freezes, the test system reboots and the output from remote disk server starts changing again. This is an unending loop.

[PulkoMandy]

Ok, we’re making progress!

The next step would be to grab a syslog, but I’m not sure if you can access the early boot menu for that when using the PXE loader. The alternative would be to get it from the COM port of the target machine, if there is one (they are getting rare), or maybe the system got far enough to write the syslog to haiku.image?

[6foot3]

I tried the spacebar trick while booting to see if I could get a boot menu and that works so, I have access to all sorts of goodies from the boot menu. What do I start with? The system is based on Intel’s H67 chipset and a 3.3 GHz Sandy Bridge processor and is essentially “legacy free” so no PS/2 or COM or printer ports. I was able to find the point at which the remote_disk_server output freezes and the system reboots and have included the out put just before and after the reboot for what it’s worth. I tried a x86_64 nightly and a ggc2 hybrid nightly I downloaded, and an image I built myself and they all displayed similar behavior, except that the read offset when the boot process freezes is different. I suspect that nothing was written to the syslog since the output from the remote disk server does not include any write requests but, I can access the syslog from the previous boot according to the boot menu. I edited this post after I realized that I had not moved my keyboard to the test system so I was hitting the spacebar on a keyboard that wasn’t connected to the system!

“READ request: offset: 227640320, 1024 bytes
READ request: offset: 227641344, 1024 bytes
READ request: offset: 227642368, 1024 bytes
READ request: offset: 227643392, 1024 bytes
READ request: offset: 227644416, 1024 bytes
HELLO request
READ request: offset: 0, 512 bytes
READ request: offset: 2048, 1024 bytes
READ request: offset: 3072, 1024 bytes
READ request: offset: 268435456, 1024 bytes
READ request: offset: 268436480, 1024 bytes
READ request: offset: 4096, 1024 bytes
READ request: offset: 5120, 1024 bytes
READ request: offset: 268437504, 1024 bytes”

[PulkoMandy]

From the boot menu you can enable on screen debug. This will log on screen what the system is doing.
Then you can see if there is an error message or how far it gets before running into problems.
You can take pictures of at least the last screen before a reboot or freeze and share it with us for further analysis

[6foot3]

Last screen of current boot loader log

Last screen of syslog from previous session

Second to last screen of debug output before reboot

I had to use a video and do a screen capture from it to get the last screen of debug output before reboot.

Should I file a bug report?

[PulkoMandy]

Yes, you should file a bugreport if there isn’t one already for this. I think this is as far as I got last time I tried a network boot as well. It seems the package manager filesystem does not like being mounted from the network for some reason.

[6foot3]

There are seven bug reports with "PXE in the summary and the last and only open one is ticket #10198. While this is vaguely similar it is four years old and the person who reported it seems not to have gone any further than I have in terms of investigation. I did notice lines in the debug output starting with packagefs that suggest that there is a problem with the package fs. Should I file a new bug report or add something to ticket #10198?

[PulkoMandy]

I think the existing ticket is enough then. It would be nice to fix this but I’m not sure what to do next.

[6foot3]

I downloaded the most recent build I could find that did not have package management (hrev46101) and tried to boot that. All seven boot icons lit up but, it did not boot. I am currently downloading the first build I could find with package management included and will be checking to see what happens with that build.

Edit: I downloaded and attempted to boot hrev46154. It reverted to the behavior of the most recent builds, hanging with the last three boot icons grayed out and the rebooting.