UEFI guide


#1

Hi. Looking at this Haiku guide on Uefi booting, there appeared some questions.
ESP (EFI System Partition) Size.You recommend 32 MB, telling it’s “more than enough”. It depends on many factors if that would be enough at all. The Uefi documentation frees itself of the responsibility to even advice “common” size, that’s true. It’s left to the choice of OS vendors. This is how it explains it:

Note, there is a “multiple OS” mentioning. That’s the point. When you format (in GPT) a raw disk, you create an ESP as part of Haiku installation. But this ESP is not for Haiku only! And given the stage of Haiku, it’s well reasonable to expect many people using it alonside with other OSes, - a multiboot environment. There are chances, that the ESP creation could be made by Haiku. Then OSes installed after it, may use the same ESP. 32 MB could be not enough then. For many reasons. And not all are reasonable, but still they are real. A good example of a not so reasonable one. Linux. It has taken the worst ever possible approach of doing UEFI. It embedded an UEFI OS Loader into its kernel (what a surprize!). Meaning that you need to put the whole linux image into an ESP and a bunch of other linux stuff. Knowing how bloated linux images are, you do realize that 32 MB might be problematic… There are other, saner reasons - ESP can hold a lot of data, resources from more than 1 vendor. Chapter 12 (2.4.errata.B) is a good source of hints on this.
I’ve seen in many places, that setting ~100MB of the ESP size is quite good. It’s still a tiny portion of a rather smallish 256 GB hard disk, not to mention larger ones, that are more common nowadays.

Finally the second, which especially makes me wonder. When I see attempts to interact with UEFI on the OS development level (communities, hobbyists), I always see using this BOOT{MACHINE_TYPE}.EFI approach. As if it is the only one. Again, maybe you do so just because UEFI support is too fresh and unpolished yet, but I wanted to say on this. The approach, when you put your OS Loader (OSL) in the \efi\boot\boot{machinetype}.efi is the last resort for the FW. It’s intended to be used normally only for removable devices (even here, it’s not the only option). And for persistent, non-removable storage, where an OS could reside, this approach is the last resort of the UEFI Boot Manager to boot something (a so called “default boot behavior”):

When no valid load options worked out, nothing, only then it may try that. It’s left to the implementation what set of media to try. Just read carefully section 3.4.1.2 (2.4.errata.b):

“Optionally” may occur. Or may not. This is already a sufficient reason to get rid off of this approach for the installation disk, since it makes running (installing) Haiku less successful. But also, it has other dangers. For example if you will install another OS alongside and after Haiku, it might overwrite Haiku’s BOOTXXX.EFI file.
It’s so easy to see from the specification, the right way to do stuff - you create your own folder under \efi\boot, in this example, let it be “haiku”, and put the OSL there, - voila! This is how it should work.

\efi
 \boot
  \haiku
   \haikuldr.efi

See section 12.3.1.3 “Directory Structure” (2.4.errata.b), they wrote that down clearly.
During the installation, your installer creates the needed load option, asking UEFI Runtime Services - any UEFI compatible OS not only should preserve UEFI RTS, but also - may make use of them! This is one of the best examples of what it can be used for. This way, Haiku will appear in the Boot Manager menu and users can see/set/edit its load option as well (e.g. changing the order). It’s a normal way, whereas BOOTXXX.EFI approach is for an “exceptional” case and the quote above shows - it is even “optional”!


#2

Nice explanation, thanks!
Please remember, the UEFI support is pretty basic in Haiku, with many limitation, which will be eliminated later. Currently Haiku won’t:

  • create any ESP partition
  • copy the .efi loader
  • set anything in nvram

So basically one should do everything manually - again: currently.
So we let the users read the UEFI docs and set everything up as they wish.
32 mb is - in my opinion - way too small.


#3

@val, your description is so inexact in some places, that I considered to make some clarifications.

The purpose of a bootloader is to load the kernel image into memory and to transfer the control to it. UEFI is not an exception. How do you imagine the UEFI loader is embedded into Linux kernel, if at the stage of executing of UEFI bootloader the kernel is not yet loaded in memory? And what is the purpose of the bootloader if during its execution (it is already loaded in memory) the Linux kernel (that is assumed embedded into it) is in memory as well? Linux NEVER installs kernel or initramfs (each of several MB) into ESP partition. On the other hand, the GRUB bootloader stage 0 is installed here (other stages are installed in Linux partition). The code of this stage is really tiny. For comparison: when GRUB is installed on MBR systems, this stage is completely fits into 512 Bytes of MBR (i.e. 4096 bits), and there is plenty of space there yet.

OSL (OS Loader, or stage 1 boot) resides in the first sector of partition where OS is installed, not in ESP partition, which is shared across all the OSes. Its size is larger, but is not shared and occupes no space in ESP.

Now, let do some math. The GPT standard allows up to 128 partitions in default partition scheme (it can be modified to almost unlimited number though). I personally use 7 OSes on my desktop, but usually people are using no more than 2-3 OSes installed. Let us imagine some extremist who installs 127 OSes (1 partition is reserved for ESP). Each has the stage 0 bootloader of less than 1 MB, total is less than 127 MB in this extreme use-case. Microsoft recommends 256 MB for ESP partition, and this is more than enough. 32 MB is absolutely sufficient for almost all use-cases. Of course, people are free to keep their movies of several GB in ESP partition, but it is out of its purpose.

As of naming convention and possible conflict. BOOTHAIKU.EFI is the name, which probably will not be used to boot anything other than Haiku. But suppose some malicious OS creator use BOOTHAIKU.EFI to boot his OS, and you are going to install it without damage the Haiku boot. Because ESP partition is shared across all the OSes and the OS installation process writes to it, there is no possibility to ensure perfect naming uniqueness. If you want to use \efi\boot\haiku\haikuldr.efi, there is no technical obstacle for using exactly the same loader file name (including its complete path) by other OS, and thus rewriting Haiku loader.


#4

The MacBook Pro system after bootcamp allocated a 200Mb partition as Fat32 EFI. You can have many directories with different .efi images. I also have EFI/Haiku/boot_x64.efi, as well as OSX, Windows, Grub and rEFInd (default boot). Using rEFInd I can boot Haiku, but using Mac’s own Option-key boot menu (or Windows boot manager) I cannot boot Haiku. Once I got a process which worked, I did not play with it any more.

EDIT: accidently typed Refind instead of rEFInd. Hard to type on autocorrect phone


#5

You are using rEFIt to boot haiku or the newer rEFInd (since the first isn’t maintained for many years)?


#6

T2-chip inside or not ?

Even without (or with but in no security setup), I won’t be surprised if the Mac UEFI only support well known paths for macOS and Windows .efi bootloaders and that’s all.


#7

rEFInd :slight_smile: replied from autocorrect phone. Thx


#8

Pre T2. MBP 2014 (11.3)


#9
  1. They call it “efi boot stub”. From their scarce documentation (Documentation/x86/boot.txt), we learn:

Even having separate OSL, it can “defer initialisation” to the embedded into the kernel “efi boot stub”, which is basically yet another OSL.

  1. From the their file Documentation/x86/x86_64/uefi.txt, line 26:

As of what the purpose, they probably know it. I was talking about ARM though, because on x86 they use some loaders but still require putting their kernel and stuff into what they call VFAT partition.
(Added later). On ARM (64 bit), there is also that stub, which is at the beginning of their gigantic kernel, basically a PE file fused with everything else (see Documentation/arm64/booting.txt). for 64 bit ARM, they devised a special “image” and its header has a field pointing to the “stub”. and for now, on ARM SBCs, UEFI is not broadly used, but the approach is the same: you have a small (~100 MB) “boot” volume and linux image & Co reside there (or, as a bonus of the crappy building process, for some vendors, - a bunch of images, initrds, and device trees). now uboot loads linux image & Co through a special linux specific command “booti”, :poop: but uboot is getting into UEFI and has something implemented of it already. So uboot has two ways of transfering control to it - jumping into code of the header (see referenced “documentation” above), or, - calling the entry poing of the “efi boot stub”. and if that stub makes use of UEFI, it will call uboot back (through the System Table passed to it). I don’t know which way they use now, OpenBSD for ARM e.g. already requires UEFI. Now you see, that in fact, linux turns itself into a UEFI OSL that is loaded (as a whole) by uboot (that is “kinda” UEFI). This is not UEFI compatible. They need to provide yet a separate OSL, that would be messing around with their invented wheels (arm64 “image”). uboot follows linux obediently keeping on on the antipattern of forging linux specific commands, but a normal UEFI implementation won’t do this. Then, you will need elilo, grub or whatever. but still, this doesn’t change the fact linux has internally embedded UEFI OSL. Yet once - OSL is a UEFI application that calls ExitBootSevices(). Guess who does this if “efi boot stub” has been called? :wink: The only thing why linux image can’t be loaded directly by UEFI is because UEFI expects a proper PE file, whereas linux has it somewhere behind those “headers” inside. Anyway, very often linux images end up laying inside “boot” volumes of any kind. Don’t expect ESP won’t suffer the same fate. I don’t, so I’d allocate a little bit more than 32 MB. just in case.

Not in the UEFI context, What you retell is the BIOS way (MBR code, boot sector code etc). UEFI ignores BIOS MBR code and any boot sector code completely. They are not used, you are wrong. There is no “stages” of loaders in UEFI either. Instead, there is notion of an OS Loader - a special type of UEFI application (it’s specialty is in that normally it doesn’t return to UEFI, uses ExitBootServices()). It’s a payload of UEFI, it loads it into memory (according to the load option related to it) and transfers control to it. OSL is a PE file that is most expected to reside in the ESP just like any executable file. But this file may reside on other volumes, where FW can reach to. For example the Boot Manager “boot from file” option is an example. There could be non-ESP FAT volumes from where you could load it. Or even non-ESP non-FAT volumes! In the latter case, a UEFI driver is needed, that “materializes” that file system and installs the FS protocol on it. What is invariant - UEFI operates with PE files as its payloads. No “boot sectors” or alike.

Actually, there is no limit on the partition count, the specification just requires the minimum size for the Partition Array, to be able to create this minimum number of partitions. It’s not “default”. I’d say, that correlating with the disk size, would be a normal way of the decision, when creating GPT on it.
Your math forgets about:

  • bitmap images for logos (with freaking animation! :smiley: ), other similar resources (bitmap fonts maybe, because there are moments when UEFI Boot Services aren’t available already, but the kernel isn’t ready yet to draw the screen, and you still want to show something, this is graphics, its resources can take a bit of space)
  • utilities (memory testing, storage testing etc)
  • backup files used by OSLs and utilities during their operations
  • configuration files
  • logs of different kinds

finally, again - linux all in one pile of it. :smiley:

As of naming conventions. It’s not about preventing “malicious” users to do bad things. Nor about providing an “ubreakable” way of organizing ESP. For this there is “teh security”. :smiley: It’s about avoiding clashing with others. I referenced the chapter of the specification, where it’s described better, than I did. The intention of this is for well behaved OS vendors to have their own apartment in the UEFI house, that lets them integrate into platform the best way.


#10

Dear @val, I don’t intend to offend you, and in no way I want to transform the technical discussion into emotionally charged polemic. I would stop here, but for those interested to boot an OS in UEFI mode rather than to show their superior knowledge of the specification, I will add some more details.

Linux image has ELF (Executable and Linkable Format) as well as Haiku, *BSD (modern) and several other OSes. You can find it out by issuing a command from terminal:

file linux-XXXXX

and see the result, something like:

linux-XXXXX: ELF executable x86-64, for Linux

PE executable format is used by Microsoft executables and by UEFI. Again, you can try:

file boot_linux.efi

(or whatever you have as part of GRUB installation) and see:

boot_linux.efi: PE32 executable x86, for UEFI

or something like this.

EDIT:
On my Linux installation:

file vmlinuz-4.9.0-8-amd64
vmlinuz-4.9.0-8-amd64: Linux kernel x86 boot executable bzImage, version 4.9.0-8-amd64 (debian-kernel@lists.debian.org) #1 SMP Debian 4.9.144-3.1 (2019-02-19), RO-rootFS, swap_dev 0x4, Normal VGA
ls -lh *-4.9.0.8-amd64
-rw-r--r-- 1 root root 183K Feb 19 11:05 config-4.9.0-8-amd64
-rw-r--r-- 1 root root  18M Mar 22 12:10 initrd.img-4.9.0-8-amd64
-rw-r--r-- 1 root root 3,1M Feb 19 11:05 System.map-4.9.0-8-amd64
-rw-r--r-- 1 root root 4,1M Feb 19 11:05 vmlinuz-4.9.0-8-amd64
file boot.efi
boot.efi: Universal EFI binary with 2 architectures, i386, x86_64
ls -lh boot.efi
-r--r--r-- 1 alex alex 224K May  6  2017 boot.efi

You can see the difference between file format of Linux kernel and GRUB EFI image, as well as between their sizes.

The UEFI system only knows how to load / execute PE format, specially compiled for it, including GRUB EFI. GRUB executable knows how to load Linux kernel (among some others, *BSD, IlluminOS/Solaris, macOS) as a file image, or (in case some OS kernel is part of file system and is not a file, such as MS Windows, Haiku) to chain load that partition, which loads partition loader, or OSL in your terminology. In all cases, an OS cannot pass the control back to UEFI, for this you always need to restart the computer. On the contrary, until Linux is not yet got the control, GRUB EFI can call other EFI executables directly, without restarting the computer.

Think of some Linux distribution, which doesn’t know what system do you have: BIOS/MBR or UEFI/GPT (or any other combination thereof). The same Linux image is installed in all cases. There is no UEFI specific or BIOS specific Linux image. So, the UEFI stuff is not related to Linux per se, but to the bootloader. You can do the following experiment (I did it): copy all the content of Linux partition with installed there Linux from a BIOS PC to UEFI PC, and then reinstall the GRUB. Linux will boot the same Linux image without any problem.

Now, where to keep the Linux image? You choose. Again, you are free to keep it in ESP partition, but this is not its intention (and FAT32 file system doesn’t keep important for Linux meta-information, like the access permissions and creation/modification timestamps). But usually ESP partition is not mounted in Linux. Now, consider you get an update, which also includes the Linux image. Where will it be written to? To Linux partition, or to ESP partition? Again, you can manually configure whatever to work, and update program cannot know your plans. But it will most probably install new kernel image into Linux partition, and you will be forced to repeat your setup to be able to boot Linux again. Where to keep all supporting files (fonts, splashes, animations, etc)? You decide. But just as well GRUB EFI image can access Linux partition to read Linux kernel image file, it also can read all the other files, including all the supporting stuff.

Backup files are usually kept outside of ESP partition. MS Windows UEFI installation creates separate partitions for booting and for backup, of different sizes.

I also played with other architectures than x86/x86-64. They all have their specific partition scheme, have specific bootloaders, which act differently and sometimes are implemented as image to be flashed (unlike file in file system). You probably will not want to have several OSes on ARM based computer, but it is also possible.

Regarding MBR vs GPT partition scheme. MBR takes 512 bytes at the very beginning of the disk and allows up to 4 primary partitions (logical partitions are implemented by repeting the MBR scheme in the beginning of so called extended partition). GPT keeps the partition layout in table of variable length at the beginning of the disk, and its copy at the end of the disk. Because GPT partitioning doesn’t limit the number of partitions on disk, this partition table may grow or shrink. By default, whatever tool you use to create GPT partition scheme (gdisk, gparted, etc), the partition table will be enough to hold 128 partitions. After that, if you create necessary partitions (not more than 128), and then want to create more than 128 partitions, you will need to re-create or adjust the partition table. In the former approach you will lost all your partitions, in the later approach the partition table will take physically more space out of the very first and very last partition, so their size will need to be adjusted.


#11

Interesting. Windows and Systemd-boot put their loaders in their respective folder in /EFI directory, not /EFI/BOOT.

That being said, I think it doesn’t matter outside of UEFI guidelines. (For some reason) I absolutely refuse to install anything like rEFInd since I believe OS should be able to load without it, and I stumbled upon efibootmgr Linux utility that can add arbitrary entries to UEFI. I’m not sure what modern Linux installers use for this purpose, but well, it just works, and it worked perfectly for Haiku bootloader - I just pointed it to .efi file in my arbitrary location (that being said, Haiku’s bootloader don’t see my main partition, but I guess that’s whole other problem).

So maybe functionality of this utility needs to be ported to Haiku separately or as a part of installer?


#12

Of course, they do right. It’s me, making reference to that chapter (12.3.1.3 Directory Structure), looking at it, still, messed it up. It’s really doesn’t matter (except for clarity and being well organized), because OSL image is being looked through the path to it, so it may reside anywhere, where FW could reach. It’s guidelines, you are right.
The proper way, would be something like that:

\efi
 \haiku
  \haikuldr.efi

Or even

\efi
 \haiku
  \v1
   \haikuldr.efi
   \v1.1
    \haikuldr.efi
   ...

For having any combination of OSL and OS version pairs. I mean, say OSL v1 can’t boot newer OS versions, and newer ones dropped support of loading old OS versions; so you create a set of loaders and load options, that can cover every possible way, just on the same machine, without any clashing. :slight_smile:

OS Loader still is needed to load OS. I don’t know what exact part of the job rEFInd and Haiku loader do, but the way it is supposed to have been done in UEFI is this:
UEFI Boot Manager -> OS Loader -> OS. OS Loader here is surely OS specific, because only such can know all the OS internals and requirements. That’s why all those GRUBs etc are irrelevant in this context, they only add unneeded overhead and lengthen boot chain needlessly. Or they need to understand requirements of all OSes over there.
If you mean mechanics of how to add that entry (load option, pointing to the OSL), then this could be done best during the OS live session under installation by the OS itself. Asking user what he/she wants and then calling appropriate UEFI Runtime Services. This functionality might be exposed by the OS for later uses, but something more that adding a load option (which belongs to the installation, obviously), would be better to do with the interface provided by the Boot Manager itself and this is where user goes and sets it up manually. This is because a lot of things in Boot Manager policies left implementation defined.


#13

Of course. I’m honestly not familiar with rEFInd, but I believe it is used, for example, on Macs, for the purpose of launching, for example, Linux distributions. That’s why I mentioned it and why I don’t want it - if I already can run any EFI code using my regular UEFI menu, why do I need another layer?

Honestly, the point of my post was to share my discovery of efibootmg utility, not so much to join argument about UEFI (since I’m far from being knowledgeable there).


#14

Hiya, thank you to all contributors of thid thread, it really has taught me what the functioning of UEFI is really about. I boot Haiku OS from an independent EFI partition on a HDD shared with Manjaro on a PC that has a windows10 SDD and a Hackintosh SDD. I use REFInd as boot loader and Clover for the Hackintosh. I just wish to get rid of the “Search all volumes …” message when booting Haiku, everything else works just fine!