Memory allocation in C fails past a certain size

Hello, I’ve been trying, mostly for fun, to build the compiler for the Zig programming language on Haiku R1 Beta 4 (x86_64). I’ve faced issues, but a team member for the language has been looking into it: haiku: get a cross-compiled compiler working by jacobly0 · Pull Request #19406 · ziglang/zig · GitHub

They were able to identify the following issue:

bootstrap isn’t working because single large allocations > 0x3f60e740 bytes fail, even though many allocations of that size succeed.

Through some testing, I’ve found that calls to malloc, realloc, calloc… fail for any value above 1063315264 (or 0x3f60e740) bytes

#include <stdlib.h>

int main() {
  void *p = malloc(1063315264);
  return p == NULL;
}
~/test> cc -o maxmem maxmem.c
~/test> ./maxmem
~/test> echo $?
0

Succeeds, but

#include <stdlib.h>

int main() {
  void *p = malloc(1063315265);
  return p == NULL;
}
~/test> cc -o toomuchmem toomuchmem.c
~/test> ./toomuchmem
~/test> echo $?
1

Fails.

However, directly calling mmap for amounts greater than 1063315264 seems to work:

#include <sys/mman.h>
#include <unistd.h>

int main() {
    void* p = mmap(NULL, 1349910528, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
    if (p == MAP_FAILED) return 1;
    return munmap(p, 1349910528);
}
~/test> ./testmmap
~/test> echo $?
0

Do you know what could be causing this limitation? Is it intended behavior?

If I can provide any additional information for debugging, don’t hesitate to ask me.

To rule out the obvious, Haiku does not allow overcommiting memory by default. How big is your physical ram (+swap)?

I’m currently running Haiku in a VMWare Workstation VM with 16GB of RAM dedicated to it, I don’t have any swap space

I suspect that it is a restriction of old Hoard v1 allocator that is currently used for malloc/free implementation.

1 Like

That seems to be the case. I was able to build mimalloc: GitHub - microsoft/mimalloc: mimalloc is a compact general purpose allocator with excellent performance. and use it as a drop-in replacement for malloc(), realloc() and calloc() and the example snippets I’ve provided run correctly when using LD_PRELOAD. I’m also getting farther in building Zig thanks to it, I’m currently stuck on stage3 but it seems to be an issue with the compiler’s code.

2 Likes

The question for the core devs is, is it safe and reasonable to replace malloc() et al with mimalloc() given that support for haiku has already been added to it?, personally i had never heard mimalloc() before today.

I’ve never heard about it until today either (though it does seem to be used pretty extensively on Microsoft platforms) and I think it would definitely have to be tested intensively to make sure it’s safe. Additionally, support for Haiku was not functional in the stable build, I could only make it work by building the dev branch.

I don’t know what you expect from the core devs here. Try it and see if it works? There is nothing special or magic about a merory allocator.

The reason we are currentlyestaying with hoard is that a previous replacement attempt (with rpmalloc) did not go well. Rpmalloc had fragmentation problems, especially on 32bit systems.

There is no reason upfront that other allocators would have the same problems. The only way to gauge an allocator is to try it, see how it behaves, and decide if it’s good for our use or not. This is because the usage patterns matter a lot (do you do only a few large allocations? Thousands of small one? How long lived are they?), and so it is difficult to extract relevant information from existing benchmarks.

I’m not expecting anything in particular, I just wanted to know if there were anything I could do to progress in my attempts to build the Zig compiler that the people on this forum had any insight on. @X512’s suggestion that this was a limitation of the implementation used in Haiku was the necessary push for me to look if it were possible to use a different one. I didn’t know it was so trivially doable.

I’m sorry if I didn’t make my intentions clear or if it seemed I was demanding actions on your part or fundamental changes in Haiku, that’s not what I was trying to do. Maybe I miscategorized the thread.

It seems our messages crossed, I was replying to khallebal who wanted advice from the core developers.

Oh, my bad, then!

Well i don’t expect anything actually, it was just a question for the devs, because they’re the ones who make this kind of decisions, whether i try it or not won’t have any bearing on the decision making.

Wether we want to replace may be a discussion point, however you are free to test this, report on it, and create a review on gerrit. If it makes sense we can include it : )

3 Likes

According to the files in the repository, our release of hoard is 24 years old.
Maybe an update wouldn’t hurt.

The review on Gerrit to switch to mimalloc already exists…

https://review.haiku-os.org/c/haiku/+/6952

It probably needs to be updated, build for gcc2 fixed, and get some more testing.

But that wasn’t the point in this thread.

As Siph already wrote:

So, it’s already possible for individual applications to switch to mimalloc in this way without any changes to the OS. And the specific problem here is already solved.

Also, it is probably not the best time now to switch the memory allocator if we want to have a beta 5 release sometime soon. Let’s revisit that after the beta5 release :slight_smile:

4 Likes

It need some kernel changes such as address space reservation for better integration.

I guess the other question is, can hoard be fixed (or maybe it has already been fixed upstream?)

There are new versions of Hoard, but they work differently from the current one we use.

The version we use is old and designed for 32 bit systems. It reserves about 3GB of address space upfront and then fills it as needed. Newer versions (like lost other allocators) switched to allocating smaller areas progressively, which works better on 64bit systems and also leaves memory available for other uses. But there is also a risk that memory gets too fragmented and then large allocations start failing. Or, that applications pre-reserve a lot of memory that they don’t really need, and then other applications can’t allocate anything.

That’s why switching to a new allocator requires some testing on various configurations. And newer versions of Hoard are so much different than the current one that it’s the same effort as switching to something else.

3 Likes