The crate supports Protection::NONE, which implies that the area cannot be read from or written to. This causes a few tests to fail when they try to drop the area when it goes out of scope. This is because we use the area_for() function to find the area_id of an area, but Haiku is designed to not return the area_id for an area that cannot be read from or written to. See around line 3789 in src/system/kernel/vm/vm.cpp.
Four approaches to this:
You could try to argue that the behavior of area_for() is wrong, and that it should also be able to find areas that do not have read or write access. You would need to investigate the history though. This would be a Haiku change.
You could avoid area_for() in the free() function, and instead just iterate through all areas and try to match the address yourself. This is not super optimal, but it will work.
You could refactor the Allocation struct in alloc.rs to support platform-specific handles. This would store the area_id in haiku, and store nothing in other platforms.
You could just ignore the broken tests. area_for() is also used in the platform specific protect() function, so that will break too if the area is initially set up for Protection::NONE.
THANKS! That helps a lot! The Windows iterator just tries and finds out what is possible but the OS doesnāt supply an iterator function. I may be able to do something like that!
A second thing you may want to look at is the implementation of QueryIter::new(). The first test you do there is to use area_for() on the start address. This seems to be incorrect with how this method may be used, as the documentation for the query_range() function that calls this OS implementation explicitly allows to use ranges that are outside of the existing ranges.
Actually your whole implementation of QueryIter does not make sense. You need to write something that uses get_next_area_info() to loop through all the areas. The QueryIter::new() function should only set up the iterator, it is the QueryIter::next() that should return an area id, or return None if there are no more values within that address range.
I think option 3 is probably best, but option 2 will probably be easiest for now and allow you to go forward with testing the rest of the implementation.
If this is only for deallocation, I think you can call unmap instead of delete_area(area_for(...)) and it will work just as well. (Iām pretty sure we take advantage of this ourselves in some places.)
Thanks, but I found a spot in src/lib.rs that lets me change the dereference method of the AllocatedPages structure. All I had to do was exclude Haiku from the POSIX implementation and write custom Haiku code based on the Area commands.
Update
The query may work the same way. Iām ditching their wrapper and writing my own, I think.
After trying to replace the area_for() function with a static hash map as would be required by option 3, it seems this was a dead end. Rust, in all of its safety, refuses to create a static handle to a hash map structure. Even if the key and value members are all safe behind mutexes, Rust canāt do it anyway. Iām not sure option 2 would be much easier because it would still require some sort of memory handle. Iām thinking making a wrapper around the std::slice datatype in Rust might be easier at this point. It would still need to be a wrapper though, because std::slice also doesnāt support Protection::NONE.
Apart from the Rust compiler refusing to make a HashMap with static visibility, Iāve gotten the rest of the code in the Region crate to compile on Haiku. I canāt get the unit tests to run without that last bug solved though.
Update2
Using a third-party dependency, it is possible to do it.
Why is this written the way that it is? Rust is concurrent multithreaded and it needs to be told when to execute the static initializer so that it doesnāt run in multiple threads. Using a mutex isnāt enough. Tomorrow Iāll try out these solutions and hopefully get past this roadblock.
Now that Iāve added the lazy_static crate to the dependencies (duplicating the effort with it was not worth the time in the end) and copied the in-code documentation from alloc.rs, the code builds again. There are 11 succeeding unit-tests and 16 that fail again but now it only has 2 allocation problems and those may be related to failing queries or protection tests. Iām on the right track. Thanks, @nielx for getting me on the right track!
Upon closer examination 11 of the 16 failed unit-tests are giving the same error message: InvalidParameter(āCould not find any allocated pagesā) which suggests there may be a race condition in accessing the mutex to the hash map. Another possibility is that the allocations are failing silently for some reason. I always check the return codes from the Haiku subroutines so the former is looking to be more likely than the latter. If thatās the case it may be something as simple as adding a āfenceā to make sure the write to the hash map gets through the mutex before the read does.
Update
I tried locking the mutex at the start of every allocation but it made no difference. Itās possible that the create_area() function is not allocating Protection::NONE protected regions.
Problem Found
fix_protection() is incompatible with the region crate altogether. It is called by create_area() which is called by all user code.
How is fix_protection incompatible with Region? All it does is make sure the kernel has the same access to an area that userland does. How can this cause any problems for userland applications?
It appears to require that memory being allocated is at least readable. The unit tests of the Region crate need memory allocated with the no permission to read, write nor execute. It may just be a problem with the unit tests themselves in the long run.
WebKit does the exact same thing (allocating areas with PROT_NONE.) Thatās perfectly valid and it works just fine.
The kernel will give itself read permissions, yes, but these are a totally separate permissions system from the userland ones, and userland cannot see them at all.
Thanks! That eliminated the likeliest cause of failure. Now I have to find the actual cause.
It seems that the hash map I put in to replace the AreaFor() commands are reporting allocations as not found. Itās the only place that error could have been reported from. The allocations must have succeeded, I think.
It occurred to me this morning that Wasmer was written for Region crate version 2.x so if it isnāt using the 3.0 features Iām trying to fix, it might just already work. Some Wasmer crates already do.
Now Iām working on the actual source of Wasmer proper. Itās forked and Iām working on the vm. Iām down to the point where the Wasmer VM is allocating an alternate stack for the event handling. The Unix-like operating systems need 64k while the stack supplied by Windowsā default is big enough already and the code does nothing. Iāll assume the latter of Haiku for now and see if I can get it to run.
Update2
It built! Itās just the rust crate so far but Iāve made more progress with it today than I have for a month!
Speaking of moon-shots, I have a second interview for a position at Wasmer on Monday. WIsh me well. (Considering all the examples I sent are incomplete snippets and Rust crates, I may need some well-wishing and divine guidance.)
