What else is needed to upstream Haiku support into .NET, if that’s on the table?
1 Like
I’m doing some tests with the Layout Builder and have come across an issue with the SplitView.
I have this code in C#
fStringView = new BStringView("stringView", "BStringView");
fButton = new BButton("BButton", new BMessage(kMsgButtonClick));
BOutlineListView outline = new("");
BSplitView split = new BSplitView(Orientation.Horizontal);
split.AddChild(fStringView);
split.AddChild(fButton);
BGroupLayout group = new(Orientation.Vertical);
SetLayout(group);
group.Owner().AdoptSystemColors();
group.AddView(split);
group.AddView(outline);
It is roughly equivalent to the following C++ code (which works):
auto fStringView = new BStringView("stringView", "BStringView");
auto fButton = new BButton("BButton", new BMessage(kMsgButtonClick));
auto outline = new BOutlineListView("");
auto split = new BSplitView(B_HORIZONTAL);
split->AddChild(fStringView);
split->AddChild(fButton);
auto group = new BGroupLayout(B_VERTICAL);
SetLayout(group);
group->Owner()->AdoptSystemColors();
group->AddView(split);
group->AddView(outline);
The application crashes with a segment violation:
thread 7902: w>Main Window
state: Exception (Segment violation)
Frame IP Function Name
-----------------------------------------------
0x7ff9823d9c70 0x16cd78f3a5e BAbstractLayout::IsVisible() + 0xe
Disassembly:
BAbstractLayout::IsVisible():
0x0000016cd78f3a50: 55 push %rbp
0x0000016cd78f3a51: 4889e5 mov %rsp, %rbp
0x0000016cd78f3a54: 4154 push %r12
0x0000016cd78f3a56: 53 push %rbx
0x0000016cd78f3a57: 488b9fd0000000 mov 0xd0(%rdi), %rbx
0x0000016cd78f3a5e: 488b03 mov (%rbx), %rax <--
Frame memory:
[0x7ff9823d9c50] ...j......)j.... 00 da 04 6a 0d 12 00 00 00 f4 29 6a 0d 12 00 00
[0x7ff9823d9c60] ..=.....P...l... a0 9c 3d 82 f9 7f 00 00 50 a5 91 d7 6c 01 00 00
0x7ff9823d9cb0 0x16cd791a54d BGroupLayout::PrepareItems(orientation) + 0x5d
0x7ff9823d9ce0 0x16cd7984d6b BTwoDimensionalLayout::CompoundLayouter::_PrepareItems() + 0x3b
0x7ff9823d9d10 0x16cd79851e2 BTwoDimensionalLayout::CompoundLayouter::ValidateMinMax() + 0x32
0x7ff9823d9d30 0x16cd7985a19 BTwoDimensionalLayout::LocalLayouter::ValidateMinMax() + 0x49
0x7ff9823d9dd0 0x16cd7985c68 BTwoDimensionalLayout::DoLayout() + 0x18
0x7ff9823d9e30 0x16cd792ec96 BLayout::_LayoutWithinContext(bool, BLayoutContext*) + 0x76
0x7ff9823d9e70 0x16cd798bccd BView::_Layout(bool, BLayoutContext*) + 0x9d
0x7ff9823d9ee0 0x16cd798bdf7 BView::Layout(bool) + 0x27
0x7ff9823d9f50 0x1d2d2b9054e ?
0x7ff9823d9fe0 0x1d2d2b904b7 ?
0x7ff9823da010 0x1d2d2b90378 ?
0x7ff9823da050 0x1d2d2b8fbe8 ?
0x7ff9823da090 0x1d2d2b8fb34 ?
0x7ff9823da130 0x16cd7999ac3 BWindow::task_looper() + 0x1d3
0x7ff9823da150 0x16cd78dafdb BLooper::_task0_(void*) + 0x1b
0x7ff9823da170 0x107f0183dc7 thread_entry + 0x17
00000000 0x7fc871a7e258 commpage_thread_exit + 0
Am I doing something wrong or is it a bug in the Haiku API binding?
My question is either for @trungnt2910 or one of the Haiku devs (maybe @PulkoMandy or @waddlesplash may help?).
Like I have said somewhere before, patches are split into branches:
For repositories with more than one patch applied, such as
trungnt2910/dotnet-runtime, each patch is kept in a separate branch,dev/trungnt2910/{feature_name}. Currently, the active branches are:
dev/trungnt2910/haiku-config: Tracking unmerged pull request #86391.dev/trungnt2910/haiku-pal: Native (C++) support for the runtime on Haiku.dev/trungnt2910/haiku-lib: Managed (C#) support for system libraries on Haiku.This separation of branches makes opening and keeping track of pull requests easier.
#86391 has been merged after that. Now, we’re tracking dev/trungnt2910/haiku-pal at #93907.
However, to get any progress in that pull request, we need to fix the failing CI builds. Currently, some Linux-specific crypto tests are failing, and I have no idea why since I haven’t touched any crypto code.
I’m having a feeling that this has something to do with garbage collection.
It might have been the case that group has been passed and stored in some native code yet deleted by the managed runtime after being out of scope for a while.
Can you confirm it by creating a class that inherits from BGroupLayout and overriding the finalizer:
public class BTestLayout : BGroupLayout
{
~BTestLayout() { Console.WriteLine("Finalized"); }
}
(I haven’t tested this code yet so it might have a few compile errors, but the idea should be the same).
I’ve subclassed both BGroupLayout and BSplitView but their finalizers never get called.
I’ve also tried to set GC.KeepAlive() for the relevant objects to no avail so I tend to exclude a garbage collection problem.
AFAIK from .NET 5, finalizers don’t get called at the exit of the program but if I call GC.Collect() tha app crashes due to an invalid instance because the layout objects retain ownership of their children. This may cause the opposite problem that is the resources are not finalized until the very end, maybe.
Please notice that tests on other Layout API objects have succeeded, the only one causing troubles is BSplitView.
I’m instead inclined to say that it must be declared or used in a different way, maybe?
With native resources, you really need to keep a reference to the native handle around. Different platforms handle this differently - for example, I worked on a project where we integrated native code in to a Xamarin Forms project, and when we were bringing it up, the app would crash because we passed in a pointer to a managed callback and the garbage collector was collecting the memory, even though the C interface still had a reference to the function:
int ACallBack(int input) => return input;
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
delegate int ACallBackDelegate(int input);
// in an actual method
void Main()
{
native = new NativeWrapper(new ACallBackDelegate(ACallBack)); // this will lose the pointer
}
Where as:
int ACallBack(int input) => return input;
ACallBackDelegate callbackReference = null;
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
delegate int ACallBackDelegate(int input);
// in an actual method
void Main()
{
callbackReference = ACallBack;
native = new NativeWrapper(callbackReference); // this will not lose the pointer
}
Looking at the code, it doesn’t look like you definitely need to create an instance of the delegate, I believe that is done by assigning it to the field. But, yeah - native to managed boundaries are weird. The same code worked fine under Windows .Net 4.6, but not under Linux Mono (Android).
I’m instead inclined to say that it must be declared or used in a different way, maybe?
To be honest, I don’t have much experience with the Haiku UI API. I work directly more with the lower-level parts of the OS.
Without a closer analysis of the crash dump, I don’t really know what’s going on. Is the pointer null? Is it invalid? Or does it hold an object of the wrong type?
Crashing while dereferencing rdi in a C++ method mostly means an invalid this pointer, but who knows what else is going on.
As long as I keep a reference with a static field I do not experience any dump by the GC. For example:
private static BMenuBar? menuBar;
Whatever combination I use, it all falls short when I use the BSplitView.
The following code works ok until I attempt to add the split view:
fSplit = new BSplitLayoutBuilder(Orientation.Horizontal);
// fSplit.AddChild(leftGroup);
// fSplit.AddChild(rightGroup);
mainGroup = new BGroupLayoutBuilder(Orientation.Vertical);
SetLayout(mainGroup);
mainGroup.RootLayout().Owner().AdoptSystemColors();
mainGroup
.AddGroup(Orientation.Vertical)
.Add(menuBar)
.End()
// .Add(fSplit)
.AddGroup(Orientation.Horizontal)
.AddGroup(Orientation.Vertical)
.Add(fOutline)
.End()
.AddGroup(Orientation.Vertical)
.Add(fStringView)
.Add(fButton)
.End()
.End()
.AddGlue()
.End();
1 Like
It’s hard to say, I would probably need a debug version of libbe.so which I am not able to build (my Haiku source does not build anymore since a few weeks and I don’t know why).
I’ll keep looking into it.
I wrote one of the first BeAPI wrappers for object pascal and I remember there were all sorts of oddities with the BeAPI about creating UI elements in the right thread/team. I forget which control, but maybe the text input stuff? The newer auto layout APIs didn’t exist so I have no experience with them.
Another part that might be complex is this, quoted from my 4th progress report:
BLooper problem
BLooperobjectsdeletethemselves afterQuit()is called. When owning the object memory, the managed instance should know about this and dispose of the native pointer to prevent a doublefree().To solve this problem, two lines of code has been injected into
_QuitDelegateHook. This is the function whose address will be stored directly in thevtable. It acts as a trampoline between native C++ code and the managedBLooper.Quit()method:
private static void _QuitDelegateHook(__IntPtr __instance)
{
var __target = global::Haiku.App.BLooper.__GetInstance(__instance);
__target.Quit();
__target.__ownsNativeInstance = false;
__target.Dispose(false, callNativeDtor: false );
}
When C++ code calls
BLooper::Quit(), the function above will forward the call to managedHaiku.App.BLooper.Quit(). After that, the managed looper will renounce its ownership of the pointer and callDispose().Note that this only affects calls from the C++ side. Calling the
Haiku.App.BLooper.Quit()function from .NET does not dispose the looper. Managed callers should therefore make sure thatDispose()is called right afterQuit().Also note that attempting to use a managed
BLooperafter it has quitted would result in a segmentation fault due to aNULLpointer access, not a managed, catchable exception. This is because CppSharp does not handle wrapper objects withNULLpointers. Every wrapped function calls P/Invokes native C++ functions without conducting any sanity checks on thethispointer. The problem is therefore notBLooper-specific. This may be fixed (by me, if needed) in a future CppSharp version.
Looking back at the bindings, I seem to not have added anything to prevent the managed runtime from deleting the native BLooper instance before quitting. Don’t know if it’s causing your problem though; the docs do not explicitly mention anything against delete-ing a BLooper.
Wrongly generating interop code for BLooper can mess up a lot of things for the C# application, since it is the base class for many Haiku UI elements.
My problem is not caused by the garbage collector which causes other effects. These can be witnessed for example when the BMenuItems attached to a menu suddenly disappear without further notice.
As said before, to avoid this all the UI objects must be kept in static fields otherwise the GC will sweep them out. The layout system retains ownership of these instances but only at the unmanaged pointer level, all managed objects are susceptible to garbage collection making the pointer invalid anyway.
The problem with the BSplitView is maybe due to how it is used and declared and to the use of BGroupLayoutBuilder which is deprecated in favour of the templates in the BLayoutBuilder namespace. These cannot be exported by CppSharp, I think and they use private APIs anyway.
I would suggest at this stage that we export all the APIs available including the private ones or at least those that can be handled by CppSharp.
This way we will have an API ecosystem roughly equivalent to that available from C++ and we could make up for the missing templates by recreating them in C#, the C# way…
I managed to reproduce the same code both in C++ and C# and the BSplitView finally gets addedd to the Group layout . It shows the splitter but does not show any control/group either in C++ or in C#. This why I think the problem is with the use of these classes and not the GC.
This will likely crash the application and a BLooper should never be deleted directly as it deletes itself when calling Quit() (except a BApplication).
From time to time, I have experienced some crash which might be caused by the BLooper being prematurely disposed. Not 100% sure about that, though.
However, I think that the BLooper must be preserved from being disposed by the GC.
Just to keep you updated, I think I’ve figured out why I experienced the issue with the BSplitView and my guess was correct: it was a problem with how the split view was constructed and used.
Having said that, I’m working on reimplementing the BLayoutBuilder classes.
So far so good, except a few issues with the Garbage Collector which randomly nukes some instances…
As it stands now, I’m trying to replicate the behaviour of the corresponding C++ classes in LayoutBuilder.h but I plan to make them more “C#-ish”.
Here’s an example:
(mainGroupBuilder = new Group<RootBuilder>(this, Orientation.Vertical, 0))
.Add(fMenuBar)
// .AddSplit(B_HORIZONTAL, B_USE_SMALL_SPACING)
.AddGroup(Orientation.Horizontal)
.AddGroup(Orientation.Vertical, B_USE_SMALL_SPACING)
.Add(fStringView)
.Add(fButton)
.AddGlue()
.End()
.AddGroup(Orientation.Vertical)
.Add(fOutline)
.End()
.SetInsets(B_USE_SMALL_INSETS)
.End();
One of the many annoying things is that C# does not allow for unbound generic types like C++. SO I had to create my own “void” parent type like this:
using System.Runtime.InteropServices;
namespace Haiku.Interface.LayoutBuilder;
[StructLayout(LayoutKind.Sequential, Size = 0)]
public struct RootBuilder { }
Stay tuned…
7 Likes
I’ve created a complete rewrite of the BLayoutBuilder classes in C# here:
The classes mirror almost 100% the builders available from the C++ world (LayoutBuilder.h), more info in the README.md file.
fMainGroupBuilder = new Group(this, Orientation.Vertical, B_USE_SMALL_SPACING);
fMainGroupBuilder
.Add(fMenuBar)
.AddSplit(Orientation.Horizontal)
.AddGroup(Orientation.Vertical)
.Add(fOutlineScroll)
.End()
.AddGroup(Orientation.Vertical)
.AddCards()
.GetLayout(out fCardLayout)
.AddGroup(Orientation.Vertical)
.Add(fStringView)
.End()
.AddGroup(Orientation.Vertical)
.Add(fButton)
.End()
.SetVisibleItem(0)
.End()
.AddGlue()
.End()
.End()
.SetInsets(B_USE_SMALL_INSETS)
.End();
Menus are supported, too:
fMenuBar = new BMenuBar("menu");
fMenuBuilder = new Menu(fMenuBar);
fMenuBuilder
.AddMenu("Menu1")
.AddItem("Item1", fMessages[kMsgShowCard1])
.AddItem("Item2", fMessages[kMsgShowCard2])
.End()
.AddMenu("Menu2")
.AddItem("Item3", new BMessage())
.AddItem("Item4", new BMessage())
.End()
.End();
Please let me know if you find bugs or issues and if you have any suggestion and/or improvement.
9 Likes
After a while I’m experimenting with the .NET bindings, I sadly have to say that they are not usable as they are today.
The problem is the Garbage Collector which wipes out all the instances randomly. The reason is well known, the bindings do not retain any reference to the instances passed as parameters or constructed inside the classes.
Let’s take two examples: BMessages and BOutlineListView.
For BControls that accept an invocation message (e.g. BButton and BMenuItem) the message must have a reference retained in the calling class.
Same for a BOutlineListView where all the BStringItems must be retained in the caller.
@trungnt2910 I don’t have visibility of the code generated for Haiku.dll and HaikuGlue.a as I can’t generate them by myself. Would you be able to send it over?
Anyway, it seems that the bindings do not use GCHandle.Alloc() to properly retain the reference to the managed instance across the managed and unmanaged boundaries or these are freed prematurely.
The bindings should follow the behaviour of the wrapped classes. Most of the times they retain ownership of the instances (for example BoutlineListView retains ownership of the BStringItems), sometimes they do not. So the managed classes should do the same.
There are two possible routes here.
One is to let everything as is and create another layer of classes that wraps the managed bindings and implement the correct retaining strategy. I’m doing this regardless to implement the Async Programming and the event handler/delegate patterns. One example is as follows:
fAddButton = new CButton("Add item");
fAddButton.Click += new(AddButton_Clicked);
...
public async void AddButton_Clicked(object sender, EventArgs e)
{
Console.WriteLine("Add Button pressed!");
await Task.Run(() => {
LockLooper();
fStringView?.SetText("Add Button pressed!");
// string str = "item" + (fStringItems.Count()+1).ToString();
BStringItem item = new("str", 0, true);
// fStringItems.Add(item);
fOutline.AddUnder(item, fOutline.ItemAt(fOutline.CurrentSelection()));
UnlockLooper();
});
}
So I can definetely add the necessary boilerplate code to retain messages, BStringItems, etc.
Funny but not so manegeable in the long run. I have to say thet the Haiku API is quite stable and does not change much often so it’s still feasible.
Another route is to enforce the use of GCHandle on a case by case basis depending on whether the API retains ownership or not and freeing the resources appropriately.
Any thoughts?
HaikuGlue.a is a library that contains a bunch of functions that should have been inlined. It looks like this (the file containing the AppKit):
#define protected public
#include <AppDefs.h>
#include <Application.h>
#include <Clipboard.h>
#include <Handler.h>
#include <Invoker.h>
#include <Looper.h>
#include <Message.h>
#include <MessageFilter.h>
#include <MessageQueue.h>
#include <MessageRunner.h>
#include <Messenger.h>
#include <PropertyInfo.h>
#include <Roster.h>
#include <new>
namespace Haiku_App_symbols
{
struct compound_type::field_pair& (compound_type::field_pair::*_0)(struct compound_type::field_pair&&) = &compound_type::field_pair::operator=;
struct compound_type& (compound_type::*_1)(struct compound_type&&) = &compound_type::operator=;
struct property_info& (property_info::*_2)(struct property_info&&) = &property_info::operator=;
struct value_info& (value_info::*_3)(struct value_info&&) = &value_info::operator=;
extern "C" void app_info_app_info___1__S_app_info(void* __instance, const app_info& _0) { ::new (__instance) app_info(_0); }
struct app_info& (app_info::*_4)(const struct app_info&) = &app_info::operator=;
extern "C" void BRoster_BRoster___1__S_BRoster(void* __instance, const BRoster& _0) { ::new (__instance) BRoster(_0); }
class BRoster& (BRoster::*_5)(const class BRoster&) = &BRoster::operator=;
}
As for the code, I have a somewhat outdated copy here. Newer builds are generated wholly on GitHub Actions but should not have significant differences in the object lifetime. To explore the exact code, you can always use dnSpy or a similar .NET decompiler.
Since we’ve already gotten the API surface in the binaries, we can use Roslyn code generators that take the old binaries as one input, and an XML declaration of APIs that should reference/dereference Haiku objects as the other. Then, the new wrapper can properly pin and unpin objects using GCHandle.Alloc() and related APIs. This method can also replace the hacky regexes used to deal with BLoopers self-destroying as well.
That said, such a solution is a lot of work and is itself a project that can take a couple of months.
Yes. From my experience with other language bindings, though, I think you have a pretty good start on this problem - there aren’t an infinite number of such cases. Is the problem just tracking them down, or is there a problem dealing with the ones you know about?
I’m not asking for a detailed explanation, just wondering how close .NET is to stuff I’ve been doing for my amusement. Some day I’d love to see the headers/os/ include files generated from a more complete function data set that has information like this that C++ doesn’t care about. Such a data set could generate minimally adequate documentation as well.
I’ve just put the Layout stuff aside, for my purposes, thinking it will be better implemented directly in a hgher level language, than called as a foreign interface from that higher level language.
The root cause is known as well as the solution(s).
There are two typical cases:
- BMessage
- Other data that the Haiku API retain ownership of (a list of items for example or the BView passed to the layout builder)
I do love C# and .NET but the Haiku API is clearly C++ centric. Simple bindings like the ones we have now are not sufficient or even suitable but they are a solid foundation to build something that is more idiomatic in C# and fits better with the CLR.
I’ve already created two subclasses of BButton and BOutlineListView by implement event handlers, delegates and the async programming pattern plus the logic to retain a reference of the managed objects (BMessage and BStringItem).
Although it would be better to generate these automatically, I think I will add classes to the namespace as I need them.
I’m not sure I have understood. Could you elaborate?
Yes and no. My esperiment with the layout builder is successful, it works as it leverages the underlying layout API by retaining a reference to the managed objects.
I don’t think we should delegate the entire layout system to another language. Haiku API is functional yet not perfect but it serves the purpose well. You will need to reimplement things like a fluent builder because the target language has its own way to deal with that.
An include file like, say, interface/Menu.h, prescribes a set of constraints relative to the defined functions. If it puts “const” before a function parameter, then C++ will object if calling code tries to modify that area of memory. C++ doesn’t care, though, what you do with storage allocated to the BMenuItem you add to the Menu - that’s your problem, not the compiler’s - so it has no way to express the storage semantics. The Rust include file, for example, will express another set of constraints that Rust enforces.
There really aren’t all that many issues involved, but the source we’re working from - headers/os/interface/Menu.h - is C++, so it’s missing a lot of them. If the original source were a machine readable file where the parameters for BMenu accounted for the needs of Rust and maybe one or two other languages (maybe Rust would suffice for everything, I don’t remember but it enforces a lot!), then the C++ include files could be generated from that. In whatever format pleases the core developers. And Rust, Python, C#, Haskell, Ocaml, what have you, can generate competent interfaces from that data as well.
It may not make sense for C#, but Layout has a lot of “new C++” scaffolding that solves problems that other languages solve in very different ways. The basic API - the application and view classes etc. - is what you really need, and as it’s old C++ it’s more trivially adaptable to high level foreign interfaces.
I’m not familiar with Rust but I assume that similar problems may arise with other languages (Swift, for example).
This is more or less what @trungnt2910 suggested by enriching the API definition with details that ease the job of an automatic generator.
I don’t know if there’s anything similar around that can be reused but it’s not a trivial task.
The Haiku API is clearly C++ centric and it wasn’t designed with the purpose of being available to foreign languages. Moreover, each language have their own peculiarities that it would be very difficult to take into account in an automatic way that fits them all.
IMHO anyone who is interested in using a certain language should take care of shaping a customized API which leverages a common set of “basic” bindings.
This is the route I’m taking by (manually) creating wrappers around the existing bindings to cope with GC and implementing idiomatic constructs, patterns and techniques in C#.
This requires some knowledge of the three key elements: C++, the Haiku API and C#. And I’m not a champion in any category 
1 Like