My progress on WebKit2 port

I’ve upvoted the ff-sync one and created a ticket for the vertical tabs.

https://dev.haiku-os.org/ticket/17471

4 Likes

I have to congratulate the involved developers for their progress with Web+ over the last 6 months. Stability, performance and compatibility have moved foward in leaps and bounds - I have gone from mostly using other browsers to mostly using Web+. I think this progress indicates the Haiku dev community’s capacity to achieve their goals - a 1st-class, robust, feature-rich Haiku-first, Haiku-native browser is possible. We have made it a lot of the way there. Getting there will make Haiku much more compelling for everyday users.

18 Likes

Oh wow, did I not touch this for over a year?

Anyway, I have rebased the haiku-webkit2 branch over the current version of WebKit. It is still as broken as it was last year, that is, it compiles, but the connection between the different processes does not establish.

However, the factorized code I was talking about earlier has now been made a bit more flexible and I think it should be possible to use it now. We’ll have to find a way to pass a BMessenger to another app through a command-line argument, but if nothing else works, we can always serialize the BMessenger into a BMessage, flatten that, and then convert it to base64? That’s the worst case solution, there are probably less stupid ways to do this.

But first, I have two more preparation tasks to do:

  • I have switched to a 64bit install now, which means I can’t use the old BeDC “developer console” from BeOS for debugging. I have to write a replacement for it that will run on 64bit systems.
  • The haiku-webkit2 branch has more than a hundred small commits with build fixes I had to make accross multiple git rebase steps, I want to clean that up and extract fixes that can already be merged into the main Haiku branch. I hope to have all these things merged in the main branch but I want to make sure they don’t break the currently in use WebKitLegacy, and also it’s convenient to keep the “work in progress” parts at the top of the branch for future reference.
25 Likes

Can you serialize the BMessenger and store it in an Area, and then pass the area id on the cmdline?

I rewrote a new implementation of it, named DevConsole, you can get it here for now: devconsole - Gitiles

And I wrote a blogpost about it on my personal blog.

I am now at under 80 commits. Some things got squashed, some things got removed because they were wrong, and some were merged in the main branch already.

To get there, unfortunately I ran into some issues with gitk, which is ported to Haiku currently using undroidwish, a Tk implementation over SDL. Waddlesplash wasn’t able to get Tk to work completely with Xlibe. This would be OK, except it exposed two regressions in our SDL2 port. I fixed one (the handling of the current directory when starting apps was not correct) and wored around the other (resizing the window crashes, but fortunately undroidwish allows to specify a window size on the command line).

The next steps are continuing a bit with this commit cleanup/squashing effort and integrating the “done” parts in the main branch, so I can more clearly find and study the “work in progress” parts and resume the work on getting the interprocess communication going. Of course by the time I’m done, there will be several more months of work in the main WebKit branch that I will have to integrate as well. And I guess at some point I should start on trying to upstream all these things to WebKit, as well…

25 Likes

Oh yeah! I’ll be using your devconsole a lot, so thanks for that one @PulkoMandy!

Edit: for those wondering how to clone that repo anonymously (couldn’t sign-in with oauth):

git clone https://pulkomandy.tk/gerrit/devconsole

worked for me.

1 Like

Can this repo be cloned via https as well?

Looks like a really useful app, I´m looking forward to test it. Comes in especially handy when debugging an application that redirects stdout/stderr and reads from there.

EDIT: Just saw that my question has been answered in the post above :slight_smile:

I have done enough cleanup and rebasing for now. A few more things were integrated in the main branch and I’m now at 50-60 commits, most of which I am not sure are correct and still quite experimental or outright broken code. I will continue cleaning up after working on it a bit, since there’s no point in cleaning up code that will eventually be rewritten or deleted.

This week I am reworking the log system so that all logs get sent to BeDC/DevConsole. Previously only the “notImplemented()” function would log there, and all the other logs were disabled, not giving me much info about what was happening.

Now the logs are controlled as intended in WebKit, by default they are completely disabled, you can build a version of WebKit with logs enabled and then from there you can configure which logs you want using an environment variable. I have to adjust the process launching to forward the environment variable to the web and network processes which are started in the background using BRoster::Launch (I think the environment is not forwarded in that case).

After all this preparation work I should be able to start debugging the initialization process and make sure all the processes can talk to each other…

33 Likes

This weekend progress:

  • The cleanup of existing commits continue, I am down to less than 40 commits now which is almost manageable. There are still some things in the commit history that are added and then removed again in later commits, so I will continue cleaning this up. It really helps to have a cleaned up history so that I can see what was changed and is still considered work in progress easily.
  • The log system with BeDC is working well now
  • The network and web processes (the background apps that will do the actual work of downloading and rendering web pages) are starting but not doing anything. This isn’t surprising as I had to temporarily delete a bit of the code there, because it was very hacky, and the hacks are not needed anymore in newer WebKit versions (or at least, I hope so). So, I did a bit of investigation to why they are not doing anything (I expected they would at least complain that some methods are not implemented). I found and fixed some issues with the “RunLoop” management. In the Haiku port this is basically a BLooper, but the typical all-in-one creation of a BLooper does not work well, because WebKit wants to spawn a thread by itself, and then use that thread as a run loop. I had already patched BLooper to be used in this way, but there are some special case, in particular, for running in the main thread, we should create a BApplication instead of just a BLooper. So I got that working. Now the WebProcess and NetworkProcess are both BApplications. The next step should be that they receive some messages from the UI process (the web browser) asking them to load and render some page. But they don’t seem to receive anything for now. That’s the next thing to investigate (again, it is not surprising because a lot of the code needed for this was removed or commented out during the rebase of the changes on an up to date version of webkit, and needs to be rebuilt).
32 Likes

It is possible to tell BLooper to run in existing thread.

Yes, I know, I added BLooper::Loop() to BLooper to allow it a few years ago because it was needed in WebKit.

Finally making some progress!

I spent some time re-designing the code for inter process communication and today I managed to get it running again. The WebProcess and NetworkProcess are started and they start sending messages to and from the UIProcess.

One of the first few messages makes things crash, however. Tomorrow I will clean up all debug printf and puts I had tp add everywhere to follow how things worked.

The code is not as clean and simple as I’d like it to be for now, so I may do another pass of re-architecturing it now that I understand how things work.

Things I have done to get it working:

  • Create a temporary BHandler on the UIProcess
  • Create a BMessenger targetting it
  • Serialize that to an hex string I can pass as a command line argument
  • Rebuild a BMessenger in the Web or NetworkProcess from that argument
  • Create a BHandler on the other side
  • Use the BMessenger to send a message containing a BMessenger targetting the BHandler we just created
  • On the UI process side: “finalize” the connection which creates yet another BHandler, and delete the initial temporary one (this is the step I’d like to delete)
  • Send a message back to the other process to tell it where the new handler is
  • Finally, the connection is established and we can start sending messages back and forth

That’s it for today. Tomorrow I’ll fix the crash and see how far I can get things to run :slight_smile:

46 Likes

Kudos for all your work PulkoMandy👌🏼

2 Likes

Nice Work!. Please keep us updated!

Great process, nice to see you working on the WebKit2 port again.
Who else is involved and helping you?

What would we do without your programming skills?

In general @madmax and me also contribute to the webkit port (I haven’t lately, but plan to do so again soonish)

5 Likes

To be fair, I’ve only sent a few small patches, none of them targeting WebKit2 specifically. I think a quick look at Commits · haiku/haikuwebkit · GitHub shows very clearly whom to thank for work in that branch.

2 Likes

Today I have released HaikuWebKit 1.9.5. This is still on the WebKitLegacy branch. There were some problems because the windows version of WebKitLoegacy was removed by Apple, and with it they removed support for Curl in WebKitLegacy, so I had to undo that.

There will be some regressions in this version due to all these changes. I have already noticed that scrollbars are not rendering anymore, and websocket seem to use a lot of CPU while doing nothing. Your help is welcome in testing and finding all the problems, and over the week and next weekend I will try to fix some of them and make another release.

30 Likes

Things fixed:

  • Missing scrollbars, I accidentally deleted a function returning the scrollbar size, and so we got 0 pixel scrollbars as that’s the default.
  • Found a bug in WebKit core while attempting to run the testsuite, related to shrinking the favicon and cookie databases when things get deleted from them, the patch is being reviewed upstream. Interestingly this had been fixed in 2021, and then broken again in late 2022. Anyway, it was a simple fix, and I noticed it by chance because we have an error message in case this function fails, where other WebKit ports don’t.
  • Fixed missing code for drawing rounded rectangles, which led to a lot of drawing problems (square avatars instead of rounded on this forum, pretty much all widgets and boxes on Trac, …). For WebKit, everything is a rounded rect!
  • Fixed a missing refresh of the display when the selection is changed, so selected text would not be visible (the WebKitLegacy version for Mac/iOS had the same problem, so once I found that I just had to copy their code in the right place in our port)

I will wait a day more in case there are any more problems, but since these are quite bad overall, I will do a new release tomorrow if no one finds anything more.

34 Likes

Those two issues are unrelated to the latest release.

3 Likes