The current state and the future of Python on Haiku

It isn’t trivial. Some of the key parts of my “ghc knowledge” are just a collection of things I’ve stumbled across and use in a kind of trial-and-error fashion without really understanding, and dynamic loading platform support sure falls in that category.

And the builds routinely crash. Less often after I increased the RAM on my build host. My suspicion has always been that GHC exposes some problem in Haiku’s mmap implementation, but I can’t imagine how to track that down. (GHC has its own memory allocation pools, using mmap to get the memory from the system, and the amount of memory allocated in this manner can be rather large. I’ve never had this fail in my own applications, though.) Anyway, this is one of the little things that could make it difficult to get into the depot system - more complicated to run automatic builds. Then there’s the circular dependency issue.

That’s was never the goal of BeOS either of Haiku… That’s the way you need to go for innovate… and doing the things from scratch is the rigth thing to do, that include custom kernel and custom API if needed… If you dont belive me go and ask Google with their Fucshia OS project… even having two well mature OS products like Android and Chrome OS… KEEP YOUR EYES WIDE SHUT and stay away from this hard working and small community of Devs that they have had the constancy in principles, fidelity of purpose and the humility of so many years of voluntary work to fulfill the true objective raised since the creation of OpenBeOS. Haiku is not a Hobby OS, it is a piece of art in restoration and recreation of the ideals that BeOS defended and innovated in its time…!

3 Likes

I should probably give some info about the state of GHC since I was involved at least in packaging it.

My goal was indeed to port pandoc (but eventually I found a way to not need it, for now…).

I worked from jessicah’s effort to port GHC. She first cross compiled it from Linux, and then I wrote a recipe to package her cross compiled version. It should be usable to build more things, but I’m yet to manage to write recipes for cabal and the other often used packages. So right now you get just the compiler. Probably updating that to a newer version would not be a bad idea.

This use of mmap is known to be problematic when using ASLR on 32bit machines (we quickly run into address space limitations then). Disabling ASLR may help then (IIRC it can be done by setting some xattr on the executables?)

This is an interesting theory, how cloning a 20 years ago OS with ABI compatibility is going to be the way for innovate?

Yeah, in fact Fuchsia project will take lots of parts from android, libstagefreight and a whole lot of other things.

Bah. I really question myself why I keep discussing with this kind of people. I spent countless hours of my life to support this project. I spent countless nights of sleep to support the beta. I am pretty sure I spent much more time for the beta than you, so, eventually, you’d have to keep your eyes wide shut not me. I am proud of what I do.

However now that beta is out I promise to never get again into this kind of discussions, as you can see from the commit log I already begun innovating.

1 Like

Not a mystery, the recipe points at someone :thinking:

IIRC, the builds try to get cabal compiled (bootstrapped?) too. May be wrong. Also, all is done with cabal in current versions, so it’s a one way only, get cabal running.

Didnt think about using cross compilation from a GHC in Linux tho, i try to do the “all in Haiku, with Haiku tools” way for now…:thinking:

ok, u know, i don’t approve your way of discussing :smiley: but this doesn’t mean i’m not going to read and discuss what you write (have u tried with some chamomile? :stuck_out_tongue:)

I think the truth is almost in the middle. Haiku carries out a very strong heritage and has a very idealistic heart. This is a good thing and it must be preserved because makes cohesion among the community members. Think for example how similar projects (in essence) has failed on this and look at their status. AROS is the first coming in mind, but even ReactOS suffers this lacks of ideals (in facts, they advertise it as “nag free, open source alternative to NT”. Which is good maybe, but it’s not perfect)

(I’m not saying they’re abandonware. I’m just saying they’re cold).

But on top on it innovation could come. I’m not involved into the project so deeply to judge if binary compatibility and drop-in intended usage were good decisions. Probably they were. Shall we have proof of that? How many users in % uses closed-source binary app from the BeOS era?

The real point here: where to innovate first? I mean…innovate is much more difficult than replicate…

2 Likes

I think the premise of your question is incorrect. It assumes Haiku cannot be both useful for legacy compatibility and innovative in building new features and services on top of this base.

Personally, I won’t be spending any time worrying about 32 bit support or BeOS app compatibility in what I do in the OS day to day. (Although I’m sure people will want the Paladin IDE to be able to build 32 bit BeOS compatible apps…)

Whatever we add though will have a maintenance overhead so we need to be sure it’s worth the effort. Certainly ensuring existing groups’ efforts work well together rather than duplicate functionality is a good thing (Think Koder and Paladin IDE here - I have no real interest in adding YetAnotherCodeEditor to Paladin).

I’m sure there are a group of functions that we could add to an API on Haiku to benefit all app developers and users of those apps. As an example, someone mentioned adding scripting support to apps for Hey. Well, is there a way of making that much easier so it’s “on by default”? (E.g. automatically export all Menu Items as scriptable integration points, or similar for BMessages being sent to apps)

Replicant samples and recipes may also be a good way to expose value-add services to users without a lot of additional code bloat and associated maintenance cost.

Building all this in to the Paladin IDE so an app developer just has to write purely unique code for their app would be good. So I write a clock and notification app, but automatically I have replicants for a clock and upcoming events, and hey scripted commands for adding/listing/getting/removing events.

2 Likes

That is unlikely, but 32 bit Haiku apps (built with modern GCC, and working on Haiku, not on BeOS) — definitely.

2 Likes

Define your protocol for remote controlling an editor, and people writing text editors will have to implement it if they want their app to be taken seriously. Probably other apps will make use of the feature and it will become a standard thing expected from any text editor.

This is how we build an Haiku ecosystem, and also why it is hard to fit ported applications in it.

There already is an “on by default” export of all controls of every window for scripting. But it is a pain to use. Scripting an app by locating windows, buttons and menus is not the best way to go about it. The point of scripting, ideally, would be that you get direct access to the application internals and “business logic” and manipulate this.

For example, you should be able to say “hey text editor, open file.cpp and put a red, wavy underline at line 100, colums 20 to 35. And scroll the view so that part is visible to the user”. You should not be doing this by invoking menus programatically (in fact there may even not be a menu at all to do that).

Likewise, “hey web browser, open a window at (100, 100, 600, 400), hide the navigation controls and show just the manpage for strcmp”. And later on “hey web browser, close that window you opened earlier”.

Note how I used generic “text editor” and “web browser” in these examples. The idea is you would find the preferred app for sourcecode (resp. URLs) and send it generic commands without worrying about how its menus and buttons are laid out. This means if you replace WebPositive with NetSurf, or Pe with Koder or Gvim, things still work as expected. Likewise if you update to a new version of the apps with a reworked UI.

We have the tools to achieve this, yet no one has really put them to good use yet.

1 Like

Yeah perhaps I should have been more clear. I actually meant ‘export the event handler behind the menu item’, rather than perform some evil UI location dependent x,y co-ordinate based click faking.

We have this already. Literraly, the command line would look like “hey application invoke item 3 of menu 5 of window 3”. It’s not x/y click faking, but this is still not what’s needed. There is no way to access menus by names (I think they don’t even have a name, only a label which is locale-dependant so unusable for this).

Controls and windows do have a name so that’s slightly better, but still not enough for something generic that can work by sending the same message to different interchangeable applications.

Right gotcha. So really what we need is a way to name and describe an incoming event that is handled by the app, along with its invocation options. That mechanism could then be used by menu items as well as hey integration, and possibly also invoked via a standard BMessage mechanism. So a slight separation from what we have now.

Dare I say it, kinda like iOS app development works. There you can define an integration point in Swift with an annotation, and then it’s automatically visible in the visual development IDE to use.

I might try and mock this up at BeGeistert just to see how it ‘feels’ from a programmers standpoint.

We have this already, it’s called “scripting”: The Be Book - System Overview - The Application Kit

It’s reachable using the “hey” command line tool but also usable with BMessages from one app to another.

The problem is while we have the whole protocol already defined, very few apps actually make use of it. It’s about time we start actually doing things with it.

4 Likes

Heh that’s what I deserve for not reading the legacy documentation…

So thinking of Paladin and Editors, we could define a standard protocol suite suite/vnd.Haiku-ValidatingEditor that had a “HighlightProblem” command. This command would have a FileRef specified, a Line Number specifier, and a ProblemDescription property that could be set by Paladin should a file fail compilation.

In Pe/Koder/GVim, this file window would be given focus, the cursor and view moved to the line, and an error flag set on the line with a popup description.

This could be implemented by any editor, and could be used by any compiler/code checking tool.

Is my understanding correct?

My last question would be, the documentation on the page you sent says that suites are discovered from live running apps by sending a message. Is there a more static way to discover these? I.e. are the suites supported discoverable from the app binary or another file distributed with the app? Is a ‘library’ of suites built up by the OS anywhere (E.g. from installed hpkg file introspection)?

If not, would it be useful to have a Macro to say “This next function should be exposed as a command in a suite for this app”, which generates such metadata for use by a scripting tool. (Or does this already exist somewhere???)

Apart from the integration discussion above, on the topic, seems like there is a Haiku package contradiction with python (iirc), as the python package is for gcc2, pyqt for python 2.7 is for gcc7 (x86) , so you couldnt invoke QT apps from the bundled python2.7 or you will trigger the infamous __tls_get_addr bug :thinking:

I am able to do so in my tests, because i installed my custom python2 version for x86 so I can work with it, but…

Gotta upstream this to the haiku depot version(s).

Experienced this bug being on default arch on 32bits, when trying to run wcgbrowser with pyqt. I’m testing this to compare its browser backend (qt) with Webpositive. @PulkoMandy I didnt think of using Web+ in kiosk mode, but having something like this browser does (it’s python code) for kiosk configuration (whitelisting, autodisabling popups, …) may worth it in the long future.

Added a capture of py-failing:

There is an issue about splitting libpython from the python package. https://github.com/haikuports/haikuports/issues/133

If you want to see how it’s done, see the example run documented in AGMSScriptOCron Documentation

Talking about python 3+, I tried several times to install python 3.6 from the repositories and it always stops at around 95% (my guess). Am I the only person having that problem? I am living in China, therefore weird things can happen in regards to connection speed, but I can see the download progressing in the progress bar.

Thanks for any help.

Yes, that’s the idea, define a “standard” protocol, and then have multiple editors implement it.

Currently the discovery is only dynamic, which indeed is not perfect. What I was thinking is that you would ask the MIME database for an app handling the text/sourcecode MIME type, and expect that application to handle the matching scripting protocol. But that would just be a non-enforceable agreement between application developers. Maybe we need something stronger. In that case, exporting the scripting structure as a symbol with a known name would allow to detect it by load_image and peeking at the data structure.

Adding a metadata entry to packages certainly is possible. How would that one be used? I guess the Paladin package would REQUIRE “scripting:validating_editor” to make sure at least one such editor is available?

I suspect true enlightenment for me on this issue will come after a few beers at BeGeistert. 8o)

Until then, I’ll keep my thinking cap on.