The current state and the future of Python on Haiku

I don’t know. I’m having this problems when other’s code use python (Like gobject-introspection), and may or may not work with python further than 2.7.

But in any case, “move to x64” and/or “move to N-further version” doesnt solve userland issues for a newcomer developer :thinking: You can have a kernel-land implementation going ok-ish, and a userland going ok-ish, but shouldnt both.

Following the DIY principles here, I made the recipe changes to have a x86 version of python2, let’s see if that keeps the wheels moving, it doesnt work or something else happens.

On any case, the packages depending on gcc2 version of python are like… 8. Shouldnt take so much time (dev or tools time)to migrate them to gcc7.

Epoll native version calling to be present, so libuv works “as is”.

Required by almost all X11 calls on cairo and gtk, unless you disable shm modules manually. Broadway implementation solves this opening fdescriptors and using mmap.

https://wiki.python.org/moin/Python2orPython3

Python 2 is deprecated and 2.7 (from 2010) is an end-of-life release. No new code should be written in Python 2 at this point. If you have good reasons for sticking with Python 2 (for example you depend on a library that still has not migrated to Python 3, or you deploy your code to BeOS or something as old, which does not have Python 3 available yet), then it’s fine. But otherwise, I see no point.

I could say the same of 32 vs 64 bit. Use 32 bit only if:

  • You deal with existing codebases that are not compatible with 64bit yet
  • You run old BeOS apps
  • You have old hardware without 64 bit support

The main difference is in the last case. Replacing hardware costs money. Replacing Python 2 with Python 3 costs only time (so it is a bit like making code compatible with 64bit versions). This does not mean Python 2 is going away or that your patches will be rejected. It just explains why no one did the work so far.

2 Likes

I know about that deprecation, i also code in python for “real” day job (lotta languages, lotta code).

What I mean is that you shouldnt say “that is broken in Haiku 32, use 64bit version while it is fixed”. My ticket 14641
I know there’s no better answer to say than Diver said, and you cannot review what every newcomer moans about but… doesnt look great.

My hardware is 64bit, but i took the 32bit image to create ports, as imagined it would have most common user base (i suppose), hence why I care.

Haiku 32b packages provided for the users should be (imo) updated whenever possible to the latest gcc.

Btw, tuning the recipe for x86, compiling and installing fixed the issue (as “expected”), despite having to simulate a version of “ldd” for the linking of gobject-introspection :thinking:

Will try to merge the changes if nothing is broken with this version of python.

He just provided a possible workaround in case you can’t wait for the issue to be fixed. If you can actually fix it yourself, that’s indeed a lot better.

And yes, things should be moved away from gcc2 when no BeOS compatibility is involved.

Agreed, i try to fix my stuff because I’m stubborn and have the skills, but other devs may not.

Also, seems like there are further problems (on the topic) if a package/library/make tries to generate a pyenv virtual environments.

Normal environments have a bin+lib folder, whereas Haiku likes to have them in a “non-packaged” directory being created too, and python goes nuts when trying to install something in libs (a directory above). So you end with bin+lib and non-packaged/bin+lib

Fixable by hand (moving files / doing soft links) but harder to do in recipes (like waiting for the error on creating the pyenv asuming it will fail, fix and redo the process).

Shit happens :man_shrugging:

That’s what I was going to ask right next. The current status of virtual environments. I can see now that the 64 bits version has some of my currently used packages available through HaikuDepot, and I am lucky enough to be able to choose the 64 bit version now. But with the install process for packages being split between pip and HaikuDepot, how well do virtual environment utilities work?

I can see this being a problem for Python developers, since HaikuDepot does not seem to integrate with pip, pipenv and the others, and many Python applications depend on packages that my be incompatible with the dependencies of other applications present in the same system. It reminds me of the kind of limited environment we have in Pythonista for iOS, where we can only have one version of each package, and then all applications must adapt to conform.

Dont be confused here, they work, but you have to fix that non-packaged … “bug” / “feature”. Once you do that, it becomes a normal (“unlimited”) and working pyenv.

pyenv

Replacing kqueue with kernel ports (or build a wrapper) doesn’t look that hard IMHO. I’d suggest to explore this way.

Bug, definitely. “non-packaged” is needed for the system directories, but virtualenvs should work as usual on other systems.

As for getting pip to integrate with haikuporter, I’m not enough of a Python dev yet to comment about this. Help welcome improving the situation, then.

Need one with experience with py-env to fix that, i’m not.

If this isnt fixed, python has a side effect of installing packages in the pyenv to one of the two directories (non-packaged and vanilla) and modules might not be found when importing. Experienced that while working with the building process for one famous program that required pyenv.

I dont see it being done soon. There are a lot of problems of non native compilation on Haiku (better said lot of flags from gcc “failing”), so most pips that need some c/cpp code might fail. I just tweak things while testing or throw my hacc proxy to bypass non compatible flags, but that doesnt solve all like magic.

Not like you dont have those problems in Linux Debianized world, where repository brings you one version (plus dependencies) and pip does too (never cared much about that, but i guess that you wont get the library integration between APT & pip).

Out of curiosity, what is this “one famous program” that requires pvenv?

Not ready yet to mention, not the topic, needed to have gtk3 as a recipe for haiku, which is almost ready (the oven is burning) and required python for a dependency (gobject-introspection package), which in the end wasnt required because you can… cough disable introspection with a flag cough.

So found “bugs” in the current state if python (on this thread topic) for dependencies that might not even going to to the public :confused:

What’s that, Darcs? I had to check in to see what you might have meant by “updating” Haskell, and am pleased to see GHC 7.8 is available. I’ve been able to build 8.2 on a nightly, have been meaning to check that on the beta, but … if someone else has taken on ghc support, that’s the best thing. There’s a lot to like about Haskell, but the creeping featuritis that makes it impossible to build what you need without upgrading to the latest and greatest … you can have it.

No. I tested compiling several versions of ghc (using 7.8.3) to bootstrap up to a newer version (i was unable to, maybe you had more luck or ghc knowledge, and stopped when the bootstraping said that a dyn version wasnt able to find yadayada hs packages ). Not specially a fan of haskell (apart from using it at uni), but i thought that having pandoc was a good feature to craft documentation(s).

It isn’t trivial. Some of the key parts of my “ghc knowledge” are just a collection of things I’ve stumbled across and use in a kind of trial-and-error fashion without really understanding, and dynamic loading platform support sure falls in that category.

And the builds routinely crash. Less often after I increased the RAM on my build host. My suspicion has always been that GHC exposes some problem in Haiku’s mmap implementation, but I can’t imagine how to track that down. (GHC has its own memory allocation pools, using mmap to get the memory from the system, and the amount of memory allocated in this manner can be rather large. I’ve never had this fail in my own applications, though.) Anyway, this is one of the little things that could make it difficult to get into the depot system - more complicated to run automatic builds. Then there’s the circular dependency issue.

That’s was never the goal of BeOS either of Haiku… That’s the way you need to go for innovate… and doing the things from scratch is the rigth thing to do, that include custom kernel and custom API if needed… If you dont belive me go and ask Google with their Fucshia OS project… even having two well mature OS products like Android and Chrome OS… KEEP YOUR EYES WIDE SHUT and stay away from this hard working and small community of Devs that they have had the constancy in principles, fidelity of purpose and the humility of so many years of voluntary work to fulfill the true objective raised since the creation of OpenBeOS. Haiku is not a Hobby OS, it is a piece of art in restoration and recreation of the ideals that BeOS defended and innovated in its time…!

3 Likes

I should probably give some info about the state of GHC since I was involved at least in packaging it.

My goal was indeed to port pandoc (but eventually I found a way to not need it, for now…).

I worked from jessicah’s effort to port GHC. She first cross compiled it from Linux, and then I wrote a recipe to package her cross compiled version. It should be usable to build more things, but I’m yet to manage to write recipes for cabal and the other often used packages. So right now you get just the compiler. Probably updating that to a newer version would not be a bad idea.

This use of mmap is known to be problematic when using ASLR on 32bit machines (we quickly run into address space limitations then). Disabling ASLR may help then (IIRC it can be done by setting some xattr on the executables?)

This is an interesting theory, how cloning a 20 years ago OS with ABI compatibility is going to be the way for innovate?

Yeah, in fact Fuchsia project will take lots of parts from android, libstagefreight and a whole lot of other things.

Bah. I really question myself why I keep discussing with this kind of people. I spent countless hours of my life to support this project. I spent countless nights of sleep to support the beta. I am pretty sure I spent much more time for the beta than you, so, eventually, you’d have to keep your eyes wide shut not me. I am proud of what I do.

However now that beta is out I promise to never get again into this kind of discussions, as you can see from the commit log I already begun innovating.

1 Like

Not a mystery, the recipe points at someone :thinking:

IIRC, the builds try to get cabal compiled (bootstrapped?) too. May be wrong. Also, all is done with cabal in current versions, so it’s a one way only, get cabal running.

Didnt think about using cross compilation from a GHC in Linux tho, i try to do the “all in Haiku, with Haiku tools” way for now…:thinking:

ok, u know, i don’t approve your way of discussing :smiley: but this doesn’t mean i’m not going to read and discuss what you write (have u tried with some chamomile? :stuck_out_tongue:)

I think the truth is almost in the middle. Haiku carries out a very strong heritage and has a very idealistic heart. This is a good thing and it must be preserved because makes cohesion among the community members. Think for example how similar projects (in essence) has failed on this and look at their status. AROS is the first coming in mind, but even ReactOS suffers this lacks of ideals (in facts, they advertise it as “nag free, open source alternative to NT”. Which is good maybe, but it’s not perfect)

(I’m not saying they’re abandonware. I’m just saying they’re cold).

But on top on it innovation could come. I’m not involved into the project so deeply to judge if binary compatibility and drop-in intended usage were good decisions. Probably they were. Shall we have proof of that? How many users in % uses closed-source binary app from the BeOS era?

The real point here: where to innovate first? I mean…innovate is much more difficult than replicate…

2 Likes