The current state and the future of Python on Haiku


#1

I have arrived to Haiku recently (or, more precisely, I started exploring it more deeply) and have been trying to get it running natively in my Mac and exploring the currently available software packages, in order to see how it fits me. I have been programming in Python for some time, including GUI applications using tkinter/ttk. I have already found a Python 3.6 package and the corresponding pip3.

For me, that don’t speak C++, it’s a bummer that we don’t have tk yet, but I totally understand it. What about other GUI toolkits compatible with Python 3? Is the recently launched QT for Python available in Haiku? Or any other options?

Also, I did a quick benchmark using pystone, and was a bit surprised to see that Python 3.6 in Haiku got only half the score I got in the same machine using macOS El Capitan. I thought Haiku was faster that macOS, but that seems not to apply to Python 3.6, at least in my case.

Python package management feels very weird, to say the least. Some packages are available through pip, others through HaikuDepot, others either elsewhere or completely unavailable. I understand the fact that it’s a work in progress and that many binary packages still need to be ported, but I am intrigued by the fact that Python packages are split between pip and HaikuDepot. Why is that?

What about Python 3.7? Is it being ported?

Sorry for all my questions. I am really excited with the beta release and just want to know how things really are right now.


#2

Seems like bindings are up https://depot.haiku-os.org/#!/pkg/pyqt_python3/haikuports/5/9/1/-/2/x86_64?bcguid=bc376-FLYE

Cant get an accurate benchmark with that. As it’s an interpreter for a language, the times for doing operations will probably vary, because the layers that get to the OS level are different. Then there’s the possibility of macOS libraries being compiled with flags for speed instead of smaller code/debug, and others that i cannot think about now. even with that, half the score is a big change.

I think it may have relation to one simple topic: binary code not matching Haiku natively when compiling with pip.

Like:
ok i’m pip and i’m going to compile supercoolpackage with -pthreads flag … ooh fuck, i failed kkthxbye.

Whereas you can have a package on the system that has no -pthreads flag in the compilation, and further options.

In any case, I’m glad that the topic “X language state in Haiku” is brought up.

I’m in a personal war (failed battles for now) on having the Haskell language updated to have some of its tools along. Not specially for the language itself :thinking:


#3

Haiku currently doesn’t PGO compile anything as far as I know, and for python that is usually worth about 10% in performance, LTO may also be worth a few more percent.

Also there are some python bindings to the BeAPI https://github.com/HaikuArchives/HaikuLanguageBindings


#4

Haiku feels snappier, but since we are using custom kernel, and don’t have too many kernel developers around anymore, it is slower than mainstream OSes.


#5

Do the Python bindings work with Python 3.6?


#6

No idea, they were written when only python 3.2 was out though… and probalby written for python 2.7. They are generated though so it should be possible to update the generator to output 3.x compatible bindings if it doesn’t already.

A good project for someone that knows python and perl.


#7

And one crazy amount of time, and luck.

I’m stuck right now, because i compiled a python module from source (x86), and then the makefiles try to load it with python 2.7 (x86_gcc2) and that crashes by libroot cross architecture symbols mismatch.

So i think that it might happen when working with this api binding too. Maybe it’s unrelated.


#8

About package availability:

Ideally we would get everything into haikudepot. But there is a manual process to add a python package there (writing a recipe for it). We do this when they are needed for other packages in Haikudepot.

pip should work and bring more packages, but sometimes things will not compile on Haiku, and there we can only rely on the package maintainers to fix it (while when using haikudepot we can have our own patches).


#9

This is definitively not something we’re going to highlight into our marketing activities :roll_eyes:


#10

Yes, let’s continue to keep eyes closed, without realizing that Haiku has to choose between supplying a custom kernel or a custom API. Doing both is going to relegate the project into something that will never get a point of serious usability.


#11

Is there a central high level list of areas of the kernel that need work? I would ask for specifics, but I guess given there’s so few kernel devs the list is quite long!?!

I eventually would like to get in to kernel development (talking potentially after Christmas) as it’s an area that has always interested me. If there was a known bottleneck in a relatively small area of the kernel perhaps that would be a good way for me to become familiar with the kernel whilst simultaneously helping everyone out.

Is there such an area like this that you know of @KapiX ?


#12

What is your contribute in changing this, If I can ask? Aside of ranting and blaming everyone on the forum, ofc.


#13

Now, now children…


#14

You can go in my github of course.

Can I ask what’s your latest code contribution?


#15

No, you can’t.

I’m not here to explain you how many different ways one can contribute to a project without writing C++ code.

One of these ways is be polite, supportive and respectful of others’ efforts. This is most valuable than 10k lines of code commit.


#16

Well, then don’t ask me what I am contributing. Tell me that you don’t like my way of discussing. This is a different subject. Have a nice day.


#17

On the real topic here, This is the status of python (2.7):

resolve symbol "___tls_get_addr" returned: -2147478780

Whenever I want to run something in x86 but some parts are compiled in gcc2, which is a dead end for some porting and tests.

As python 3 is already on x86 only, why is the 2.7 version still on gcc2, and not compiled in gcc7?


#18

Python buildsystem in the 2.7 days, and our early haikuports system, made it hard to build python with the “alternate” compiler. This may need some more patching to the Python sources. And we have little interest in Python 2 so no one has done it. Can’t you switch to Python 3?


#19

The reasons for the relatively slow kernel are:

  • If you run a nightly, there is a lot of “debug mode” things enabled. These make it much easier to investigate problems but have a performance hit.
  • The general idea is to first get things working, and optimize only when we identify a bottleneck that is actually a problem in a given use case. As long as the OS feels like it replies in time to user input, there is no need to make it faster than that, or at least there are more important things to work on.

For example, if you want to get into kernel development, one nice task would be adding something similar (or comatible) to BSD kqueues, which would be useful for porting node.js and possibly other modern software. A complete implementation of POSIX shared memory would also be nice. There are probably other features like that which are needed.

On the side of performance, it’s more about picking a specific problem and then trying to fix it. For example, jua has done this for WebPositive (leading to additions to app_server in order to speed up and simplify drawing operations) or HaikuDepot (leading to mostly rework of the application). Recently another bottleneck identified was in the HTTP implementation in the network kit, which does a lot of BMessage passing and is quite inefficient. A redesign of the HTTP code is more likely to provide improvements than an attempt to optimize delivery of BMessages here (so, no need to get into improving the kernel this time).


#20

Ah this might be a good area for me. I’ve done some work in C++ on the app and API wrapper side - creating wrappers in C++ to communicate with NoSQL databases over HTTP. Wouldn’t take me long to get up to speed on the code that does this in the network kit.

I’ll add that to the list of many things I want to have a look at in Haiku. 8o) Might be one for me to upskill at BeGeistert, if there’s anyone versed in Network Kit going?