Pip3 not working - No module named 'pkg_resources'

HelgeCPH · 2019-06-11 09:58:41 UTC

Dear all,

I am currently experimenting with Haiku. As I am mainly programming in Python, I tried to see if I could switch to work in Haiku.

I am running Haiku version R1/beta1 (Revision hrev52295+129) at the moment currently in Virtualbox via the (official?) Vagrantbox, see https://app.vagrantup.com/haiku-os/boxes/r1beta1-x86_64

I installed Python 3 and the corresponding pip via

~> pkgman install python3
~> pkgman install pip_python3

The installation process works fine and Python itself seems to work. But pip cannot be used as it cannot locate the module pkg_resources

~> python3 --version 
Python 3.7.3
~> pip3 --version
Traceback (most recent call last):
  File "/bin/pip3", line 6, in <module>
    from pkg_resources import load_entry_point
ModuleNotFoundError: No module named 'pkg_resources'

What do I have to do to fix that issue? Thanks in advance for any help.

extrowerk · 2019-06-11 12:26:45 UTC

Probably you have to install setuptools_python3 too.

HelgeCPH · 2019-06-11 12:49:00 UTC

Thank you for the reply.

setuptools_python3 is already installed, see below. I think it was pulled in automatically when installing pip_python3.

~> pkgman install setuptools_python3
100% repochecksum-1 [65 bytes]
Validating checksum for Haiku...done.
100% repochecksum-1 [64 bytes]
Validating checksum for HaikuPorts...done.
setuptools_python3-28.8.0-2 from repository system is already installed.
Nothing to do.

extrowerk · 2019-06-11 13:39:54 UTC

I think booth pip and setuptools built against Python 3.6.x and not against 3.7.x.

Try to install python36

HelgeCPH · 2019-06-11 14:38:47 UTC

Thank you, I tried that. But it does not seem to work:

I uninstalled Python 3.7 as in the following:

~> pkgman uninstall python3
The following changes will be made:
  in system:
    uninstall package pip_python3-9.0.0-1
    uninstall package setuptools_python3-28.8.0-2
    uninstall package python3-3.7.3-2
Continue? [yes/no] (yes) : [system] Applying changes ...
[system] Changes applied. Old activation state backed up in "state_2019-06-11_14:30:03"
[system] Cleaning up ...
[system] Done.

After that, I still had a Python 3.6 installed:

~> python3.6 --version
Python 3.6.8

Now, I installed pip3, the only one that I can find with pkgman, which has Python 3.7 set as dependency.

~> pkgman search pip3
Status  Name         Description
-----------------------------------------------------------------------------
        pip_python3  The PyPA recommended tool for installing Python packages
~> pkgman install pip_python3
100% repochecksum-1 [65 bytes]
Validating checksum for Haiku...done.
100% repochecksum-1 [64 bytes]
Validating checksum for HaikuPorts...done.
The following changes will be made:
  in system:
    install package python3-3.7.3-2 from repository HaikuPorts
    install package setuptools_python3-28.8.0-2 from repository HaikuPorts
    install package pip_python3-9.0.0-1 from repository HaikuPorts
100% python3-3.7.3-2-x86_64.hpkg [9.78 MiB]
Validating checksum for https://eu.hpkg.haiku-os.org/haikuports/master/x86_64/current/packages/python3-3.7.3-2-x86_64.hpkg...done.
100% setuptools_python3-28.8.0-2-any.hpkg [506.79 KiB]
Validating checksum for https://eu.hpkg.haiku-os.org/haikuports/master/x86_64/current/packages/setuptools_python3-28.8.0-2-any.hpkg...done.
100% pip_python3-9.0.0-1-any.hpkg [2.08 MiB]
Validating checksum for https://eu.hpkg.haiku-os.org/haikuports/master/x86_64/current/packages/pip_python3-9.0.0-1-any.hpkg...done.
[system] Applying changes ...
[system] Changes applied. Old activation state backed up in "state_2019-06-11_14:31:16"
[system] Cleaning up ...
[system] Done.

That is, I do not know how to install pip3 for Python 3.6 with pkgman.

extrowerk · 2019-06-11 15:19:19 UTC

Sorry, then the modules depending on P3.7, my mistake.
Afaik we currently generating packages only for Python2 and Python3.7, so no 3.6 modules available.

So the problem lies somewhere else.

Oh, btw, what do you planning to do with pip? Maybe you could use easy_install instead pip?

Pahefu · 2019-06-12 07:15:27 UTC

Reproduced the same behavior on a nightly.
Upgraded python2 and python3.6 through pkgman (having to reboot for haiku to get them recognized as command), and tested some simple pip usages.

Pip for python2.7 works normally whereas pip3 fails. After playing a bit with pkgman and uninstalling python3.6 you can see that pip3 dependencies link to python 3.6 and not current system one (3.7). So i guess pip3 is calling a wrong version of python and so on…

Pip beats all the stuff. Just like pkgman is easier than downloading hpkgs. Also probably nowadays most python virtualenvs will use pip, if i’m right.

HelgeCPH · 2019-06-12 08:24:25 UTC

Thank you. Your comment on pip3 calling the wrong Python version let me wonder and I double checked.

You are right. It seems that the pip_python3 package is currently built wrongly as the shebang in the script that is placed under /bin refers to the Python 3.6 interpreter.

~> head -1 /bin/pip3
#!/bin/python3.6

I tried to manually change that to point to 3.7 but realised that I am not allowed to as everything under /bin seems to be read only… I could neither change the permissions with chmod so that I could write to that file. How would I do that manually?

While searching for a possible solution yesterday, I found the Github repository (https://github.com/haikuports/haikuports) of the Haiku Ports people and it seems that raising an issue there is the right thing to do. Is this how it is done in the Haiku world? If so, then I will just write an issue there later today.

@extrowerk: the reason for not using easy_install is that I try to reproduce my current workflow and that it is deprecated, see https://setuptools.readthedocs.io/en/latest/easy_install.html.

extrowerk · 2019-06-12 09:02:42 UTC

You can’t directly change packaged directories, as the files are not there at all.
Read the user guide about package manager for more info.
Yes, creating an issue at haikuports is the right thing to do, and you can even send a PR if you are able to fix it.

The Python modules are in pretty bad shape after the switch to Python 3.7. Somebody really have to adjust all the recipes to add P3.6 support and fix 3.7. It would involve the following as far as i see:

One should add python36 here: https://github.com/haikuports/haikuports/blob/master/dev-python/pip/pip-9.0.0.recipe#L23
3.7 should be added here: https://github.com/haikuports/haikuports/blob/master/dev-python/pip/pip-9.0.0.recipe#L24

Probably the REPLACES, but maybe not.
This changes should be done on every recipe. Are you guys up to the task?

HelgeCPH · 2019-06-12 10:37:35 UTC

Thank you for the advice and thank you a lot for your help!

I started by writing an issue over there.

Currently, I did not sent a pull request as I think there are at least two versions of solving the problem and I do not really know how the second would be implemented Let’s see what the folks over there think.

During the weekend I will try to see if I can get the HaikuPorter to run and to understand what is going on…