Until the issue with root certificates that breaks my network connection to Git and other important sites, how can I downgrade the affected package?
Ideally I’d like to “pin” this version to protect it from being updated during pkgman update, like it’s possible in Linux package managers, but I think that’s not supported.
I understand that you can “update” to an older version by installing the relevant package, but in the case of ca_root_certificates-2024_07_02-1-any there are a lot of (unecessary imo) hard dependencies that prevent me from installing the older certificates.
Also, pkgman wants to go online to check, so I have to enable at least one repository (but most of them cannot be reached to the certificates).
More importantly, it won’t me let install the older version unless I uinstall a lot of other packages, which makes no sense to me in this case.
Normally this would be a safe and sane thing to do, but if the package dependencies are too strict or you know what you’re doing, there should be a way to ignore this.
I don’t think these packages would need to be uninstalled just to swap out my root certificates… (looks like the entire OS depends on root certificates…)
It looks like pkgman only looks for broken dependencies when the existing package is uninstalled but does not take into account that they will be fulfilled by installation of the new package.
> pkgman install /boot/system/packages/administrative/state_2025-04-23_23:06:12/ca_root_certificates-2024_07_02-1-any.hpkg
100 % repochecksum-1 [64 bytes]
Validating checksum for HaikuPorts...done.
Encountered problems:
problem 1: package curl-8.13.0-2 requires ca_root_certificates>=2024_11_26, but none of the providers can be installed
solution 1:
- allow deinstallation of curl-8.13.0-2
- allow deinstallation of haiku-r1~beta5_hrev58831-1
- allow deinstallation of webpositive-r1~beta5_hrev58831-1
- allow deinstallation of haiku_devel-r1~beta5_hrev58831-1
- allow deinstallation of haiku_datatranslators-r1~beta5_hrev58831-1
- allow deinstallation of openssl3-3.5.0-2
- allow deinstallation of libgit2_1.8-1.8.4-1
- allow deinstallation of openldap2.4-2.4.48-5
- allow deinstallation of git-2.48.1-1
- allow deinstallation of openssh-9.8p1-2
- allow deinstallation of xmlsec-1.2.37-3
- allow deinstallation of libevent-2.1.12-5
- allow deinstallation of libcmis0.6-0.6.2-2
- allow deinstallation of libreoffice-24.8.1.1-1
- allow deinstallation of haikuwebkit-1.9.21-1
- allow deinstallation of neon-0.34.0-1
- allow deinstallation of libarchive-3.7.2-2
- allow deinstallation of qt6_base-6.7.2-7
- allow deinstallation of ffmpeg7-7.1.1-1
- allow deinstallation of ffmpeg6-6.1.2-5
- allow deinstallation of wget-1.24.5-2
- allow deinstallation of libssh2-1.11.1-1
- allow deinstallation of raptor-2.0.15-10
- allow deinstallation of qt6_declarative-6.7.2-1
- allow deinstallation of wpa_supplicant-2.11.haiku.0-1
- allow deinstallation of openssl3_man-3.5.0-2
- allow deinstallation of openssl3_devel-3.5.0-2
- allow deinstallation of poppler24_qt6-24.12.0-1
- allow deinstallation of cmake-3.31.5-1
- allow deinstallation of clipdinger-1.2.4-2
- allow deinstallation of poppler24-24.12.0-1
- allow deinstallation of redland-1.0.17-9
- allow deinstallation of rasqal-0.9.33-6
- allow deinstallation of iceweasel_bin-137.0.1-1
- allow deinstallation of qt6_imageformats-6.7.2-1
- allow deinstallation of qt6_shadertools-6.7.2-1
- allow deinstallation of qt6_multimedia-6.7.2-1
- allow deinstallation of qt6_svg-6.7.2-2
- allow deinstallation of ffmpeg7_debuginfo-7.1.1-1
- allow deinstallation of ffmpeg6_debuginfo-6.1.2-5
- allow deinstallation of xmlsec_nss-1.2.37-3
- allow deinstallation of libgit2_1.8_devel-1.8.4-1
- allow deinstallation of fribidi-1.0.16-1
- allow deinstallation of dav1d-1.5.0-1
- allow deinstallation of makefile_engine-r1~beta5_hrev58831-1
- allow deinstallation of make-4.4.1-1
- allow deinstallation of vulkan-1.4.311-1
- allow deinstallation of mpfr-4.2.0-3
- allow deinstallation of gdk_pixbuf-2.42.9-5
- allow deinstallation of libxkbcommon-1.7.0-1
- allow deinstallation of haiku_svg_icon_theme-5.15.2.38-1
- allow deinstallation of openexr30-3.0.5-2
- allow deinstallation of coreutils-9.6-1
- allow deinstallation of grep-3.11-2
- allow deinstallation of libmspub-0.1.4-6
- allow deinstallation of lcms-2.16-1
- allow deinstallation of gutenprint9-5.3.4-2
- allow deinstallation of libwps-0.4.14-1
- allow deinstallation of libwpg-0.3.4-1
- allow deinstallation of libwpd-0.10.3-2
- allow deinstallation of wayland-1.21.0~git-3
- allow deinstallation of pe-2.4.5-11
- allow deinstallation of libnumbertext-1.0.6-1
- allow deinstallation of clucene-2.3.3.4-4
- allow deinstallation of m4-1.4.19-1
- allow deinstallation of bash-5.2.037-1
- allow deinstallation of speexdsp-1.2.1-1
- allow deinstallation of tiff-4.7.0-1
- allow deinstallation of libvpx-1.13.1-1
- allow deinstallation of gmp-6.3.0-1
- allow deinstallation of glu-9.0.0-8
- allow deinstallation of which-2.21-6
- allow deinstallation of bc-1.07.1-2
- allow deinstallation of libunibreak-5.1-1
- allow deinstallation of xz_utils-5.6.2-2
- allow deinstallation of findutils-4.9.0-3
- allow deinstallation of gawk-5.3.0-1
- allow deinstallation of perl-5.40.1-1
- allow deinstallation of boost1.83-1.83.0-3
- allow deinstallation of p7zip-17.05-2
- allow deinstallation of gcc-13.3.0_2023_08_10-4
- allow deinstallation of libexecinfo-1.1-6
- allow deinstallation of libxml2-2.12.9-1
- allow deinstallation of argon2-20200709-2
- allow deinstallation of libcroco-0.6.13-2
- allow deinstallation of liblangtag-0.6.3-1
- allow deinstallation of lame-3.100-4
- allow deinstallation of netcat-1.10-4
- allow deinstallation of cairo1.18-1.18.0-1
- allow deinstallation of giflib-5.2.2-1
- allow deinstallation of libuuid-1.3.1-5
- allow deinstallation of libffi-3.4.6-1
- allow deinstallation of qt6_translations-6.7.2-1
- allow deinstallation of libgcrypt-1.10.2-2
- allow deinstallation of autoconf-2.72-1
- allow deinstallation of fdk_aac-2.0.2-4
- allow deinstallation of automake-1.16.5-3
- allow deinstallation of lexilla-5.2.4-1
- allow deinstallation of libtheora-1.1.1-8
- allow deinstallation of zstd-1.5.6-2
- allow deinstallation of tar-1.35-2
- allow deinstallation of bison-3.8.2-1
- allow deinstallation of libmysqlclient-6.1.6-4
- allow deinstallation of gobject_introspection-1.78.1-1
- allow deinstallation of libodfgen-0.1.7-1
- allow deinstallation of jsoncpp-1.9.5-3
- allow deinstallation of unixodbc-2.3.11-1
- allow deinstallation of libpng16_devel-1.6.44-1
- allow deinstallation of libopenmpt-0.7.11-1
- allow deinstallation of tcpdump-4.99.5-1
- allow deinstallation of sed-4.9-1
- allow deinstallation of wayland_server-0.1.20250303-1
- allow deinstallation of libpcre-8.45-3
- allow deinstallation of vision-0.10.6-2
- allow deinstallation of snappy-1.1.10-1
- allow deinstallation of libvisio-0.1.7-6
- allow deinstallation of speex-1.2.1-2
- allow deinstallation of harfbuzz_glib-8.3.0-2
- allow deinstallation of adwaita_icon_theme-42.0-2
- allow deinstallation of hunspell-1.7.2-1
- allow deinstallation of libtool_libltdl-2.4.7-1
- allow deinstallation of libpcap-1.10.5-2
- allow deinstallation of libcdr-0.1.8-1
- allow deinstallation of harfbuzz-8.3.0-2
- allow deinstallation of libraw-0.20.2-2
- allow deinstallation of editorconfig_core_c-0.12.6-1
- allow deinstallation of mandoc-1.14.3-2
- allow deinstallation of patch-2.7.6-2
- allow deinstallation of libedit-20230828_3.1-1
- allow deinstallation of libqxp-0.0.2-5
- allow deinstallation of libepubgen-0.1.1-3
- allow deinstallation of expat-2.7.1-1
- allow deinstallation of libebook-0.1.3-4
- allow deinstallation of glib2-2.78.0-2
- allow deinstallation of llvm12_libs-12.0.1-8
- allow deinstallation of gzip-1.12-2
- allow deinstallation of atk-2.38.0-3
- allow deinstallation of libass-0.17.3-2
- allow deinstallation of libpsl-0.21.5-1
- allow deinstallation of openal-1.21.1-5
- allow deinstallation of texinfo-7.1-2
- allow deinstallation of llvm20_libunwind-20.1.0-4
- allow deinstallation of nghttp2-1.63.0-1
- allow deinstallation of poppler_data-0.4.12-1
- allow deinstallation of sqlite-3.47.2.0-1
- allow deinstallation of rhash-1.4.4-3
- allow deinstallation of graphite2-1.3.14-2
- allow deinstallation of libabw-0.1.3-1
- allow deinstallation of xkeyboard_config-2.41-1
- allow deinstallation of libuv-1.48.0-1
- allow deinstallation of ixion0.18-0.19.0-1
- allow deinstallation of jasper-2.0.33-1
- allow deinstallation of unzip-6.10c23-5
- allow deinstallation of fontconfig-2.13.96-2
- allow deinstallation of opus-1.3.1-2
- allow deinstallation of nspr-4.36-1
- allow deinstallation of libogg-1.3.5-2
- allow deinstallation of libpng16-1.6.44-1
- allow deinstallation of gsettings_desktop_schemas-43.0-2
- allow deinstallation of gettext_libintl-0.22.5-1
- allow deinstallation of flex-2.6.4-4
- allow deinstallation of rav1e-0.7.1-1
- allow deinstallation of libpagemaker-0.0.4-3
- allow deinstallation of libmwaw-0.3.22-2
- allow deinstallation of nss-3.110-1
- allow deinstallation of dbus-1.12.20-6
- allow deinstallation of libidn2-2.0.5-3
- allow deinstallation of brotli-1.1.0-2
- allow deinstallation of librevenge-0.0.5-2
- allow deinstallation of libvorbis-1.3.7-1
- allow deinstallation of bzip2-1.0.8-3
- allow deinstallation of yaml_cpp0.8-0.8.0-2
- allow deinstallation of libtasn1-4.19.0-1
- allow deinstallation of libetonyek-0.1.12-2
- allow deinstallation of libicns-0.8.1-9
- allow deinstallation of mesa-22.0.5-3
- allow deinstallation of mpc-1.2.1-2
- allow deinstallation of libexttextcat-3.4.6-1
- allow deinstallation of double_conversion-3.2.0-1
- allow deinstallation of libmng-2.0.3-5
- allow deinstallation of libgpg_error-1.51-1
- allow deinstallation of media_helpers-0.1-1
- allow deinstallation of qsystray-5.15.2.14-1
- allow deinstallation of icu74-74.1-6
- allow deinstallation of libxslt-1.1.39-2
- allow deinstallation of libepoxy-1.5.8-3
- allow deinstallation of cdrtools-3.02~a09-2
- allow deinstallation of zlib-1.3.1-4
- allow deinstallation of libiconv-1.17-4
- allow deinstallation of game_music_emu-0.6.4-1
- allow deinstallation of pkgconfig-0.29.2-4
- allow deinstallation of hyphen-2.8.8-4
- allow deinstallation of gcc_syslibs-13.3.0_2023_08_10-4
- allow deinstallation of nasm-2.15.05-2
- allow deinstallation of libfreehand-0.1.2-5
- allow deinstallation of nano-8.1-1
- allow deinstallation of readline-8.2.013-1
- allow deinstallation of shared_mime_info-1.15-2
- allow deinstallation of pango-1.54.0-2
- allow deinstallation of mythes-1.2.4-5
- allow deinstallation of freetype-2.13.3-1
- allow deinstallation of ncurses6-6.5-2
- allow deinstallation of libzmf-0.0.2-7
- allow deinstallation of zip-3.0-4
- allow deinstallation of soxr-0.1.3-2
- allow deinstallation of libjpeg_turbo-2.1.5.1-1
- allow deinstallation of librsvg-2.50.7-4
- allow deinstallation of libwebp-1.5.0-1
- allow deinstallation of gtk3-3.24.36-2
- allow deinstallation of libpcre2-10.45-1
- allow deinstallation of libjxl-0.6.1-4
- allow deinstallation of less-668-1
- allow deinstallation of openjpeg-2.5.3-2
- allow deinstallation of box2d-2.4.1-2
- allow deinstallation of orcus0.18-0.19.2-1
- allow deinstallation of binutils-2.42-1
- allow deinstallation of libunistring-1.2-1
- allow deinstallation of gflags-2.2.2-2
- allow deinstallation of mkdepend-1.7-5
- allow deinstallation of jam-2.5_2021_10_29-2
- allow deinstallation of sharutils-4.15.2-3
- allow deinstallation of pixman-0.42.2-1
- allow deinstallation of libavif1.0-1.1.0-1
- allow deinstallation of woff2-1.0.2-2
- allow deinstallation of libstaroffice-0.0.7-1
- allow deinstallation of diffutils-3.10-3
- allow deinstallation of lpsolve-5.5.2.5-3
- allow deinstallation of libpcre2_devel-10.45-1
- allow deinstallation of llvm20_libunwind_devel-20.1.0-4
- allow deinstallation of mesa_devel-22.0.5-3
- allow deinstallation of mesa_swpipe-22.0.5-3
- allow deinstallation of zlib_devel-1.3.1-4
- allow deinstallation of yaml_cpp0.8_devel-0.8.0-2
- allow deinstallation of editorconfig_core_c_devel-0.12.6-1
- allow deinstallation of lexilla_devel-5.2.4-1
- allow deinstallation of libjpeg_turbo_devel-2.1.5.1-1
- allow deinstallation of zstd_devel-1.5.6-2
- allow deinstallation of gflags_devel-2.2.2-2
solution 2:
- do not install "pkg:ca_root_certificates-2024_07_02-1-any"
The certification package has been updated recently, but after curl was updated earlier, so best bet (I think), would be to rebuild the older version with haikuporter so it can use the new curl package?
For security reason
It make sense to upgrade any networking package to depends on latest network security related packages available, in particular root CAs.
It help boost the deprecation of an older version of packages on which security flaw was identified.
For instance, a CA authority that was found since as not trustworthy at all.
One solution is indeed to get from haikuports the latest curl recipe, edit it to change the requirements regarding certificates package minimal version, and build your own version of latest curl package via haikuporter tool.
But the more important question is why do you need to rollback to an older ca certificates package?
It is very likely that it breaks connectivity to several pages, including Github and haiku EU repository, as I’ve found out here:
When I rollback to before the certificated update to 2025-04-23 state, everyhing works fine in the same environment.
So the better option would be to find out the root cause of the root certificate issue;-) and ship a more recent, fixed version.
Is 11/24 really current?
This will report way more useful information about why the connection fail than Curl output will.
For sure, if something was broken in the latest ca-certificates packages, it has the potential to break everything doing SSL networking, or even just verifying certificates, whatever the way they were obtained, against its CA chain.
Before removing the offending package, I would try to put the one you made in home…packages directory. There it won’t be updated but you will have to keep in mind that it is there.
Download it (yes, you will need to have a working way to do https for that, how ironic…), and try to see what give the openssl command when you use the -CAfile option to point at this specific CA certs file.
Seems I was wrong, curl was the last one updated here, the certificates were updated in December. Maybe an update is at hand there? … pokes @waddlesplash
maybe it’s really some mismatch then, I’ll udpate to the latest and tell you my experience.
So I need to update the haikuports cacerts package with the latest .perm from Mozilla, right?
yeah the impact should be more severe, that’s right.
But I cannot imagine what else could have gone wrong from Haiku nightly 2025-04-23 and 2025-04-24 besides the iawifi200 driver and the certificates…
Tried Haiku repository prefs, browser, and shell - all time out.
Compiled the latest cacerts from 2025-02-25 now, but still get the same issues, sadly. @phoudoin your command gives me a timeout with this output:
> openssl s_client -connect github.com:443
Connecting to 140.82.121.3
CONNECTED(00000003)
E0011487A9000000:error:0A000126:SSL routines::unexpected eof while reading:ssl/record/rec_layer_s3.c:691:
---
no peer certificate available
---
No client certificate CA names sent
Negotiated TLS1.3 group: <NULL>
---
SSL handshake has read 0 bytes and written 1557 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Protocol: TLSv1.3
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
how can this be?
no peer certificate available
To reiterate, network itself is fine, this is my syslog:
Haikuporter does not try to be smart about this. If you set up a dependency in a recipe without a specific constraint, it will automatically set a >= constraint with the latest version available at the package build time, as this is the strictest dependency.
Anything more flexible has to be manually declared (for example by use of “compat” rules. This requires identifying what is compatible with what and declaring the right rules. While that is relatively easy to do for libraries (thanks to soname and versioning conventions), it is not so easy for ca root certificates, where identifying the incompatible changes from one version to another isn’t always easy.
No one has done this for ca_root_certificates, because in most cases, people don’t need to use older versions of that. It could be done, but until now, no one had needed it.
Yes, because, as you are experiencing, without certificates, nothing at all works.
So, currently you don’t have a way to revert just that package.
Since it seems not many other people are complaining about this, it’s possible that there is something specific to your installation. Maybe you have some old packages installed in /home or some old version of openssl in a non-packaged directory? That could explain the incompatibility. In that case, maybe you can check with listimage on a program that is failing to connect to something, and look at the list of libraries for anything that’s not in /system or that is in /system/non-packaged. If you have an old version of some library that doesn’t know how to handle the new certificates (sometimes the format or the organization of these changes), that could be the problem.
Thanks for the thorough explanation, makes sense and I understand, just trying to make sense of the issues I’m experiencing and finding the root cause so I can solve this.
I do have some unrelated libs I worked on in ~/config/non-packaged (litehtml, qpdf and gumbo) but I even cannot connect with git, and just to go sure, I did a listimage git and came up, as expected, only with the core deps:
> listimage git
\
TEAM 1337 (/bin/listimage git):
ID Text Data Seq# Init# Name
--------------------------------------------------------------------------------
9846 0x00000183f82de000 0x00000183f82df000 0 0 /boot/system/bin/listimage
9844 0x00007fd47e8f1000 0x0000000000000000 0 0 commpage
9845 0x0000007def81e000 0x0000007def83d000 0 0 /boot/system/runtime_loader
9847 0x000000342bf2e000 0x000000342c027000 0 0 /boot/system/lib/libroot.so
9848 0x0000001ea4e0c000 0x0000001ea4e2b000 0 0 /boot/system/lib/libgcc_s.so.1
I can’t even reach github.com with IceWeasel and that is more or less an isolated environment.
So all signs point to the root certificate, but it would be unlikely that I should be the only person having problems with the certificate…
So I need to find more differences between Haiku R1/Dev from 23.04.2025 and the day after for possible culprits…
Your listimage output appears to be looking at itself (/boot/system/bin/listimage).
You need to attach it to a running git process, which may not be so easy to do if git immediately fails. Or some other process that is easier to keep running (a web browser would be easier, but then, they tend to be large and have lots of dependencies).
In your openssl logs, it looks like openssl did not even receive any certificates from the server (“no peer certificate available”) so we don’t even get to the point where ca_root_certificates is used. If we don’t get a certificate from the server, there is no way to check if that certificate is trusted.
The next thing to check may be if you have any settings files for openssl that may be attempting to use some other certificate path or other authentication mechanism (pkcs11 for example). For this:
Find the OPENSSLDIR using openssl version -d
Check for any configuration files there and see if their content is correct (I don’t remember if we deploy a default config in Haiku’s openssl package)
Thanks for your help on this! Please keep in mind that everthing works fine with the 2025-04-23 Haiku build which I’m running now.
With any build after that, I cannot even use duckduckgo in WebPositive, I just get a timeout.
I think it’s more related to OpenSSL than the CA certificates packages.
It seems that OpenSSL v3 introduced a more stricter behavior in case of unexpected shutdown/weird response from the peer.
I’m not sure, but previous openssl1.x was catching the same EOF but was reporting it with an errno set to 0, and many programs using openssl1.x where, therefore, didn’t really catch them. So many programs where ignoring it.
In OpenSSL3, it’s not the case anymore, and now such programs can’t just ignore an SSL error with some errno actually set.
Curl since 7.88.something is supposed to be ready to face such situation.
But maybe not our latest git haikports.
Are you behind some proxy or VPN, BTW, that could explained why not everybody experience the same behavior than you?
Ok thanks for the pointer, as @pulkomandy also pointed towards openssl this seems very likely to cause my issues after all.
This might explain why IceWeasel - presumably using its own, more recent openssl implementation than the one until Haiku nightly from 23.04. - cannot reach Github but WebPositive can, on the same nightly.
I’m not sure how I can apply your suggestion with the openssl client EOF option in combination with external tools - I need git etc. to work, it’s not my own script or application that fails.
I am not behind a firewall (sadly, need to repair my firewalla box) besides my local Magenta router, and have no proxy. Most connections work fine, but some do not and just time out, although the site is perfectly reachable from Linux in the same network, and from my older Haiku nightly.
(looks like the entire OS depends on root certificates…)
and came up, as expected, only with the core deps: