I have the w2c2 package first draft written for HaikuPorter. It presently uses the “download as zip file” hack from GitHub but I think it would cost less work if I could do a git clone instead. Is that supported? Is there an example package that I can model mine after that will do that? Where I was running into trouble was figuring out how to generate a sha256 checksum from a directory. The draft is in my fork of HaikuDepot at https://github.com/SamuraiCrow/haikuports/blob/master/dev-lang/w2c2/w2c2-0.1.recipe.
AFAICT, the idea is to always download a “tarball” of sources. Better if it’s a release, but sometimes the “$srcGitRev” is necessary (specially for projects with no new releases but plenty of new code).
That download happens only once, unless you change the SOURCE_URI, then it’s unpacked and a new “clean git repo” gets initialized with that code as its initial commit (tagged: ORIGIN).
That’s done so you can modify that copy, adding new commits, and then using “haikuporter -e <recipe_name>” to extract a .patchset that contains all your commits (but ONLY your commits) to that “clean repo”.
Not sure how an actual “git clone” from a given “$srcGitRev” would be better than the current method.
Regarding the SHA256…
When you change the SOURCE_URI and then do “haikuporter <recipe_name>”, it will download the new file, and it will warn you that the SHA256 does not match the one from the recipe (as expected, you just used a different file).
The trick is to take the new SHA256 from that warning, and use that in your updated recipe. Next time… they will match 
(edit: the sha256 is of the download archive, not from the unpacked directory).
Regarding your draft:
Uncomment that srcGitRev=“cd90cd995501d83ce41f57bfb11b1a52de092d20”.
Then replace “cd90cd995501d83ce41f57bfb11b1a52de092d20” everywhere with “$srcGitRev”.
Next time you only need to update the commit rev in one place.
Hope it helps a bit.
(another Haikuporter/Haikuports newbie here).
Edit Note:
“haikuporter -c <recipe_name>” can be your friend in cleaning things up between recipe/build changes… but be aware: don’t use that if you made changes on the “working git repo”, unless not until you ran that “hp -e” to extract the patchset. Your commits there will be lost otherwise.
(happened to me at least twice 
)
Thanks! That helps a lot! I only had the HaikuPorts Wiki to go on so I didn’t know what all the template entries were there for. I’ve commented out the ones I didn’t need for now. I guess having GitHub zip up the files into a zip file is their problem for now. Somehow I thought the git clone option might be faster.
A few more things that came to my mind:
Always remember that while HaikuPorter is written in Python, .recipe files are bash scripts (Scorpion’s “Unholy Alliance” plays in the background:-P), thus, anything there is/expects bash syntax.
Another tidbit: “.patchsets” files are “mailboxes” that Haikuporter uses, via “git am”. So, either get comfortable with “git am” (and/or “git apply” if necessary), or do it like a caveman (read: me), and “pkgman install patchutils” (splitdiff and filterdiff can be handy). I also have a patchset_split.py script for breaking them down into one file per commit.
Pe from its HaikuArchive’s repo has support for those from the “function popup” as well.
These little things help to navigate (and work with) big patchsets (at least for me, until I get better at handling patches with git :-D)
1 Like
Thanks for the tips! I sent an updated patch file in on the early WebKit_GTK project before. I added a variable definition inside a conditional in a CMake file. I had to be pretty careful with hp -c and hp -e but I got the endian specification set correctly for both endians.
Fortunately, this new package, W2C2, needs no patching. It’s fully POSIX compliant now and the patch it would have needed was welcome upstream! It was just a Linux cached value that not every Linux filesystem supplies so the pull request made the grade after some reworking. It didn’t end up needing any Haiku-specific code at all! Not even a __HAIKU__ preprocessor macro check!
I’ll keep this thread bookmarked in case I run into trouble again!
1 Like
haikuporter itself supports downloading using git, but there is no way to generate a SHA256 checksum in that case. For this reason, it is not allowed to use that in recipes built by the builder.
The SHA256 checksum allows us to make sure that the code we build is always the same. Not checking that would introduce confusion (a patch seems to work for one developer, and then a few months later, the upstream sources changed and the patch doesn’t work anymore) and also security risks (there is an attack scenario where someone corrupts or replaces a source archive while it’s being downloaded.
So, please point to an archive that can be checksummed.
As for getting the archives generated by github, it is tolerated, but Github makes no guarantees about them. They may change the code to generate these archives and this could result in different checksums. It happened at least once and broke a lot of packaging systems relying on these archives and checksums. But there isn’t a lot we can do about this, it is a problem for people using Github as their only software distribution that they have no stable/checkable/signed releases or not often enough.
1 Like
Are tagged GitHub releases just more of the same?
I don’t know. The archives are generated using the same code as the other archives, but there is maybe more long-term caching. I’d say Github still does not guarantee anything about them, so it could possibly still run into problems if they decide to regenerate all archives someday.
Since GitHub releases can store fixed tarballs and zips files within the release, that should definitely solve the issue with the signature hashes. If I’m going to the trouble of making a release tarball, should it be a rebadged Zip, .tar.gz or .tar.xz? There’s more steps involved in the later formats but they compress better. The xz files seem to compress much better than the others at the expense of the CPU cores needed to unpack them. I’ve noticed that they all seem to work in HaikuPorter, regardless. Is it all case-by-case?
Surely people “more in the know” will correct me if I err too much but…:
- HaikuPorter has to support all (or at least most) archives types, as it has no control over what each project’s mantainer decides to use for their source releases.
- Decompression speed should be of little concern (it’s the least of my worries at least, and the buildmasters are consistently two times faster building recipes than the hardware I use while updating recipes ).
- Smaller tarballs are nice, at least considering my slow internet, but… unless it saves several megabytes (unlikely for smaller projects), I would not worry too much.
So, I would say: find the best common ground between one format that it is supported by HaikuPorter, and one you AND other potential users of those source releases will be comfortable with.
2 Likes
That seems correct, since HaikuPorter has no control (and can’t have control) on all upstream sources that we package, we just support whatever formats are commonly used by software we package.
I would also say that, if you want to make a choice, smaller files are better, but do whatever is simpler for you.
Some of the recipes contain SOURCE_URI's that contain *.tar.lz archives, these can’t be build if cmd:lzip isn’t installed, maybe it’s missing from a default Haiku install?
What’s the recommended way of doing recipes for 32-bit Haiku nowadays? Should we use x86_gcc2 or x86 secondary arch?
In other words, do we still want to compile ported software with the legacy gcc2 toolchain or can we migrate to gcc11 when updating recipes? e.g. libpcap, tcpdump - we can build from (almost) pristine source tree with gcc11 but needs some additional patching on gcc2
1 Like
On most software, x86_gcc2 is the secondary architecture because it is the one that needs the most exceptional handling.
Most of the ports at HaikuPorts target the x86 secondary architecture for 32 bit version. That allows you to build it using the modern GCC compiler.
If you check at other recipes available, most of them specifically disable the old gcc2 version:
ARCHITECTURES="all !x86_gcc2"
SECONDARY_ARCHITECTURES="x86"
(the ! before x86_gcc2 means NOT)
1 Like
No need to support gcc2 unless it’s something critical to Haiku itself (and even then, we’re happy to find alternatives) or running old BeOS apps. For everything else, just use the gcc11 secondary arch, and you can optionally move the binaries to the main “bin” directory instead of “bin/x86”
4 Likes
Where do we store more lengthy patches?
Haikuports repo or shall I create my own repo?
Recently I was looking into rebasing @mmlr’s native qemu frontend to latest qemu-7.2.0
2 Likes
That"s a lot of work 
, but i would guess for a case like that haikuArchives would be the appropriate place?
Do it as you want. Ideally get it upstreamed to qemu so they take care of hosting it 
That’s what I thought but actually not so many changes are needed
I think I’ll just append it to my haikuports PR for qemu-7.2.0 for now
1 Like