That sounds like exactly how it’s done. Be concise, as their patience is thin with any mention of AI. You may not want to not go too in depth on the actual fix. Just describing the issue and the general concept of a fix approach should generally be enough. And of course they can always ask for more details if they need it.
This can’t work that well, as you are basically trying to work backwards from a solution, instead of solving the problem.
For example, you might have a software that parses a config file wrong, an LLM might make a patch that makes it read that config file, but if you ask a dev the problem would have been “That file shouldn’t be there”
In any case, the proper workflow here would be use your step 1: find or reproduce a bug. Then write a ticket, and then stop there… Don’t go to a LLM to ask it why or how this happens, for one a LLM can’t “know” the answer, and for another in this situation you’d lack the skills to determine if the answer is correct or not.
This leads to some chocolate covered banana problems later down the path, so it’s best to avoid this alltogether.
In any case, Haiku as an open source project is a social thing aswell, asking other people for ressources to learn or asking them how or why something happens is a much better learning experience
(Anyhow, “in your own words” isn’t the problem neccesarily, it’s perfectly possible to write something in your own words, but it does not help at all if this is rewriting something the LLM generated… Here the idea would again be that developers wanting to fix some bug, with the help of machine learning, would have just used machine learning themselves already)
The example doesn’t even make sense. An individual not using LLMs may also come up with a solution to “read that config file”, and their idea could still be rejected when submitting it, because there are perceived better ways to do it. Doesn’t mean their solution was wrong. Doesn’t mean there aren’t better solutions out there.
Why not “go to a LLM to ask it why or how this happens”? They are actually very good at providing these answers. I am also weary of vibe coded slop from people with absolutely no coding knowledge who leave their AI agents in auto mode. Agentic engineering should be an iterative process where you are constantly learning and discovering new things about programming and the Haiku system.
I am not sure that there were any glaring errors in my bug report that got fixed, or in the various Gerrit CRs that were submitted with the help of @phoudoin. If I recall there was one CR that was rejected because it was based on my Beta 6 fix, and changes in Master since had made one part of it irrelevant. In the other case, where it was pointed out that there was an issue in the code, it actually turned out to be a bug in stock code that we had simply carried over. It has been fixed now on both sides. Which is the awesome thing. There are Haiku fixes coming from AI generated projects (albeit very slowly) and in that process, we are also benefitting from the input and guidance of the Haiku development team on these submitted patches. Which is exactly how it should work.
It just means they did not know the subject matter. If you ask an LLM for this and don’t know the subject matter you can not judge its output.
In the end you didn’t learn anything and wasted peoples time…
An LLM does not “think” and cannot provide any answers. It can only provide answers to precise questions in it’s training material, or heuristic answers unrelated to the question (but looking like answers). The only way investing time into an LLM makes any sense is when you already know how to do everything required by yourself, and in that case they just make you slower…
Not using an LLM and contributing directly would have been faster. Especially in this case… I don’t see that this is “how it should work” I don’t think Haiku should continue in a direction where people publish patches based on heuristics, with no way for anything to become a new core contributor, with the last few contributors trying to make sense of LLM bug reports…
You are also missing the simple part that a coin toss will also find bugs half the time. Heads for the kernel and tails for userland… If you just generate bug reports with absolutely no access to the source the expected result is valid bug reports… atleast some of them
I hardly think a short sentence or two on how an LLM solved the issue is going to waste too much time. On a bug report, the developer who takes it will come up with their own code and ideas. But a quick “LLM did this” could A) serve as a verification that your initial idea for a fix was indeed the right one B) may actually give you an idea you may had never considered, still with the freedom to implement as you feel is correct.
A does not apply. Many bugs can be worked around, and the problem space that can be worked around is much broader than what can be fixed.
as for B it might just aswell lead you down a wrong path… and inicidentally this is also one of the reasons why LLM driven development can be much slower, especially when you don’t know the subject well enough in the first place
I then write, in my own words, a description of the problem and a high-level explanation of how to fix it
A sufficiently detailed description of a problem already would suggest a solution, you don’t have to explain how to fix it. Just write a good description and point everything relevant.
I want to second this point. I have seen numerous LLM suggested patches for various terminals and terminal utilities that “work” in that the original behavior seems to be different, but sadly they are flat-out wrong. Understanding the right way often requires referencing things way outside the code itself – such as very old standards from the 60’s/70’s/80’s, developments and deadends in other terminals/applications, or simply gut engineering reasons to say “no, not here, not yet”.
However, I have also had very positive experiences with LLM-using developers who were able to provide good descriptions and occasionally screenshots of what wasn’t working right, and then have a discussion about what should happen. Then we can talk about the domain and let the hairy details come out. Afterwards, the code has been very straightforward to do. I like to think we all came away happier.
My so-far very-limited experience in submitting patches to Haiku has been that my reviewers are making me a better contributor. I still have a lot of patches to queue in for Terminal, so I am excited about where I might be in another few months.
There are tools like LLDB/GDB, Debugger, and Valgrind. But, the point is in knowing the tools to get the right information so you are not led down wrongful paths of easy deduction (or AI seduction, take your pick).
AI is a guide but better you the master craftsperson than leaning much on the tool itself. Then again, we do have our tools that make our lives easier…
I still have a lot of patches to queue in for Terminal, so I am excited about where I might be in another few months.