This assumes that using an LLM allows to develop better software or write it faster. This also remains to be seen, and was not discussed in this run of LLM discussion. As far as I know, the debate on this is still open. You can generate a lot of code very quickly, but it remains to be seen if this doesn’t lead the architecture to become spaghetti code or degrade in other ways.
Also, persnally my goal in Haiku is to understand how things work, and learn new things. I have no interest in delegating the interesting parts to an LLM.
If you want to bring new elements to the discussion, such as the fact that Haiku may be “falling behind”, please do not do so in a post that’s supposed to be a summary of previous messages.
Don’t forget about the option three: use LLMs for finding bugs and their causes in the source code.
100500 features is not the only reason why one may prefer one OS over another. More so if you remember that the OS is modular, so these 100500 features (including hardware support!) can be implemented externally.
But stability matters a lot more and can’t be implemented outside.
While I do share the concern and have to some extent observed it in practice, I expect spaghettification to be a much smaller issue on an OS than another complex software like a browser since an OS is quite modular by nature. Many pieces like individual drivers, kits and command line tools are quite self-contained.
However, one issue that I don’t think has been mentioned yet is maintenance. When someone develops a software or a piece of it by hand, they are invested in the project, simply because it is a lot of work to do so. That provides some motivation to also maintain it in the future.
If, however, the development is just a few hours of chats with a chatbot, that investment is significantly lower. Yet even with an LLM, development is not an automated process and requires substantial knowledge and architectural decisions to make a high-quality change. All that background information needs to be understood.
Without the original creator, the burden is entirely on the maintainers of a project.
Of course this is not a new problem and quite common with open source, but the lower barrier of entry with LLMs could significantly increase this type of hit-and-run pull requests where there is no intention at all to ever maintain the code.
This is the same for software developed using AI. I have been working on an IMAP email client since February, and while I have not released it yet, once I do put it out there, I would hate to see it rot.
Whether someone has the will to maintain a piece of software is not really a matter of whether one uses AI or not; there is plenty of open source software created by humans that has been abandoned by their authors after all.
A browser is also quite modular. There is an HTML5 parser, a CSS parser, a javascript runtime, a DOM model, all these things are very well separated.
But that modulrity didn’t happen by accident. It happened because people thought about the architecture and designed it this way. And that was done because it keeps the code more accessible to humans, you can fit one small part of it in your head at a time and understand how it works.
I don’t think this happens naturally when writing code. It’s always tempting to take shortcuts, make some quick hack that solves a small problem locally, rather than embark in a large refactoring. But the result of that tends to be that things are not done in the right module.
I think you write it better than I do
Many projects are drowned in low-effort, low-quality merge requests made with or even by LLMs. The result is not faster progress for these projects: it’s developer burn out, code going closed source or not allowing any outside contributions, people just giving up.
I’m sure there are success stories as well, but it takes a very different shape than how Haiku currently is developed. I think in that situation, the existence of forks is a good thing, and we will see what their approaches are and where they end up, how they manage to maintain architecture and documentation, how they onboard new contributors, which projects and challenges they take on, what they ship in releases, etc.
My experience with agentic coding and LLMs is that if you give an established code base to them and ask for specific tasks to be completed, they are very good at this. They do not rewrite the existing code unless there are bugs or they need to change something because of a request (interface, feature etc.)
If you ask them to plan something they will give you a well reasoned plan, which you can then tweak if you are unhappy.
If you ask for the plan to be implemented, they really don’t stray off that plan, unlike humans who have a tendency to “fix” things they see while reading code.
The issue of copyright and licensing is valid, but it is possible to write nearly identical code as a human solving the same problem without ever seeing the original code that was allegedly “copied”, so yeah.
I think planning and reviewing code is the most useful thing Haiku could do if actually writing code is not allowed. Reviews obviously include bugs and criticism of poor implementation. A good linter can also get you some of the way (for example the Sonar products), but LLMs go that extra mile by being able to give a specific explanation of the issues found.
It’s much easier to avoid taking shortcuts when using LLMS, because they aren’t afraid of moving code from a file to another, refactoring, writing interfaces, etc.
You can refactor a whole application in minutes with LLMs, without breaking stuff, because you can make it write tests before doing any changes.
Usually it will decide to write tests itself, without even asking you.
It would take a very long time to do that manually, and often humans just drop the ball instead.
One of the odd quirks of behaviors that you see over and over across the human experience is resistance to change. Those in the technology industry sees it more frequently than most, but the pattern is very real, and regardless of the change, the nature of the resistance is the same, and often fierce.
A group of invested people who thrive in change push something new.
A group of people who are not emotionally invested in the old embrace the new ( and push it on everyone else ).
A group of people who ARE emotionally invested in the old resist the new.
No one agrees. Tribes are born. No one wins these arguments, and they serve little more than to entrench the tribes. What brings change is not an argument in a forum, it is when products of the new bring enough compelling interest to the other tribes that resistance breaks down and they choose to move forward.
The path forward for the pro-LLM tribe it for them to use them responsibly and within the policies, while proving that statements made by the anti-LLM tribe do not necessarily hold up to a longer inspection. Knowing all the while, that there will be slop. Lord knows there has always been code slop, and purist behavior in the development world.
Need I remind some of the older folks reading this of the arguments made between the C developers on Windows, and the absolute meltdown and tribal war of words against the ‘Slop’ that Visual Basic brought by enabling non-programmers to create Windows programs cheap, easy and quick.
This transition is just another technology transition. LLM’s aren’t going away, the challenges will be resolved and the world will move on. Or it won’t, and we will all be slaves to our robot overlords.
I think it is well worth adding a 3. Monitor LLM enabled forks for code that can be merged to the main once the legal situation clears itself up.
I say this, because one of the things that makes Haiku so fascinating is that you can wholesale replace entire parts of the OS with drop in replacements after it’s is installed. Want to use an LLM to implement a device driver? fine, build it, package it, add after install. need to replace a service? do it, after install. There are a lot of options here that allow the use of LLM’s to enhance the Haiku experience, they just cannot be part of the core distribution at this time.
( FWIW, I’m about to go this route with a driver that I cannot for the life of me figure out what I’ve borked )
With the probably long-overdue change of removing the 10 minute wait between posts on the “Proprietary & Other” category, I’ll also go ahead and remove it here as well. Really, what it was meant for was the fear in the early days of the category that we would have a lot of heated conversations, but that mostly hasn’t happened.
And it has the cost of not allowing you all to edit your posts. Discourse makes it so easy for me to edit my posts on this topic. It doesn’t have any explicit reminder that regular users can’t edit posts! So, whilst I try to restrict myself to usually only doing what a regular user can do, I have forgotten and edited my posts quite a bit here! Now everybody should be able to. That should help level the playing field.
I think I’ll keep that summary just as a summary of the past conversation. But yes, thank you for pointing that out. It’ll be useful should I decide to write a new summary. Or, perhaps, more ambitiously, I’ll use it to write a wiki page summarizing the entire LLM controversy.
Edit: I have also added a disclaimer to the summary to make it clearer that it is not an authoritative summary.
I’m not sure what the point would be? If the forks are a Hard-fork that deviate massively then they are as good to port changes from as any other OS, if they are a soft fork (that tracks haiku upstream) they would just have additional stuff ontop of Haiku.
Pretending that differences in values are merely resistance to change is merely resistance to change is a common propaganda narrative that does not need to be exercised here.
Consider the Luddites: it was a workers right movement that used sabotage as their fighting method. Later worker movements used labor strikes instead. Surely all these people were fighting against the “progress”. The “progress”, where one had to work 11-12 hours per day.
A lot of people died fighting for their rights. The right to work only 8 hours per day was earned with blood.
The problem with that argument is the fact, that AI is inherently not deterministic. You cannot guarantee a reliable output from it. Even AI assisted coding it could put slight bugs in there. Even with all the documentation and code review. And the thing is it’s missing the point of what Haiku is, an alternative operating system. It’s not meant to use every new tool, what’s worked since 2001, has been working. Why fix, if not broken? (I agree with suhr here, sometimes new tech can be extremely detrimental like AI.)
I apologize if you read it that way. My point was simply that change is hard, and there is always pushback. Sometimes it is justified. Personally, I respect the position they have taken in the core repo. I am no way advocating that they adjust their position right now, only pointing out that at some point, most of the concerns that exist will be addressed, and only at that time does there need to be a reevaluation ( and the choice at that time may remain the same )
Easy, a human, an AI can fake logic and reasoning, a human has consciousness (typically.) Which means a human can doubt reevaluate to their skill level then submit. An AI once trained is a fixed skill and is unstable. (At least for me.)
P.S: I don’t know why I put typically there, don’t think unconscious or brain dead people can contribute to Haiku anyway.
PP.S To answer your question @Zardshard I think an AI produces more bugs, but feel free to prove me wrong! It’s not a easy to find out question.
AI may produce more bugs, but it is also capable of fixing those bugs. In my experience, you might go through 2 or more cycles to get a working solution with AI, but that is 1-2 hours. Humans might take considerably longer and regularly do and may still introduce bugs.