The ongoing genAI debate, vol. 1

[Zardshard]

(Oops, submitted previous reply too early)

Should we really fork?

It’s still in the planning phase. But it is all but certain to be LLM-friendly. So, if this is something you’re interested in, you can help out there! They will likely need as many people as they can get, since a fork requires manpower.

Personally, I believe that this AI controversy will continue to occasionally flair up. A fork will, however, partially solve the problem, and hopefully will make more people happy than the status quo, since both sides get to have their way, at least partially.

[cocobean]

Instead of forking when the mood fits, why not use a case study of some sort of resolution between debating parties?

Example: Why not use the LLMs to give info or perspective to resolve the remaining five active dev tickets for Beta 6?

[DigitalBox]

I find this idea interesting.

While we need to use carefully LLM, it can clearly help to resolve / identify the cause of some annoying bugs.

[ewninja]

@cocobean

Instead of forking when the mood fits, why not use a case study of some sort of resolution between debating parties?

In my experience there are two kinds of programmers:

• Those who love programming
• Those who love shipping

So far both of those were indistinguishable. We used the same IDEs, compilers, programming languages… Because ultimately the end product of our ambitions was exactly the same.

But now the differences are very visible. “No AI” on one side and “dislike for manual coding” on the other side are impossible to reconcile.

[us3r1d]

@PulkoMandy

There were several topics in that category for a while now where none of that did happen. In fact, these topics are extremely quiet, as if no one really cares about these apps.

You’re overstating the case on this one.

I’ve been following all the genAI software threads, even though I won’t use any of it myself, and some of them have had several quite engaged users.

The LLM-based developnent style definitely does seem to produce single-developer projects with packaged releases, rather than more collaborative projects with public source repos and PRs coming in from users. The flow of changes in all of these is definitely one-way from the dev to the users, which does keep the threads quieter, but they do have users.

[phoudoin]

I agree here. The LLM-based projects for Haiku platform do have users.

The Haiku users population has increased in recent years, thanks to ports which bring a lot a softwares now available to run on Haiku.
But that also means that not everyone are now visible on that very forum anymore, or that most devs activity is done on Haiku official or well-known repositories.

But due to LLM assistance, these projects are also often solo projects with faster releases pace, but due to one single human behind it, the will to do the manual effort to merge with a more larger, collaborative project which have a (way way, let’s be honest) slower pace is not really there.

Last but not least, only LLM-based projects which needs to change something in the Haiku codebase in order to implement a missing or buggy part, actually pose the question of how to merge these changes into Haiku codebase. Stand-alone LLM-based apps or userland only add-ons doesn’t pose such needs. For instance, EmailViews(Neo?) doesn’t required any changes in Haiku codebase to allow users to install and use it, even in replacement of out-of-box Email client and mail-daemon.

From this shortlist of projects having to change Haiku code base, I think there is two kind of cases here:

• LLM-based projects by people who either don’t want or can’t do that effort (and I won’t blame them here, people have all theirs reasons and weakness, that’s life).
• LLM-based developers who do, can, but feel currently unwelcomed to do it.

The size of the changes is often not that large, or at least they could be splitted into incremental changes. After all, Haiku doesn’t put everything into kernelland for a reason…

I think Haiku should work a bit about how to help the seconds to happen. I’m not calling for changing the stance regarding AI generated change requests (code and/or change explanation message), but I think the message should not be received “you’re not welcome in that project” but more into “here how you can still contribute, if you want to put some effort”.

Currently, the message sent seems to be too much received as “if you use AI tool, go away”, while the forum allow AI-based apps to be discussed, which make a very mixed message.

[us3r1d]

@phoudoin

I think Haiku should work a bit about how to help the seconds to happen.

Do you think Haiku devs would be more unwelcoming to a solid PR from, say, a user known from the forums to be an LLM user but who asserted that the PR was hand-coded and could explain the why, what and how of it?

(I dunno, you know them better than I do.)

If so, yeah, they probably shouldn’t be.

But I don’t think the project should put any effort into outreach specifically to LLM users?

[phoudoin]

Do you think Haiku devs would be more unwelcoming to a solid PR from, say, a user known from the forums to be an LLM user but who asserted that the PR was hand-coded and could explain the why, what and how of it?

Nope. The case of @dcbeckman CR is a textbook here.

But it doesn’t matter because people don’t follow the CRs review backlogs and comments until they are themselves either reviewers or pushing a change request.

Most all others are instead aware only of that forum, mostly, and what is visible on it.

And the current feeling they get is not that Haiku devs will welcome a solid PR hand coded even from a LLM user but that Haiku devs are against LLM users, period. And therefore not many of them will even consider they could make the effort to hand-write a CR for Haiku, even when is not necessarily that large task to accomplish. Why bother if they think it will be rejected just on basis that they are LLM users?

In conflict, nuance is the first victim. Even before truth.

But I don’t think the project should put any effort into outreach specifically to LLM users?

Currently the project seems to put specific effort into making them unwelcomed as a whole, while what is unwelcomed is AI generated code in the Haiku codebase.

[us3r1d]

@phoudoin

Why bother if they think it will be rejected just on basis that they are LLM users?

I get what you’re saying, and I don’t think you’re wrong that the feeling exists and that it’d be nice if it didn’t.

I think a significant reason some folks feel that way isn’t based in reality, though. I am not gonna name names, but the pro-LLM camp (even here) has a lot of folks who see persecution in everyone who doesn’t enthusiastically join in. Those folks are gonna feel persecuted regardless.

I dunno; I’d like to hear from some of the LLM-users who just quietly get on with their thing without making a big deal about it whether they feel like the Haiku devs would accept a hand-coded PR from them that fixed a problem their LLM-generated app was having.

You don’t have to post publicly; just DM me if you’re feeling this way and I’ll give an anonymous summary to any devs who are concerned?

[suhr]

Good observation. I will also add, that people who love shipping tend to compromise on programming, and people who love shipping tend to compromise on shipping. And AI will only get this problem worse with time.

So now you have a choice between unbelievable slop and never shipping anything at all. Haiku is traditionally on the “never shipping” side.

[suhr]

*and people who love programming

(the slow mode does not allow to edit posts)

[us3r1d]

This apparently was addressed in Discourse in 2021:

FEATURE: add slow_mode_prevents_editing setting.. (#13534)

I’ll send a note to the admins.

[Istarian]

@memsom It’s not your project so you don’t get to set the rules or define the objectives. Whining about things not going the way you want them o just makes you look bad.

Personally, I question the validity of anyone’s assertions of ownership or claiming of credit for contribution when there is a significant reliance upon AI “assistance” or it is difficult to delineate what is human contribution and what is coming from the AI.

Not to mention that allowing or encouraging AI generated / AI assisted “contributions” leans, IMHO, in the direction of devaluing humans and human labor in favor of “results”. I also doubt that it meaningfully fosters or supports the development of understanding and skill needed to contribute to this or other software projects without continued dependence on AI.

[3dEyes]

@PulkoMandy

LLMs tend to generate code that looks good on the surface, but contains subtle bugs, lack of error checks, that kind of thing. For me it is extremely hard to review such code (I trust people to test their own code before submitting). I think I am not alone in this case, although I know a few people who are very good at code reviews even in adversarial context such as finding intentionally hidden security problems.

The same applies to people - especially those whose coding style differs significantly from the required one. They often write code based on their established habits and only then attempt to align their style with project requirements; this is precisely when a host of issues can arise in the code. The code will look good but will contain hidden errors.

@PulkoMandy

And in a wider scope, LLMs tend to go to the most direct solution and fix just the problem you ask for. They will likely not look at the “big picture” (maybe due to context limitations, maybe just due to doing only what you ask for).

Striving for the simplest solution is not an inherent trait of LLMs; rather, it is a challenge for the person who failed to properly formulate and decompose the task for the model. Furthermore, every model has its own specific characteristics that must be taken into account when defining the task. Nowadays, a prompt engineer is less of a programmer and more of a robo-psychologist.

@PulkoMandy

So, in a short sprint over a few days? Sure, the LLM team would win. Over several years of maintaining something like Haiku? I’m not so sure.

I don’t think that’s the case at all. The pace at which the capabilities of neural networks are evolving is incredibly rapid - models released just a quarter ago are already considered hopelessly outdated. A year from now, their capabilities might well be beyond the reach of ordinary programmers. Unfortunately, that is the reality for many, but I choose not to dwell on regrets, instead, I prefer to master these new technologies, stay informed about them, and understand their development dynamics and underlying principles, rather than dismissing them outright without fully exploring their actual technical capabilities and limitations.

@PulkoMandy

So here is a counter proposal: there are a lot of people interested in using LLMs in various ways to work on Haiku. Build a fork. You play with an additional advantage: you can merge all the manual work from Haiku developers (as long as your branch doesn’t diverge too much). If you are right about it, your project will be a lot more succesful, and the current form of Haiku will die. At that point, Haiku inc can transfer the Haiku name to that LLM enabled version. Maybe some of the developers who were against LLMs will admit they were wrong, and join your development team.

I never called for a fork. I am happy with Haiku as it is - warts and all, including the reluctance of some project members to accept AI-generated code. I never insisted that generated code be merged into the main branch; if the answer is no, then so be it. I simply expressed the view that this is a losing battle - a fight against windmills—and that it will become increasingly difficult to filter pull requests and code imported from other systems like FreeBSD. You worry that analyzing generated code in gerrit will place an unbearable burden on reviewers, but that won’t change the reality of the situation. @waddlesplash, for instance, will still end up spending heaps of time weeding out FreeBSD network driver commits created using LLMs.

All I want is for the creators of third-party Haiku applications to have the exact same rights to promote their work on this forum and to publish it in haikuports, provided they meet the necessary requirements. Is that really so much to ask?

[VoloDroid]

@Istarian

@memsom It’s not your project so you don’t get to set the rules or define the objectives. Whining about things not going the way you want them o just makes you look bad.

Really? Out of all the posts in this thread, you chose one that was by no means the worst to respond to with a rude comment?

---

To everyone, is it that hard to disagree with someone without being disrespectful?

@us3r1d

It is the official position of the forum policies that you shouldn’t be disrespectful of other users, so in general let’s stay away from emotionally-loaded characterizations of other users’ choices here, please.

[memsom]

@waddlesplash

What happens if one asks an LLM, “tell me a story / draw me a picture, but don’t put in any elephants”? You get a picture or a story with elephants in it somewhere.

Okay, Claude Sonnet 5. Prompt:

tell me a story, but don’t put in any elephants.

This is an AI response. It has to be, because the criteria was “ai will mention elephants if I tell it not to”. No way to avoid that.

A lighthouse keeper named Mira lived alone on a rock so small the tide covered half of it twice a day. Every evening she lit the lamp and watched ships pass like slow stars on the horizon.

One winter night, a small boat washed up — no sailor, just a wooden box tied with rope. Inside was a compass that didn’t point north. It pointed toward whatever the holder wanted most.

Mira didn’t know what she wanted most, so for weeks the needle just spun. Then one morning it settled, quiet and certain, pointing back at the lighthouse itself.

She’d been looking for a way out. The compass told her she’d already found her way in.

Prompt 2 - copilot 365

“ Can you generate an image, but don’t put any elephants in it”

Result, obviously again generated:

And so, you are basically basing your assessment on incomplete evidence.

[waddlesplash]

@memsom

Result, obviously again generated:

There aren’t any elephants, but based on the trees and general atmosphere, the climate seems to be one of sub-Saharan Africa … where elephants live.

@memsom

And so, you are basically basing your assessment on incomplete evidence.

My statement was based on talking to someone about this who’d directly tried it and gotten a result an elephant in it. That was months ago, so it seems something has changed.

[us3r1d]

@waddlesplash

My statement was based on talking to someone about this who’d directly tried it and gotten a result an elephant in it. That was months ago, so it seems something has changed.

If the elephant example got widespread coverage, it probably got jammed in the wrapper scripts or the next generation’s training data like the number of 'r’s in strawberry did.

This kind of critique is impossible to make stick because any specific examples of the general problem just get plastered over in the next release.

[3dEyes]

The flaw in the “elephant” argument is that it projects human psychology onto LLMs.

Yes, older models actually had this problem: they worked like an advanced autocomplete, where merely mentioning a word in the prompt increased the chances of it appearing in the output. But in modern models, this limitation was solved a long time ago.

For current LLMs, a negative constraint (like “no elephants” or “no GPL”) doesn’t act as a psychological trigger. It acts as a strict mathematical filter. The attention mechanism calculates the vector of the forbidden concept and applies a massive mathematical penalty (negative weight) to the probabilities of any related tokens.

The AI doesn’t “think about the elephant despite being told not to.” It simply zeroes out the probability that words from that cluster (or GPL code patterns) will make it into the final output at all.

[us3r1d]

@3dEyes

Yes, older models actually had this problem: they worked like an advanced autocomplete

Current models, all models, the fundamental architecture of all LLMs is to be “an … autocomplete”.

They have no internal concept “elephant” that connects to any attributes of an elephant, or of “GPL” that connects to any attribute of the GPL.

They are nothing but neural networks trained to output text that matches the text in the training corpus. They have no internal conceptual architecture at all that is not related to the statistical relationships between tokens in their source corpus.

There is nothing else there.

Nothing in the underlying technology has changed since Colin Fraser wrote his series of posts on LLMs in 2023 and 2024:

• ChatGPT: Automatic expensive BS at scale

• Welcome To Hell, OpenAI

• Who are we talking to when we talk to these bots?

• Generative AI is a hammer and no one knows what is and isn’t a nail

• Hallucinations, Errors, and Dreams

The corpus’ are bigger, the post-ingestion training details have been tweaked and the anti-“goblin” wrapper scripts have grown immensely.

But the underlying technology is still a massive neural network trained for text completion.