On The Merits Of LLMs

Amnesty International puplished a report where they conclude that current LLMs and generatiâe AI systems are detrimental to human rights for various reasons, and they call for a ban (at state or UN level) as well as several regommendations for companies which we could consider (the recommendations are at the end of the report): https://www.amnesty.be/IMG/pdf/20260528_rapport_unlawful_by_design.pdf

3 Likes

The same organization that had a loud voice when the police shoots a criminal, but keep silent when hundred of innocents are robbed or killed by those same criminals… They have 0 credibility, and hope they got kicked out of my country.

5 Likes

@3dEyes gave another example that we can use to compare classical and LLM based translators, so here’s one more example, thanks to @3dEyes :slight_smile:

Example that caused a misunderstanding

Original:

Google Translate classical translation (when given full context):

You have a poor understanding of the capabilities of modern LLMs.

Google Translate LLM translation (when given full context):

You have a very limited understanding of the capabilities of modern LLMs.

What is meant by the phrase:

I apologize for confusing an Italian word with an Esperanto word.

1 Like

My only exposure to Esperanto is Harry Harrison’s “Stainless Steel Rat” novels, so I’m not gonna judge.

A couple of belated points. English is fortunately my native language, but I use Google translate regularly with Portuguese. My Portuguese is just barely adequate, automated translation helps a lot.

  • Let’s not add translation to the forum and post in other languages. I usually edit translator output. Sometimes that’s really necessary (Google introduces stupid errors), sometimes it’s just a matter of style or something. What I post is my words, and I need this control over the version that people will read.
  • We’re talking about a technology that is very much in transition. Its performance accuracy today will not be relevant by the time the policy has been sorted out.
  • 3DEyes translation B appears to have uniquely gone with “firmly believe” at one point, where no one else did. That’s an example of what I believe is the kind of defect we can expect with AI outputs - like, “here, let me help you make this more impactful!” People who use these tools need to keep an eye on them to make sure they’re getting genuine translations, that are not juiced up.
2 Likes

People forget that there is a downside to everyone having instant access to every culture. With the rise of highly-efficient automated translation, we are now seeing a massive increase of harassment from westerners against other cultures, which can only be described as a form of cultural imperialism. You got people minding their own business, doing what is normal in their culture, and they get attacked by people from another country. This was rare in the past, because you actually had to take the time to learn a new language in order to harass people efficiently. The most ridiculous people were usually filtered out by this.

Now the game is on. If you don’t behave 100% like an American, no matter what remote place you live in the world, billions of people will come for you.

1 Like

This is a poor indicator of the “humanity” of the text. I can ask my local Gemma to write all the texts, including typos and minor errors, and even imitate my writing style, manner of speech, and other nuances, and you will have no way of telling whether I wrote it or not. You have a very poor understanding of the capabilities of modern LLMs.

1 Like

I read it as being a joke?

Regardless, it’s not helpful to the discussion to dismiss anyone who disagrees with you as being ignorant this way. Stating that you don’t think spelling errors are a useful indicator is fine; insulting the understanding of the other person is not.

1 Like

Here are the first victims of Google Translate’s incorrect translation (this and the previous post were translated using it). In Russian, the last sentence was completely innocent and meant something like “modern LLMs can do things much more complex than you can imagine.” But a literal translation without understanding the idioms and context ruined everything and turned an innocent statement into something offensive. I apologize if this sounded rude, but maybe it’s better to focus on the essence of the matter rather than try to take offense at everything possible?

3 Likes

Ah, thanks for clarifying. The tone that came across in the other message read more like a personal attack and something that could lower the productivity of the conversation here.

By the way, if you don’t mind, can we have the original version of the text? It might help the discussion in the other thread. :folded_hands:

1 Like

Hah, yeah. Nice to have a current example of that problem. :slight_smile:

Original

Это слабый показатель “человечности” написания текста. Я могу попросить свою локальную Gemma писать все тексты с опечатками и мелкими ошибками и даже имитировать мой стиль письма, манеру речи и прочие нюансы и вы не сможете никаким образом понять писал это я или нет. Вы вы слабо представляете себе возможности современных LLM.

In Russian, the phrase “вы слабо представляете себе” is a soft, conversational idiom that means something closer to “you might not fully realize” or “it’s hard to imagine.” It is not an attack on someone’s knowledge.

However, Google Translate translated it literally as “You have a very poor understanding,” which in English sounds like a direct, rude attack on someone’s intellect.

If I had translated that exact same Russian text using an LLM (which understands context and politeness), it would have output something like:
"You might not be fully aware of the true capabilities of modern LLMs.

So, ironically, by using a “classic” non-LLM translator, I accidentally generated a toxic, insulting post that almost caused a moderation issue. If I had used an LLM, the conversation would have remained perfectly polite.

This is exactly why I advocate for the freedom to use LLMs for translation. They don’t just translate words; they translate intent and tone, saving us from these exact types of cultural and linguistic misunderstandings.

6 Likes

Thanks for writing it up here.

I also admit to my mistake in not double-checking the text before sending it for possible misinterpretation - blaming the automatic translation for everything is a convenient excuse )

6 Likes

“Greatly underestimate” would be the right translation I think. “Слабо представляете” is not that soft.

Though one tends to speak a bit differently in different languages, because a culture is a part of a language.

3 Likes

LLMS generate “acceptable” text in the target language, with the major downside of matching the source much worse.

Beeing able to “tell” something is translated is a big bonus, as it avoids misunderstandings, it is expected that something like idioms are not perfectly understood or translated. Making something that “sounds” like it was perfectly written in english just makes something that seems insulting that much more prominent.

I would argue that the issue you mentioned here will be much worse with LLMs than without it, the issue here isn’t that you used a translation tool but rather that you knew this particular idom was translated badly and then chose to still post it… that’s not a “victim” of a translation tool, but rather your own choice…

I moved this discussion over to this thread since it deals with the merits of LLMs

Indeed it was a joke. I am quite aware that you can order an LLM to make spelling errors.

have we reached the point where humans can no longer detect humour without emojis to point it out? Then we might as well surrender to the LLMs.

Yes, that was another joke.

1 Like

I have not looked into the forums for the last few days and certainly don’t have time to really partake in this discussion a lot since I need to hand in some work very soon. Still, I just wanted to add something from my experience here:

Well, I might give you a very personal thing I associate with LLM usage:
At the beginning of the year, I participated at a game jam with two friends of mine. We formed teams who each tried to develop a game and there where quite a lot of people that have a lot of experience working on game design. I on the other hand had previously never really touched godot, which my team, like most others used and I had to learn it in parallel to the project. As a result, the stuff I produced was buggy, hardly usable and I struggled to get stuff to work the way I wanted it to. But could ask the people around me for help and across teams there was a lot of discussion, sharing and helping. This gave me both a feeling of joy and I now am much more proficient in programming with godot. I at no point actively used an LLM myself.

Thing was, at the final presentation, there was one person, who did a game by themselves. Whilst this by itself is fine of course, they did disclose that they “had some help by AI” writing the game as they did not have the knowledge about that. The game did not really have a noticable concept and on being pressed by a person in the audience they disclosed that the entire music was generated by a model. This was demotivating and frustrating for multiple reasons: The person tried to pass (the music especially) of as their own work,
next to teams which genuinely tried to produce stuff to the best of their abilities and even more:
The person could have just asked for help or join a team, as both participants and organizers tried their best to be open to new people. And the way the person presented it, they tried to pass this of as their own work. In a collaboration and learning environment like that game jam (there was no prize, no rating and finding the place where the game is published is quite difficult tbh) this is dangerous to its general atmosphere. It feels, in my opinion, demotivating to see someone in the same environment as you, given the same task and being expected the same effort go ahead to cheese the entire point of that game jam. The person was not required to participate in the game jam either and the main point there was to learn, collaborate and connect with other people. Sure, that person created a game, but I doubt that using an LLM to program really helped in really learning something and I very much doubt that they learned much in music production using an LLM.

Haiku is an open source project on which most contributors work in their spare time. I found the community to be really open and helpful if someone has trouble in both system usage or trying to contribute to the system. If someone wants to help but is unsure how, I’m quite certain that there will be a lot of people that help that person or try their best to point them
into the right direction. I have and will continue learning a lot of stuff from this project. Of course, someone could point an LLM on the Haiku codebase, quickly let it skimp over it and then “contribute”. Formally this would be contributing. However, this circumvents something which I see as crucial: learning, collaborating and connecting with other people. LLMs don’t really have a vision of a project, they don’t try to share knowledge and they don’t have a limit on how much they can do (a person on the other hand can only do so much work, thus
requiring to collaborate and offload work, giving others the chance to share their knowledge actively by for example applying different concepts to the same task).

For me, this is crucial to this project and one of the big reasons why I like contributing here. I love the collaborative, open and constructive atmosphere that Haiku has and as far as I can see, LLMs tend to break this atmosphere, nullifying the need for human interaction and thus destroying a huge part of what makes Haiku fun and rewarding to work on.

5 Likes