Humans are responsible for machine-generated content we post, and we need to attribute it honestly

[jyasskin]

"AI" systems can produce incorrect assertions, long low-content rambles, and copies of other people's text. If people then share that output, it can mislead other people, waste their time, or steal credit from the actual authors. I suggest that all of these should be code-of-conduct violations, and I don't see existing elements of the CoC that apply directly.

• For hallucinations, "Deliberate misinformation" is close, but LLMs aren't conscious and so can't deliberately do anything, and the human generally also wasn't trying to misinform. Perhaps "negligent" misinformation should also be an unacceptable behavior?

• When LLM-generated text is longer than necessary, it unfairly offloads work from the author to readers. The ban on Gish galloping is similar, but the problem here isn't always about "many weak arguments", and this isn't an intentional "attempt to cause others to waste time"—it's just the side-effect.

• I think we're entirely missing a ban on plagiarism, and that's probably straightforward to add. However, plagiarism implies some amount of intent to "pass[]-off of another's work as one's own", and that's generally absent when a person pastes machine-generated text into a post. There are also gradations in responsibility here: When LLMs are asked to generate text, they're known to paste in portions of their input, so a person who generates text ought to take explicit steps to prevent that from happening. Attributing to "ChatGPT" is probably also not sufficient. But machine translation systems, even if they incorporate LLMs, aren't expected to incorporate quotes from other people, and if they do that instead of producing an accurate translation of the source, I wouldn't want to blame the human.

[torgo]

I support us to doing something here. I've seen uncritical use of LLMs causing major problems in adjacent domains - see Daniel Stenberg's great talk on the problem with AI-generated bug reports on the Curl project for one example. The common threads here seems to me to be that if people use this technology (LLMs), they still need to take responsibility for their work (that is, they stand behind their contribution / review / comment, including checking that they are not plagiarising another's work); and that if one person's use of AI creates additional work for other people, this should be considered a violation along the lines of what you've proposed, adjacent to gish galloping.

I think we can separate the use of LLMs for translation as a different class of problem?

[LJWatson]

I think this is worth recognising in the CoC. As a datapoint, at TetraLogical it's part of our AI policy rather than our CoC, but the need to acknowledge when AI has substantively contributed to the generation of information/content is there, and failing to do so can even lead to disciplinary action in egregious cases.

[hidde]

If people then share that output, it can mislead other people, waste their time, or steal credit from the actual authors.

Yes. I think all of the above consequences are in the way of our collective efficiency making standards.

Apart from the mentioned unacceptable behaviours, the expected behaviours elimate bias and confidentiality and privacy may also conflict with machine-generated content, as many machines that can generate content reflect biases that were in their training data, and some vendors are known to reuse input data for training.

For hallucinations, "Deliberate misinformation" is close, but LLMs aren't conscious and so can't deliberately do anything, and the human generally also wasn't trying to misinform. Perhaps "negligent" misinformation should also be an unacceptable behavior?

I agree with adding ‘negligent’. Maybe we can change “7. Deliberate misinformation. ” to “7. Deliberate misinformation, or misinformation due to negligence, including responding with unchecked output from language models.”

When LLM-generated text is longer than necessary, it unfairly offloads work from the author to readers.

+1, we could add one more example to 14, which does not have to be LLM-specific, as longer text than necessary is also disruptive when hand-written (we may have to remove the “bad faith” bit in the sentence before the examples, as this example would also disrupt when in good faith).

Something like “Overwhelming a debate with unreasonably long answers.”

I think we're entirely missing a ban on plagiarism, and that's probably straightforward to add.

Would not mind adding that, but think we should leave LLM's out of it, as the code of conduct applies to the humans in our commnunity, and I agree with you, we should blame the humans. There is often (forms of) plagiarism at play, but that is arguably committed by the LLM vendors.

I also want to +1 to @LJWatson's point there is a need for acknowledgement of when AI substantively contributed to information. This too could be an expected behaviour, maybe combined with @torgo's suggestion re taking responsibility… “Disclose when large language models substantively contributed to your contribution, and review contributions before bringing them up.”

[dbooth-boston]

The common threads here seems to me to be that if people use this technology (LLMs), they still need to take responsibility for their work

+1

Another thread is that plagiarism is not okay: it's not okay to pass off someone else's work -- or in this case, an AI tool's work -- as your own. Incidentally, this anti-plagiarism concept is slightly at odds with US copyright law, which allows the individual to claim copyright ownership of a work that was made-for-hire by someone else. Some other countries provide greater recognition for moral rights to a work that was made for hire.

“Disclose when large language models substantively contributed to your contribution, and review contributions before bringing them up.”

I like the first part of that sentence, but I'm uncomfortable with "and review contributions before bringing them up", because I fear that it subtly undercuts responsibility. I could well imagine someone claiming: "But I did review it before I posted it! I just didn't notice that little part where it says the holocaust didn't happen!" In other words, I think "reviewing" is not strong enough. People need to be held responsible for what the post -- regardless of what machine or human help they have in drafting it.

[hidde]

I'm uncomfortable with "and review contributions before bringing them up", because I fear that it subtly undercuts responsibility

Hm, point taken, and agreed, it's not as much about reviewing as it is about taking responsibility for the full text.

[amyvdh]

I wonder if it's useful to consider the W3C contribution, IP and document guidelines? Eg: is LLM content considered a derivative work? To whom would LLM text be attributed (or by whose rights would it be included)?
https://www.w3.org/copyright/intellectual-rights/

[jyasskin]

I wonder if it's useful to consider the W3C contribution, IP and document guidelines? Eg: is LLM content considered a derivative work? To whom would LLM text be attributed (or by whose rights would it be included)? https://www.w3.org/copyright/intellectual-rights/

FWIW, I think we should keep the policies for Contributions and text that goes into specifications, separate from the code-of-conduct discussion about text that gets sent to other humans, for example in issue discussions and mailing lists. We should be significantly stricter about Contributions to defend against all the legal uncertainty in this area, and I think we don't need to be that strict in the code of conduct.

[hidde]

Another aspect that we did not really cover here, is how throwing in LLM output can demoralise discourse (which could lead to people leaving discussions altogether).

Pre-LLMs, people would share their thoughts, and because putting something into words requires at least a minimal amount of consideration and deliberation, there would be some consideration and deliberation. A comment could be badly informed or ignore what others said, but that's about it.

Now, when someone can use an LLM to write a comment (from scratch, eg not cleanup/translation), they likely bypass a degree of consideration and deliberation. That is demoralising to the discourse, because considering and deliberating is core to what we're trying to do here when we make and discuss standards. We should want to hear their own voices, perspectives, thoughts, that's what makes the contribution meaningful instead of meaningless.

Saying people “are responsible for their output” and “should not waste” cover some of this, but I feel like there's something to be captured around valuing people's perspectives over what they made a machine output, with the goal of optimising for quality of the discussion and the resulting standards.

[TzviyaSiegman]

Discussed at PWE. @hidde will propose PR about misinfo. @torgo and @cwilso will propose a PR about disruption of discussion. Copryight/IP issues belong in Process or IP documentation. We will open issues there.