AI-generated Answers experiment on Stack Exchange sites that volunteered to participate

This remains troubling.

• Committed to the Stack Exchange network being a place for human-curated knowledge and information. This experiment explores how that can remain the case in a world with GenAI, while investigating possible new workflows that could benefit existing community members and the next generation of users.

• Committed to building solutions that add value for users on the platform. LLMs are part of the world now, and any potential integration must be explored responsibly, in ways that not only provide value (task completion, closing knowledge gaps, etc) but also create transparency, keep humans in the loop, and encourage human contributions.

I fail to see how adding generative AI to the platform demonstrates a commitment to the "network being a place for human-curated knowledge and information" or the company's commitment to "building solutions that add value".

I don't see evidence that expert humans are interested or willing to edit or curate AI slop. If you have information from people - and not just regular or high reputation users, but people who have truly demonstrated deep subject matter expertise who are qualified to curate generated content for completeness and correctness - that says otherwise, I (and likely the broader community) would be interested in seeing that.

This also seems to be a deviation from the SE mission. The basis seems to feed the idea that the SE network is a place for people to get answers to their questions. That isn't the purpose. The network is a knowledgebase of questions and answers and we don't exist to answer every question that a person may have. Feeding the idea that if humans haven't written and vetted answers in 72 hours seems to placate the idea of getting fast(ish) answers to a person's specific question rather than building a knowledgebase of human-curated content.

Just because you're limiting the view to moderators and high-reputation users doesn't mean that those people can effectively validate the content.

It’s true to say that many of the people involved in those conversations expressed concerns about moving forward with this experiment. We also heard from people cautiously optimistic about the concept. The feedback heard was instrumental in getting us to how the experiment is designed today.

I would recommend breaking this down a little more. Who were the people who were concerned? Who were the people who were cautiously optimistic. If my suspicions are correct, the knowledgable curators are the ones who are concerned and people tending more toward answer seeking were cautiously optimistic. The weights of these two groups of people should not be the same, with greater preference toward the curators.

Right now it’s not included since the GenAI output does not consistently provide that. We’ve stated that attribution is non-neogitiable and that goes both ways. We are determining what sourcing data will be delivered from LLM providers along with the private AI-suggested answers. For the purposes of this limited experiment, we feel it’s still worthwhile to test out the concept and user interactions.

If attribution is non-negotiable, why are you investing anything until that non-negotiable thing has been demonstrated? It feels like attribution isn't non-negotiable if you are proceeding with testing a tool that doesn't (and likely can't) provide the level of attribution expected from SE network answers.

I just saw this after talking to a recently resigned mod...and this makes me want to leave too, honestly. I could ask an AI myself, I don't need Stack Exchange for that. You are making this site useless.

If attribution is really non-negotiable, then why have you decided to ignore it (for the time being) and publish this without it?

Do you have any way planned to prevent robo-reviewers who mark all AI answers correct? And a way to prevent just plain wrong reviewers? 50 rep is enough that someone who has never visited the site before is somehow trusted to review an AI answers. Why is it that it takes 350/500 rep to review in some queues, up to 1k/3k in others, but you trust 50 rep users to review this?

Why do you feel it okay for you guys to post "checked" AI generated answers while it is suspendable for users to post checked AI generated answers (thinking about this incident specifically)?

Most importantly, why should I stay here? Tell me, why shouldn't we all leave and never come back? What value is Stack Exchange now providing I couldn't get myself by asking an AI? Why should I put effort into asking a question to get the same thing I would get if I asked to do an LLM?

Oh, and why won't you tell us what AI you are using? That's pretty weird and I can't imagine why you wouldn't be able to share that.

Stack Exchange is...Committed to the Stack Exchange network being a place for human-curated knowledge and information

In that case, why are you adding AI answers which will inevitably replace at least some of the human-curated knowledge on this platform?

Not interested in any outcomes that might dilute the value of the platform.

AI answers, even if correct, would still dilute the value of the platform.

Currently, Stack Exchange is useful because it provides value I can't get somewhere else. Codidact, while great, is quite small, and there is really nowhere else where there are humans answering questions in a formal, curated, high-quality manner. That's what makes this place unique, that's why I'm here.

If you make this site so it provides nothing I can't get from asking an LLM, you've destroyed this site, not just for the community, but for you guys as well. You make no money if users don't go to your website, do you?

Questions that meet the following criteria...Posted in 2024 or 2025

Why? These are actually much less likely to receive value from an AI answers (as it is very likely the user who posted this was aware of the option of asking an AI and decided against it or the AI wasn't able to solve their problem)

We want to test if this feature could help improve the answer experience

This won't improve the answer experience, it will just make it totally non-existent in same cases. Do you even understand what you wrote there?

We remain open to concluding the experiment early if we find the results unfavorable for any reason.

Who is we who "find[s] the results unfavorable" here? The company? Mods? The community? ChatGPT?

If it is the company, as I suspect, is "everyone is really mad at you and thinks its terrible" considered an unfavorable result?

If you are asking us to edit and review AI written answers, then why shouldn't I just write an answer of my own and take the reputation for myself? If I can review that the answer is correct, then it means I can also write an answer myself.

(Trying to ignore the severely ironic banner above the edit box as I write this, which says "Reminder: Answers generated by artificial intelligence tools are not allowed on Meta Stack Exchange. Learn more"...)

At the risk of making this all sound like a non sequitur, I would like to point out that the way these "experiments" (i.e. soon-to-be-features-network-wide) are managed shows lots of parallels with the situation of content creators facing today's AI apocalypsoid.

There's this viral tweet that succinctly says

You know what the biggest problem with pushing all-things-AI is? Wrong direction. I want AI to do my laundry and dishes so that I can do art and writing, not for AI to do my art and writing so that I can do my laundry and dishes.

I've also come across this more verbose blog post by comic artist group War and Peas. It explains one of many instances when they were contacted by some AI grifter company, offering to have their creative work outsourced to an AI slop factory.

I won't quote the whole piece (it's not long, and worth reading in my opinion), but here's the artists' reasoning from their polite middle-finger reply:

[...] we must politely decline. Here are some reasons why:

• The surge in AI has been built on the backs of creative people like us. Artists’ work has been harvested in order to train large language models and they have not been informed or compensated.
• Social Media platforms cannot grow without engaging and original content, made by people. This is the basis for any user to join. Creators have – in large part – not been privy to the rewards that these platforms have generated over the years even though they are the reason for platform growth. Any business model that does not acknowledge this and does not seek proper compensation for artists is not of interest to us.
• While being creative is surely not always fun, most artists love their work and do not want to outsource their passion to a machine that does this for them. They would much rather be properly compensated for their work in order to continue to be able to do said work. Frankly, storytelling “without the need for personal content creation” sounds horrible. Storytelling is personal. It is a connection between the writer and the reader. Without this personal connection, storytelling loses its purpose.
• Artists would much rather have an AI that actually helps with grunt chores, such as writing invoices, or helping with taxes in order to focus more on fulfilling creative tasks. Such an AI-tool would be of much more value to the artist community.

Why am I pointing to the above? Most of that also applies to the Stack Exchange network, and Stack Overflow where I'm mostly active. It baffles me how tone deaf the company is regarding this matter. I'm way past assuming good faith, all that could be said to raise our concerns has been repeated ad nauseam already. You the company keep pushing AI garbage onto the network, blatantly disregarding both how much harm it does to the platform and how detrimental this is to the communities that make the platform (your product) viable. All in the name of generating more profit, because frankly, there can be no other reason. And yes, I still don't think the company is just trying to pay the bills (link to comment thread about hypothesized intentions of a company that acts like only profit matters; sorry if the comment thread gets deleted eventually, you can find an archived version of the page here, look for the comments under the answer from TylerH).

To make my point perhaps a bit more obvious, let me paraphrase the artists' points above, applied to our your network:

• The surge in AI has been built on the backs of experts (and creative people) like us. Contributors' work has been harvested in order to train large language models and we have not been informed or compensated.
• The network cannot grow without original and correct content, made by people. This is the basis for any user to stick around and keep the network alive. Contributors have not been privy to the rewards that the AI platforms have generated over the years even though we are in large part the reason for their success (especially Stack Overflow for chatbots trying to solve programming problems). Any business model that does not acknowledge this and does not seek sustainable collaboration for contributors is not of interest to us.
• While solving other people's problems is surely not always fun, contributors love this hobby and do not want to outsource their passion to a machine that does this for them (especially when it does it much worse). They would much rather be empowered to curate the knowledge base effectively in order to continue to be able to do said work. Frankly, solving problems "without the need for personal content creation" sounds horrible. Answering questions is personal. It is a connection between a problem and an answerer. Without this personal connection, problem solving loses its purpose.
• Contributors would much rather have an AI that actually helps with grunt chores, such as educating new users, triaging new questions or finding duplicates in order to focus on more fulfilling problem solving tasks. Such an AI tool would be of much more value to the Stack Exchange community (honestly, it needn't be AI, please don't make it AI).

Expecting that the contributors will start curating AI garbage when we're already wary of any content written with suspiciously elaborate English, willingly and for free, is an affront to the intelligence and passion of the people who keep this network afloat (but rapidly sinking). It completely chooses to miss the point of all the pushback you've been seeing due to the mindless/unscrupulous plugging of AI garbage on the network. It ignores all the reasons why AI was banned by the community in the first place. I hope the inevitable graduation of this "experiment" to a feature that defies all objective feedback you will have received (and eventual implementation on Stack Overflow) will finally put the network out of its misery.

I think it's worth asking, why do users answer questions? Reputation points aren't that valuable of a commodity. I'm not getting paid to answer. I'm doing it because I want to share my expertise and help people learn. I enjoy writing up explanations and helping people with their problems, and interacting with a community of other human beings who answer my questions in turn when I'm confused about something.

Now StackExchange wants me to spend my free time curating AI slop instead. My question is, why should I? Why should I spend my free time reading these answers, which nobody cared enough about to bother writing them in the first place? What do I get out of this that's worth my time and effort?

If anyone wants an answer from some GenAI, they'll ask a GenAI. This service is already available all over the web, by approximately 999 other companies.

First of all, many people come to SE explicitly because they want an answer from a human expert and not from an AI.

As one version of that, SE has in latter years turned into the place where people come after they already tried to ask an AI but only got nonsense as an answer. It has become common that people write things in their questions like "I tried to ask ChatGPT but it didn't work".

And the reason that didn't work is that GenAI is rather useless for moderately advanced questions and topics, for example about programming. It cannot be trusted because it is simply too bad. The state of GenAI is not what you think it is.

You get back garbage, lies and hallucinations mixed in with portions of correct information - aka the best kind of lies. Such a messy AI reply often can't be salvaged by editing because what's written simply doesn't make sense. Why edit things that hold no value?

Furthermore, the GenAI often blatantly contradicts itself in many places, so even when editing there's a chance of some of that madness slipping through. Ask it a somewhat advanced question and you might find statements in the answer along "A equals B.... B does not equal C... A equals C." The GenAI isn't even capable of proof-reading such basic things, because it just generates text, which doesn't necessarily connect with previously written text in the same answer. This happens more often than not - what the AI is lacking specifically is intelligence.

The subjective sales argument made by many is something like: there are a few lost souls out there who have never heard of GenAI but will get pleasantly surprised when a low quality answer is provided from one at SE. Do you remember that moment when you want to ask tech support why your Internet isn't working and you got pleasantly surprised that you get to talk to a chat bot instead of a human? No? Me neither.

From there, common sense leaves us with the following conclusions for this new feature:

• Potential target audience: nobody.
• Purpose of salvaging AI replies: pointless busy-work and likely more work that just writing a completely new answer.

We can't have intelligence, artificial or otherwise, if we don't even apply basic human common sense during project idea evaluation.

I don't support this experiment at all.

I get the point of experimenting and verifying whether some idea can pass in real life. The problem is when you start with a bad idea (and you get a feedback about the idea itself) and then you still pursue the bad idea with hope that some magic will happen during the experiment. Sorry, but running experiments based on bad ideas are only a colossal waste of time and money, and in this case, a good will of your users.

Following is my response from the Moderators SOfT. Not much has changed in the meantime. This experiment was a bad idea then, and it is still a bad idea now. Reading some of other answers, it is clear that we share similar concerns.

The first problem with this experiment is the difference in starting points. You (the company) are starting from the position "We want to do this. How can we make it work?" and we are at the position "There is no way this will work, even as an experiment".

It will be very hard to find common ground in such circumstances. You are standing on one cliff, we on the other and all we can do is yell at each other over the abyss.

The issues with AI answers are multifold and even conducting the experiment itself will be highly problematic.

What do you currently have?

The purpose of the SE network is to create large repository of (human) knowledge where people can easily find answer to their questions. While I cannot say how well is that goal achieved in other areas, your technical sites, most notably Stack Overflow, have been successful in achieving that goal. You have the largest community of experts in their respective fields in the world. People can interact and get their answers directly form the very people that are developing technologies they are working with. You have employees from the largest tech companies like Microsoft, Google, Apple. Book writers and educators, MVP-s and tech gurus, professionals with decades of experience.

In two words you have knowledgeable PEOPLE nobody else has in such concentration in one single place.

What will you have with AI answers?

If you start posting AI answers on sites, you will be just another place on the Internet with AI something slapped on.

What would be the difference between such Stack Overflow and using Copilot directly in some IDE? Or chatting with AI in the browser?

Why would anyone come to Stack Overflow and other sites in the network to get AI generated content they can get it more easily and faster anywhere else?

Why would someone type question in Stack Overflow and then wait few days to get AI answer that may not even be correct, when they can just ask AI directly?

If you put the AI answers on SE sites, you will no longer have then single best selling point: answers written by people. You will become just another AI dumpster fire.

What is the purpose of this experiment and expectations?

There are three aspects of every problem with this experiment:

1. What AI companies want?
2. What SE as a company wants?
3. What SE communities want?

When it comes to SE communities, we may also need to separately look at differently sized sites and their topics and we also need to take into account different categories of users as issues will vary.

• Sites based on size

  • Stack Overflow
  • larger SE sites with enough traffic
  • smaller SE sites with poor traffic

• Users (communities)

  • moderators
  • curators
  • answerers
  • askers

1. What AI companies want

Note: This was written with the presumption that AI companies with which SE is in partnership are also somehow involved in this experiment, even though this was not explicitly stated

This is simple, AI training companies probably want to have some additional input and feedback process that will help the assessing the correctness of AI responses which they could possibly use to further improve and tweak AI models and algorithms.

But, the results they will get highly depend on two things: traffic and user engagement with AI answers.

Which immediately surfaces several problems: the only site with significant question traffic is Stack Overflow. On all other sites traffic is too low to run any viable analysis. Another issue that makes this issue worse is that AI answers will be posted on unanswered questions which reduces the numbers even further.

This also means that primary target for this experiment (and further application) is Stack Overflow. Which is highly problematic, as Stack Overflow is probably the last site in the network where this experiment would be welcomed.

And I suspect that it is not only Stack Overflow where community has such stance, and it is likely that even talking about plans in the public will have immediate negative consequences for users' participation.

2. What SE as a company wants?

The answer here is probably not a clean cut. SE probably wants to get some money from AI companies, and at the same time increase the engagement and lost traffic on sites, which also translates into a money loss.

There is also a question how much money is in play, and for how long. In other words, are AI companies willing to pay only for a short period (during some preliminary experiment), or is there some long term interest.

It is very likely that taking money for this experiment will be an equivalent of drilling down and destroying the foundations of your house because someone will pay next winter's heating bill. You may be warm for one season, but soon after when the house collapses, you will no longer have a roof over your head.

How can that happen, we can see after we analyze what the communities want and how would they be impacted by this experiment and subsequent permanent implementation.

3. What SE communities want?

Moderators

We absolutely don't need additional work, especially on Stack Overflow, where moderators are still clearing up pending AI flags. AI answers experiment will certainly increase our work load as there will be more users who will take this experiment as a permission to post AI answers on their own. No matter in how many places you write that AI answer are not allowed, once you have them yourself, people will also start posting them.

The argument with AI being forbidden because it is not vetted will not hold water because plenty of AI posters claim they have vetted their answers (even when this is blatantly false). We will also have hard time explaining why community can validate AI answers posted by a bot, but is not able or willing to verify those posted by users.

The scale arguments will also not mean much as if posted on unanswered question, there will be significantly more AI bot answers posted than ones people are posting. If you want to say that those will not do much harm because they won't be visible, then what will be purpose of having them if most people will not be able to see them (get answers to their questions).

Curators and Experts

Curators, like moderators will have similar issues. While they will not be drowned in flags, they will have more posts they will have to flag and spend their time on. Considering that most curators are also experts in their fields, that means they will spend less time posting valuable content.

Overall, moderators and curators stance against AI answers, regardless of their origin, is well known. Even announcing this experiment clearly shows that company does not want to take input from us about AI features implemented on sites. This feature goes against everything we have been doing in last two years since AI LLM models came to existence. It directly goes against our wishes.

Pushing this experiment can seriously backfire. People are tired of fighting against company for no avail. Time after time, company has shown that it does not have community and our interest at heart. You (the company) only want to go forward with your own vision, regardless of the impact it may have.

There will be no next strike. People will just leave for good. We have already lost moderators and valuable members of the community over the previous issues, but nothing will compare to this. It will be just the last drop (more like a flood) which will make them leave. It might not happen immediately, but with time participation from those most active members will certainly die out.

Answerers

When it comes to people posting answers there are very different kinds and for some that are just concerned with their own reputation, this may not mean much. But, those are not really concerned by quality and they are also the ones that are most likely to post AI answers of their own.

However, the most valuable answerers are the ones that are not here just for the imaginary Internet points, but also for teaching and learning experience. The ones that really have something to offer either now, or in the future when they become experts themselves.

But if the other experts and curators start leaving, they will no longer be around to provide guidance to the new answerers. Users that are just starting on sites and also might not have enough knowledge to write the best answers at that time, but will be able to improve after receiving comments and suggestions.

Askers

While it seems that having answers posted on unanswered question may be a good thing for askers that have received none, I doubt that this feature will be wll received among them either.

Many people who are coming on SE sites have already tried to solve their problem with AI, but they failed and they have turned to the experts. They don't want another AI answer that does no solve anything. You can find many comments form low reputation users that are just starting on sites, being angry when someone posts some AI nonsense answer on their question, because they have already tried AI, but it haven't helped.

Also receiving possibly incorrect AI answer after few days, is too little, too late. If they wanted to use AI, they would have already used it directly and with much greater success as they can interact and refine AI responses.

If the question was on topic, but hard one, AI will probably not be of much help. If the question is off-topic or low quality, AI answer will just bring more attention to it, leading to closure and down votes and there will certainly be users who will notice that correlation and will be less than happy about it.

Chances that someone will get genuine help from such AI answer is minimal and it will not increase participation.

The experiment

Now let's focus on the experiment itself and potential problems.

Numbers

As previously mentioned, except for Stack Overflow, other sites have very low traffic, taking into account that this will be posted on unanswered questions, the potential number of AI answers will be low enough that it will be hard to get any relevant results out of this experiment.

My further focus is on Stack Overflow as it only has significant traffic.

Unless, there is a AI answer queue, visibility on those answers will be low. Most of those questions will be unanswered because they have flown under the radar and are not the ones worth answering. I doubt you will get more than single feedback from the OP on most of them.

This will be not enough feedback either for AI training companies nor for bringing answer to the public view where it may potentially be helpful some day to some passer-by.

Overall, the gains of this experiment are extremely low.

And when it comes to the sites with low participation, I cannot see how would this work at all. There is not enough people there in the first place, let alone those that will be reviewing AI answers. I don't see how this can help bring those sites to life. Especially, when you consider that some users will inevitably leave as the result of this experiment.

Another important thing here, is considering that long term goal would be adding AI answers across sites as I don't see the justification of doing this "just as an experiment" that will be scraped if the experiment fails. With this there is also a question of interpreting results and trying to bend them as much as possible to show that experiment is successful. I don't think that the results would be intentionally changed, but that not all relevant factors would be taken into account.

Answer visibility

Using very minimal 15 (50) reputation as a baseline for users that can give feedback is one of such things that can have negative impact on the experiment. Majority of users with such low reputation are nowhere near being qualified to judge and validate AI generated answers.

Since low reputation users are generally more willing to upvote just about anything, and there is way more of them than qualified experts, this alone can significantly alter the results of the experiment.

Even if the reputation increases, there are other issues. To get some relevant results, this experiment would need to be a long running one. Sometimes it takes months and years for answers to accumulate votes. This issue will only get worse for the AI answers that will be posted on unanswered questions. Chances that they will be seen is extremely low (unless there is a queue).

If you want the get more accurate results, this "experiment" would have to be implemented for so long, that it can be hardly called an experiment any more.

AI answer queue

To solve some issues with longevity of the experiment, and number of reviewers, you may want to introduce AI answer queue. However, this has issue on its own.

First (on Stack Overflow), we don't need more review queues. We are already having problems with handling existing ones. Any new queue that is taking away from the core ones needed to maintain sites' health will not be well received.

Second, even if you attract users to review, they will most likely fall into two categories. Robo-reviewers that will accept just about anything, and seriously annoyed curators that will decline just about anything. Both will ruin the results.

I suspect that number of users that will be genuinely willing to review AI answers and that can actually do that correctly is extremely low. I would be tempted to say almost non existent.

Correctness of the answers

You have to start with the premise that AI answers will be incorrect and you will have to take into account the damage they can do if posted on sites (to public). Even if they are just visible to reviewers that doesn't meant that they cannot cause harm.

Human answers can also be wrong, but for those there are commonly other signs from which you can more easily detect possibly bad or lower quality answers that might require more caution.

AI answers sound very convincing and plausible and for some, it may be even hard for the experts to determine whether there is some incorrect information or not.

There will be a difference between sites about what kind of potential damage can incorrect answers cause. For instance, having incorrect answer on Role-playing Games or Worldbuilding will not cause any harm, while incorrect and misleading answers on technical and science based sites can cause plenty of harm. Even the sites that deal with social sciences, religion, and history would want to have correct facts instead of potential AI hallucinations.

Ability to verify facts and how much time is needed to do so, can also significantly vary. If the correctness and facts are hard to verify, you may get skewed results through feedback because people might mark answers as correct simply because they will sound plausible, but they will not have any means to truly verify the correctness.

Correctness, may seem easier to verify on some technical sites, for instance you can more easily check whether some piece of code solves the problem or works as expected, but even then explanations about inner working can be wrong and code event though it "works" may have some other serious flaws.

For instance, if someone posts code susceptible to SQL injection and AI corrects the non working part but does nothing to correct the security flaw, such answer can be easily marked as correct while it suffers from critical flaws. And some flaws may not be so obvious.

Except in most trivial cases where AI might accidentally get correct answer on the first try, AI answers tend to be mostly incorrect. It takes several iteration between person and AI to get something usable and correct (if the correctness can be easily verified).

This can be seen on AI answers posted on Stack Overflow even by high reputation users where they often require several revisions to get to the correct one. And this is also experience of users that are already using AI off sites, where AI is mostly useful when they are stuck and need some idea about direction and hints, but often raw AI output will not be usable as such.

Editing AI answers

Just like you will have problem with getting enough users to validate AI answers, editing will also be a problem. Most of the time, you will need experts to edit such answers and I don't think there will be too many of those that will be willing to edit AI answer instead of posting their own.

Human participation

Even if we ignore issue with users that might leave or decrease their participation as form of protest, there may be additional decrease in participation as posting answer after AI posted one may look like some sort of plagiarism.

Currently, I can see some reluctance from people (and I have even been in such situation myself) to post answer to the question that previously received AI answer as some parts of their answer may look like being inspired by AI one.

Additional concern is that seeing AI answers posted on site, may encourage other users to post AI answers themselves which is not allowed and this will only create more work for moderators and curators.

Impact on curation

Since plenty of questions that might receive AI answer will be off-topic or low quality, there is a question what kind of impact will AI posted answers have on curation. More specifically, on roomba. Will having those answers prevent automatic deletion of such questions and under which circumstances?

Conclusion

Overall, the potential gains from this experiment are so low, that I don't see how proceeding further is justified. You have nothing or very little to gain, and everything to lose.

You will alienate the core of your communities (moderators, curators, experts) even further, while at the same time you will not add new value to people that need solutions.

I can only see further decline of participation and traffic as the result.

I will not repeat all the excellent responses you've gotten to this post. I will only give you my opinion as an "expert in my field" who has been active on SE for about a decade.

Why on God's Green Earth do you think I might want to voluntarily fact-check an AI generated answer? If the question interests me (and isn't asked with a gimme attitude), I'm happy to answer myself, mostly because I like to help people, and it keeps me involved in the field post-retirement in a way that I alone get to determine.

I'm completely uninterested in helping LLMs to learn to give better answers. If experts are what is needed to check the answers, I imagine many share my feelings.

What a disappointment this is. Again.

Vote To Close: How to Master the Art of Coffee Brewing: Techniques and Tips?

Staff finds this part unnecessarily rude and thus blanked it out. The gist is: The question is not a suitable or representative testcase for StackExchange questions and belies a fundamental misunderstanding of what questions should be asked on StackExchange.

Flag as Not an Answer: AI answer

Credit to the AI for giving a terse answer. However, it decided to answer a different question about how to operate a drip coffee machine, as opposed to explaining the differences between drip, french press, espresso, etc.

It is wild to me not only that you wouldn't just demonstrate the output on a real question from one of the participating sites, but particularly that this would be your example, given how obviously flawed both the answer and question are.

(Finally, please make the transcript of the generated answer available to users with a screenreader. See: please do not upload images of code/data/errors.)

After this "experiment" (and let's be honest it's not, it's a feature you are going to impose regardless of whether we like it or not) is introduced, the last piece of value that the Stack Exchange network had over AI slop will be gone. This is the killing cut to the goose that lays the golden eggs.

Farewell my fellow curators, my fellow pursuers of knowledge, my fellow librarians. We had something unique and good, and like so many good things it was destroyed by incompetence, greed, and refusal to listen to the people who actually understood and cared about it.

So, if I may summarize this wall-of-evasive-text more succinctly:

As an "experiment" you're going to allow a few "volunteer" SE sites be flooded with the garbage output of a glorified autocomplete, in the hopes that the network's volunteers will curate the mess for you for free, after which your still-unnamed-because-privacy-is-important-for-people-who-give-us-money business partners hoover up the curated data to refine their model.

I knew you were planning to screw us all over but you could at least have offered to buy us a drink first.

What about attribution/citation/sourcing on the answers that are suggested?

Right now it’s not included since the GenAI output does not consistently provide that. We’ve stated that attribution is non-neogitiable and that goes both ways. We are determining what sourcing data will be delivered from LLM providers along with the private AI-suggested answers. For the purposes of this limited experiment, we feel it’s still worthwhile to test out the concept and user interactions.

Sorry, but that is extremely difficult to understand. If you must test a concept, you could do so in private, but not in public. If attribution is truly non-negotiable, you cannot make exceptions because the output does not provide it. You must instead make the output providing it. If you don't, it's very much negotiable (obviously). Are you trying to cheat on yourself here?

I think this generated AI content without attribution on this platform is a breach of the CC-BY license which requires attribution. All the creative people who provided content to the network are deprived of their reward, which is to be attributed. If I would be one of them, I would feel very unhappy about it.

Provide attribution or wait until you can.

A little reminder words from February 2024:

All products based on models that consume public Stack Overflow data are required to provide attribution back to the highest relevance posts that influenced the summary given by the model. With the lack of trust being felt in AI-generated content, it is critical to give credit to the author/subject matter expert and the larger community who created and curated the content being shared by an LLM. This also ensures LLMs use the most relevant and up-to-date information and content, ultimately presenting the Rosetta Stone needed by a model to build trust in sources and resulting decisions. Sharing credit ties to attribution and with attribution—trust is at the heart.

You see the irony?

P.S.: What is the license the AI generated answers are published under? If they are to be edited, what is the license of the human edited AI generated answers? I think there might be some legal problems coming, which may not have been thought through enough.

P.P.S.: Now if you would have proposed a Duplicate Finder bot that automatically comments about possible duplicates, I might actually have had sympathy. But unfortunately the Answer bot will do the exact opposite. It will happily answer every question even if it has been asked before, especially if it has been asked before.

At a minimum, the reputation requirement should either be higher, or the association bonus shouldn't be factored into it. As it currently stands, a user can join a site with this feature enabled and mark suggestions as correct even if they have limited interaction with that site.

For example, I can vote on suggestions on Raspberry PI even though I have yet to do anything on the site other than join and talk in chat. This seems wrong as I don't have the knowledge needed to vote on these AI answers.

Update 2025-02-06: It now appears that I am no longer able to see these answers on Raspberry Pi with just 101 rep from the association bonus. The main post also has been updated to reflect 150 rep as the new minimum.

Opt out

Please make a toggle so that users do not see these at all. I do not want to review them. I do not want to see them on my questions. I do not want to downvote them when they are invariably wrong.

Approval of such AI-generated answers should be limited, at minimum, to users with a silver badge in the question's main language tag. If it seems difficult to code for that, let this be your signal to implement parent language tags as a system feature (something we've been asking for for 5+ years, at least), then it will be trivial.

Otherwise it will just be the same level of garbage that generative AI tools today put out.

This honestly sounds like you want the community to help some undisclosed AI company refine their model, for free.

People don’t want to do this for money, why do you think they will do it for free?

What about attribution/citation/sourcing on the answers that are suggested?

Right now it’s not included since the GenAI output does not consistently provide that. We’ve stated that attribution is non-neogitiable and that goes both ways.  We are determining what sourcing data will be delivered from LLM providers along with the private AI-suggested answers. For the purposes of this limited experiment, we feel it’s still worthwhile to test out the concept and user interactions.

The training process used by LLMs based on the GPT architecture does not and cannot remember its information sources. It's operating on the level of tokens (words, word fragments, and special symbols like emojis), not on the level of concepts.

When a human reads, we attempt to build some internal model of the facts we are learning. We create mental structures that somehow mirror the concepts we're reading about, and how they all fit together. We also retain some information on our sources.

In contrast, a LLM assimilates its training data to create a weighted neural network which enables it to emit pseudorandom sequences of tokens that are statistically consistent with the current prompt and that training data. It doesn't remember its sources, and it doesn't explicitly operate on the level of concepts, it's just operating on tokens. All token pair transition probabilities get consumed by the training process, regardless of their source, and regardless of what concepts they happen to be associated with. An LLM can give the impression that it understands concepts, but that's just a side-effect of statistical correlations in token sequences.

Now some of the latest GPT-based LLMs can appear to give attributions for their utterances, but these are fakes. The GenAI system simply generates an utterance and then does a search on a relevant data pool (eg, the whole Internet, or the Stack Exchange network), looking for close matches to the utterance, and then claims that those matches are its sources.

So it's simply not possible for current GenAI models to provide genuine attribution for their utterances. We need a paradigm shift to a more advanced architecture for that.

That's a big problem for the Stack Exchange network, since we demand that answers provide legitimate attribution for all substantial claims, especially on scientific and technical sites. It is fundamental to academic integrity that you always cite your sources and references. If you attempt to just spout unsupported opinion you are considered to be a charlatan or a crank.

Don't get me wrong. Current GPT-based LLMs which have undergone extensive RLHF fine-tuning are very impressive, and can be very useful tools, if you're aware of their limitations, and you know what you're doing. But they are no substitute for a human expert who actually knows what they're talking about, and who can cite their sources.

I fail to see the point of this.

If the answer is posted after being modified by users, then what value is the site integration, the whole feature that you are announcing here, actually adding? Right now, most sites here do not allow AI-generated answers, for good reason, but if they did, users are perfectly capable of copying the question into some chat interface, copying the answer that it returns, and adding their own modifications before publishing it here. Likely more capable, since they can add additional context to the question to try to get a better answer. If that is the case, then the whole team responsible for the integration might as well resign, because they will have spent a lot of work to produce something that adds no value.

If an answer created by some sort of generative model instead is posted without any modification, then what is the value of Stack Exchange as a company or a business? People are very much capable of copying and pasting their questions into ChatGPT, DeepSeek, or whatever the latest interface will be when you are reading this answer. As such, you would be doing nothing but returning the same result that a user would have gotten anyway with extra steps and a delay. If that is the case, then the entire company might as well disappear, because it would be providing the same service with more difficulty.

If, instead, your goal is to market upvotes and downvotes here as a cheap input to improve the quality of the models, I would suggest that you may overestimate the willingness of your users to do so, their skill at actually identifying the bad answers (bad generated answers tend to be written very professionally, which I have seen mislead voters), and the long-term interest of companies like OpenAI to contract out something that they already have integrated into their systems.

This experiment is indicative of the profound misunderstanding about what LLMs do which infests the entire industry.

LLMs are designed to produce output that looks like English. The models do not "know" the content of the answer being produced. They are accessing and manipulating tokens that do not have any assigned semantic values. E.g. when LLMs are called on to "summarize" long papers, they merely shorten the papers. They can't summarize. They cannot pick out the key topics, follow arguments, or guarantee that the conclusion of the paper will be accurately represented in the output.

It astounds me that a site such as Stack Exchange, which encourages users to cite sources for their answers, would do an about-face and run experiments using tools which deliberately obscure the source of the (largely pirated) training set materials which are being blended and regurgitated for the end user. Do you want answers with real citations, or not?

"Yes, but you can challenge ChatGPT to write a citation" and what you do you think it does? It writes something that looks like a citation. It is extremely time-consuming to chase down (to choose an example relevant to GenealogySE) a citation for a newspaper article from a newspaper that never existed.

You don't seem to realize why people are willing to contribute to SE sites. Count me in among the many people who have already said they have no desire to clean up AI slop. If someone can't be bothered to write an answer, why should I or any of my community members waste valuable time to edit it?

The AI companies are taking advantage of end-users' human nature to believe that the underlying algorithm must "know what it is talking about" in order to produce the output it does. That doesn't mean we need to fall for the con.

Please don't waste resources on this rubbish.

• When ChatGPT summarises, it actually does nothing of the kind
• ChatGPT is Bullshit
• The LLMentalist Effect: how chat-based Large Language Models replicate the mechanisms of a psychic’s con

Which Stack Exchange sites are participating in the experiment? Arts & Crafts, Raspberry Pi and User Experience (UX) are currently participating in the experiment. Web Apps was a participant in an earlier stage.

Why did Web Apps withdraw from the experiment? E.g., was it due to poor answer quality? I see Answer Bot received a few downvotes:

Update: the question has been posted on WebApps meta and one of the mods addressed it (thanks Berthold for the pointer to it!).

What would need to be in place for you to feel comfortable seeing Answer Assistant implemented as a controlled experiment in your Stack Exchange community?

For me to be OK with this "answer assistant" experiment, it would need to actually be an assistant and not some repackaged LLM pretending like it knows things.

It wouldn't generate an answer; It would help me quickly write a good answer based on my expert knowledge. Some random ideas of how it could do that:

• Help me find related questions on the network
• Summarize all of the comments on the question and its answers
• Help me find good supporting sources for my answer
• Format the answer according to the community norms of the site
• Suggest ways to make the phrasing of the answer clearer, expand thoughts into full sentences and otherwise wordsmith my content
• Warn me that the question may be off-topic or a duplicate

A true answer assistant would encourage humans to write answers by making it easier to write them well. This AI generated answer experiment is pushing humans into the tedious work of reinforcement training an AI while taking away the more rewarding activity of helping people by answering their questions.

Unanswered questions have traditionally been a way for new users to start getting involved with the site. Having AI steal that opportunity seems counter to the goal "... to build and support a healthy ecosystem of active users and community contributors." Unless of course the community the company is trying to build is one willing to curate data for and reinforcement train an AI without compensation.

If an AI can answer a question, I can ask it myself to an AI. If I come to a StackExchange web site in 2025, chances are that an AI is not able to answer my question properly, or that I don't want to risk to get an answer that seems correct on the surface but contain factual errors (e.g., invented although plausible Win32 APIs, like I had once). Having people "moderate" answers generated by AI instead of empowering them with writing content make the site more user-hostile in some way, and doesn't bring anything to end users.

So, in my opinion this is not a feature I'd feel would have its place here. It would just be ruining the reputation of the sites.

Questions that meet the following criteria may receive an AI-generated answer:

• Older than 72 hours, to leave time for human curation

• Posted in 2024 or 2025

• Net positive score (0+)

• Unanswered, defined as having no upvoted or accepted answer

I can think of a few reasons a question might fit these criteria:

• it is badly tagged and almost no human is seeing it
• it is borderline in terms of quality / interest (not bad enough to be downvoted, but also not good enough to get upvotes)
• it is (potentially very) difficult to answer

The Answer Assistant is not going to help with any of these.

• low visibility of question: no human will see the answer either. It will never be curated, and never be posted.
• low-ish quality: will likely lead to a low-ish quality answer. This will flood the site with mediocrity in the best case.
• difficult to answer: if experts can't answer it, why would a language model suddenly be able to do so?

The Answer Assistant is not going to be useful for experts writing answers. An expert is not going to need AI assistance to write a good answer. It might lower the "barrier of entry" to answer a question due to saving some amount of time (reviewing a suggestion is probably faster than writing a full answer from scratch), but I feel like if a question is interesting/high-quality and unanswered then an expert is not going to need that kind of "push".

All I see this accomplishing is driving existing users away, and preventing more experts from joining, due to the network getting the reputation of being filled with AI slop.

LLMs are part of the world now,… that not only provide value but also… keep humans in the loop, and encourage human contributions….

What a detached and oddly dystopian phrase. It offers nonetheless an invaluable glimpse into the future and how the Stack Exchange company will view the flesh and blood members of its communities.

AI, the new pioneer, continually evolving and improving itself, now sits in the council of knowledge whereas humans, who will increasingly depend their livelihood on these sterile learning machines–are still, somehow, required to clean up the muck left behind by these compassionless and predictable so-called models of intelligence.

A private answer [i.e. generated by AI] becomes public if multiple users mark it as ‘correct’. A private answer moves to a deleted state, visible only to site mods, if multiple users mark it as ‘incorrect’

Welcome to the upgraded "Let the humans eat cake" era.

We remain open to concluding the experiment early if we find the results unfavorable for any reason.

Is the second "we" meant to be the same as the first "we", which I believe refers to the company? If so, then to what extent, if any, would the community's reactions be considered if they find the results unfavorable, e.g., many (such as a majority) of the site mods are against it, a meta post indicates the strong dislike from the members of the site, etc.?

Also, should you stop an experiment on a site, what sort of cleanup, if any, would be available, e.g., delete all of the AI bot answers, just remove the ones that are currently being processed, etc.? In addition, how would the type(s) of cleanup be determined?

Note that Berthold's comment below states

The second "we" is communal, since feedback from the test site communities and the broader community is a big factor. ...

However, as requested in Starship's comment, the "communal" aspect should be more clearly specified, e.g.,

... Communal means what exactly? Mods agree? General users agree? How is this measured? Who gets to vote? What percent of a vote is enough to turn it off? How would one start just a request to remove this? Details please.

Finally, regarding Berthold's comment that feedback from the "broader community" is also a big factor, then with so much negative feedback (e.g., with this question currently having only 25 upvotes compared to 335 downvotes, as well as 43 answers which are almost all very much against this idea), I hope and trust this is sufficient feedback to show the "broader community" does NOT want this "answer assistant".

an experiment in which AI-generated answers are verified, edited, and curated by the community before becoming publicly visible. We want to test if this feature could help improve the answer experience and encourage knowledge sharing by helping users get unstuck or get a jump-start on content curation while maintaining quality.

This adds no value. Let me add one more assenting voice: if I thought that "verifying, editing and curating" an AI-generated answer could produce something better or faster than what I do on my own, I could just use any existing generative AI (subject to site regulations). But I don't believe that anyway.

we want the Stack Exchange community to know that the team is: committed to the Stack Exchange network being a place for human-curated knowledge and information; committed to building solutions that add value for users on the platform; and not interested in any outcomes that might dilute the value of the platform.

The team's actions suggest otherwise. The community has repeatedly told you as clearly as possible that these kinds of generative AI integrations are not wanted: especially because attribution is non-negotiable, it doesn't add value to have automatically generated text that requires manual verification. It would take at least as long to ensure the answer meets requirements as to do a different answer from scratch - it's far easier to figure out attribution when you were the one who looked up the information in the first place.

If the goal is simply to inspire the community to collaborate on a canonical answer, that is what the "community wiki" feature is for. Everyone can mark their questions and answers to lower the bar required for editing and to disclaim personal interest in the content.

If the goal is to engage with the community and collaborate with human curators to find a legitimate use for generative AI, that would start with a) asking the community for suggestions instead of having a steady stream of proposals shut down; b) being willing to take no for an answer. Quite a few of us, myself included, can't see any legitimate use for any kind of generative AI integration on the site, full stop - and improvements to the quality of the output of such systems are entirely beside the point.

What about attribution/citation/sourcing on the answers that are suggested?

Right now it’s not included since the GenAI output does not consistently provide that. We’ve stated that attribution is non-negotiable and that goes both ways.

In other words: by your own admission, attribution must be included, and generative AI cannot be trusted to attribute its own content correctly. Therefore, this isn't just a question of editing the content; you would be imposing upon the community to come up with correct attribution for content that didn't originally come from those community members. This is what's so destructive about the idea.

No amount of improvement to generative AI systems could fix this fundamental problem. You wouldn't ask one human Stack Exchange user to attribute another's content, either. You would flag the unattributed content for plagiarism instead.

You might wonder why we’re moving forward with the experiment, in the face of concerns and sensitivity around AI-generated content across the network. Put simply – in this time of foundational change, we must prepare for many possible futures.

Avoiding corporate speak like "in this time of foundational change" (when were we ever not in such a time?) would go a long way towards improving the appearance of sincerity. But it wouldn't change my personal assessment that this is yet one more attempt to get a foot in the door with generative AI. How many more times must we slam that door in the company's face before the message is clear?

User expectations around seeking and contributing knowledge are rapidly shifting

No, they aren't. What is shifting is the expectation of ordinary people who have experienced these technologies personally. That has no bearing, and shall not have any bearing, on how Stack Exchange works. Phrasing like this is exactly why the team continues to lose the community's trust.

You know that our expectations are not shifting, because community reaction to any suggestion with even the faintest whiff of AI about it remains just as negative as ever. This post contains many paragraphs that seem aimed at addressing the community's concerns about AI, yet it still misses the point completely.

I'm not going to repeat arguments other people have made in this answer. Yes, this new feature seems more cumbersome than anything else, and it feels disrespectful towards the community to continue trying to push AI despite the fact that you are aware of the negative reception of your previous attempts. Sadly, this post confirms that this is going to continue no matter what:

Committed to building solutions that add value for users on the platform. LLMs are part of the world now, and any potential integration must be explored responsibly, in ways that not only provide value (task completion, closing knowledge gaps, etc) but also create transparency, keep humans in the loop, and encourage human contributions.

Why must new ways to integrate AI be explored? I'm not necessarily against AI tools in general, and this is probably also the case for most users of this platform, but the Company seems to think that exploring AIs should be the top priority, and nobody understands why. Most users want this website to be a reliable way to obtain answers from trustworthy individuals, and they don't see why AIs should be used here.

Anyway, I wanted to talk about the three Stack Exchange sites that are a part of this experiment: Arts & Crafts, Raspberry Pi and User Experience. It seems to me that some of these websites would not benefit at all from using an AI as a default answer provider. For Raspberry Pi, this is at least a bit reasonable, if a user wants to ask about how to do X with their Raspberry Pi for example, an AI could provide a reasonable answer to such a question if a task is common enough (although the question would probably have already been answered in this case, as many people pointed out). This is a bit more blurry for Arts & Crafts, because while the AI can answer basic questions about the crafts themselves, it cannot really guide you when you ask a more complex question in most cases. However, I don't understand how your AI is going to improve anything for User Experience. I'm not going to pretend like I'm any kind of expert on this topic, but most questions asked on this site are not "How can I ...?" questions, but rather "How should I ...?".

This distinction is quite important. When you ask an AI about how to do something, it will generally give you a straight answer. However, if you ask instead if you should do that thing, the AI will generally give a vague answer that only lists considerations you should take into account before doing the thing. This is what I like to call a "shopping list" answer, when the AI just lists a bunch of stuff in order to avoid answering definitively the question. Why is it doing that? Because it would be a very bad idea if the AI started to form its own opinions, especially if it is hallucinating. I need a good example for this... I know, let's ask an AI about this topic:

Should Stack Exchange publish answers generated by AI when no user has published an answer and when these answers are double-checked by other users willingly?

Generated by GPT-4o mini

The decision for Stack Exchange to publish AI-generated answers, even when double-checked by users, involves several considerations:

1. Quality and Accuracy [...] 2. Community Standards [...] 3. Transparency [...] 4. Attribution and Licensing [...] 5. Encouraging Human Participation [...] 6. Use Cases [...].

In summary, while there could be potential benefits to publishing AI-generated answers under certain conditions, it would require careful consideration of quality, community standards, transparency, and the overall impact on user engagement.

As you can see, this answer is not really an answer to my question. After reading this, have I really obtained a definitive answer that would satisfy me? And this is a good thing, because answering "Oh yeah, sure, this is a great idea" would have been the worst available answer. Also note that this is the kind of answer that cannot be easily edited. How are you suppose to edit this in order to answer the question? You can't. So I hope that some kind of system was implemented in order to avoid these pitfalls, but only time will tell I guess. One of the many advantages this website has is that two users are able to post different and sometimes contradictory answers, and these answers are able to complement each other, because the user can now make a choice by reading each solution. This is not really the kind of thing AI can do well. This is not to say that the Raspberry Pi site is not going to have these problems, quite the contrary actually, but it's obvious that this system is not effective at all for some topics.

The last problem I want to talk about is the fact that AIs will not be able to read or write comments. Because I want every site to be filled with bots 24/7. Comments are more important that you may think. If a question is a bit too broad, I can just ask the user to modify their question in order to be able to provide a better answer. I can do that. An AI can't, and will instead just answer the question no matter what. An AI also can't read comments, so it's impossible for the user to ask for further clarification. Hope some users are ready to babysit the AI, because this is currently the only way to provide more information once an answer is posted. This last point can potentially be "fixed", but even then it would not be a great solution to this problem.

Brilliant plan.

You haven't got enough people actively creating content, so you just go to AI generated content for which you haven't got enough people to check the content.

Flood the stack system with BS AI "answers" and flush the system down the toilet even faster.

I hope the owners are getting a good paycheck out of this because it sure isn't doing anything for people who need the stack system for information.

I've been going through the questions with LLM answers on Raspberry Pi. These tend to be troubleshooting questions, and the LLM fairly consistently takes the "shotgun" approach to answering them: throw out a whole bunch of suggestions, and hope that one of them hits the target.

Take the answer to this question: it's got five "steps" and two "tips". Of them, I know from reading the question that both "tips" and "step 5" are irrelevant, and I'm nearly sure that "step 1", "step 2", and "step 4" won't fix it. "Step 3" might work, but I don't have a Pi 5 with Ubuntu to test it on, and if it's wrong, it might open up a security hole.

Honestly, the whole answer reads like a condensed version of what happens when a troubleshooting question hits Hot Network Questions, and is unlikely to actually be of use to anybody.

AI answers should always look different

Before I saw the image captions, I thought the red-background answer was how AI answers would always look, and that felt like a good compromise.

Honestly, as a user, I don't hate the idea that alongside the human answers there is one, clearly distinguishable, AI generated answer. I kind of don't even mind if it isn't quality controlled first. It's another perspective, another option.

But having AI generated answers blending in amongst human answers...no thank you very much. "AnswerBot" is nowhere near enough of a signal.