-66

TL;DR What if Stack Exchange utilized an LLM integration to add ONE standard answer for questions with low activity, while encouraging the community to review said answers?

It could help revitalize the platform -- or at least make it more useful until the Singularity & inevitable subsequent AI Uprising!

Though I haven't followed the LLM-related drama too closely, I'm aware that it's been a hot (perhaps volatile?) topic on Meta and on Stack Overflow, and for good reason.

I am NOT a proponent of the widely popular AI-to-All-the-Things mentality pervading the Interwebs these days. I do use LLMs frequently to get quick, plausible feedback on sticky issues in order to (hopefully) point me in the right general direction. But I also know from painful experience to always assume there's a flaw in the response if not an outright lie.

This is a core reason I still trust and appreciate Stack Overflow. While I can get quick answers either from SO, or from GPT et al, with SO I can often gauge accuracy to a much higher degree of confidence thanks to the variety of answers and the feedback on those answers.

Unfortunately not every question on SO has a viable answer, either. And a significant portion are, if not severely outdated, then at least no longer considered best practice.

Would it be worthwhile for Stack Exchange to introduce a dedicated LLM integration with one of the major providers to suggest an additional answer (at least for questions with low response count or activity)?

This would NOT be yet-another LLM chatbot. I'm talking about a feature that generates and attaches a single answer in the same way any user would -- *though clearly designated as an "official" LLM-generated response. And it would be (presumably) the SOLE exception to the ban on LLM-generated answers.

The intriguing aspect of this to me is not merely having a potential answer for every question -- but that those answers can then be properly reviewed and vetted by the venerable SO community using the tried-and-true SO formula we all know.

I'd even wager that a few SO veterans (a category which I'd very generously allow myself to hover on the edges of) would feel just PO'd enough to drift on over and put the high & mighty LLM in its place with informed rebuttals, further strengthening the quality of the platform for everyone involved.

I am NOT shouting "This is a good idea!"

But if this were implemented, I do have the sense that I'd find even more value in posting my questions here knowing not only that an answer would always be provided, but also that the generated answer would be subjected to the community's keen and critical eye (unlike on ChatAnthroGemAI).

9
  • 30
    No, it should not. It was tried already by the way: meta.stackexchange.com/q/406307 Commented Aug 3 at 20:55
  • 14
    They tried, and it was a failure, as expected. Commented Aug 3 at 21:19
  • 29
    Hard no. Anyone who wants an AI answer can get one elsewhere. People come here for answers from experts. Commented Aug 3 at 21:20
  • 14
    "the generated answer would be subjected to the community's keen and critical eye" Sigh Please read at least a good sample of everything in "Policy: Generative AI (e.g., ChatGPT) is banned". As has been the case from the very beginning, the time that would be required from our volunteer Subject Matter Experts in order to do that just doesn't exist. Subject Matter Experts are necessary for such review, because LLM generated "answers" can easily sound just fine/good to someone who doesn't already know the subject matter in detail, even when the answer is complete crap. Commented Aug 3 at 21:27
  • 12
    I'd even wager that a few SO veterans ... "A few" is probably accurate, but it would take more than a few to review the enormous number of AI-generated answers your proposal would generate. Speaking for myself, I'd have no interest in such activity. Commented Aug 3 at 21:49
  • 4
    See also e.g. Will OverflowAI really feature dumping AI code explanations onto Stack Overflow? to understand just how opposed we are to this sort of thing. Commented Aug 3 at 22:25
  • 1
    I don't understand "for user review" in the/your Title...? // => " User" = Asker, or any Visitor (even 10 years later...?) // And "Review", => what (kind of) review, en by whom...? (Related to the "Review Queues"...?) // Reacting on the current Title: "Should Stack Overflow officially integrate an LLM feature to suggest answers for user review?" Commented Aug 4 at 2:32
  • 1
    “ What if Stack Exchange utilized an LLM integration to add ONE standard answer for questions with low activity, while encouraging the community to review said answers?” - I would immediately delete my account. That is precisely how I feel about LLM features suggesting solutions to questions. Answers generated by a LLM will never be considered to be helpful by this user. Commented Aug 4 at 5:11
  • 1
    No not an LLM. Wrong tool for the job. It would have to be a pseudo-AI specifically built and trained for the job, and even then... when you have to make a call to a service desk and you get faced with some stupid chat bot, is your experience good? Or are you tearing your hairs out just thinking about it? Commented Aug 5 at 11:31

2 Answers 2

Reset to default
23

The company tried something very, very similar this year on some exchanges (not this one, see AI-generated answers experiment on stack exchange sites that volunteered to participate) and discontinued the experiment afterwards. Mods of affected sites described the quality of the given answers as too low (not enough meat), the effort of curating the content as too high. And there were other problems like licensing of edited content for example as well as attribution of the created content and giving sources for the underlying statements.

The company later introduced AI assist, a dedicated page for using an LLM that is only lightly integrated. It faced considerably less backlash from the community.

My guess is that this new AI assist feature is currently seen as the best way to integrate LLMs into the network, by the company as well as the users. And it can be used for what you want. If you encounter a question with low activity and without satisfiable answer you can use AI assist to suggest an LLM answer for you.

What you cannot do is using the LLM/AI assist generated content to review it and post an answer directly to the visiting Q&A on exchanges that ban the use of AI. SO is one of those exchanges. On SO, it could at most only be an inspiration anyway.

5
  • 5
    SO AI is just ChatGPT, it’s not even the current and most efficient model, and isn’t even trained on the SE community website. So it’s essentially useless, it offers nothing, that free ChatGPT doesn’t already offer Commented Aug 4 at 5:16
  • The premise of the question is flawed. An AI as currently understood can not advance the total sum of knowledge by answering a question. A true AI perhaps could but humans are much better at this. Commented Aug 4 at 7:18
  • @SecurityHound It's somewhat integrated, with more or less related Q&A showing. And ChatGPT is trained on the SE community content. It may not be very useful but Brian Lacy didn't ask for the best LLM, whatever that is, just for some LLM. A comparison of the performance of different LLMs on unanswered Q&A might be interesting. Commented Aug 4 at 7:19
  • @MT1 I don't think this is the premise, but we could ask the OP here about it. I understood it as increasing the speed of advancing the sum of all knowledge by giving human reviewers useful cues (guesses) to work with. Many programming questions aren't exactly rocket science. A horde of monkeys with computers trying out random things and reporting back might also work. What AI currently needs in my view is access to a compiler. Commented Aug 4 at 7:25
  • 1
    @NoDataDumpNoContribution - I asked it a question, admittedly how to disable itself, and it hallucinated an incorrect answer. Which is hilarious but not helpful but also suggests it wasn’t trained properly n the functions of the network or any of the community websites. But it truly is just ChatGPT on a SE wrapper Commented Aug 4 at 9:40
20

If someone wants an AI-generated answer, they can easily ask AI themselves. They don't need Stack Overflow for that.

Stack Overflow is where you go if you want a human-generated answer that's been vetted by other humans.

With regards to answers with inadequate or outdated information, I would be in favor of having a way to surface these and provide more visibility to the fact that they need updates to their answers. That would be much better than simply dumping some AI slop on the site.

2
  • 1
    There is one exception. AI generated code snippets are okay if you have verified them. SO is the place for human or AI generated code verified and documented by humans. Commented Aug 4 at 16:18
  • Exactly my point why I am turning off various Google AI overview and such. I can easily ask AI myself and I don't trust your insert_company_name AI at all, it's worser than the one I can pay for, it hallucionate, it generate answer from a poor prompt (targeting human), likely incorrect, nor working, for a wrong library, whatever AI decide to mix into its answer. I use AI and SO in parallel, if SO become AI itself there will be no reason to ever use SO. Commented Aug 11 at 11:38

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.