19

New users frequently report to us that they view Stack Overflow as a slow place to get answers to problems that they face. Their perceptions are often rooted in the belief that many posts go unanswered indefinitely or may take days to answer. More broadly, new users often say that it feels hard to get seen by someone who can answer. In reality, most questions that will be answered get their first answer within just a few hours of posting because users are reading their questions.

Based on this discrepancy between users’ beliefs and reality, we came up with the idea that we could show users an indicator that live activity is taking place on the site so they can easily recognize that the community is alive and vibrant even when viewing a static page. In turn, we hypothesize that influencing this underlying belief will increase the rate at which users submit their first contribution to the platform.

Beginning Monday, June 2nd, users with one rep and no questions or answers may see the following panel on the question ask page:

A new panel on the right sidebar of the Ask Question page is highlighted with a drawn red arrow. The panel is called "Community Activity," and shows three stats over the last hour: the number of users online, the number of questions asked, and the number of answers shared.

We may modify this panel over the course of the experiment. Once we have acquired enough data about user behavior to complete an analysis, we will turn the experiment off. It will probably take a week or two to get the data we need. After we complete the analysis, we may deploy this panel (or a variant of it) to Stack Overflow, provided the results show promise.

If you are eligible to see this panel but do not want to see it, you may opt out by switching off “Enable experiments” on your profile. Further, if we modify this experiment to incorporate a live feed of some kind (incorporating usernames and post times), we recognize that some users may not want their usernames listed in the feed. If this feature proceeds to a full deployment, we will consider how we could address this desire in the final design.

28
  • 40
    "New users frequently report to us that they view Stack Overflow as a slow place to get answers to problems that they face." Maybe improve search. Most of their questions are already answered before they even ask. They just don't find them. Commented May 29 at 18:12
  • 2
    @KevinB It should appear on both versions of the ask question page. I only included one screenshot because the module is the more important bit. Since it's in testing and the state is a bit wonky, I can't grab a screenshot of how it'll look on the Ask Wizard right now, but I should be able to add one soon if you want to see it. (It's not really very different, though.) Commented May 29 at 18:15
  • 21
    That 'Community Activity' Widget would actually be "interesting" for all Users, logged in or not, and all levels of Reputation, not only for 1-Rep Users with 0 Questions, I would quite like it myself, placed on the Homepage for example, just above the 'The Overflow Blog' Widget... Commented May 29 at 18:15
  • 4
    @chivracq Interesting, I'll bring it up with folks and see what they think :) Commented May 29 at 18:17
  • 5
    "In reality, most questions that will be answered get their first answer within just a few hours of posting because users are reading their questions." Is it reasonable to assume that the people who are getting answers within a few hours are generally not the people who are complaining about slow answers? If so, why is this other group of questions not receiving quick turnaround? Are the questions uninteresting, off-topic, or poorly tagged? Do they lack user engagement (from either the OP or other users)? Do they require specialized knowledge at the intersection of disciplines? Commented May 29 at 18:46
  • 3
    @beaker they aren't getting answers, because either it's off topic in one way or another, a dupe, low quality, or advanced enough (or poorly asked/debugged) that they need an expert who simply wasn't around to see it. Commented May 29 at 18:48
  • 1
    though... i suppose the ads issue is moot because we certainly did stick collectives links above the blog to push ads further down the page... maybe it isn't so set in stone. Commented May 29 at 18:52
  • 9
    I almost wonder if it would be good is the reverse is true, and the answerers can see if the author is active. It can be frustrating for us too when a asker posts a question that needs clarifying, and even if you post a comment within 30 seconds, it takes them 24+ hours to respond (and you've lost interest). It's a two way street. Commented May 29 at 19:27
  • 10
    @samcarter_is_at_topanswers.xyz I don’t know that I agree with that. I’ve found that unless you smack them over the head with it, users don’t think any search result is a duplicate. “In that question, they called their variable foo. Mine is called bar, so…” Commented May 29 at 19:45
  • 1
    Those, unfortunately, are the users here only for the bespoke solution, not an answer with an explanation, @AaronBertrand . There's rarely anything the community can do to appease them when someone correctly closes their duplicate question. Commented May 29 at 19:51
  • 6
    @ThomA right, and I’m saying improving search (and I’m not sure how you would, in any way that is not merely hand-wavy) isn’t going to fix that problem, and I think that’s a large majority of the angered-by-duplicate (or won't-even-read-the-duplicate) crowd, at least in the tags you and I frequent. Commented May 29 at 19:53
  • 9
    I like this experiment in contrast to some of the ones happening lately. We got advanced noticed, a clear description into why the change is being done, it is fairly unobtrusive, and already hints of taking feedback in the comments. Just wonder what is the success criteria for this, less users exiting the ask page before submitting their question? Commented May 29 at 20:19
  • 2
    @ThomA Honestly, I think that's a good idea. It'd be out of this particular team's scope, but I'm going to make a note of it and bring it up when I can Commented May 29 at 20:41
  • 1
    @DanielBlack Thanks for the feedback. That's what the team is looking to influence with the experiment, yeah. Commented May 29 at 20:57
  • 7
    "[new users] view Stack Overflow as a slow place to get answers to problems that they face" - great! This isn't supposed to be a helpdesk, and if they're exhausting the faster ways of solving their problem (searching for existing posts, for example) before posting a question that's ideal. Discouraging that belief seems likely to lead to more noise, if anything. Commented May 29 at 23:09

6 Answers 6

15

Interesting hypothesis, but I can easily see this leaving a negative impression with newer users as opposed to a positive one for two reasons:

  1. Site activity is not the same as question views, so users may be left wondering why nobody is viewing their question.
  2. In the same line as number one, there's no correlation between site activity and expertise, so a lot of users online doesn't mean that anyone can actually answer the question.

Furthermore, as has fairly persistently been pointed out elsewhere, writing a quality answer takes time, practically when someone has a well written question that points to an esoteric problem. It seems to me that SO is trying to address the wrong problem - instead of possibly giving the impression that a question may be answer quickly why not be more explicit that "Quality answers take time!"

11
  • 1
    Yeah, there's undoubtedly some risk that users walk away with a worse impression. I think we're keeping an eye on how it presents thru data, but I'll mention it to the team and see if we can track it in a better way. Re estimated time to answer -- actually surprisingly difficult. It's dependent on time of day, day of week, user reputation, random chance, and what kind of question it is. Estimates get a bit hairy. (A second issue is that an estimate is a kind of promise. We'd have to be really careful with how it affects expectations, and what happens when those expectations aren't met.) Commented May 29 at 18:23
  • yeah. I like this experiment, but I think managing expectations is important too. It's important enough to have its own Help Center page: /help/no-one-answers. cc @Slate Commented May 29 at 18:24
  • 7
    I'd have upvoted this if it didn't have the "calculate an estimated time to answer" bit. Please don't suggest such a feature! We'll then have people complaining if they don't get an answer in such time period. Also having an ETA would give off the feeling that the site is some kind of help desk (which it is not). Commented May 29 at 18:27
  • 4
    Moaw..., I find posting/indicating "an estimated time to answer" pretty "dangerous", to be honest, that completely varies per Tags and depends heavily on the quality of every Question... Plus 'Meta' will get flooded by Askers complaining that they "still didn't get an Answer, while the "System" told/guaranteed them they should have already gotten an Answer"... (Same like 'Abdul' apparently, posted at the same time...) Commented May 29 at 18:28
  • 1
    Ha! Fair about the suggestion, moving it into the comments: "Given all of the data that SO is collecting, would it not be possible to calculate an estimated time to answer?" Commented May 29 at 18:29
  • 1
    @Slate The estimates are challenging problem for sure, but it's also the type of day-to-day problem that data scientists are commonly expected to address. However, I think you are correct that the bigger problem is likely the perception of those estimates - practically if you have medians that are measured in hours (i.e., the "Why haven't I gotten an answer!" problem) or days (i.e., the "SO is useless!" problem). Commented May 29 at 18:32
  • @AbdulAzizBarkat "time to answer" removed. Commented May 30 at 0:05
  • @philipxy Yeah-yeah, I had noticed directly when 'rizii' had edited their Answer, but I then "made the call" to leave my Comment, at least for the time being, some other Answers and Comments in the Thread are talking about "expectations" (for Askers), so I think our both Comments (from 'Abdul' and myself) might become relevant again... // Flag our 2 Comments as 'NLN' if you don't agree... (And I'll check again (later/"tomorrow") also if 'Abdul' removes theirs, then I think I will probably follow...) Commented May 30 at 1:25
  • 1
    Maybe something like "recent average time to first N views" is a useful statistics? Commented May 30 at 1:43
  • 1
    @chivracq Just wanted you 2 commenters to know about the change. The comments make good points though. Commented May 30 at 1:55
  • The natural follow-up to this widget is to work to encourage quicker responses. For example, there's no real content algorithm suggesting good posts to check out next (compare with social media proper). As another example, there's only a few options for email alerts, and they're put in a place nobody can find, which can't be good for tags that don't have enough activity to keep experts checking the homepage. Commented May 31 at 22:20
13

Remember, Stack Overflow has a large number of tags, and activity levels vary significantly across them.

You might find it helpful to include in your experiment the observation of the effects of not showing or showing the most active tags.

2
  • 4
    I anticipate tag activity display will attract inappropriate use of them. Commented May 30 at 0:02
  • 2
    Thanks, I'll pass this along to the team. I think something like this has been considered, but wasn't pursued for a first test due to complexity. But it doesn't appear in my notes so don't hold me to that. I'll double check to make sure. Commented May 30 at 1:03
12

This is the wrong place for that information

New users frequently report to us that they view Stack Overflow as a slow place to get answers to problems that they face.

First off, thank you for taking feedback from the previous post and making sure to describe the experiment in advance this time.

However, I want to make sure we're on the same page regarding the validity of trying something like this.

Quoting myself from a comment there, with minor editing:

We aren't offended if new users view Stack Overflow as "slow" — nor do we feel disrespected. However, we don't care if users have that perception, because being "fast" in the sense they're talking about - i.e., responding promptly to questions and ensuring that they get a personally relevant answer quickly (and without additional time expenditure on their part) - is explicitly not part of what we do.

We achieve speed by being a) direct and b) searchable. The way to address new users' perception of slowness is to get them to not ask a question unless really necessary.

Which is to say: we do new users a service when we close their questions as duplicates. We link them directly to an existing answer, which is even faster than writing one; we can do this without insisting on polishing the question to meet other standards and avoid closure (since the question does get closed anyway); and we do it without any more drama than necessary. (In the most egregious cases of laziness, one might comment along the lines of "I copied and pasted your question title into a search engine and found the linked duplicate as the first result", assuming that's true. For me, it has been true many times.)

The way that new users are intended to get fast results is to find an existing question, not to ask their own. The primary purpose of asking a new question is not to get a direct, personalized response (since we after all explicitly don't offer "help" and feel no sense of urgency). Rather, it's to expand the supply of high-quality questions with high-quality answers, specifically so that searching will work better in the future.

A new question is a kind of bug report against the site - an implicit claim that the question being posed doesn't appear to be there already, and furthermore that it should be. This is, of course, actionable only when the underlying problem has been correctly identified, the corresponding question has been properly scoped, any necessary context is provided, and the absence of prior questions has been evaluated appropriately.

And in 2025, on a site which has over three times as many publicly visible questions about programming as Wikipedia has articles about literally anything notable, it is vanishingly unlikely that a beginner to programming has a question that actually matches that description.

I also want to pick at your phrasing a bit, and make sure it's clear: "problems" cannot be answered; only questions can. The distinction is vital because a question is what you end up with after analyzing the problem you've encountered, so as to:

  1. Figure out whether you need to ask about how to do something or about why something you don't understand is the way it is;

  2. Figure out what that thing is based on the problem you encountered;

  3. Phrase a question in a way that has enough context to make sense, but is still clearly a question that could be asked by others who go through the same process.


Therefore, if the goal is to convince new users that Stack Overflow is full of recent activity, the information about online user count etc. should appear all the time, not just on the Ask Question page - because new users should usually not see that page. (And you could perhaps also make a show of, for example, how many curation actions were taken in the last hour or so. Yes, including duplicate closures. If you can't put a positive spin on that, you haven't "gotten it" yet.)

And if the goal is to convince new users (especially ones not already skilled at programming) that Stack Overflow can provide quick answers, the right approach is to openly discourage asking a new question, and instead prominently encourage them to:

  1. Follow standard debugging advice, if they are trying to fix a problem in existing code;

  2. Break complex tasks down into logical steps, if they are trying to figure out how to do something;

  3. Either way, make sure they can identify specific, atomic things to ask about (and if they need to learn more than one thing, they should be prepared to have to worry about each one separately);

  4. Put effort into making sure they understand and can properly use the relevant terms and jargon (because this is essential to searching existing Q&A and also important for clearly posing new questions);

  5. Search the site both with the site's internal search and with external search engines (and explicitly include useful tips, such as adding site:stackoverflow.com to external searches).

Because every programmer needs these skills anyway, no matter what; and having learned them, they are permanent — and, in my experience, highly transferable to every other aspect of one's life.

Certainly they are necessary to "get help" from Stack Overflow - because they're part of the process of finding existing Q&A, and also (a smaller, but still essential) part of the process of asking new questions.

5
  • 4
    "it is vanishingly unlikely that a beginner to programming has a question that actually [is new]" Just another fundamental onboarding notice that new users should be given in caps with a stop sign to click to pass. Commented May 30 at 2:03
  • "Figure out whether you need to ask about how to do something or about why something you don't understand is the way it is" You've said this before multiple times (I was saying it independently) but I was surprised lately that you commented to disagree when I commented that those are different questions in reply to a meta request re improving a main question. Commented May 30 at 2:08
  • @philipxy "how should I write the code instead" isn't necessarily different from "why does this code do the wrong thing", depending on the details of why the code does the wrong thing (in particular: whether the problem is closely tied to the attempt to do the right thing, or whether it's simply one example of a gotcha that's unrelated to the right thing). But "how do I do the right thing", absent any previous attempt, typically is distinct. Commented May 30 at 2:10
  • Thanks. I still don't understand what distinction you're making re "how should I write the code instead". Maybe I agree with you but don't quite follow your last comment. I would agree that it can be ambiguously trying to say either question. (I think the question in question had a title tilting the ambiguity towards 1 of the 2.) Commented May 30 at 2:12
  • I vaguely recall what you're referring to. I'll say that this kind of thing is often more subjective and practical and may require SME about the specific question to judge. I'm also not infallible. But the general principle, as I see it, is to make sure that answers are in the best, most logical place for the most people who need them. (See also the "evaluated appropriately" link; there are others I probably could have linked, too.) Commented May 30 at 2:23
9

It kind of reminds me of Reddit's activity thing for subreddits. I wonder if you'd want to lean into that and factor in the aspect of tags somehow.

My one minor gripe is that it shifts the guidance down. Is there some way to make that not happen? There's a lot of basically unused space at the top right of the page.

1
  • 3
    Factoring in tags is a good idea. Re guidance - tbh, probably not a good way to do it for the first test. If we proceed to deployment, I'll make sure placement concerns and guidance prominence are registered as relevant concerns for the final design. Commented May 30 at 1:05
7

I think this communicates the opposite, this indidicates an expectation of 'fastness' that the site does not (and cannot) offer.

On many social media sites with algorithmic feeds that people are used to your post gets attention in the first hour or even ten minutes or it is buried. Advice for getting that initial momentum usually includes making your post at the right time of the right weekday, and piggy-backing on tags of some form that are 'trending' in some way -- This activity window gives users the impression that they need to do this on here, too.

But both of these techniques are useless and unnecessary on this site.

We don't want newbies to rush out posts or leave them waiting to hit some specific astrological time window, we don't want newbies to choose the tags of their question by what gets attention, we don't want newbies to look for short-term trends in tag-activity.

We don't want to give new users the impression that a new users lack of answers is due to not playing the algorithm and attention game correctly.

The simple reason that people are not getting 'fast' answers is that new questions nowadays are either bad (duplicate, low-quality, unanswerable, lacking understanding) or hard (niche, deep). The bad ones are not answered out of principle, the hard ones just take time naturally. The questions that are both easy and good are mostly gone, because they already exist on here.

4

A few minor notes on the UI of the box:

  • I'm unsure why it uses @Svg.SpeechBubbleRightStar for '36 answers shared' as opposed to using @Svg.Answer, especially considering the questions one uses @Svg.Question.
  • It's not wrong, per se, but the blue color on last 1 hr feels like it should be a link (maybe to let the user select a different time span, although I don't know that such a dropdown would be useful to 1-rep users)
  • Should 'Community Activity' be 'Community activity' for consistency with 'More helpful links'
  • The horizontal rules between the stats are fine, but it might be good to remove the one under the heading

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.