Questions tagged [hypothesis-testing]
Hypothesis testing assesses whether data are inconsistent with a given hypothesis (usually a null hypothesis of no effect).
11,119 questions
5
votes
3
answers
241
views
Why does the p-value increase when I add more observations to my t-test?
I am comparing the means of two groups using an independent two-sample t-test in R. Initially, I had the following samples:
Group A: n = 15, mean = 52.3, sd = 4.8
Group B: n = 15, mean = 48.1, sd = 5....
- 51
5
votes
3
answers
289
views
Assumptions when conducting hypothesis test for population mean
Consider the following problem:
A report in USA Today stated that the average age of commercial jets in the U.S. is 14 years. The population of commercial jet age is known to be normally distributed ...
- 391
14
votes
6
answers
923
views
Why does classical testing privilege Type I error (vs. the symmetric Bayesian view), why is status quo given privilege in traditional decision theory?
I'm trying to reconcile the classical Neyman-Pearson (NP) approach to hypothesis testing with the more symmetric, decision-theoretic view we use in my 6.437 (Inference & Information) class.
In our ...
- 7,318
1
vote
0
answers
43
views
Wilcoxon signed one sample test and asymmetry
Hi i would like to prove that wilcoxon signed rank (one sample) statistics measure asymmetry. The setup of Wilcoxon test is given a sample from a distribution $X_1,\dots, X_n$ and given a fixed value ...
- 11
1
vote
1
answer
43
views
Guarantees of different experiments with the same p-value
Frequentists often explain that:
If we construct a 95% confidence interval for the mean of some unknown distribution using n random samples from a population, the ...
- 153
6
votes
1
answer
223
views
Asymptotic consistency of hypothesis tests
We’d like for our hypothesis tests to be statistically consistent, meaning that as my sample size grows infinitely large, I will arrive at the right answer. But from my understanding, usually the ...
- 63
13
votes
3
answers
2k
views
Why is the Bonferroni correction needed?
Intuitively it makes sense that extracting multiple hypotheses from the same dataset is a bad idea because the probability of a false positive goes up. But when I think more about it, it doesn't make ...
8
votes
3
answers
817
views
Trouble with the definition of p-value
I'm following a statistics course and our professor gave us a definition of p-value I'm having some trouble understanding. First, some preliminary notations I want to set:
$\Omega = A \cup A^\...
- 81
5
votes
3
answers
327
views
Analyzing test result using test statistic vs p-value?
I am trying to understand how/when to use a test-statistic and/or p-value to draw a conclusion on a test.
As far as I understand, when running a statistical test we must decide whether to use the test ...
- 75
1
vote
0
answers
40
views
Compare two independent meta-analysis
I extracted the overall effect from an existing meta-analysis that I would like to compare with the overall effect of my meta-analysis by performing an equivalence test (These are two independent meta-...
- 83
3
votes
2
answers
145
views
Reporting p-values as an inequality or an exact value? [duplicate]
Should p-values be reported as an inequality (e.g. p<0.05) rather than the actual value (e.g. p=0.04)? If it is a binary decision as to whether results are significant, wouldn't stating p<0.05 ...
- 101
2
votes
1
answer
208
views
Help with original test of two medians
I am fairly new to statistics and for some reason I decided it was a good idea to come up with my own test to try and answer a research question. I'm wondering if this test I came up mostly via ...
- 21
1
vote
2
answers
97
views
How can I compare the average change between two unpaired groups?
I have two groups of participants, a test group and a control group, both of whom have completed pre/post activity surveys. For the test group, I can match their pre and post surveys to get change at ...
6
votes
1
answer
259
views
Do you need to do correction on multi-factor ANOVAs?
Assume a 2-way ANOVA with no post-hoc comparisons.
Do you need to use e.g. Bonferroni correction to determine the correct alpha?
I have seen anecdotally that some don't do this. I guess you can ...
- 381
4
votes
3
answers
587
views
Statistical test for independence
The Chi Square and Fisher's Exact tests have as the null hypothesis $H_0$ that the variables are independent. I understand and agree that, in these tests, if we reject the null, we have evidence of ...
Chris
5
votes
2
answers
301
views
Which tests to use when comparing small sets of proportions between two groups?
The question is similar to this
Which test to use when comparing multiple sets of proportions (with unknown sample size)?
except for the sample size being smaller (10 for each of the two conditions ...
- 679
9
votes
1
answer
769
views
Simpler explanation of p-values
I came across this fact sheet from the Data Analytics Services Unit at Western Michigan University (https://wmed.edu/dataanalytics):
https://wmed.edu/sites/default/files/P-VALUES%20SIMPLIFIED.pdf
They ...
- 93
3
votes
1
answer
79
views
Determining Sample Representativeness of Lat/Long Points
Let's say I have 100,000 points where each point represents a customers location in a given city. A third party has visited 10,000 customers and recorded information from them. I do not have access to ...
- 161
2
votes
2
answers
197
views
How does the significance level affect the probability of a Type I error? [duplicate]
Let's say I am conducting a (one sided, for simplicity) hypothesis test and have chosen my significance level to be $\alpha$. Then this means that if the probability of my data occurring assuming the ...
- 675
1
vote
1
answer
79
views
Suggestions for evaluating survey results using Factor analysis
currently I am tasked with evaluating the survey responses as part of ongoing marketing research and am hoping to get some suggestions as to how to evaluate them. There are 6 surveys in total (2 ...
- 11
1
vote
1
answer
120
views
Hypothesis test. Comparing two population parameters given a threshold (difference) value. Given paired means
We want to test whether two measurements start deviating from each other in time.
Situation
We want to know the quality of water by measuring an indicator (i.e. Concentration of pollutant). There is ...
- 111
3
votes
1
answer
336
views
Is it correct that in principle, both Bayes factor and posterior odds ratio can be used to perform hypothesis test?
Is it correct that in principle, both the Bayes factor and posterior odds ratio can be used to perform hypothesis test?
My textbook tells me that the hypothesis test should be determined by comparing $...
user398751
8
votes
3
answers
417
views
Definition of equivalence of statistical tests
I first encountered the notion of equivalence in the context of the Wilcoxon-Mann-Whitney-Test, where one can use either consider the Wilcoxon two-sample rank sum statistic $W$ or the Mann–Whitney $U$...
- 283
15
votes
8
answers
2k
views
Do we reject null if we have a pvalue of 0.0503 at alpha = 0.05? What way to validate the rejection, if ever?
I know it is more than 0.05, but I'm just wondering since rounding it to two decimals will give 0.05. I just wanted to ensure I am not falsely accepting the null hypothesis. Is there any way to say ...
- 153
6
votes
1
answer
223
views
Test if two coefficients in a linear model are both significant
Is there a single test one can perform that would be significant if two coefficients in a linear model are both different from 0, but would not be significant if only one of them differs from 0? Yes ...
- 2,087