Questions tagged [hypothesis-testing]
Hypothesis testing assesses whether data are inconsistent with a given hypothesis (usually a null hypothesis of no effect).
11,119 questions
5
votes
3
answers
220
views
Why does the p-value increase when I add more observations to my t-test?
I am comparing the means of two groups using an independent two-sample t-test in R. Initially, I had the following samples:
Group A: n = 15, mean = 52.3, sd = 4.8
Group B: n = 15, mean = 48.1, sd = 5....
- 51
4
votes
2
answers
295
views
How to test i.i.d. assumption?
Given a sample $X_1,\ldots X_n$, how can I test the hypothesis that these are i.i.d. samples from a fixed (unknown) distribution?
To add context, assume this is a time series and I want evidence ...
- 159
1
vote
0
answers
43
views
Wilcoxon signed one sample test and asymmetry
Hi i would like to prove that wilcoxon signed rank (one sample) statistics measure asymmetry. The setup of Wilcoxon test is given a sample from a distribution $X_1,\dots, X_n$ and given a fixed value ...
- 11
1
vote
1
answer
43
views
Guarantees of different experiments with the same p-value
Frequentists often explain that:
If we construct a 95% confidence interval for the mean of some unknown distribution using n random samples from a population, the ...
- 153
0
votes
0
answers
14
views
How can I test the proportional odds assumption for cumulative logit models estimated with svyVGAM?
I'm trying to estimate a cumulative logit model with partial proportional odds while taking into account sampling weights (to make the sample representative of the population). My best option seems to ...
14
votes
6
answers
923
views
Why does classical testing privilege Type I error (vs. the symmetric Bayesian view), why is status quo given privilege in traditional decision theory?
I'm trying to reconcile the classical Neyman-Pearson (NP) approach to hypothesis testing with the more symmetric, decision-theoretic view we use in my 6.437 (Inference & Information) class.
In our ...
- 7,318
2
votes
0
answers
42
views
Two ideas to include measurment errors in a Binomial Test with different results
I have a measurement with two possible outcomes, let's say 0 and 1. If the outcome of the measurement is 1, the true value is always 1. If the outcome of the measurement is 0, there is a chance of 1% ...
- 83
1
vote
0
answers
40
views
Compare two independent meta-analysis
I extracted the overall effect from an existing meta-analysis that I would like to compare with the overall effect of my meta-analysis by performing an equivalence test (These are two independent meta-...
- 83
0
votes
1
answer
21
views
Handling multiple comparison corrections in association rule mining
We’re working on a project applying association rule mining to environmental DNA (eDNA) datasets, and run into some uncertainty regarding how to best handle multiple comparison corrections in this ...
1
vote
2
answers
95
views
How can I compare the average change between two unpaired groups?
I have two groups of participants, a test group and a control group, both of whom have completed pre/post activity surveys. For the test group, I can match their pre and post surveys to get change at ...
2
votes
1
answer
208
views
Help with original test of two medians
I am fairly new to statistics and for some reason I decided it was a good idea to come up with my own test to try and answer a research question. I'm wondering if this test I came up mostly via ...
- 21
1
vote
1
answer
79
views
Suggestions for evaluating survey results using Factor analysis
currently I am tasked with evaluating the survey responses as part of ongoing marketing research and am hoping to get some suggestions as to how to evaluate them. There are 6 surveys in total (2 ...
- 11
3
votes
1
answer
79
views
Determining Sample Representativeness of Lat/Long Points
Let's say I have 100,000 points where each point represents a customers location in a given city. A third party has visited 10,000 customers and recorded information from them. I do not have access to ...
- 161
13
votes
3
answers
2k
views
Why is the Bonferroni correction needed?
Intuitively it makes sense that extracting multiple hypotheses from the same dataset is a bad idea because the probability of a false positive goes up. But when I think more about it, it doesn't make ...
0
votes
0
answers
47
views
Sampling counterfactual posterior to mitigate error autocorrelation in event studies
I have question regarding event studies (pre-event data is observed, an event occurs at $t=e$, then following the treatment is assumed to be in-effect.)
There are multiple approaches to event study ...
- 3,970
3
votes
2
answers
145
views
Reporting p-values as an inequality or an exact value? [duplicate]
Should p-values be reported as an inequality (e.g. p<0.05) rather than the actual value (e.g. p=0.04)? If it is a binary decision as to whether results are significant, wouldn't stating p<0.05 ...
- 101
1
vote
1
answer
120
views
Hypothesis test. Comparing two population parameters given a threshold (difference) value. Given paired means
We want to test whether two measurements start deviating from each other in time.
Situation
We want to know the quality of water by measuring an indicator (i.e. Concentration of pollutant). There is ...
- 111
0
votes
0
answers
53
views
Testing if aggregated statistics have different distribution for each group in time series data
This post is continuation of these (Test for significant difference between the groups of the time series based on the aggregated statistics), but with more detailed explanation (a lot of changes were ...
8
votes
3
answers
484
views
Proportions of groups are aligned but the T-statistic is large?
CONTEXT
I have the following data. Everyone was sent an invitation to a party. I wanted to test whether or not sending an invitation late would change the respondent's attendance on a statistically ...
- 223
1
vote
2
answers
197
views
Intuitive understanding of multiple hypothesis testing
Can anyone explain to me, in simple language, why multiple hypothesis testing correction is important if I test multiple set of outcomes?
It is hard for me to wrap my head around this. This is because ...
- 31
1
vote
0
answers
80
views
Minimal conditions to define rejection regions in hypothesis testing
I have a conceptual question about hypothesis testing. I would like to know what are the minimal conditions that a statistic and its rejection regions must satisfy in order for a test of a null ...
- 210
0
votes
0
answers
60
views
Is the binary hypothesis testing's minimum error probability monotonically decreasing with KL divergence?
For the following binary hypothesis testing problem
$$
\begin{aligned}
H_0: \boldsymbol{y} \sim f(\boldsymbol{y} | H_0)\\
H_1: \boldsymbol{y} \sim f(\boldsymbol{y} | H_1)
\end{aligned}
$$
where $\...
- 1
1
vote
2
answers
198
views
confusion about superiority trial sample size
I have looked for info on this before posting but couldn't find anything. I'm a biostatistician working in cancer research so trials are pretty common, but I don't have a ton of previous experience.
...
- 11
6
votes
1
answer
223
views
Test if two coefficients in a linear model are both significant
Is there a single test one can perform that would be significant if two coefficients in a linear model are both different from 0, but would not be significant if only one of them differs from 0? Yes ...
- 2,087
4
votes
1
answer
63
views
Seeking Advice: Analysis Strategy for a 2x2 Factorial Vignette Study (Ordinal DVs, Violated Parametric Assumptions)
I am seeking guidance on the most appropriate statistical methodology for analyzing data from my research investigating public stigma towards comorbid health conditions (epilepsy and depression). I ...
