Questions tagged [hypothesis-testing]
Hypothesis testing assesses whether data are inconsistent with a given hypothesis (usually a null hypothesis of no effect).
11,119 questions
0
votes
1
answer
125
views
Statistical inference for L2-penalized cox proportional hazard regression
First and foremost, thank you for reading my problems. This is my first time asking on this platform so I apologize if some requirements are not met.
I am trying to estimate the effect of multiple ...
2
votes
0
answers
67
views
Resources for UMP Hypothesis Tests
There are many examples in mathematical statistics books like this; "Let $X_1, \ldots, X_n$ be a random sample from some distribution with parameter(s) θ. A UMP (or UMP unbiased) test for testing ...
- 165
3
votes
1
answer
94
views
If a hypothesis test with level $\alpha$ rejects for my realized sample, will all tests with level > $\alpha$ also reject the same sample?
Question: If a hypothesis test with level $\alpha$, rejects for a realized sample, will all tests with level > $\alpha$ also reject the same sample?
Formally: let $\hat{\psi}_{\alpha} \in \{0, 1\}$ ...
- 804
1
vote
2
answers
79
views
Evaluating total effect on population after conducting a/b test
I conducted an A/B test where the treatment involved offering discounts to customers. However, after the test concluded, I discovered that the discount was not consistently displayed to customers in ...
- 11
2
votes
2
answers
166
views
Null hypothesis significance testing with non-independent observations
I am working with a dataset with measurements that we classify into distinct categories. The primary objective is to determine whether the measurements within each category differ significantly from ...
- 25
0
votes
1
answer
119
views
Significance of interaction term in GAM
I have a generalized additive model of the following form:
$E[Y] = \beta_0 + f(X_1) + \beta_2X_2 + f(X_1)\times X_2$
Where $f(X_1)=\sum^n_i\beta_i\boldsymbol{B}_i(X_{1i})$ is a smoothing function and $...
3
votes
1
answer
111
views
Can flipping a coin give a hypothesis test an arbitrary $\alpha$?
Does the following hypothesis test meet all the definitions?
Select an $\alpha$ (typically $0.05$)
Let $A := \lceil -\log_2(\alpha) \rceil$ (where $\lceil x \rceil$ is the smallest integer greater ...
- 252
6
votes
2
answers
687
views
Why do we define the standard error to ignore bias (unlike MSE which includes bias)?
Why is standard error of an estimator $\hat \theta$ defined as $$se = \sqrt{Var(\hat \theta)}$$ and not $$se = \sqrt {MSE(\hat \theta)} = \sqrt{Bias^2(\hat \theta) + Var(\hat \theta)}.$$
That is, ...
- 252
1
vote
0
answers
73
views
Hypothesis tests and p-values of Bayesian parameter estimates
In frequentist statistics it is common to do hypothesis tests on estimated parameters of (generalized) linear models, e.g. to test whether one of the coefficients significantly differs from 0 or has a ...
4
votes
1
answer
257
views
How to calculate standard deviation when only mean of the data, sample size, and t-test is available?
I found a study that reports the following data for two groups:
Mean (7.5 and 8.68)
Sample size of each group (26 and 22)
P-value from the independent one-tailed t-test (0.055)
Is there a formula to ...
- 41
1
vote
0
answers
72
views
Proper A/B test analysis with related segments
I have the data of an online A/B experiment (e.g. a redesign) and there's a concern about a different effect on different platform - web and mobile. The users was splitted by some identificator, and ...
- 147
3
votes
1
answer
74
views
Comparing multiple means from a dependent variable against multiple means from an independent variable -- Predict mean scores based on mean scores?
I have about 300 students who replied to a psychological test formed of several resilience and vulnerability issues. A typical Likert scale, with "mean scores" as the result. From these ...
- 244
1
vote
1
answer
99
views
Circular Statistics - Looking at wind selectivity in shorebirds and want to select the correct statistical test
I completed my Master's degree looking at shorebird migration ecology, specifically looking at their behaviour during southbound migration. One component was looking at the conditions upon departure ...
ahjort
2
votes
0
answers
60
views
Test of variance component in one-way random effects ANOVA
I have a question regarding the adaptation of the following to the test of single variance components laid out in Raudenbush and Bryk (2002, p. 63-64). The hypothesis is about a single component of ...
- 3,789
0
votes
0
answers
70
views
Number of runs needed for Probability of Improvement metric in Deep RL
I'm working with the Probability of Improvement (POI) metric described in [1], Section 4.3.
The paper introduces various aggregate metrics in Section 4.3, and for most of these metrics (IQM, mean, ...
- 682
1
vote
0
answers
41
views
Sequential probability ratio test for variance
An exercise problem from Hogg and McKean's book "Introduction to Mathematical Statistics" is the following
Exercise 8.4.1. Let $X$ be $N(0,\theta)$ and, in the notation of this section, let $...
4
votes
4
answers
1k
views
How to permute p-values?
Recently, my colleague encountered a problem while working with a dataset, df_a, which contains gene data. Each row represents a ...
- 163
0
votes
0
answers
80
views
Not computed tests in gofstat in R
I am working on a survival analysis project. For this project, I use this dataset:
https://archive.ics.uci.edu/dataset/519/heart+failure+clinical+records
I began by importing these libraries :
...
- 1
1
vote
2
answers
128
views
Suitable statistical test
I have 2 groups - people that were ever active on my website and people who were never active. Activity is a metric based on some actions - both groups of people visited the website.
Each group used a ...
0
votes
0
answers
81
views
How to find the point where a ridge 'statistically' dissipates?
Here is an image of a surface $\mathcal{M}$ of my interest.
The surface is estimated from a family of curves $z=f_x(y)$ (each curve is a measured physical profile of some object) assigned with $x$ ...
- 101
1
vote
1
answer
112
views
Chi-squared test of independence
Consider a contingency table which contains observation counts that are poisson distributed. In particular, $O_{ij}$ is $Poi(\lambda_{ij})$. If we apply CLT assuming $\lambda$ is large, then we get ...
- 135
0
votes
0
answers
43
views
How to establish statistical significance of changes in strength & direction of confounding/mediators/risk factors/effect modifiers for paired survey?
First of all, I'm new to Biostatistics (took one Intro to BioStats class more than a year ago), so please be patient with me :-) ...
I have a paired survey taken before (F0 survey) and after the ...
- 35
1
vote
0
answers
43
views
Nonparametric extrema hypothesis test based on ranks
Does there exist a non-parametric test testing the following hypothesis:
given $N$ points in $\mathbb{R}^m$ and their ranking $Y$ in $\mathbb{N}$ answer:
$H_0:$ there is no minimum or maximum (extrema)...
- 41
0
votes
0
answers
41
views
Compare two variances
I am reading this paper
I have difficulty understanding Section 6: A Linear Time Statistic and Test.
At the beginning, they claim that $\text{MMD}^2_l$ has higher variance than $\text{MMD}^2_u$ (we ...
- 141
15
votes
4
answers
1k
views
In a Frequentist setting, how are we able to condition on the null hypothesis being True/False?
Paraphrasing Casella and Berger (2002): A hypothesis test is defined by a null hypothesis $H_0: \theta \in \Theta_0 $ and an alternative hypothesis $H_1: \theta \in \Theta_0^c = \bar{H_0}$, where $\...
- 804