7
$\begingroup$

I stumbled upon the definition of a continuous random variable when I studied Mathematical Statistics. It confused me badly.

By a Continuous Random Variable I think they mean the following:

If the sample space $\Omega$ is a continuum, then the the random variable defined on $\Omega$ is called a continuous random variable.

However, in this case, it seems that the range of a continuous random variable is not necessarily uncountable. For example, the random variable $X:\Omega \to \Bbb R$ defined by $X(\omega)=1,\forall \omega\in \Omega$ has a finite range $\{1\}$, I doubt it would be considered a continuous random variable. This precisely creates a confusion for me.

My question Are the range of continuous random variables necessarily uncountable by definition? What is the exact definition of a continuous random variable?


Some references:

The book from where I am studying all these i.e, Mathematical Statistics by John E. Freund, has a chapter dedicated to the continuous random variables in Section 3.3 where no formal definition of continuous random variables is given but some introductory notions are given, for example, as follows:

In the continuous case, where random variables can take on values on a continuous scale, the procedure is very much the same. The outcomes of experiments are represented by the points on line segments or lines, and the values of random variables are numbers appropriately assigned to the points by means of rules or equations.

Also, in this website, I found this definition:

A continuous random variable is one which takes an infinite number of possible values. Continuous random variables are usually measurements. Examples include height, weight, the time required to run a mile.

Again, this appears unclear to me and makes me more confused as I am not sure whether they mean both countably infinite and uncountably infinite when using the word 'infinite' in the above definition.

All in all, this seem to contradict the notion of continuous random variables given on Freund's book.

$\endgroup$
12
  • 2
    $\begingroup$ You can remove the word "unique" in your definition of random variable. It is a function that assigns each outcome in the sample space to a real number (not necessarily unique). For example, $X(\omega)=0$ for all $\omega \in \Omega$ is a valid random variable that is always-zero. Also, I don't think anyone talks about "continuous sample space." Instead, people talk about sample spaces with countably many outcomes (meaning finite or countably infinite), and sample spaces with uncountably many outcomes. It is common to define discrete random variables on an uncountably infinite sample space. $\endgroup$ Commented Jul 27, 2024 at 15:20
  • 5
    $\begingroup$ The final website you mentioned that says "A continuous random variable is one which takes an infinite number of possible values" is dumbing down the material. It is not giving standard definitions and it will not be helpful in clarifying anything. $\endgroup$ Commented Jul 27, 2024 at 15:35
  • 2
    $\begingroup$ The always-zero random variable is a good example of a discrete random variable that can be on any sample space, including a sample space that is a “continuum.” We can define $S_X$ as the set of all values a rv $X$ can take. Indeed a good definition of “discrete random variable” is one where $S_X$ is finite or countably infinite. This is not quite the same as Amir’s more inclusive definition, but is close enough for all practical purposes. As Amir notes, a random variable that is not discrete type could be of continuous type, or mixed. $\endgroup$ Commented Jul 27, 2024 at 19:39
  • 1
    $\begingroup$ Strictly speaking, this depends on the linguistic conventions of the person you are talking to. One usage says that a continuous random variable is what is more precisely described as an absolutely continuous random variable, meaning that its law is given by integration of a function against the Lebesgue measure. Without the measure theory lingo, this means the CDF is given by $\int_{-\infty}^{x_1} \int_{-\infty}^{x_2} \dots \int_{-\infty}^{x_n} f(y_1,y_2,\dots,y_n) dy_n dy_{n-1} \dots dy_1$ for some $f$. $\endgroup$ Commented Jul 28, 2024 at 17:18
  • 1
    $\begingroup$ (Cont.) Some authors use the word "continuous" to refer to all random variables whose CDF is continuous, and then use the words "absolutely continuous" and "singular continuous" to split things up further. In my recent experience this latter usage is not common, and most usages of the term "continuous random variable" really mean "absolutely continuous random variable". You can immediately tell that this is going on if the author starts talking about PDFs. $\endgroup$ Commented Jul 28, 2024 at 17:18

3 Answers 3

10
$\begingroup$

There are basic definitions of continuous, discrete, and mixed random variables, that can be understood by who are less familiar with measure theory (see PS1 for more technical details).

Continuous random variables

Two equivalent definitions for continuous random variables are as follows:

First definition:

A random variable $X$ is continuous iff $\mathbb P(X=x)=0$ for any $x\in\mathbb R$.

Second definition:

A random variable $X$ is continuous iff its cdf $F_X(x)=\mathbb P(X \le x)$ is continuous on $\mathbb R$ (equivalently, the image of the cdf inculdes $(0,1)$).

Note that the image of any cdf is a subset of $[0,1]$, but for a continuous random variable it is a superset of $(0,1)$ (it may not include $0$ or $1$, for example, consider normal and exponential distributions). A deep analysis of the image of a cdf can be found here [1].

Discrete random variables

Similarly, two equivalent definitions for discrete random variables are as follows:

First definition:

A random variable $X$ is discrete iff $\sum_{x \in S_0}\mathbb P(X=x)=1$ for a countable set $S_0$ of points, where $\mathbb P(X=x)>0$ for $x\in S_0$ and $\mathbb P(X=x)=0$ for $x\notin S_0$ (equivalently, $\mathbb P(X\in K)=1$ for some countable set $K$).

The set $S_0=\{ x \in \mathbb R: \mathbb P(X=x)>0 \}$ is a subset of the image of $X$, denoted by $X(\Omega)$ (when $X$ is viewed as a Borel measurable function on the sample space $\Omega$), which is also called the support of $X$ (see this answer [2] for two equivalent definitions of the support of a random variable); in fact, there are some pathological cases that the image $X(\Omega)$ is larger than the set $S_0$ appeared above and is equal to the union of $S_0$ and a null set $N$ (see PS4 for more details). Note that $K$ is not unique and can be any countable superset of $S_0$, which is the smallest possible $K$. In sum, the image of a discrete random variable is the union of a countable set (its support) and a null set (so the image can be countable or uncountable), and the image of any non-discrete (continuous or mixed) random variable is always uncountable.

Second definition:

A random variable $X$ is discrete iff its cdf $F_X(x)=\mathbb P(X \le x)$ is discontinuous only on a countable set of points in $\mathbb R$ whose jump values sum to $1$ (equivalently, the image of the cdf is a null set).

Indeed, the cdf has jumps only on the set $S_0$, and is continuous on $ \mathbb R \setminus S_0$ where $S_0$ is the set appeared in the first definition. When $S_0$ is finite or can be ordered with respect to the usual order of numbers, the cdf is a staircase function (also called step function [3]). If $S_0$ is a densely ordered set [4] such as the set of rational numbers in $[0,1]$, the cdf can be strictly increasing with no flat areas; see here [5] for more details. Hence, the image of the cdf of a discrete random variable can be uncountable set (as well as its pushforward measure). The equivalent condition given in the parentheses follows from the fact that any increasing function is discontinuous at most on a countable set, and thus $F_X$ has a null image (its Lebesgue measure is zero) iff its jump values sum to $1$. In fact, it is not required to mention that the set of discontinuities is a countable set in the first part of the definition, but I kept it since some people may be not aware of such a result (i.e., the set of discontinuities is either empty or countable for any increasing function).

Mixed random variables

Simply, a random variable $X$ is called mixed when it is neither continuous nor discrete.

An example of a mixed random variable is $X=\min(U,0.5)$ where $U \sim \mathcal U(0,1)$. Indeed, for $x=0.5$, $\mathbb P(X=0.5)=0.5>0$ whereas $\mathbb P(X=x)=0$ for any $x\neq 0.5$. Similarly, the cdf $F$ is given by

$$ F(x) = \begin{cases} 0 &\quad x<0\\ x & 0\le x<0.5 \\ 1 &x\ge 0.5\\ \end{cases}$$

which is neither a continuous nor a staircase function (note that the set of discontinuities is finite).



PS 1: It should be noted that measure theory is required to properly define the concept of random variable $X$, and probabilities $\mathbb P(X=x)$ and $\mathbb P(X \le x)$ used in the above definitions. Given the probability space $(\Omega, \mathcal F, P)$, $X:\Omega \to \mathbb R$ is called a random variable if it is Borel measurable, and the probabilities are computed as $$\mathbb P(X=x)=X_*(P)(\{x\})=P\left (X^{-1}(\{x\})\right)$$ $$\mathbb P(X \le x)=X_*(P)\left((-\infty,x] \right)=P\left (X^{-1}((-\infty,x])\right)$$ where $X_*(P)$ denotes the pushforward measure of $P$ by random variable $X$.

PS 2: Some authors call continuous, discrete, and mixed random variable, defined above, as a random variable with continuous, discrete, and mixed distribution. The terms "absolutely continuous distribution" and "absolutely continuous random variable " may be also used for a continuous random variable with a pdf, that is, there is a non-negative function $f:\mathbb R \to \mathbb R_{\ge 0}$ such that its cdf can be represented as $F_X(x)=\int_{-\infty}^x f(t) \text{d}t$ for any $x\in \mathbb R$.

PS 3: According to the classification given in Cramér's book (Random Variables and Probability Distributions), there are four mutually exclusive types of distributions: discrete, absolutely continuous, singular continuous, and a mixture of some of the preceding types. In fact, absolutely continuous and singular continuous, and their convex combinations form the class of continuous distributions.

The above classification is based on the fundamental Lebesgue's decomposition theorem [6] , which states the cdf $F_X$ of any random variable $X$ can be decomposed as follows for some non-negative weights $\alpha_{AC},\alpha_{SC},\alpha_D$ with $\alpha_{AC}+\alpha_{SC}+\alpha_D=1$:

$$F_X(x)=\alpha_{AC} (x)+\alpha_{SC} F_{SC}(x)+\alpha_D F_DF(x)$$

where $F_{AC}(x)$, $F_{SC}(x)$, and $F_D(x)$ are cdfs of some absolutely continuous, singular continuous, and discrete distributions, respectively. For a continuous random variable $X$, we have $\alpha_D=0$, and for a continuous random variable $X$ with a pdf, we have $\alpha_{SC}=\alpha_D=0$.

PS 4: It should be noted that a continuous random variable $X:\Omega \to \mathbb R$ cannot be defined only based on $\Omega$ or the image $X(\Omega)$ since it also depends on the probability measure $P$ defined on $\Omega$. In fact, the sample space $\Omega$ can be uncountable, but $X$ is a discrete random varaible, e.g., when $X$ is a constant $c$, $X(\omega)=c, \forall \omega\in\Omega$. Moreover, the image of $X$ can be uncountable, but $X$ is classified as a discrete random variable, e.g., when $X$ is a constant $c$ with probability one, defined as $X(\omega)=c, \forall \omega\in [0,1]\setminus N$ and $X(\omega)=\omega, \forall \omega \in N$, where $N$ is an uncountable null subset of $[0,1]$ such as the Cantor set [7] in $[0,1]$ and where the Lebesgue measure is considered as the probability measure on the sample space $\Omega=[0,1]$.

In sum, random variables defined on an uncountable sample space (such as a continuum) can be of any type whereas any random variable defined on a countable sample space is discrete. Indeed, a random variable $X$ cannot be classified only based on the sample space $Ω$ or the image $X(Ω)$ because the classification also depends on the probability measure defined on $Ω$, unless $Ω$ is a countable set for which $X$ is always discrete.

PS 5: Last but not least, the terminology used for continuous and discrete random variables may be confusing since random variables are formally defined as special functions. Indeed, a random variable $X$ is a (Borel measurable) function from the sample space $\Omega$ to $\mathbb R$, for which generally we cannot define continuity unless the sample space $\Omega$ is an open subset of some topological/metric space. Even in this case, a continuous random variable $X$ can be a non-continuous function on $\Omega$. For example, let us define $X(\omega)= 0, \omega \in (0,1)\cap \mathbb Q$ and $X(\omega)= \omega, \omega \in (0,1)\setminus \mathbb Q$ where the probability measure is the Lebesgue measure on $\Omega=(0,1).$ Indeed, $X$ is a continuous random variable with distribution $\mathcal U(0,1)$, but it is not a continuous function over $\Omega=(0,1)$. On the other hand, $X(\omega)= 0, \omega \in (0,1)$ is a discrete random variable, but it is a continuous function on $\Omega=(0,1)$. In fact, the adjectives "discrete", "continuous", and "mixed" have been used to approximately describe the set of values retuned by random variables, not to describe them as some functions.

$\endgroup$
19
  • 3
    $\begingroup$ @QiaochuYuan Thanks for your comment! Yes it is standard. A random variable with the Cantor distribution is considered as a famous example of a continuous random variable for which no pdf can be defined (because its cdf is only continuous and not absolutely continuous). $\endgroup$ Commented Jul 27, 2024 at 8:23
  • 1
    $\begingroup$ Also, in the Wikipedia link on the Cantor distribution, in the statement "It is thus neither a discrete nor an absolutely continuous probability distribution, nor is it a mixture of these. Rather it is an example of a singular distribution", singular distributions are a type of continuous distributions and there is no disagreement (please see the Wikipedia link on Singular distribution). $\endgroup$ Commented Jul 27, 2024 at 8:24
  • 1
    $\begingroup$ Sorry to be pedantic, but neither of those pages refers to the Cantor distribution or singular distribution using the term "continuous random variable," and the Wiki pages I've seen appear to be very careful about this. For example en.wikipedia.org/wiki/… does not even define "continuous probability distribution" but only "absolutely continuous probability distribution." $\endgroup$ Commented Jul 27, 2024 at 8:36
  • 1
    $\begingroup$ They cannot be used in this way. You may see that in my first comment I stated "A random variable with the Cantor distribution is considered as a famous example of a continuous random variable for which no pdf can be defined", which can be re-stated as "The Cantor distribution is considered as a famous example of a continuous distribution for which no pdf can be defined". Both terms "continuous random variable" and "continuous distribution" are used interchangeably in most elementary and advanced text books and research papers, and I think this should be clarified in Wikipedia. $\endgroup$ Commented Jul 27, 2024 at 8:56
  • 2
    $\begingroup$ The accepted terminology is very poor: a r.v. of discrete type (referring to its distribution) may be a continuous function (e.g. the identity function on $\Bbb R$). I think this is due to historical usage of "random variable" as a variable rather than a function; e.g. Cramér's classic book Random Variables and Probability Distributions categorizes a r.v. by the "type" of its distribution function (c.d.f.), the four mutually exclusive possibilities being (1) discrete, (2) absolutely continuous, (3) singular continuous, (4) a mixture of some of the preceding types. $\endgroup$ Commented Jul 27, 2024 at 14:58
5
$\begingroup$

This is annoyingly subtle, and pinning down the details precisely depends in part on how far you're willing to dig into the measure-theoretic weeds. First of all, as far as I know there is not a completely precise formal definition of "continuous random variable." Here are two completely precise formal definitions of different words:

Definition 1: A real-valued random variable $X$ is a measurable function $X : S \to \mathbb{R}$ where $S$ is a probability space.

This incredibly general definition makes no assumptions about the function $X$ other than measurability (which is a very mild technical assumption needed to avoid horrible monsters) and no assumptions about the sample space $S$ other than that it is equipped with some probability measure; the range of $X$ could be an arbitrary (measurable) subset of $\mathbb{R}$, and in particular could be uncountably infinite, countably infinite, or finite. If you are only interested in statistics you don't need to understand this definition, but this is how mathematicians set up the foundations of random variables in a maximally general way, and it naturally includes both "discrete" and "continuous" random variables as a special case, as well as mixtures of these and other examples that aren't obviously either "discrete" or "continuous."

To get much more concrete, we can produce interesting "continuous" examples via the following construction.

Definition 2: A real-valued random variable $X$ has a probability density function if there exists a measurable function $f : \mathbb{R} \to \mathbb{R}_{\ge 0}$ such that

$$\mathbb{P}(X \in [a, b]) = \int_a^b f(x) \, dx.$$

Often in practice the function $f$ will be continuous or at least piecewise continuous; for example this is enough to get us the uniform distribution on an interval, the normal distribution, etc. Every example I can think of is piecewise continuous; I think that will generally be true in statistics.

This is a pretty good way to cash out what people in practice mean when they say "continuous random variable," and as written it excludes any random variable such that the probability that $\mathbb{P}(X = r)$ is nonzero, because it's not possible to produce such nonzero probabilities by integrating a function. So in fact the range of a random variable with a pdf is necessarily uncountable. The precise distinction between arbitrary random variables and ones that have pdfs is formalized by the notion of absolute continuity but again, we're getting into the measure-theoretic weeds here. I have personally never needed to learn about this in any detail.

Also, this is less relevant to the main thrust of your question, but your definition of "continuous sample space" is quite awkward, since it excludes any higher-dimensional Euclidean space $\mathbb{R}^n, n \ge 2$. Even statisticians want to be able to talk about the multivariate normal distribution!

how does the time required to run a mile, qualifies as a valid example of this, as is mentioned in the quoted definition above. The time required to run a mile by a particular person is a fixed time that may be measured in seconds, hours, etc.

In the context of statistics I assume people are thinking in terms of survey data, so you're thinking of survey results (e.g. times required for $1000$ people to run a mile) as samples from an underlying random variable (the time required for a "random person" to run a mile).

$\endgroup$
5
  • $\begingroup$ I disagree with your statement "First of all, as far as I know there is not a completely precise formal definition of "continuous random variable." They can be defined precisely. You may check my answer. $\endgroup$ Commented Jul 27, 2024 at 7:25
  • $\begingroup$ @QiaochuYuan I have edited the original post as well with some more details so that I may sound more clear about the issue. It would be very helpful if you consider checking out the edited post. $\endgroup$ Commented Jul 27, 2024 at 8:48
  • 1
    $\begingroup$ @Qiaochu : Minor adjustment, I think your Definition 1 should say that $S$ is the "sample space" rather than the "probability space," since probability space is usually a triplet that includes the sample space, sigma algebra, and probability measure. $\endgroup$ Commented Jul 27, 2024 at 15:24
  • $\begingroup$ @Michael: this is an ordinary abuse of language in mathematics. It's very standard to refer to a tuple of a set $S$ with a bunch of extra structure on it using only the carrier set $S$, e.g. when I say "the ring $\mathbb{Z}$" I don't usually specify "the ring $(\mathbb{Z}, 0, 1, +, \times)$" and so forth. $\endgroup$ Commented Jul 27, 2024 at 18:07
  • 1
    $\begingroup$ Then I do not think you can characterize your definition as “completely precise,” as you have currently described it. $\endgroup$ Commented Jul 27, 2024 at 20:01
5
$\begingroup$

(From your description of "continuous sample space", I guess the book you are using is not the best choice. It was written mostly for non-mathematical undergrad students equipped with some basic calculus training. )

When we say continuous random variables, we typically refer to the CDF. Usually we don't care too much about the continuity of "mapping from $S$ to $\mathbb{R}$" itself, which is what people typically consider as a "continuous function". For the latter, we have to have some way to quantify what is a small change in $S$, but probably no one really bothers to define that for a probability space.

As @Amir answered, the continuous random variable is usually defined according to the continuity of the CDF of the random variable. It's a direct property of / qualifier for the CDF, which is only indirectly related to the random variable (as a function).

In nearly all statistical commmunications, people would interpret "continuous" as "absolutely continuous", except in discussions directly concerning the measure theoretic foundations of probability.


PS: A similar terminology issue/convention is "convergence in distribution". When people say R.V.'s $X_n$ converges in distribution to $X$, they really mean the convergence of the corresponding CDF's, not the convergence of R.V.'s.

$\endgroup$
2
  • 4
    $\begingroup$ I think this should be more emphasized: the accepted terminology is very poor, as a random variable of discrete type (referring to its distribution) may be a continuous function (e.g. the identity function on $\Bbb R$). $\endgroup$ Commented Jul 27, 2024 at 14:04
  • 1
    $\begingroup$ A tangential comment on "...but probably no one really bothers to define [small change] for a probability space" - AFAIC, the existence of regular conditional probabilities and disintegrations relies on topological properties (e.g. having a Borel space based on a Polish space); some authors make statements only about conditional distributions of variables, while others also speak of conditional probability distributions per se (without attachment to a specific variable). In the latter case one has to define a topology in the base probability space $(S,\Sigma,P)$. $\endgroup$ Commented Jun 28 at 20:09

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.