22 Quantile Regression
22.1 Towards Estimation
A common theme in the identification results of the previous sections is that we express the parameters of interest in terms of the conditional distributions or quantiles of the observed outcome \(Y_i\). Thus, to estimate these objects, it is sufficient to estimate these conditional quantities.
When all conditioning variables are discrete, estimation is straightforward: we can simply split the sample and use sample quantiles or empirical distribution functions within each subsample. However, when continuously-distributed variables are present, splitting the sample is not feasible. Instead, we must approximate the quantiles or the distribution function.
Quantile and distribution regression are two primary methods for this purpose:
- Quantile regression directly models conditional quantiles of the type \(Q_{Y|X, W}(\tau|x, w)\).
- Distribution regression instead models the conditional distribution function \(F_{Y|X, W}(y|x, w)\).
In this section, we provide an essential introduction to quantile regression, while distribution regression is covered in the next section. Finally, in section 24, we discuss how to use these methods to estimate QTEs and DTEs.
22.2 Introduction to Quantile Regression
22.2.1 Setting
We now give a brief introduction to quantile regression, temporarily disregarding the causal framework. For a more detailed treatment, see the monograph by Koenker (2005) and the Handbook of Quantile Regression (Koenker et al. 2017).
As always, let \(Y_i\) be some outcome variable and \(Z_i\) be a vector of covariates (which may include both the treatment \(X_i\) and other controls \(W_i\)). We assume the conditional CDF \(F_{Y|Z}(\cdot|z)\) is continuous and strictly increasing for all \(z\).
Our goal is to estimate the \(\tau\)th conditional quantile \(Q_{Y|Z}(\tau|z)\) of \(Y_i\) given \(Z_i=z\) given a sample of \(N\) i.i.d. observations \(\{(Y_i, Z_i)\}_{i=1}^N\).
22.2.2 Population Quantile Regression
The starting point for quantile regression is the characterization of quantiles as minimizers of a specific loss function. Under our assumptions on the CDF of \(Y_i\), the conditional quantile uniquely minimizes the check risk: \[ Q_{Y|Z}(\tau|Z) = \argmin_{q(Z)} \E[\rho_{\tau}(Y-q(Z))], \tag{22.1}\] where the minimum is taken over the class of all measurable functions and \(\rho_{\tau}(u)\) is the check loss function: \[ \rho_{\tau}(u) = \begin{cases} \tau u, & u \geq 0, \\ (\tau-1)u, & u<0. \end{cases} \tag{22.2}\]
22.2.3 Sample Objective Function
The population characterization in Equation 22.1 cannot be used directly in practice for two reasons:
- We only have access to a finite sample;
- The class of all measurable functions is too large to search through.
The first problem is straightforward to address by replacing the population expectation with its sample counterpart to obtain the sample objective function: \[ \E_n[\rho_{\tau}(Y-q(Z))] \equiv \dfrac{1}{N} \sum_{i=1}^N \rho_{\tau} (Y_i - q(Z_i)). \tag{22.3}\]
22.2.4 Linear Specification
To address the second problem, we restrict the quantile function \(q(\cdot)\) to a simpler, typically finite-dimensional space. The most popular approach restricts the quantile function to be linear in parameters: \[ q(z, \beta(\tau)) = \psi(z)'\beta(\tau), \tag{22.4}\] where \(\psi(z)\) is some known transformation of covariates (e.g., polynomials, interactions, or basis functions).
22.2.5 Quantile Regression
After specifying the functional form, the parameters \(\beta(\tau)\) are estimated by minimizing the sample objective function for each \(\tau\) of interest. In the linear case, this yields the canonical quantile regression estimator of Koenker and Bassett (1978): \[ \hat{\beta}(\tau)= \argmin_{b} \dfrac{1}{N}\sum_{i=1}^N \rho_{\tau}(Y_i-\psi(Z_i)'b). \tag{22.5}\] This problem can be represented as a linear program and solved efficiently even for large \(N\) (Koenker 2017). The resulting estimator for the conditional quantile is: \[ \hat{Q}_{Y|Z}(\tau|z) = \psi(z)'\hat{\beta}(\tau). \]
22.2.6 Moment Representation
While the check loss is non-smooth, the population minimizer \(q(Z, \beta(\tau)) = \psi(Z)'\beta(\tau)\) satisfies the following system of moment conditions: \[ \E\left[ \left( \tau - \mathbb{I}\{Y_i \leq \psi(Z_i)'\beta(\tau)\} \right) \psi(Z_i) \right] = 0. \tag{22.6}\]
22.2.7 Asymptotic Properties
Under standard regularity conditions, the estimator \(\hat{\beta}(\tau)\) is \(\sqrt{N}\)-consistent and asymptotically normal. Specifically, it holds that \[ \sqrt{N} (\hat{\beta}(\tau) - \beta(\tau)) \Rightarrow \mathcal{N}(0, H^{-1} J H^{-1}), \tag{22.7}\] where \[ \begin{aligned} J &= \tau(1-\tau) \E[\psi(Z_i)\psi(Z_i)'], \\ H &= \E[f_{Y|Z}(\psi(Z_i)'\beta(\tau)|Z_i) \psi(Z_i)\psi(Z_i)']. \end{aligned} \] See chapter 4 of Koenker (2005) for a detailed discussion of the regularity conditions and proofs.
The asymptotic variance depends on the conditional density of \(Y\) at the \(\tau\)th quantile, \(f_{Y|Z}(Q_{Y|Z}(\tau|z)|z)\). This “sparsity” term in \(H\) implies that estimation is typically less precise in the tails of the distribution where data is sparse (\(f_{Y|Z}\) is small), which leads to a “flatter” sample objective function.
One can also establish the joint distribution of \(\hat{\beta}(\tau)\) across different values of \(\tau\) or a continuum of \(\tau\) values, which is useful for inference on the entire conditional quantile function. See Koenker (2005) and Koenker et al. (2017) for details.
22.3 Estimation and Identification Perspectives
As with mean regression, the choice between parametric and nonparametric specifications involves a trade-off between bias and variance, as well as different identification targets under misspecification.
22.3.1 Parametric Specification
In a parametric setting, we typically assume that \(Q_{Y|Z}(\tau|z) = \psi(z)'\beta(\tau)\) for a fixed-dimensional \(\psi(z)\).
- If the linear model holds, \(\hat{\beta}(\tau)\) consistently estimates the true coefficients.
- If the linear model is misspecified, Angrist et al. (2006) show that QR identifies a “best” linear approximation to the true conditional quantile function. Specifically, \(\beta(\tau)\) minimizes a weighted mean squared error between the true quantile function and the linear index, where the weights are determined by the conditional density of the outcome.
22.3.2 Nonparametric Specification
In a nonparametric approach, we allow the complexity of the model to grow with the sample size \(N\) to ensure consistency without functional form assumptions. Common strategies include:
- Series estimation: The number of basis functions in \(\psi(Z_i)\) (e.g., polynomials, B-splines) increases with \(N\) at an appropriate rate.
- Local methods: Approaches such as quantile smoothing splines (Koenker et al. 1994) or local polynomial quantile regression approximate the quantile function locally around each \(z\).
- Quantile random forests (Meinshausen 2006) provide a flexible, data-driven way to partition the covariate space and estimate conditional quantiles within the resulting leaves.
22.4 Operational Challenge: Quantile Crossing
A known issue in both parametric and nonparametric quantile regression is quantile crossing. Since \(\hat{\beta}(\tau)\) is estimated separately for each \(\tau\), the resulting surfaces \(\psi(z)'\hat{\beta}(\tau)\) and \(\psi(z)'\hat{\beta}(\tau')\) may cross for \(\tau \neq \tau'\) in finite samples (or under misspecification).
This violates the monotonicity of the quantile function and renders the resulting “distribution” invalid (e.g., non-monotonic CDFs). For operationalizing QTEs and DTEs, this is often addressed using rearrangement methods (Chernozhukov et al. 2010), which monotonically transform the estimated curves to ensure they satisfy the basic properties of a quantile function.
Next Section
In the next section, we introduce distribution regression as an alternative tool for estimating the distribution of potential outcomes.