Functional regression

Functional regression is a version of regression analysis when responses or covariates include functional data. Functional regression models can be classified into four types depending on whether the responses or covariates are functional or scalar: (i) scalar responses with functional covariates, (ii) functional responses with scalar covariates, (iii) functional responses with functional covariates, and (iv) scalar or functional responses with functional and scalar covariates. In addition, functional regression models can be linear, partially linear, or nonlinear. In particular, functional polynomial models, functional single and multiple index models and functional additive models are three special cases of functional nonlinear models. __TOC__

Functional regression is a version of regression analysis when responses or covariates include functional data. Functional regression models can be classified into four types depending on whether the responses or covariates are functional or scalar: (i) scalar responses with functional covariates, (ii) functional responses with scalar covariates, (iii) functional responses with functional covariates, and (iv) scalar or functional responses with functional and scalar covariates. In addition, functional regression models can be linear, partially linear, or nonlinear. In particular, functional polynomial models, functional single and multiple index models and functional additive models are three special cases of functional nonlinear models. __TOC__ Functional linear models with scalar responses can be obtained by replacing the scalar covariates X and the coefficient vector \beta in model ( ) by a centered functional covariate X^c(\cdot) = X(\cdot) - \mathbb{E}(X(\cdot)) and a coefficient function \beta = \beta(\cdot) with domain \mathcal{T} , respectively, and replacing the inner product in Euclidean space by that in Hilbert space L^2 , where \langle \cdot, \cdot \rangle here denotes the inner product in L^2 . One approach to estimating \beta_0 and \beta(\cdot) is to expand the centered covariate X^c(\cdot) and the coefficient function \beta(\cdot) in the same functional basis, for example, B-spline basis or the eigenbasis used in the Karhunen–Loève expansion. Suppose \{\phi_k\}_{k=1}^\infty is an orthonormal basis of L^2 . Expanding X^c and \beta in this basis, X^c(\cdot) = \sum_{k=1}^\infty x_k \phi_k(\cdot) , \beta(\cdot) = \sum_{k=1}^\infty \beta_k \phi_k(\cdot) , model ( ) becomes Y = \beta_0 + \sum_{k=1}^\infty \beta_k x_k +\varepsilon. For implementation, regularization is needed and can be done through truncation, L^2 penalization or L^1 penalization. In addition, a reproducing kernel Hilbert space (RKHS) approach can also be used to estimate \beta_0 and \beta(\cdot) in model ( ) Adding multiple functional and scalar covariates, model ( ) can be extended to {{NumBlk|::| Y = \sum_{k=1}^q Z_k\alpha_k + \sum_{j=1}^p \int_{\mathcal{T}_j} X_j^c(t) \beta_j(t) \,dt + \varepsilon, | }} where Z_1,\ldots,Z_q are scalar covariates with Z_1=1 , \alpha_1,\ldots,\alpha_q are regression coefficients for Z_1,\ldots,Z_q , respectively, X^c_j is a centered functional covariate given by X_j^c(\cdot) = X_j(\cdot) - \mathbb{E}(X_j(\cdot)) , \beta_j is regression coefficient function for X_j^c(\cdot) , and \mathcal{T}_j is the domain of X_j and \beta_j , for j=1,\ldots,p . However, due to the parametric component \alpha , the estimation methods for model ( ) cannot be used in this case and alternative estimation methods for model ( ) are available. {{NumBlk|::| Y(t) = \beta_0(t) + \int_{\mathcal{S}} \beta(s,t) X^c(s)\,ds + \varepsilon(t),\ \text{for}\ t\in\mathcal{T}, | }} where X^c(\cdot) = X(\cdot) - \mathbb{E}(X(\cdot)) is still the centered functional covariate, \beta_0(\cdot) and \beta(\cdot,\cdot) are coefficient functions, and \varepsilon(\cdot) is usually assumed to be a random process with mean zero and finite variance. In this case, at any given time t\in\mathcal{T} , the value of Y , i.e., Y(t) , depends on the entire trajectory of X . Model ( ), for any given time t , is an extension of multivariate linear regression with the inner product in Euclidean space replaced by that in L^2 . An estimating equation motivated by multivariate linear regression is r_{XY} = R_{XX}\beta, \text{ for } \beta\in L^2(\mathcal{S}\times\mathcal{S}), where r_{XY}(s,t) = \text{cov}(X(s),Y(t)) , R_{XX}: L^2(\mathcal{S}\times\mathcal{S}) \rightarrow L^2(\mathcal{S}\times\mathcal{T}) is defined as (R_{XX}\beta)(s,t) = \int_\mathcal{S} r_{XX}(s,w)\beta(w,t)dw with r_{XX}(s,w) = \text{cov}(X(s),X(w)) for s,w\in\mathcal{S} . Regularization is needed and can be done through truncation, L^2 penalization or L^1 penalization. Various estimation methods for model ( ) are available. When X and Y are concurrently observed, i.e., \mathcal{S}=\mathcal{T} , it is reasonable to consider a historical functional linear model, where the current value of Y only depends on the history of X , i.e., \beta(s,t)=0 for s>t in model ( ). A simpler version of the historical functional linear model is the functional concurrent model (see below). Adding multiple functional covariates, model ( ) can be extended to {{NumBlk|::| Y(t) = \beta_0(t) + \sum_{j=1}^p\int_{\mathcal{S}_j} \beta_j(s,t) X^c_j(s)\,ds + \varepsilon(t),\ \text{for}\ t\in\mathcal{T}, | }} where for j=1,\ldots,p , X_j^c(\cdot)=X_j(\cdot) - \mathbb{E}(X_j(\cdot)) is a centered functional covariate with domain \mathcal{S}_j , and \beta_j(\cdot,\cdot) is the corresponding coefficient function with the same domain, respectively. In particular, taking X_j(\cdot) as a constant function yields a special case of model ( ) Y(t) = \sum_{j=1}^p X_j \beta_j(t) + \varepsilon(t),\ \text{for}\ t\in\mathcal{T}, which is a FLM with functional responses and scalar covariates. Assuming that \mathcal{S} = \mathcal{T} , another model, known as the functional concurrent model, sometimes also referred to as the varying-coefficient model, is of the form where \alpha_0 and \alpha are coefficient functions. Note that model ( ) assumes the value of Y at time t , i.e., Y(t) , only depends on that of X at the same time, i.e., X(t) . Various estimation methods can be applied to model ( ). Adding multiple functional covariates, model ( ) can also be extended to Y(t) = \alpha_0(t) + \sum_{j=1}^p\alpha_j(t)X_j(t)+\varepsilon(t),\ \text{for}\ t\in\mathcal{T}, where X_1,\ldots,X_p are multiple functional covariates with domain \mathcal{T} and \alpha_0,\alpha_1,\ldots,\alpha_p are the coefficient functions with the same domain. Y = \alpha + \int_\mathcal{T}\beta(t)X^c(t)\,dt + \int_\mathcal{T} \int_\mathcal{T} \gamma(s,t) X^c(s)X^c(t) \,ds\,dt + \varepsilon, where X^c(\cdot) = X(\cdot) - \mathbb{E}(X(\cdot)) is the centered functional covariate, \alpha is a scalar coefficient, \beta(\cdot) and \gamma(\cdot,\cdot) are coefficient functions with domains \mathcal{T} and \mathcal{T}\times\mathcal{T} , respectively, and \varepsilon is a random error with mean zero and finite variance. By analogy to FLMs with scalar responses, estimation of functional polynomial models can be obtained through expanding both the centered covariate X^c and the coefficient functions \beta and \gamma in an orthonormal basis. Y = g\left(\int_{\mathcal{T}} X^c(t) \beta_1(t)\,dt, \ldots, \int_{\mathcal{T}} X^c(t) \beta_p(t)\,dt \right) + \varepsilon. Taking p=1 yields a functional single index model. However, for p>1 , this model is problematic due to curse of dimensionality. With p>1 and relatively small sample sizes, the estimator given by this model often has large variance. An alternative p -component functional multiple index model can be expressed as Y = g_1\left(\int_{\mathcal{T}} X^c(t) \beta_1(t)\,dt\right)+ \cdots+ g_p\left(\int_{\mathcal{T}} X^c(t) \beta_p(t)\,dt \right) + \varepsilon. Estimation methods for functional single and multiple index models are available. Given an expansion of a functional covariate X with domain \mathcal{T} in an orthonormal basis \{\phi_k\}_{k=1}^\infty : X(t) = \sum_{k=1}^\infty x_k \phi_k(t) , a functional linear model with scalar responses shown in model ( ) can be written as \mathbb{E}(Y|X)=\mathbb{E}(Y) + \sum_{k=1}^\infty \beta_k x_k. One form of FAMs is obtained by replacing the linear function of x_k , i.e., \beta_k x_k , by a general smooth function f_k , \mathbb{E}(Y|X)=\mathbb{E}(Y) + \sum_{k=1}^\infty f_k(x_k), where f_k satisfies \mathbb{E}(f_k(x_k))=0 for k\in\mathbb{N} . Another form of FAMs consists of a sequence of time-additive models: \mathbb{E}(Y|X(t_1),\ldots,X(t_p))=\sum_{j=1}^p f_j(X(t_j)), where \{t_1,\ldots,t_p\} is a dense grid on \mathcal{T} with increasing size p\in\mathbb{N} , and f_j(x) = g(t_j,x) with g a smooth function, for j=1,\ldots,p Linear predictor \eta = \beta_0 + \int_{\mathcal{T}} X^c(t)\beta(t)\,dt ; Variance function \text{Var}(Y|X) = V(\mu) , where \mu = \mathbb{E}(Y|X) is the conditional mean; Link function g connecting the conditional mean and the linear predictor through \mu=g(\eta) .

Text derived from https://en.wikipedia.org/w/index.php?oldid=1314009887 under CC BY-SA 4.0; modifications have been made.