Univariate regression, polynomial regression, orthogonal polynomials, nonlinear
OverviewUnivariate regression is an area of curve-fitting which, given a function depending on some parameters, finds the parameters such that provides the best fit to a series of two-dimensional data points, in a certain sense. It is called univariate as the data points are supposed to be sampled from a one-variable function. Compare this to multivariate regression, which aims at fitting data points sampled from a function of several variables. Formally speaking, consider a series of data points and, for the sake of simplicity, consider that , i.e. the points are distinct and are in increasing order with respect to . By doing least squares fitting on these data points we mean finding the parameters of a function such that the sum of squared residuals
Polynomial RegressionOne linear regression method is that of polynomial regression, which refers to finding the polynomial function that provides the least squares fitting to a series of data points. More precisely, if the polynomial function of degree is given by Regression/Linear. The optimal parameters for the case can be found in a similar manner. Instead of going into further technical details, let us look at the following graph which shows the quadratic regression polynomial in red for the series of data points shown in blue, sampled from a sine-like function. Notice that the resulting fitting curve looks more like a parabola and this is due to the fact that we may have chosen the wrong family of functions to provide the fit (polynomial ones), or perhaps the degree of the regression polynomial is too small. Parabolic regression is available as Regression/Parabolic. For the general case of regression polynomials of arbitrary degree, the component Regression/Discrete is available.
Orthogonal Polynomial RegressionThis is an extension of polynomial regression, in the sense that instead of using as the factors in the fitting function , some special kind of functions are used and thus the expression for becomes: Regression/Orthogonal. Below is a graph of a series of data points, shown in blue, and their corresponding fitting function, determined by the method of orthogonal polynomial regression. We may as well use certain types of orthogonal polynomials, instead of general ones as in the above method. For instance, we can use the Forsythe orthogonal polynomials, leading to the method of Forsythe polynomial regression, available as Regression/Forsythe. In the image below, we have used Forsythe regression on the same series of data points as in the previous graph.
Nonlinear RegressionAs mentioned at the beginning of this reference page, nonlinear regression refers to the cases when the fitting function depends nonlinearly on its parameters. Therefore can virtually be any function with this property. Depending on the experiment from which the data points were obtained and the statistical properties of the phenomenon which took place, we may choose between various families of nonlinear regression functions. To name a few nonlinear regression models, logistic regression is given by the fitting function Regression/Logistic. In the following image you can see the logistic regression curve in red, for the series of data points shown in blue. Other nonlinear regression models include the Gompertz model, given by the fitting function
- Franklin A. Graybill, Hariharan K. Iyer, Regression Analysis. Concepts and Applications, Duxbury Press, Belmont, California.