From MM*Stat International
The objectives of regression analysis
The main objective of regression analysis is to describe the expectation and dependence of a quantity on quantities . A one-directional dependence is assumed. This dependence can be expressed as a general regression function of the following form: The symbol used indicates that the regression function of observed values is does not correspond to an observed value , but rather with the average value of given the ’s, which lies on the regression function. The random variables are referred as regressors, explanatory variables or independent variables. The random variable is referred as regressand or dependent variable. An example is the simple linear regression with a dependent variable “Time working” and one independent variable “Amount of production.” Notice that this regression is referred to as simple because there is a single independent variable and is a linear regression since the function is assumed to be linear.
If the dependence of Y on X can be represented by a linear function, the regression value does describe the value of . It follows that the value of any observation can be decomposed as follows: The difference between the observed values and the value of the regression function is called a residual . It contains those influences on that cannot be described by means of the regression function; alernatively, this means that deviations of the observed values from the regression function cannot be explained by the independent variables employed in the regression function. Regression function The regression function is a representation of average statistical dependence of a dependent variable on one or more independent variables. The dependence is described by a function based on observations. In what follows, we assume only the case when a variable Y depends on a single variable X. A form of the regression function f(x) always depends on the specific application and the purpose of an analysis. Examples of possible regression functions include: