Regression Analysis

The main objective of regression analysis is to describe the expectation and dependence of a quantity $Y$ on quantities $X_{1},X_{2},\ldots$ . A one-directional dependence is assumed. This dependence can be expressed as a general regression function of the following form: $E(y|x)=f(x_{1},x_{2},\ldots ).$ The symbol $E(y|x)$ used indicates that the regression function of observed values $x_{1},x_{2},\cdots$ is does not correspond to an observed value $y$ , but rather with the average value of $y$ given the $x_{i}$ ’s, which lies on the regression function. The random variables $X_{1},X_{2},\ldots$ are referred as regressors, explanatory variables or independent variables. The random variable $Y$ is referred as regressand or dependent variable. An example is the simple linear regression with a dependent variable “Time working” and one independent variable “Amount of production.” Notice that this regression is referred to as simple because there is a single independent variable and is a linear regression since the function $f(Amount\,\,of\,\,production)$ is assumed to be linear.
If the dependence of Y on X can be represented by a linear function, the regression value $E(y_{i}|x_{i})$ does describe the value of $Y$ . It follows that the value of any observation $i$ can be decomposed as follows: $y_{i}=E(y_{i}|x_{i})+u_{i}\quad i=1,\ldots ,n$ The difference between the observed values $y_{i}$ and the value of the regression function $E(y_{i}|x_{i})$ is called a residual $u_{i}$ . It contains those influences on $y_{i}$ that cannot be described by means of the regression function; alernatively, this means that deviations of the observed values from the regression function cannot be explained by the independent variables employed in the regression function. $u_{i}=y_{i}-E(y_{i}|x_{i})\quad i=1,\cdots ,n$ Regression function The regression function is a representation of average statistical dependence of a dependent variable on one or more independent variables. The dependence is described by a function based on $n$ observations. In what follows, we assume only the case when a variable Y depends on a single variable X. A form of the regression function f(x) always depends on the specific application and the purpose of an analysis. Examples of possible regression functions include:
 Linear function $E(y|x)$ $=b_{0}+b_{1}x$ Quadratic function $E(y|x)$ $=b_{0}+b_{1}x+b_{2}x^{2}$ Power function $E(y|x)$ $=ax^{b}$ Exponential function $E(y|x)$ $=b_{0}{b_{1}}^{x}$ Logarithmic function $E(y|x)$ $=kl(1+e^{a+bx})$ 