# Regression Analysis

The main objective of regression analysis is to describe the expectation and dependence of a quantity ${\displaystyle Y}$ on quantities ${\displaystyle X_{1},X_{2},\ldots }$. A one-directional dependence is assumed. This dependence can be expressed as a general regression function of the following form: ${\displaystyle E(y|x)=f(x_{1},x_{2},\ldots ).}$ The symbol ${\displaystyle E(y|x)}$ used indicates that the regression function of observed values ${\displaystyle x_{1},x_{2},\cdots }$ is does not correspond to an observed value ${\displaystyle y}$ , but rather with the average value of ${\displaystyle y}$ given the ${\displaystyle x_{i}}$’s, which lies on the regression function. The random variables ${\displaystyle X_{1},X_{2},\ldots }$ are referred as regressors, explanatory variables or independent variables. The random variable ${\displaystyle Y}$ is referred as regressand or dependent variable. An example is the simple linear regression with a dependent variable “Time working” and one independent variable “Amount of production.” Notice that this regression is referred to as simple because there is a single independent variable and is a linear regression since the function ${\displaystyle f(Amount\,\,of\,\,production)}$ is assumed to be linear.
If the dependence of Y on X can be represented by a linear function, the regression value ${\displaystyle E(y_{i}|x_{i})}$ does describe the value of ${\displaystyle Y}$. It follows that the value of any observation ${\displaystyle i}$ can be decomposed as follows: ${\displaystyle y_{i}=E(y_{i}|x_{i})+u_{i}\quad i=1,\ldots ,n}$ The difference between the observed values ${\displaystyle y_{i}}$ and the value of the regression function ${\displaystyle E(y_{i}|x_{i})}$ is called a residual ${\displaystyle u_{i}}$. It contains those influences on ${\displaystyle y_{i}}$ that cannot be described by means of the regression function; alernatively, this means that deviations of the observed values from the regression function cannot be explained by the independent variables employed in the regression function. ${\displaystyle u_{i}=y_{i}-E(y_{i}|x_{i})\quad i=1,\cdots ,n}$ Regression function The regression function is a representation of average statistical dependence of a dependent variable on one or more independent variables. The dependence is described by a function based on ${\displaystyle n}$ observations. In what follows, we assume only the case when a variable Y depends on a single variable X. A form of the regression function f(x) always depends on the specific application and the purpose of an analysis. Examples of possible regression functions include:
 Linear function ${\displaystyle E(y|x)}$ ${\displaystyle =b_{0}+b_{1}x}$ Quadratic function ${\displaystyle E(y|x)}$ ${\displaystyle =b_{0}+b_{1}x+b_{2}x^{2}}$ Power function ${\displaystyle E(y|x)}$ ${\displaystyle =ax^{b}}$ Exponential function ${\displaystyle E(y|x)}$ ${\displaystyle =b_{0}{b_{1}}^{x}}$ Logarithmic function ${\displaystyle E(y|x)}$ ${\displaystyle =kl(1+e^{a+bx})}$