Template:Least Squares/Rank Regression Equations: Difference between revisions

From ReliaWiki
Jump to navigation Jump to search
Line 38: Line 38:
Assume that a set of data pairs (x1, y1), (x2, y2), ... , (xN, yN) were obtained and plotted. Then, according to the least squares principle, which minimizes the horizontal distance between the data points and the straight line fitted to the data, the best fitting straight line to these data is the straight line x =  + y such that:
Assume that a set of data pairs (x1, y1), (x2, y2), ... , (xN, yN) were obtained and plotted. Then, according to the least squares principle, which minimizes the horizontal distance between the data points and the straight line fitted to the data, the best fitting straight line to these data is the straight line x =  + y such that:


::<math>\displaystyle\sum_{i=1}^N(\hat{a}+\hat{b}y_i-x_i)^2=min(a,b)\displaystyle\sum_{i=1}^N (a+by_i-x_i)^2 </math>


Again, <math>\hat{a}</math> and <math>\hat{b}</math> are the least squares estimates of a and b, and N is the number of data points.


Again,  and  are the least squares estimates of a and b, and N is the number of data points.
To obtain <math>\hat{a}</math> and <math>\hat{b}</math>, let:
 
To obtain  and , let:
 


::<math>F=\displaystyle\sum_{i=1}^N(a+by_i-x_i)^2</math>


Differentiating F with respect to a and b yields:
Differentiating F with respect to a and b yields:


(5)
::<math>\frac{\partial F}{\partial a}=2\displaystyle\sum_{i=1}^N(a+by_i-x_i</math> (5)


and:
and:


(6)
::<math>\frac{\partial F}{\partial b}=2\displaystyle\sum_{i=1}^N(a+by_i-x_i)y_i</math>(6)


Setting Eqns. (5) and (6) equal to zero yields:
Setting Eqns. (5) and (6) equal to zero yields:


 
::<math>\displaystyle\sum_{i=1}^N(a+by_i-x_i)=\displaystyle\sum_{i=1}^N(\widehat{x}_i-x_i)=-\displaystyle\sum_{i=1}^N(x_i-\widehat{x}_i)=0</math>


and:
and:


 
::<math>\displaystyle\sum_{i=1}^N(a+by_i-x_i)y_i=\displaystyle\sum_{i=1}^N(\widehat{x}_i-x_i)y_i=-\displaystyle\sum_{i=1}^N(x_i-\widehat{x}_i)y_i=0</math>


Solving the above equations simultaneously yields:
Solving the above equations simultaneously yields:


(7)
::<math>\widehat{a}=\frac{\displaystyle\sum_{i=1}^N x_i}{N}-\widehat{b}\frac{\displaystyle\sum_{i=1}^N y_i}{N}=\bar{x}-\widehat{b}\bar{y}</math>(7)


and:
and:

Revision as of 17:59, 25 July 2011

Least Squares/Rank Regression Equations

Rank Regression on Y

Assume that a set of data pairs (x1, y1), (x2, y2), ... , (xN, yN), were obtained and plotted. Then, according to the least squares principle, which minimizes the vertical distance between the data points and the straight line fitted to the data, the best fitting straight line to these data is the straight line y = + x such that:

[math]\displaystyle{ \sum_{i=1}^N (\hat{a}+\hat{b} x_i - y_i)^2=min(a,b)\sum_{i=1}^N (a+b x_i-y_i)^2 }[/math]

and where and are the least squares estimates of a and b, and N is the number of data points.

To obtain [math]\displaystyle{ \hat{a} }[/math] and [math]\displaystyle{ \hat{b} }[/math], let:

[math]\displaystyle{ F=\sum_{i=1}^N (a+bx_i-y_i)^2 }[/math]

Differentiating F with respect to a and b yields:

[math]\displaystyle{ \frac{\partial F}{\partial a}=2\sum_{i=1}^N (a+b x_i-y_i) }[/math] (1)

and:

[math]\displaystyle{ \frac{\partial F}{\partial b}=2\sum_{i=1}^N (a+b x_i-y_i)x_i }[/math] (2)

Setting Eqns. (1) and (2) equal to zero yields:

[math]\displaystyle{ \sum_{i=1}^N (a+b x_i-y_i)x_i=\sum_{i=1}^N(\hat{y}_i-y_i)=-\sum_{i=1}^N(y_i-\hat{y}_i)=0 }[/math]

and:

[math]\displaystyle{ \sum_{i=1}^N (a+b x_i-y_i)x_i=\sum_{i=1}^N(\hat{y}_i-y_i)x_i=-\sum_{i=1}^N(y_i-\hat{y}_i)x_i =0 }[/math]

Solving the equations simultaneously yields:

[math]\displaystyle{ \hat{a}=\frac{\displaystyle \sum_{i=1}^N y_i}{N}-\hat{b}\frac{\displaystyle \sum_{i=1}{N} x_i}{N}=\bar{y}-\hat{b}\bar{x} }[/math] (3)

and:

[math]\displaystyle{ \hat{b}=\frac{\displaystyle \sum{i=1}^N x_i y_i-\frac{\displaystyle \sum_{i=1}^N x_i \sum_{i=1}^N y_i}{N}}{\displaystyle \sum_{i=1}^N x_i^2-\frac{\left(\displaystyle\sum_{i=1}^N x_i\right)^2}{N}} }[/math](4)

Rank Regression on X

Assume that a set of data pairs (x1, y1), (x2, y2), ... , (xN, yN) were obtained and plotted. Then, according to the least squares principle, which minimizes the horizontal distance between the data points and the straight line fitted to the data, the best fitting straight line to these data is the straight line x = + y such that:

[math]\displaystyle{ \displaystyle\sum_{i=1}^N(\hat{a}+\hat{b}y_i-x_i)^2=min(a,b)\displaystyle\sum_{i=1}^N (a+by_i-x_i)^2 }[/math]

Again, [math]\displaystyle{ \hat{a} }[/math] and [math]\displaystyle{ \hat{b} }[/math] are the least squares estimates of a and b, and N is the number of data points.

To obtain [math]\displaystyle{ \hat{a} }[/math] and [math]\displaystyle{ \hat{b} }[/math], let:

[math]\displaystyle{ F=\displaystyle\sum_{i=1}^N(a+by_i-x_i)^2 }[/math]

Differentiating F with respect to a and b yields:

[math]\displaystyle{ \frac{\partial F}{\partial a}=2\displaystyle\sum_{i=1}^N(a+by_i-x_i }[/math] (5)

and:

[math]\displaystyle{ \frac{\partial F}{\partial b}=2\displaystyle\sum_{i=1}^N(a+by_i-x_i)y_i }[/math](6)

Setting Eqns. (5) and (6) equal to zero yields:

[math]\displaystyle{ \displaystyle\sum_{i=1}^N(a+by_i-x_i)=\displaystyle\sum_{i=1}^N(\widehat{x}_i-x_i)=-\displaystyle\sum_{i=1}^N(x_i-\widehat{x}_i)=0 }[/math]

and:

[math]\displaystyle{ \displaystyle\sum_{i=1}^N(a+by_i-x_i)y_i=\displaystyle\sum_{i=1}^N(\widehat{x}_i-x_i)y_i=-\displaystyle\sum_{i=1}^N(x_i-\widehat{x}_i)y_i=0 }[/math]

Solving the above equations simultaneously yields:

[math]\displaystyle{ \widehat{a}=\frac{\displaystyle\sum_{i=1}^N x_i}{N}-\widehat{b}\frac{\displaystyle\sum_{i=1}^N y_i}{N}=\bar{x}-\widehat{b}\bar{y} }[/math](7)

and:

(8)

Solving the equation of the line for y yields:


Illustrating with an Example

Fit a least squares straight line using regression on X and regression on Y to the following data:

x

1
2.5
4
6
8
9
11
15

y

1.5
2
4
4
5
7
8
10

The first step is to generate the following table:

Table A.1 - Data analysis for the least squares method


Using the results in Table A.1, Eqns. (3) and (4) yield:


and:


The least squares line is given by:


The plotted line is shown in the next figure.


For rank regression on X using the analyzed data in Table A.1, Eqns. (8) and (7) yield:


and:


The least squares line is given by:


The plotted line is shown in the next figure.


Note that the regression on Y is not necessarily the same as the regression on X. The only time when the two regressions are the same (i.e. will yield the same equation for a line) is when the data lie perfectly on a line.

The correlation coefficient is given by: