Lisa Hacker: Created page with "{{template:LDABOOK|7.1|Exponential Parameter Estimation}} ==Estimation of the Exponential Parameters== ===Probability Plotting=== Estimation of the parameters for the exponential distribution via probability plotting is very similar to the process used when dealing with the Weibull distribution. Recall, however, that the appearance of the probability plotting paper and the methods by which the parameters are estimated vary from distribution to distribution, so there will..."

2023-03-09T00:21:03Z

Created page with "{{template:LDABOOK|7.1|Exponential Parameter Estimation}} ==Estimation of the Exponential Parameters== ===Probability Plotting=== Estimation of the parameters for the exponential distribution via probability plotting is very similar to the process used when dealing with the Weibull distribution. Recall, however, that the appearance of the probability plotting paper and the methods by which the parameters are estimated vary from distribution to distribution, so there will..."

New page

{{template:LDABOOK|7.1|Exponential Parameter Estimation}}
==Estimation of the Exponential Parameters==
===Probability Plotting===
Estimation of the parameters for the exponential distribution via probability plotting is very similar to the process used when dealing with the Weibull distribution. Recall, however, that the appearance of the probability plotting paper and the methods by which the parameters are estimated vary from distribution to distribution, so there will be some noticeable differences. In fact, due to the nature of the exponential ''cdf'', the exponential probability plot is the only one with a negative slope. This is because the y-axis of the exponential probability plotting paper represents the reliability, whereas the y-axis for most of the other life distributions represents the unreliability.

This is illustrated in the process of linearizing the ''cdf'', which is necessary to construct the exponential probability plotting paper. For the two-parameter exponential distribution the cumulative density function is given by:

::<math>\begin{align}
F(t)=1-{{e}^{-\lambda (t-\gamma )}}
\end{align}\,\!</math>

Taking the natural logarithm of both sides of the above equation yields:

::<math>\ln \left[ 1-F(t) \right]=-\lambda (t-\gamma )\,\!</math>

or:

::<math>\begin{align}
\ln [1-F(t)]=\lambda \gamma -\lambda t
\end{align}\,\!</math>

Now, let:

::<math>\begin{align}
y=\ln [1-F(t)]
\end{align}\,\!</math>

::<math>\begin{align}
a=\lambda \gamma
\end{align}\,\!</math>

and:

::<math>\begin{align}
b=-\lambda
\end{align}\,\!</math>

which results in the linear equation of:

::<math>\begin{align}
y=a+bt
\end{align}\,\!</math>

Note that with the exponential probability plotting paper, the y-axis scale is logarithmic and the x-axis scale is linear. This means that the zero value is present only on the x-axis. For <math>t=0\,\!</math>, <math>R=1\,\!</math> and <math>F(t)=0\,\!</math>. So if we were to use <math>F(t)\,\!</math> for the y-axis, we would have to plot the point <math>(0,0)\,\!</math>. However, since the y-axis is logarithmic, there is no place to plot this on the exponential paper. Also, the failure rate, <math>\lambda \,\!</math>, is the negative of the slope of the line, but there is an easier way to determine the value of <math>\lambda \,\!</math> from the probability plot, as will be illustrated in the following example.
====Plotting Example====
{{:1P Exponential Example}}

===Rank Regression on Y===
Performing a rank regression on Y requires that a straight line be fitted to the set of available data points such that the sum of the squares of the vertical deviations from the points to the line is minimized.
The least squares parameter estimation method (regression analysis) was discussed in [[Parameter Estimation]], and the following equations for rank regression on Y (RRY) were derived:

::<math>\hat{a}=\bar{y}-\hat{b}\bar{x}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}-\hat{b}\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}}{N}\,\!</math>

and:

::<math>\hat{b}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,x_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}} \right)}^{2}}}{N}}\,\!</math>

In our case, the equations for <math>{{y}_{i}}\,\!</math> and <math>{{x}_{i}}\,\!</math> are:

::<math>\begin{align}
{{y}_{i}}=\ln [1-F({{t}_{i}})]
\end{align}\,\!</math>

and:

::<math>\begin{align}
{{x}_{i}}={{t}_{i}}
\end{align}\,\!</math>

and the <math>F({{t}_{i}})\,\!</math> is estimated from the median ranks. Once <math>\hat{a}\,\!</math> and <math>\hat{b}\,\!</math> are obtained, then <math>\hat{\lambda }\,\!</math> and <math>\hat{\gamma }\,\!</math> can easily be obtained from above equations.
For the one-parameter exponential, equations for estimating ''a'' and ''b'' become:

::<math>\begin{align}
\hat{a}= & 0, \\
\hat{b}= & \frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,x_{i}^{2}}
\end{align}\,\!</math>

'''The Correlation Coefficient'''

The estimator of <math>\rho \,\!</math> is the sample correlation coefficient, <math>\hat{\rho }\,\!</math>, given by:

::<math>\hat{\rho }=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,({{x}_{i}}-\overline{x})({{y}_{i}}-\overline{y})}{\sqrt{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{({{x}_{i}}-\overline{x})}^{2}}\cdot \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{({{y}_{i}}-\overline{y})}^{2}}}}\,\!</math>

====RRY Example==== 
{{:2P_Exponential_Example}}

===Rank Regression on X===
Similar to rank regression on Y, performing a rank regression on X requires that a straight line be fitted to a set of data points such that the sum of the squares of the horizontal deviations from the points to the line is minimized.

Again the first task is to bring our exponential ''cdf'' function into a linear form. This step is exactly the same as in regression on Y analysis. The deviation from the previous analysis begins on the least squares fit step, since in this case we treat <math>x\,\!</math> as the dependent variable and <math>y\,\!</math> as the independent variable. The best-fitting straight line to the data, for regression on X (see [[Parameter Estimation]]), is the straight line:

::<math>x=\hat{a}+\hat{b}y\,\!</math>

The corresponding equations for <math>\hat{a}\,\!</math> and <math>\hat{b}\,\!</math> are:

::<math>\hat{a}=\overline{x}-\hat{b}\overline{y}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}}{N}-\hat{b}\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}\,\!</math>

and:

::<math>\hat{b}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,y_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}} \right)}^{2}}}{N}}\,\!</math>

where:

::<math>\begin{align}
{{y}_{i}}=\ln [1-F({{t}_{i}})]
\end{align}\,\!</math>

and:

::<math>\begin{align}
{{x}_{i}}={{t}_{i}}
\end{align}\,\!</math>

The values of <math>F({{t}_{i}})\,\!</math> are estimated from the median ranks. Once <math>\hat{a}\,\!</math> and <math>\hat{b}\,\!</math> are obtained, solve for the unknown <math>y\,\!</math> value, which corresponds to:

::<math>y=-\frac{\hat{a}}{\hat{b}}+\frac{1}{\hat{b}}x\,\!</math>

Solving for the parameters from above equations we get:

::<math>a=-\frac{\hat{a}}{\hat{b}}=\lambda \gamma \Rightarrow \gamma =\hat{a}\,\!</math>

and:

::<math>b=\frac{1}{\hat{b}}=-\lambda \Rightarrow \lambda =-\frac{1}{\hat{b}}\,\!</math>

For the one-parameter exponential case, equations for estimating a and b become:

::<math>\begin{align}
\hat{a}= & 0 \\
\hat{b}= & \frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,y_{i}^{2}}
\end{align}\,\!</math>

The correlation coefficient is evaluated as before.

====RRX Example====
'''2-Parameter Exponential RRX Example'''

Using the same data set from the [[The_Exponential_Distribution#RRY_Example|RRY example above]] and assuming a 2-parameter exponential distribution, estimate the parameters and determine the correlation coefficient estimate, <math>\hat{\rho }\,\!</math>, using rank regression on X.

''' Solution'''

The table constructed for the RRY analysis applies to this example also. Using the values from this table, we get:

::<math>\begin{align}
\hat{b}= & \frac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{t}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{t}_{i}}\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{y}_{i}}}{14}}{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,y_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{y}_{i}} \right)}^{2}}}{14}} \\
\\
\hat{b}= & \frac{-927.4899-(630)(-13.2315)/14}{22.1148-{{(-13.2315)}^{2}}/14}
\end{align}\,\!</math>

or:

::<math>\hat{b}=-34.5563\,\!</math>

and:

::<math>\hat{a}=\overline{x}-\hat{b}\overline{y}=\frac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{t}_{i}}}{14}-\hat{b}\frac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{y}_{i}}}{14}\,\!</math>

or:

::<math>\hat{a}=\frac{630}{14}-(-34.5563)\frac{(-13.2315)}{14}=12.3406\,\!</math>

Therefore:

::<math>\hat{\lambda }=-\frac{1}{\hat{b}}=-\frac{1}{(-34.5563)}=0.0289\text{ failures/hour}\,\!</math>

and:

::<math>\hat{\gamma }=\hat{a}=12.3406\,\!</math>

The correlation coefficient is found to be:

::<math>\hat{\rho }=-0.9679\,\!</math>

Note that the equation for regression on Y is not necessarily the same as that for the regression on X. The only time when the two regression methods yield identical results is when the data lie perfectly on a line. If this were the case, the correlation coefficient would be <math>-1\,\!</math>. The negative value of the correlation coefficient is due to the fact that the slope of the exponential probability plot is negative.

This example can be repeated using Weibull++, choosing two-parameter exponential and rank regression on X (RRX) methods for analysis, as shown below.
The estimated parameters and the correlation coefficient using Weibull++ were found to be:

::<math>\begin{array}{*{35}{l}}
\hat{\lambda }= &0.0289 \text{failures/hour} \\
\hat{\gamma}= & 12.3395 \text{hours} \\
\hat{\rho} = &-0.9679 \\
\end{array}\,\!</math>

[[Image:Exponential Distribution Example 3 Data Folio.png|center|700px|]]

The probability plot can be obtained simply by clicking the '''Plot''' icon.

[[Image:Exponential Distribution Example 3 Plot.png|center|600px|]]

===Maximum Likelihood Estimation===
As outlined in [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood equation with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the exponential distribution are covered in [[Appendix:_Log-Likelihood_Equations|Appendix D]].

====MLE Example====
'''MLE for the Exponential Distribution'''

Using the same data set from the [[The_Exponential_Distribution#RRY_Example|RRY and RRX examples above]] and assuming a 2-parameter exponential distribution, estimate the parameters using the MLE method.

'''Solution'''

In this example, we have complete data only. The partial derivative of the log-likelihood function, <math>\Lambda ,\,\!</math> is given by:

::<math>\frac{\partial \Lambda }{\partial \lambda }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right]=\underset{i=1}{\overset{14}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right]=0\,\!</math>

Complete descriptions of the partial derivatives can be found in [[Appendix:_Log-Likelihood_Equations|Appendix D]]. Recall that when using the MLE method for the exponential distribution, the value of <math>\gamma \,\!</math> is equal to that of the first failure time. The first failure occurred at 5 hours, thus <math>\gamma =5\,\!</math> hours<math>.\,\!</math> Substituting the values for <math>T\,\!</math> and <math>\gamma \,\!</math> we get:

::<math>\frac{14}{\hat{\lambda }}=560\,\!</math>

or:

::<math>\hat{\lambda }=0.025\text{ failures/hour}\,\!</math>

Using Weibull++:

[[Image:Exponential Distribution Example 4 Data.png|center|700px|]]

The probability plot is:

[[Image:Exponential Distribution Example 4 Plot.png|center|650px|]]

Exponential Parameter Estimation - Revision history