ReliaWiki - User contributions [en]

Introduction to Confidence Bounds

2023-03-22T22:51:56Z

M Spivey: /* Approximate Estimates of the Mean and Variance of a Function */

<noinclude>{{Navigation box}}[[Category: Shared Articles]]
''This article also appears in the [[Confidence Bounds|Life Data Analysis Reference]] and [[Appendix D: Confidence Bounds|Accelerated Life Testing Data Analysis Reference]] books.'' </noinclude>

== What Are Confidence Bounds?==

One of the most confusing concepts to a novice reliability engineer is estimating the precision of an estimate. This is an important concept in the field of reliability engineering, leading to the use of confidence intervals (or bounds). In this section, we will try to briefly present the concept in relatively simple terms but based on solid common sense.

=== The Black and White Marbles ===
To illustrate, consider the case where there are millions of perfectly mixed black and white marbles in a rather large swimming pool and our job is to estimate the percentage of black marbles. The only way to be absolutely certain about the exact percentage of marbles in the pool is to accurately count every last marble and calculate the percentage. However, this is too time- and resource-intensive to be a viable option, so we need to come up with a way of estimating the percentage of black marbles in the pool. In order to do this, we would take a relatively small sample of marbles from the pool and then count how many black marbles are in the sample.

'''Taking a Small Sample of Marbles'''

First, pick out a small sample of marbles and count the black ones. Say you picked out ten marbles and counted four black marbles. Based on this, your estimate would be that 40% of the marbles are black.

[[Image:estimation.png|center|200px]]

If you put the ten marbles back in the pool and repeat this example again, you might get six black marbles, changing your estimate to 60% black marbles. Which of the two is correct? Both estimates are correct! As you repeat this experiment over and over again, you might find out that this estimate is usually between <math>{{X}_{1}}%\,\!</math> and <math>{{X}_{2}}%\,\!</math>, and you can assign a percentage to the number of times your estimate falls between these limits. For example, you notice that 90% of the time this estimate is between <math>{{X}_{1}}%\,\!</math> and <math>{{X}_{2}}%.\,\!</math>

''' Taking a Larger Sample of Marbles '''

If you now repeat the experiment and pick out 1,000 marbles, you might get results for the number of black marbles such as 545, 570, 530, etc., for each trial. The range of the estimates in this case will be much narrower than before. For example, you observe that 90% of the time, the number of black marbles will now be from <math>{{Y}_{1}}%\,\!</math> to <math>{{Y}_{2}}%\,\!</math>, where <math>{{X}_{1}}%<{{Y}_{1}}%\,\!</math> and <math>{{X}_{2}}%>{{Y}_{2}}%\,\!</math>, thus giving you a more narrow estimate interval. The same principle is true for confidence intervals; the larger the sample size, the more narrow the confidence intervals.

'''Back to Reliability'''

We will now look at how this phenomenon relates to reliability. Overall, the reliability engineer's task is to determine the probability of failure, or reliability, of the population of units in question. However, one will never know the exact reliability value of the population unless one is able to obtain and analyze the failure data for every single unit in the population. Since this usually is not a realistic situation, the task then is to estimate the reliability based on a sample, much like estimating the number of black marbles in the pool. If we perform ten different reliability tests for our units, and analyze the results, we will obtain slightly different parameters for the distribution each time, and thus slightly different reliability results. However, by employing confidence bounds, we obtain a range within which these reliability values are likely to occur a certain percentage of the time. This helps us gauge the utility of the data and the accuracy of the resulting estimates. Plus, it is always useful to remember that each parameter is an estimate of the true parameter, one that is unknown to us. This range of plausible values is called a confidence interval.

=== One-Sided and Two-Sided Confidence Bounds ===
Confidence bounds are generally described as being one-sided or two-sided.

'''Two-Sided Bounds'''

[[Image:two sided bounds.png|center|350px]]

When we use two-sided confidence bounds (or intervals), we are looking at a closed interval where a certain percentage of the population is likely to lie. That is, we determine the values, or bounds, between which lies a specified percentage of the population. For example, when dealing with 90% two-sided confidence bounds of <math>(X,Y)\,\!</math>, we are saying that 90% of the population lies between <math>X\,\!</math> and <math>Y\,\!</math> with 5% less than <math>X\,\!</math> and 5% greater than <math>Y\,\!</math>.

'''One-Sided Bounds'''

One-sided confidence bounds are essentially an open-ended version of two-sided bounds. A one-sided bound defines the point where a certain percentage of the population is either higher or lower than the defined point. This means that there are two types of one-sided bounds: upper and lower. An upper one-sided bound defines a point that a certain percentage of the population is less than. Conversely, a lower one-sided bound defines a point that a specified percentage of the population is greater than.

[[Image:one sided bounds.png|center|350px]]

For example, if <math>X\,\!</math> is a 95% upper one-sided bound, this would imply that 95% of the population is less than <math>X\,\!</math>. If <math>X\,\!</math> is a 95% lower one-sided bound, this would indicate that 95% of the population is greater than <math>X\,\!</math>. Care must be taken to differentiate between one- and two-sided confidence bounds, as these bounds can take on identical values at different percentage levels. For example, in the figures above, we see bounds on a hypothetical distribution. Assuming that this is the same distribution in all of the figures, we see that <math>X\,\!</math> marks the spot below which 5% of the distribution's population lies. Similarly, <math>Y\,\!</math> represents the point above which 5% of the population lies. Therefore, <math>X\,\!</math> and <math>Y\,\!</math> represent the 90% two-sided bounds, since 90% of the population lies between the two points. However, <math>X\,\!</math> also represents the lower one-sided 95% confidence bound, since 95% of the population lies above that point; and <math>Y\,\!</math> represents the upper one-sided 95% confidence bound, since 95% of the population is below <math>Y\,\!</math>. It is important to be sure of the type of bounds you are dealing with, particularly as both one-sided bounds can be displayed simultaneously in Weibull++. In Weibull++, we use upper to represent the higher limit and lower to represent the lower limit, regardless of their position, but based on the value of the results. So if obtaining the confidence bounds on the reliability, we would identify the lower value of reliability as the lower limit and the higher value of reliability as the higher limit. If obtaining the confidence bounds on probability of failure we will again identify the lower numeric value for the probability of failure as the lower limit and the higher value as the higher limit.

== Fisher Matrix Confidence Bounds ==
This section presents an overview of the theory on obtaining approximate confidence bounds on suspended (multiple censored) data. The methodology used is the so-called Fisher matrix bounds (FM), described in Nelson [[Appendix:_Life_Data_Analysis_References|[30]]] and Lloyd and Lipow [[Appendix:_Life_Data_Analysis_References|[24]]]. These bounds are employed in most other commercial statistical applications. In general, these bounds tend to be more optimistic than the non-parametric rank based bounds. This may be a concern, particularly when dealing with small sample sizes. Some statisticians feel that the Fisher matrix bounds are too optimistic when dealing with small sample sizes and prefer to use other techniques for calculating confidence bounds, such as the likelihood ratio bounds. 

=== Approximate Estimates of the Mean and Variance of a Function ===
In utilizing FM bounds for functions, one must first determine the mean and variance of the function in question (i.e., reliability function, failure rate function, etc.). An example of the methodology and assumptions for an arbitrary function <math>G\,\!</math> is presented next.

'''Single Parameter Case'''

For simplicity, consider a one-parameter distribution represented by a general function <math>G,\,\!</math> which is a function of one parameter estimator, say <math>G(\widehat{\theta }).\,\!</math> For example, the mean of the exponential distribution is a function of the parameter <math>\lambda: G(\lambda) = 1 / \lambda = \mu\,\!</math>. Then, in general, the expected value of <math>G\left( \widehat{\theta } \right)\,\!</math> can be found by:

::<math>E\left( G\left( \widehat{\theta } \right) \right)=G(\theta )+O\left( \frac{1}{n} \right)\,\!</math>

where <math>G(\theta)\,\!</math> is some function of <math>\theta\,\!</math>, such as the reliability function, and <math>\theta\,\!</math> is the population parameter where <math>E\left( \widehat{\theta } \right)=\theta \,\!</math> as <math>n\to \infty \,\!</math>. The term <math>O\left( \tfrac{1}{n} \right)\,\!</math> is a function of <math>n\,\!</math>, the sample size, and tends to zero, as fast as <math>\tfrac{1}{n},\,\!</math> as <math>n\to \infty .\,\!</math> For example, in the case of <math>\widehat{\theta }=1/\overline{x}\,\!</math> and <math>G(x) = 1 / x\,\!</math>, then <math>E(G(\widehat{\theta }))=\overline{x}+O\left( \tfrac{1}{n} \right)\,\!</math> where <math>O\left( \tfrac{1}{n} \right)=\tfrac{{{\sigma }^{2}}}{n}\,\!</math>. Thus as <math>n\to \infty \,\!</math>, <math>E(G(\widehat{\theta }))=\mu \,\!</math> where <math>\mu\,\!</math> and <math>\sigma\,\!</math> are the mean and standard deviation, respectively. Using the same one-parameter distribution, the variance of the function <math>G\left( \widehat{\theta } \right)\,\!</math> can then be estimated by:

::<math>Var\left( G\left( \widehat{\theta } \right) \right)=\left( \frac{\partial G}{\partial \widehat{\theta }} \right)_{\widehat{\theta }=\theta }^{2}Var\left( \widehat{\theta } \right)+O\left( \frac{1}{{{n}^{\tfrac{3}{2}}}} \right)\,\!</math>

'''Two-Parameter Case'''

Consider a Weibull distribution with two parameters <math>\beta\,\!</math> and <math>\eta\,\!</math>. For a given value of <math>t\,\!</math>, <math>R(t)=G(\beta ,\eta )={{e}^{-{{\left( \tfrac{t}{\eta } \right)}^{\beta }}}}\,\!</math>. Repeating the previous method for the case of a two-parameter distribution, it is generally true that for a function <math>G\,\!</math>, which is a function of two parameter estimators, say <math>G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right)\,\!</math>, that:

::<math>E\left( G\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \right)=G\left( {{\theta }_{1}},{{\theta }_{2}} \right)+O\left( \frac{1}{n} \right)\,\!</math>

and:

::<math>\begin{align}
Var( G( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}}))= &{\left(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}}\right)^2}_{{\widehat{\theta_{1}}}={\theta_{1}}}Var(\widehat{\theta_{1}})+{\left(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}}\right)^2}_{{\widehat{\theta_{2}}}={\theta_{2}}}Var(\widehat{\theta_{2}})\\
& +2{\left(\frac{\partial G}{\partial {{\widehat{\theta }}_{1}}}\right)}_{{\widehat{\theta_{1}}}={\theta_{1}}}{\left(\frac{\partial G}{\partial {{\widehat{\theta }}_{2}}}\right)}_{{\widehat{\theta_{2}}}={\theta_{2}}}Cov(\widehat{\theta_{1}},\widehat{\theta_{2}}) \\
& +O\left(\frac{1}{n^{\tfrac{3}{2}}}\right)
\end{align}
\,\!</math>

Note that the derivatives of the above equation are evaluated at <math>{{\widehat{\theta }}_{1}}={{\theta }_{1}}\,\!</math> and <math>{{\widehat{\theta }}_{2}}={{\theta }_{2}},\,\!</math> where E <math>\left( {{\widehat{\theta }}_{1}} \right)\simeq {{\theta }_{1}}\,\!</math> and E <math>\left( {{\widehat{\theta }}_{2}} \right)\simeq {{\theta }_{2}}.\,\!</math>

'''Parameter Variance and Covariance Determination'''

The determination of the variance and covariance of the parameters is accomplished via the use of the Fisher information matrix. For a two-parameter distribution, and using maximum likelihood estimates (MLE), the log-likelihood function for censored data is given by:

::<math>\begin{align}
\ln [L]= & \Lambda =\underset{i=1}{\overset{R}{\mathop \sum }}\,\ln [f({{T}_{i}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{j=1}{\overset{M}{\mathop \sum }}\,\ln [1-F({{S}_{j}};{{\theta }_{1}},{{\theta }_{2}})] \\
& \text{ }+\underset{l=1}{\overset{P}{\mathop \sum }}\,\ln \left\{ F({{I}_{{{l}_{U}}}};{{\theta }_{1}},{{\theta }_{2}})-F({{I}_{{{l}_{L}}}};{{\theta }_{1}},{{\theta }_{2}}) \right\}
\end{align}\,\!</math>

In the equation above, the first summation is for ''complete data'', the second summation is for ''right censored data'' and the third summation is for ''interval or left censored data''.

Then the Fisher information matrix is given by:

::<math>{{F}_{0}}=\left[ \begin{matrix}
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \right]}_{0}} \\
{} & {} & {} \\
{{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} \right]}_{0}} & {} & {{E}_{0}}{{\left[ -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \right]}_{0}} \\
\end{matrix} \right]\,\!</math>

The subscript 0 indicates that the quantity is evaluated at <math>{{\theta }_{1}}={{\theta }_{{{1}_{0}}}}\,\!</math> and <math>{{\theta }_{2}}={{\theta }_{{{2}_{0}}}},\,\!</math> the true values of the parameters.

So for a sample of <math>N\,\!</math> units where <math>R\,\!</math> units have failed, <math>S\,\!</math> have been suspended, and <math>P\,\!</math> have failed within a time interval, and <math>N = R + M + P,\,\!</math> one could obtain the sample local information matrix by:

::<math>F={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{}}\,\!</math>

Substituting the values of the estimated parameters, in this case <math>{{\widehat{\theta }}_{1}}\,\!</math> and <math>{{\widehat{\theta }}_{2}}\,\!</math>, and then inverting the matrix, one can then obtain the local estimate of the covariance matrix or:

::<math>\left[ \begin{matrix}
\widehat{Var}\left( {{\widehat{\theta }}_{1}} \right) & {} & \widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) \\
{} & {} & {} \\
\widehat{Cov}\left( {{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}} \right) & {} & \widehat{Var}\left( {{\widehat{\theta }}_{2}} \right) \\
\end{matrix} \right]={{\left[ \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{1}^{2}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{1}}\partial {{\theta }_{2}}} \\
{} & {} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{\theta }_{2}}\partial {{\theta }_{1}}} & {} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \theta _{2}^{2}} \\
\end{matrix} \right]}^{-1}}\,\!</math>

Then the variance of a function (<math>Var(G)\,\!</math>) can be estimated using equation for the variance. Values for the variance and covariance of the parameters are obtained from Fisher Matrix equation. Once they have been obtained, the approximate confidence bounds on the function are given as:

::<math>C{{B}_{R}}=E(G)\pm {{z}_{\alpha }}\sqrt{Var(G)}\,\!</math>

which is the estimated value plus or minus a certain number of standard deviations. We address finding <math>{{z}_{\alpha}}\,\!</math> next.

=== Approximate Confidence Intervals on the Parameters ===
In general, MLE estimates of the parameters are asymptotically normal, meaning that for large sample sizes, a distribution of parameter estimates from the same population would be very close to the normal distribution. Thus if <math>\widehat{\theta }\,\!</math> is the MLE estimator for <math>\theta\,\!</math>, in the case of a single parameter distribution estimated from a large sample of <math>n\,\!</math> units, then:

::<math>z\equiv \frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}\,\!</math>

follows an approximating normal distribution. That is

::<math>P\left( x\le z \right)\to \Phi \left( z \right)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{z}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt\,\!</math>

for large <math>n\,\!</math>. We now place confidence bounds on <math>\theta,\,\!</math> at some confidence level <math>\delta\,\!</math>, bounded by the two end points <math>{{C}_{1}}\,\!</math> and <math>{{C}_{2}}\,\!</math> where:

::<math>P\left( {{C}_{1}}<\theta <{{C}_{2}} \right)=\delta \,\!</math>

From the above equation:

::<math>P\left( -{{K}_{\tfrac{1-\delta }{2}}}<\frac{\widehat{\theta }-\theta }{\sqrt{Var\left( \widehat{\theta } \right)}}<{{K}_{\tfrac{1-\delta }{2}}} \right)\simeq \delta \,\!</math>

where <math>{{K}_{\alpha}}\,\!</math> is defined by:

::<math>\alpha =\frac{1}{\sqrt{2\pi }}\int_{{{K}_{\alpha }}}^{\infty }{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt=1-\Phi \left( {{K}_{\alpha }} \right)\,\!</math>

Now by simplifying the equation for the confidence level, one can obtain the approximate two-sided confidence bounds on the parameter <math>\theta\,\!,\,\!</math> at a confidence level <math>\delta,\,\!</math> or:

::<math>\left( \widehat{\theta }-{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)}<\theta <\widehat{\theta }+{{K}_{\tfrac{1-\delta }{2}}}\cdot \sqrt{Var\left( \widehat{\theta } \right)} \right)\,\!</math>

The upper one-sided bounds are given by:

::<math>\theta <\widehat{\theta }+{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}\,\!</math>

while the lower one-sided bounds are given by:

::<math>\theta >\widehat{\theta }-{{K}_{1-\delta }}\sqrt{Var(\widehat{\theta })}\,\!</math>

If <math>\widehat{\theta }\,\!</math> must be positive, then <math>\ln \widehat{\theta }\,\!</math> is treated as normally distributed. The two-sided approximate confidence bounds on the parameter <math>\theta\,\!</math>, at confidence level <math>\delta\,\!</math>, then become:

::<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (Two-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{\tfrac{1-\delta }{2}}}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (Two-sided lower)}
\end{align}\,\!</math>

The one-sided approximate confidence bounds on the parameter <math>\theta\,\!</math>, at confidence level <math>\delta,\,\!</math> can be found from:

::<math>\begin{align}
& {{\theta }_{U}}= & \widehat{\theta }\cdot {{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}\text{ (One-sided upper)} \\
& {{\theta }_{L}}= & \frac{\widehat{\theta }}{{{e}^{\tfrac{{{K}_{1-\delta }}\sqrt{Var\left( \widehat{\theta } \right)}}{\widehat{\theta }}}}}\text{ (One-sided lower)}
\end{align}\,\!</math>

The same procedure can be extended for the case of a two or more parameter distribution. Lloyd and Lipow [[Appendix:_Life_Data_Analysis_References|[24]]] further elaborate on this procedure.

=== Confidence Bounds on Time (Type 1) ===
Type 1 confidence bounds are confidence bounds around time for a given reliability. For example, when using the one-parameter exponential distribution, the corresponding time for a given exponential percentile (i.e., y-ordinate or unreliability, <math>Q = 1 - R)\,\!</math> is determined by solving the unreliability function for the time, <math>T\,\!</math>, or:

::<math>\begin{align}\widehat{T}(Q)= &-\frac{1}{\widehat{\lambda }}
\ln (1-Q)= & -\frac{1}{\widehat{\lambda }}\ln (R)
\end{align}\,\!</math>

Bounds on time (Type 1) return the confidence bounds around this time value by determining the confidence intervals around <math>\widehat{\lambda }\,\!</math> and substituting these values into the above equation. The bounds on <math>\widehat{\lambda }\,\!</math> are determined using the method for the bounds on parameters, with its variance obtained from the Fisher Matrix. Note that the procedure is slightly more complicated for distributions with more than one parameter.

=== Confidence Bounds on Reliability (Type 2) ===
Type 2 confidence bounds are confidence bounds around reliability. For example, when using the two-parameter exponential distribution, the reliability function is:

::<math>\widehat{R}(T)={{e}^{-\widehat{\lambda }\cdot T}}\,\!</math>

Reliability bounds (Type 2) return the confidence bounds by determining the confidence intervals around <math>\widehat{\lambda }\,\!</math> and substituting these values into the above equation. The bounds on <math>\widehat{\lambda }\,\!</math> are determined using the method for the bounds on parameters, with its variance obtained from the Fisher Matrix. Once again, the procedure is more complicated for distributions with more than one parameter.

== Beta Binomial Confidence Bounds ==
Another less mathematically intensive method of calculating confidence bounds involves a procedure similar to that used in calculating median ranks (see [[Parameter Estimation]]). This is a non-parametric approach to confidence interval calculations that involves the use of rank tables and is commonly known as beta-binomial bounds (BB). By non-parametric, we mean that no underlying distribution is assumed. (Parametric implies that an underlying distribution, with parameters, is assumed.) In other words, this method can be used for any distribution, without having to make adjustments in the underlying equations based on the assumed distribution. Recall from the discussion on the median ranks that we used the binomial equation to compute the ranks at the 50% confidence level (or median ranks) by solving the cumulative binomial distribution for <math>Z\,\!</math> (rank for the <math>{{j}^{th}}\,\!</math> failure):

::<math>P=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}\,\!</math>

where <math>N\,\!</math> is the sample size and <math>j\,\!</math> is the order number.

The median rank was obtained by solving the following equation for <math>Z\,\!</math>:

::<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}\,\!</math>

The same methodology can then be repeated by changing <math>P\,\!</math> for 0.50 (50%) to our desired confidence level. For <math>P = 90%\,\!</math> one would formulate the equation as

::<math>0.90=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}\,\!</math>

Keep in mind that one must be careful to select the appropriate values for <math>P\,\!</math> based on the type of confidence bounds desired. For example, if two-sided 80% confidence bounds are to be calculated, one must solve the equation twice (once with <math>P = 0.1\,\!</math> and once with <math>P = 0.9\,\!</math>) in order to place the bounds around 80% of the population. Using this methodology, the appropriate ranks are obtained and plotted based on the desired confidence level. These points are then joined by a smooth curve to obtain the corresponding confidence bound.

In Weibull++, this non-parametric methodology is used only when plotting bounds on the mixed Weibull distribution. Full details on this methodology can be found in Kececioglu [[Appendix:_Life_Data_Analysis_References|[20]]]. These binomial equations can again be transformed using the beta and F distributions, thus the name beta binomial confidence bounds.

== Likelihood Ratio Confidence Bounds ==
Another method for calculating confidence bounds is the likelihood ratio bounds (LRB) method. Conceptually, this method is a great deal simpler than that of the Fisher matrix, although that does not mean that the results are of any less value. In fact, the LRB method is often preferred over the FM method in situations where there are smaller sample sizes.

Likelihood ratio confidence bounds are based on the following likelihood ratio equation:

::<math>-2\cdot \text{ln}\left( \frac{L(\theta )}{L(\widehat{\theta })} \right)\ge \chi _{\alpha ;k}^{2}\,\!</math>

where:

:*<math>L(\theta)\,\!</math> is the likelihood function for the unknown parameter vector <math>\theta\,\!</math>
:*<math>L(\widehat{\theta })\,\!</math> is the likelihood function calculated at the estimated vector <math>\widehat{\theta }\,\!</math>
:*<math>\chi _{\alpha ;k}^{2}\,\!</math> is the chi-squared statistic with probability <math>\alpha\,\!</math> and <math>k\,\!</math> degrees of freedom, where <math>k\,\!</math> is the number of quantities jointly estimated

If <math>\delta\,\!</math> is the confidence level, then <math>\alpha = \delta\,\!</math> for two-sided bounds and <math>\alpha = (2\delta - 1)\,\!</math> for one-sided. Recall from the [[Brief Statistical Background]] chapter that if <math>x\,\!</math> is a continuous random variable with ''pdf'':

::<math>\begin{align}
f(x;{{\theta}_{1}},{{\theta}_{2}},...,{{\theta}_{k}})
\end{align}\,\!</math>

where <math>{{\theta}_{1}},{{\theta}_{2}},...,{{\theta}_{k}}\,\!</math> are <math>k\,\!</math> unknown constant parameters that need to be estimated, one can conduct an experiment and obtain <math>R\,\!</math> independent observations, <math>{{x}_{1}},{{x}_{2}},...,{{x}_{R}}\,\!</math>, which correspond in the case of life data analysis to failure times. The likelihood function is given by:

::<math>L({{x}_{1}},{{x}_{2}},...,{{x}_{R}}|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{R}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})\,\!</math>

::<math>\begin{align}
i = 1,2,...,R
\end{align}\,\!</math>

The maximum likelihood estimators (MLE) of <math>{{\theta}_{1}},{{\theta}_{2}},...,{{\theta}_{k}}\,\!</math> are <math>k\,\!</math> are obtained by maximizing <math>L.\,\!</math> These are represented by the <math>L(\widehat{\theta })\,\!</math> term in the denominator of the ratio in the likelihood ratio equation. Since the values of the data points are known, and the values of the parameter estimates <math>\widehat{\theta }\,\!</math> have been calculated using MLE methods, the only unknown term in the likelihood ratio equation is the <math>L(\theta)\,\!</math> term in the numerator of the ratio. It remains to find the values of the unknown parameter vector <math>\theta\,\!</math> that satisfy the likelihood ratio equation. For distributions that have two parameters, the values of these two parameters can be varied in order to satisfy the likelihood ratio equation. The values of the parameters that satisfy this equation will change based on the desired confidence level <math>\delta;\,\!</math> but at a given value of <math>\delta\,\!</math> there is only a certain region of values for <math>{{\theta}_{1}}\,\!</math> and <math>{{\theta}_{2}}\,\!</math> for which the likelihood ratio equation holds true. This region can be represented graphically as a contour plot, an example of which is given in the following graphic.

[[Image:Examplecontourplot.png|center|450px]]

The region of the contour plot essentially represents a cross-section of the likelihood function surface that satisfies the conditions of the likelihood ratio equation.

'''Note on Contour Plots in Weibull++ and ALTA'''

Contour plots can be used for comparing data sets. Consider two data sets, one for an old product design and another for a new design. The engineer would like to determine if the two designs are significantly different and at what confidence. By plotting the contour plots of each data set in an overlay plot (the same distribution must be fitted to each data set), one can determine the confidence at which the two sets are significantly different. If, for example, there is no overlap (i.e., the two plots do not intersect) between the two 90% contours, then the two data sets are significantly different with a 90% confidence. If the two 95% contours overlap, then the two designs are NOT significantly different at the 95% confidence level. An example of non-intersecting contours is shown next. For details on comparing data sets, see [[Comparing Life Data Sets]].

 [[Image:Contourplot.png|center|450px]]

=== Confidence Bounds on the Parameters ===
The bounds on the parameters are calculated by finding the extreme values of the contour plot on each axis for a given confidence level. Since each axis represents the possible values of a given parameter, the boundaries of the contour plot represent the extreme values of the parameters that satisfy the following:

::<math>-2\cdot \text{ln}\left( \frac{L({{\theta }_{1}},{{\theta }_{2}})}{L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})} \right)=\chi _{\alpha ;1}^{2}\,\!</math>

This equation can be rewritten as:

::<math>L({{\theta }_{1}},{{\theta }_{2}})=L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}\,\!</math>

The task now is to find the values of the parameters <math>{{\theta}_{1}}\,\!</math> and <math>{{\theta}_{2}}\,\!</math> so that the equality in the likelihood ratio equation shown above is satisfied. Unfortunately, there is no closed-form solution; therefore, these values must be arrived at numerically. One way to do this is to hold one parameter constant and iterate on the other until an acceptable solution is reached. This can prove to be rather tricky, since there will be two solutions for one parameter if the other is held constant. In situations such as these, it is best to begin the iterative calculations with values close to those of the MLE values, so as to ensure that one is not attempting to perform calculations outside of the region of the contour plot where no solution exists.

 '''Example 1:''' {{Example: Likelihood Ratio Bounds on Parameters }}

=== Confidence Bounds on Time (Type 1) ===
The manner in which the bounds on the time estimate for a given reliability are calculated is much the same as the manner in which the bounds on the parameters are calculated. The difference lies in the form of the likelihood functions that comprise the likelihood ratio. In the preceding section, we used the standard form of the likelihood function, which was in terms of the parameters <math>{{\theta}_{1}}\,\!</math> and <math>{{\theta}_{2}}\,\!</math>. In order to calculate the bounds on a time estimate, the likelihood function needs to be rewritten in terms of one parameter and time, so that the maximum and minimum values of the time can be observed as the parameter is varied. This process is best illustrated with an example.

 '''Example 2:''' {{Example: Likelihood Ratio Bounds on Time (Type I)}}

=== Confidence Bounds on Reliability (Type 2) ===
The likelihood ratio bounds on a reliability estimate for a given time value are calculated in the same manner as were the bounds on time. The only difference is that the likelihood function must now be considered in terms of <math>\beta\,\!</math> and <math>R\,\!</math>. The likelihood function is once again altered in the same way as before, only now <math>R\,\!</math> is considered to be a parameter instead of <math>t\,\!</math>, since the value of <math>t\,\!</math> must be specified in advance. Once again, this process is best illustrated with an example.

 '''Example 3:''' {{Example: Likelihood Ratio Bounds on Reliability (Type 2)}}

== Bayesian Confidence Bounds ==
A fourth method of estimating confidence bounds is based on the Bayes theorem. This type of confidence bounds relies on a different school of thought in statistical analysis, where prior information is combined with sample data in order to make inferences on model parameters and their functions. An introduction to Bayesian methods is given in the [[Parameter Estimation]] chapter. Bayesian confidence bounds are derived from Bayes's rule, which states that:

::<math>f(\theta |Data)=\frac{L(Data|\theta )\varphi (\theta )}{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }\,\!</math>

where:
:*<math>f (\theta | Data\,\!</math>) is the ''posterior pdf'' of <math>\theta\,\!</math>
:*<math>\theta\,\!</math> is the parameter vector of the chosen distribution (i.e., Weibull, lognormal, etc.)
:*<math>L(\bullet )\,\!</math> is the likelihood function
:*<math>\varphi (\theta )\,\!</math> is the ''prior pdf'' of the parameter vector <math>\theta\,\!</math>
:*<math>\varsigma \,\!</math> is the range of <math>\theta\,\!</math>.

In other words, the prior knowledge is provided in the form of the prior ''pdf'' of the parameters, which in turn is combined with the sample data in order to obtain the posterior ''pdf''. Different forms of prior information exist, such as past data, expert opinion or non-informative (refer to [[Parameter Estimation]]). It can be seen from the above Bayes's rule formula that we are now dealing with distributions of parameters rather than single value parameters. For example, consider a one-parameter distribution with a positive parameter <math>{{\theta}_{1}}\,\!</math>. Given a set of sample data, and a prior distribution for <math>{{\theta}_{1}},\,\!</math> <math>\varphi ({{\theta }_{1}}),\,\!</math> the above Bayes's rule formula can be written as:

::<math>f({{\theta }_{1}}|Data)=\frac{L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}\,\!</math>

In other words, we now have the distribution of <math>{{\theta}_{1}}\,\!</math> and we can now make statistical inferences on this parameter, such as calculating probabilities. Specifically, the probability that <math>{{\theta}_{1}}\,\!</math> is less than or equal to a value <math>x,\,\!</math> <math>P({{\theta }_{1}}\le x)\,\!</math> can be obtained by integrating the posterior probability density function (''pdf''), or:

::<math>P({{\theta }_{1}}\le x)=\int_{0}^{x}f({{\theta }_{1}}|Data)d{{\theta }_{1}}\,\!</math>

The above equation is the posterior ''cdf'', which essentially calculates a confidence bound on the parameter, where <math>P({{\theta }_{1}}\le x)\,\!</math> is the confidence level and <math>x\,\!</math> is the confidence bound. Substituting the posterior ''pdf'' into the above posterior ''cdf'' yields:

::<math>CL=\frac{\int_{0}^{x}L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}{\int_{0}^{\infty }L(Data|{{\theta }_{1}})\varphi ({{\theta }_{1}})d{{\theta }_{1}}}\,\!</math>

The only question at this point is, what do we use as a prior distribution of <math>{{\theta}_{1}}\,\!</math>? For the confidence bounds calculation application, non-informative prior distributions are utilized. Non-informative prior distributions are distributions that have no population basis and play a minimal role in the posterior distribution. The idea behind the use of non-informative prior distributions is to make inferences that are not affected by external information, or when external information is not available. In the general case of calculating confidence bounds using Bayesian methods, the method should be independent of external information and it should only rely on the current data. Therefore, non-informative priors are used. Specifically, the uniform distribution is used as a prior distribution for the different parameters of the selected fitted distribution. For example, if the Weibull distribution is fitted to the data, the prior distributions for beta and eta are assumed to be uniform. The above equation can be generalized for any distribution having a vector of parameters <math>\theta,\,\!</math> yielding the general equation for calculating Bayesian confidence bounds:

::<math>CL=\frac{\underset{\xi }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }{\underset{\varsigma }{\int{\mathop{}_{}^{}}}\,L(Data|\theta )\varphi (\theta )d\theta }\,\!</math>

where:

:*<math>CL\,\!</math> is the confidence level
:*<math>\theta\,\!</math> is the parameter vector
:*<math>L(\bullet )\,\!</math> is the likelihood function
:*<math>\varphi (\theta )\,\!</math> is the prior ''pdf'' of the parameter vector <math>\theta\,\!</math>
:*<math>\varsigma \,\!</math> is the range of <math>\theta\,\!</math>
:*<math>\xi\,\!</math> is the range in which <math>\theta\,\!</math> changes from <math>\Psi(T,R)\,\!</math> till <math>\theta\,\!</math> 's maximum value, or from <math>\theta\,\!</math> 's minimum value till <math>\Psi(T,R)\,\!</math>
:*<math>\Psi(T,R)\,\!</math> is a function such that if <math>T\,\!</math> is given, then the bounds are calculated for <math>R\,\!</math>. If <math>R\,\!</math> is given, then the bounds are calculated for <math>T\,\!</math>.

If <math>T\,\!</math> is given, then from the above equation and <math>\Psi\,\!</math> and for a given <math>CL\,\!</math>, the bounds on <math>R\,\!</math> are calculated. If <math>R\,\!</math> is given, then from the above equation and <math>\Psi\,\!</math> and for a given <math>C\,\!</math> the bounds on <math>T\,\!</math> are calculated.

=== Confidence Bounds on Time (Type 1) ===
For a given failure time distribution and a given reliability <math>R\,\!</math>, <math>T(R)\,\!</math> is a function of <math>R\,\!</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. The bounds in other types of distributions can be obtained in similar fashion. For the two-parameter Weibull distribution:

::<math>T(R)=\eta \exp (\frac{\ln (-\ln R)}{\beta })\,\!</math>

For a given reliability, the Bayesian one-sided upper bound estimate for <math>T(R)\,\!</math> is:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT\,\!</math>

where <math>f(T | Data, R)\,\!</math> is the posterior distribution of Time <math>T.\,\!</math> Using the above equation, we have the following:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(T\le {{T}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \exp (\frac{\ln (-\ln R)}{\beta })\le {{T}_{U}})\,\!</math>

The above equation can be rewritten in terms of <math>\eta\,\!</math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\eta \le {{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta }))\,\!</math>

Applying the Bayes's rule by assuming that the priors of <math>\beta\,\!</math> and <math>\eta\,\!</math> are independent, we then obtain the following relationship:

::<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{{{T}_{U}}\exp (-\frac{\ln (-\ln R)}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }\,\!</math>

The above equation can be solved for <math>{{T}_{U}}(R)\,\!</math>, where:

:*<math>CL\,\!</math> is the confidence level,
:*<math>\varphi (\beta )\,\!</math> is the prior ''pdf'' of the parameter <math>\beta\,\!</math>. For non-informative prior distribution, <math>\varphi (\beta )=\tfrac{1}{\beta }.\,\!</math>
:*<math>\varphi (\eta )\,\!</math> is the prior ''pdf'' of the parameter <math>\eta.\,\!</math> For non-informative prior distribution, <math>\varphi (\eta )=\tfrac{1}{\eta }.\,\!</math>
:*<math>L(\bullet )\,\!</math> is the likelihood function.

The same method can be used to get the one-sided lower bound of <math>T(R)\,\!</math> from:

::<math>CL=\frac{\int_{0}^{\infty }\int_{{{T}_{L}}\exp (\frac{-\ln (-\ln R)}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }\,\!</math>

The above equation can be solved to get <math>{{T}_{L}}(R)\,\!</math>.

The Bayesian two-sided bounds estimate for <math>T(R)\,\!</math> is:

::<math>CL=\int_{{{T}_{L}}(R)}^{{{T}_{U}}(R)}f(T|Data,R)dT\,\!</math>

which is equivalent to:

::<math>(1+CL)/2=\int_{0}^{{{T}_{U}}(R)}f(T|Data,R)dT\,\!</math>

and:

::<math>(1-CL)/2=\int_{0}^{{{T}_{L}}(R)}f(T|Data,R)dT\,\!</math>

Using the same method for the one-sided bounds, <math>{{T}_{U}}(R)\,\!</math> and <math>{{T}_{L}}(R)\,\!</math> can be solved.

=== Confidence Bounds on Reliability (Type 2) ===
For a given failure time distribution and a given time <math>T\,\!</math>, <math>R(T)\,\!</math> is a function of <math>T\,\!</math> and the distribution parameters. To illustrate the procedure for obtaining confidence bounds, the two-parameter Weibull distribution is used as an example. The bounds in other types of distributions can be obtained in similar fashion. For example, for two parameter Weibull distribution:

::<math>R=\exp (-{{(\frac{T}{\eta })}^{\beta }})\,\!</math>

The Bayesian one-sided upper bound estimate for <math>R(T)\,\!</math> is:

::<math>CL=\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR\,\!</math>

Similar to the bounds on Time, the following is obtained:

::<math>CL=\frac{\int_{0}^{\infty }\int_{0}^{T\exp (-\frac{\ln (-\ln {{R}_{U}})}{\beta })}L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }\,\!</math>

The above equation can be solved to get <math>{{R}_{U}}(T)\,\!</math>.

The Bayesian one-sided lower bound estimate for R(T) is:

::<math>1-CL=\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR\,\!</math>

Using the posterior distribution, the following is obtained:

::<math>CL=\frac{\int_{0}^{\infty }\int_{T\exp (-\frac{\ln (-\ln {{R}_{L}})}{\beta })}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(\beta ,\eta )\varphi (\beta )\varphi (\eta )d\eta d\beta }\,\!</math>

The above equation can be solved to get <math>{{R}_{L}}(T)\,\!</math>.

The Bayesian two-sided bounds estimate for <math>R(T)\,\!</math> is:

::<math>CL=\int_{{{R}_{L}}(T)}^{{{R}_{U}}(T)}f(R|Data,T)dR\,\!</math>

which is equivalent to:

::<math>\int_{0}^{{{R}_{U}}(T)}f(R|Data,T)dR=(1+CL)/2\,\!</math>

and

::<math>\int_{0}^{{{R}_{L}}(T)}f(R|Data,T)dR=(1-CL)/2\,\!</math>

Using the same method for one-sided bounds, <math>{{R}_{U}}(T)\,\!</math> and <math>{{R}_{L}}(T)\,\!</math> can be solved.

== Simulation Based Bounds ==
The SimuMatic tool in Weibull++ can be used to perform a large number of reliability analyses on data sets that have been created using Monte Carlo simulation. This utility can assist the analyst to a) better understand life data analysis concepts, b) experiment with the influences of sample sizes and censoring schemes on analysis methods, c) construct simulation-based confidence intervals, d) better understand the concepts behind confidence intervals and e) design reliability tests. This section describes how to use simulation for estimating confidence bounds. 

SimuMatic generates confidence bounds and assists in visualizing and understanding them. In addition, it allows one to determine the adequacy of certain parameter estimation methods (such as rank regression on X, rank regression on Y and maximum likelihood estimation) and to visualize the effects of different data censoring schemes on the confidence bounds.

'''Example:'''
{{:Simulation_Based_Bounds_Example}}

Weibull Confidence Bounds

2023-03-21T18:45:22Z

M Spivey: /* Bounds on Reliability */

{{template:LDABOOK|8.2|Weibull Confidence Bounds}}

== Fisher Matrix Confidence Bounds ==
One of the methods used by the application in estimating the different types of confidence bounds for Weibull data, the Fisher matrix method, is presented in this section. The complete derivations were presented in detail (for a general function) in [[Confidence Bounds]].

=== Bounds on the Parameters ===
One of the properties of maximum likelihood estimators is that they are asymptotically normal, meaning that for large samples they are normally distributed. Additionally, since both the shape parameter estimate, <math> \hat{\beta } \,\!</math>, and the scale parameter estimate, <math> \hat{\eta }, \,\!</math> must be positive, thus <math>ln\beta \,\!</math> and <math>ln\eta \,\!</math> are treated as being normally distributed as well. The lower and upper bounds on the parameters are estimated from Nelson [[Appendix:_Life_Data_Analysis_References|[30]]]:

::<math> \beta _{U} =\hat{\beta }\cdot e^{\frac{K_{\alpha }\sqrt{Var(\hat{ \beta })}}{\hat{\beta }}}\text{ (upper bound)} \,\!</math>

::<math> \beta _{L} =\frac{\hat{\beta }}{e^{\frac{K_{\alpha }\sqrt{Var(\hat{ \beta })}}{\hat{\beta }}}} \text{ (lower bound)}
\,\!</math>

and:

::<math> \eta _{U} =\hat{\eta }\cdot e^{\frac{K_{\alpha }\sqrt{Var(\hat{ \eta })}}{\hat{\eta }}}\text{ (upper bound)}
\,\!</math>

::<math> \eta _{L} =\frac{\hat{\eta }}{e^{\frac{K_{\alpha }\sqrt{Var(\hat{ \eta })}}{\hat{\eta }}}}\text{ (lower bound)} \,\!</math>

where <math> K_{\alpha}\,\!</math> is defined by:

::<math> \alpha =\frac{1}{\sqrt{2\pi }}\int_{K_{\alpha }}^{\infty }e^{-\frac{t^{2}}{2} }dt=1-\Phi (K_{\alpha }) \,\!</math>

If <math>d\,\!</math> is the confidence level, then <math> \alpha =\frac{1-\delta }{2} \,\!</math> for the two-sided bounds and <math>a = 1 - d\,\!</math> for the one-sided bounds. The variances and covariances of <math> \hat{\beta }\,\!</math> and <math> \hat{\eta }\,\!</math> are estimated from the inverse local Fisher matrix, as follows:

::<math> \left( \begin{array}{cc} \hat{Var}\left( \hat{\beta }\right) & \hat{Cov}\left( \hat{ \beta },\hat{\eta }\right)
\\
\hat{Cov}\left( \hat{\beta },\hat{\eta }\right) & \hat{Var} \left( \hat{\eta }\right) \end{array} \right) =\left( \begin{array}{cc} -\frac{\partial ^{2}\Lambda }{\partial \beta ^{2}} & -\frac{\partial ^{2}\Lambda }{\partial \beta \partial \eta }
\\

-\frac{\partial ^{2}\Lambda }{\partial \beta \partial \eta } & -\frac{ \partial ^{2}\Lambda }{\partial \eta ^{2}} \end{array} \right) _{\beta =\hat{\beta },\text{ }\eta =\hat{\eta }}^{-1} \,\!</math>

'''Fisher Matrix Confidence Bounds and Regression Analysis'''

Note that the variance and covariance of the parameters are obtained from the inverse Fisher information matrix as described in this section. The local Fisher information matrix is obtained from the second partials of the likelihood function, by substituting the solved parameter estimates into the particular functions. This method is based on maximum likelihood theory and is derived from the fact that the parameter estimates were computed using maximum likelihood estimation methods. When one uses least squares or regression analysis for the parameter estimates, this methodology is theoretically then not applicable. However, if one assumes that the variance and covariance of the parameters will be similar ( One also assumes similar properties for both estimators.) regardless of the underlying solution method, then the above methodology can also be used in regression analysis.

The Fisher matrix is one of the methodologies that Weibull++ uses for both MLE and regression analysis. Specifically, Weibull++ uses the likelihood function and computes the local Fisher information matrix based on the estimates of the parameters and the current data. This gives consistent confidence bounds regardless of the underlying method of solution, (i.e., MLE or regression). In addition, Weibull++ checks this assumption and proceeds with it if it considers it to be acceptable. In some instances, Weibull++ will prompt you with an "Unable to Compute Confidence Bounds" message when using regression analysis. This is an indication that these assumptions were violated.

=== Bounds on Reliability ===
The bounds on reliability can easily be derived by first looking at the general extreme value distribution (EVD). Its reliability function is given by:

::<math> R(t)=e^{-e^{\left( \frac{t-p_{1}}{p_{2}}\right) }} \,\!</math>

By transforming <math>t = \ln t\,\!</math> and converting <math> p_{1}=\ln({\eta})\,\!</math>, <math> p_{2}=\frac{1}{ \beta } \,\!</math>, the above equation becomes the Weibull reliability function:

::<math> R(t)=e^{-e^{\beta \left( \ln t-\ln \eta \right) }}=e^{-e^{\ln \left( \frac{t }{\eta }\right) ^{\beta }}}=e^{-\left( \frac{t}{\eta }\right) ^{\beta }} \,\!</math>

with:

::<math> R(T)=e^{-e^{\beta \left( \ln t-\ln \eta \right) }}\,\!</math>

set:

:: <math> u=\beta \left( \ln t-\ln \eta \right) \,\!</math>

The reliability function now becomes:

::<math> R(T)=e^{-e^{u}} \,\!</math>

The next step is to find the upper and lower bounds on <math>u\,\!</math>. Using the equations derived in [[Confidence Bounds]], the bounds on reliability are then estimated from Nelson [[Appendix:_Life_Data_Analysis_References|[30]]]:

::<math> u_{U} =\hat{u}+K_{\alpha }\sqrt{Var(\hat{u})}
\,\!</math>

::<math> u_{L} =\hat{u}-K_{\alpha }\sqrt{Var(\hat{u})}
\,\!</math>

where:

::<math> Var(\hat{u}) =\left( \frac{\partial u}{\partial \beta }\right) ^{2}Var( \hat{\beta })+\left( \frac{\partial u}{\partial \eta }\right) ^{2}Var( \hat{\eta }) +2\left( \frac{\partial u}{\partial \beta }\right) \left( \frac{\partial u }{\partial \eta }\right) Cov\left( \hat{\beta },\hat{\eta }\right) \,\!</math>

or:

::<math> Var(\hat{u}) =\frac{\hat{u}^{2}}{\hat{\beta }^{2}}Var(\hat{ \beta })+\frac{\hat{\beta }^{2}}{\hat{\eta }^{2}}Var(\hat{\eta }) -\left( \frac{2\hat{u}}{\hat{\eta }}\right) Cov\left( \hat{\beta }, \hat{\eta }\right). \,\!</math>

The upper and lower bounds on reliability are:

::<math> R_{U} =e^{-e^{u_{L}}}\text{ (upper bound)}\,\!</math>

::<math> R_{L} =e^{-e^{u_{U}}}\text{ (lower bound)}\,\!</math>

'''Other Weibull Forms'''

Weibull++ makes the following assumptions/substitutions when using the three-parameter or one-parameter forms:

*For the 3-parameter case, substitute <math> t=\ln (t-\hat{\gamma }) \,\!</math> (and by definition <math>\gamma\, < t\!</math>), instead of <math>\ln t\,\!</math>. (Note that this is an approximation since it eliminates the third parameter and assumes that <math> Var( \hat{\gamma })=0. \,\!</math>)
*For the 1-parameter, <math> Var(\hat{\beta })=0, \,\!</math> thus:

::<math> Var(\hat{u})=\left( \frac{\partial u}{\partial \eta }\right) ^{2}Var( \hat{\eta })=\left( \frac{\hat{\beta }}{\hat{\eta }}\right) ^{2}Var(\hat{\eta }) \,\!</math>

Also note that the time axis (x-axis) in the three-parameter Weibull plot in Weibull++ is not <math>{t}\,\!</math> but <math>t - \gamma\,\!</math>. This means that one must be cautious when obtaining confidence bounds from the plot. If one desires to estimate the confidence bounds on reliability for a given time <math>{{t}_{0}}\,\!</math> from the adjusted plotted line, then these bounds should be obtained for a <math>{{t}_{0}} - \gamma\,\!</math> entry on the time axis.

=== Bounds on Time ===
The bounds around the time estimate or reliable life estimate, for a given Weibull percentile (unreliability), are estimated by first solving the reliability equation with respect to time, as discussed in Lloyd and Lipow [[Appendix:_Life_Data_Analysis_References|[24]]] and in Nelson [[Appendix:_Life_Data_Analysis_References|[30]]]:

::<math> \ln R =-\left( \frac{t}{\eta }\right) ^{\beta }
\,\!</math>

::<math> \ln (-\ln R) =\beta \ln \left( \frac{t}{\eta }\right) \,\!</math>

::<math>\begin{align}
\ln (-\ln R) =\beta (\ln t-\ln \eta )
\end{align}\,\!</math>

or:

::<math> u=\frac{1}{\beta }\ln (-\ln R)+\ln \eta \,\!</math>

where <math>u = \ln t\,\!</math> .

The upper and lower bounds on are estimated from:

::<math> u_{U} =\hat{u}+K_{\alpha }\sqrt{Var(\hat{u})} \,\!</math>

::<math> u_{L} =\hat{u}-K_{\alpha }\sqrt{Var(\hat{u})} \,\!</math>

where:

::<math> Var(\hat{u})=\left( \frac{\partial u}{\partial \beta }\right) ^{2}Var( \hat{\beta })+\left( \frac{\partial u}{\partial \eta }\right) ^{2}Var( \hat{\eta })+2\left( \frac{\partial u}{\partial \beta }\right) \left( \frac{\partial u}{\partial \eta }\right) Cov\left( \hat{\beta },\hat{ \eta }\right) \,\!</math>

or:
::<math> Var(\hat{u}) =\frac{1}{\hat{\beta }^{4}}\left[ \ln (-\ln R)\right] ^{2}Var(\hat{\beta })+\frac{1}{\hat{\eta }^{2}}Var(\hat{\eta })+2\left( -\frac{\ln (-\ln R)}{\hat{\beta }^{2}}\right) \left( \frac{1}{ \hat{\eta }}\right) Cov\left( \hat{\beta },\hat{\eta }\right) \,\!</math>

The upper and lower bounds are then found by:

::<math> T_{U} =e^{u_{U}}\text{ (upper bound)} \,\!</math>

::<math> T_{L} =e^{u_{L}}\text{ (lower bound)} \,\!</math>

== Likelihood Ratio Confidence Bounds ==
As covered in [[Confidence Bounds]], the likelihood confidence bounds are calculated by finding values for <math>{{\theta}_{1}}\,\!</math> and <math>{{\theta}_{2}}\,\!</math> that satisfy:

::<math> -2\cdot \text{ln}\left( \frac{L(\theta _{1},\theta _{2})}{L(\hat{\theta }_{1}, \hat{\theta }_{2})}\right) =\chi _{\alpha ;1}^{2} \,\!</math>

This equation can be rewritten as:

::<math> L(\theta _{1},\theta _{2})=L(\hat{\theta }_{1},\hat{\theta } _{2})\cdot e^{\frac{-\chi _{\alpha ;1}^{2}}{2}} \,\!</math>

For complete data, the likelihood function for the Weibull distribution is given by:

::<math> L(\beta ,\eta )=\prod_{i=1}^{N}f(x_{i};\beta ,\eta )=\prod_{i=1}^{N}\frac{ \beta }{\eta }\cdot \left( \frac{x_{i}}{\eta }\right) ^{\beta -1}\cdot e^{-\left( \frac{x_{i}}{\eta }\right) ^{\beta }} \,\!</math>

For a given value of <math>\alpha\,\!</math>, values for <math>\beta\,\!</math> and <math>\eta\,\!</math> can be found which represent the maximum and minimum values that satisfy the above equation. These represent the confidence bounds for the parameters at a confidence level <math>\delta\,\!</math>, where <math>\alpha = \delta\,\!</math> for two-sided bounds and <math>\alpha = 2\delta - 1\,\!</math> for one-sided.

Similarly, the bounds on time and reliability can be found by substituting the Weibull reliability equation into the likelihood function so that it is in terms of <math>\beta\,\!</math> and time or reliability, as discussed in [[Confidence Bounds]]. The likelihood ratio equation used to solve for bounds on time (Type 1) is:

::<math> L(\beta ,t)=\prod_{i=1}^{N}\frac{\beta }{\left( \frac{t}{(-\text{ln}(R))^{ \frac{1}{\beta }}}\right) }\cdot \left( \frac{x_{i}}{\left( \frac{t}{(-\text{ ln}(R))^{\frac{1}{\beta }}}\right) }\right) ^{\beta -1}\cdot \text{exp}\left[ -\left( \frac{x_{i}}{\left( \frac{t}{(-\text{ln}(R))^{\frac{1}{\beta }}} \right) }\right) ^{\beta }\right] \,\!</math>

The likelihood ratio equation used to solve for bounds on reliability (Type 2) is:

::<math> L(\beta ,R)=\prod_{i=1}^{N}\frac{\beta }{\left( \frac{t}{(-\text{ln}(R))^{ \frac{1}{\beta }}}\right) }\cdot \left( \frac{x_{i}}{\left( \frac{t}{(-\text{ ln}(R))^{\frac{1}{\beta }}}\right) }\right) ^{\beta -1}\cdot \text{exp}\left[ -\left( \frac{x_{i}}{\left( \frac{t}{(-\text{ln}(R))^{\frac{1}{\beta }}} \right) }\right) ^{\beta }\right] \,\!</math>

== Bayesian Confidence Bounds ==
=== Bounds on Parameters ===
Bayesian Bounds use non-informative prior distributions for both parameters. From [[Confidence Bounds]], we know that if the prior distribution of <math>\eta\,\!</math> and <math>\beta\,\!</math> are independent, the posterior joint distribution of <math>\eta\,\!</math> and <math>\beta\,\!</math> can be written as:

::<math> f(\eta ,\beta |Data)= \dfrac{L(Data|\eta ,\beta )\varphi (\eta )\varphi (\beta )}{\int_{0}^{\infty }\int_{0}^{\infty }L(Data|\eta ,\beta )\varphi (\eta )\varphi (\beta )d\eta d\beta } \,\!</math>

The marginal distribution of <math>\eta\,\!</math> is:

::<math> f(\eta |Data) =\int_{0}^{\infty }f(\eta ,\beta |Data)d\beta =
\dfrac{\int_{0}^{\infty }L(Data|\eta ,\beta )\varphi (\eta )\varphi (\beta )d\beta }{\int_{0}^{\infty }\int_{-\infty }^{\infty }L(Data|\eta ,\beta )\varphi (\eta )\varphi (\beta )d\eta d\beta }
\,\!</math>

where: <math> \varphi (\beta )=\frac{1}{\beta } \,\!</math> is the non-informative prior of <math>\beta\,\!</math>. <math> \varphi (\eta )=\frac{1}{\eta } \,\!</math> is the non-informative prior of <math>\eta\,\!</math>. Using these non-informative prior distributions, <math>f(\eta|Data)\,\!</math> can be rewritten as:

::<math> f(\eta |Data)=\dfrac{\int_{0}^{\infty }L(Data|\eta ,\beta )\frac{1}{\beta } \frac{1}{\eta }d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(Data|\eta ,\beta )\frac{1}{\beta }\frac{1}{\eta }d\eta d\beta } \,\!</math>

The one-sided upper bounds of <math>\eta\,\!</math> is:

::<math> CL=P(\eta \leq \eta _{U})=\int_{0}^{\eta _{U}}f(\eta |Data)d\eta \,\!</math>

The one-sided lower bounds of <math>\eta\,\!</math> is:

::<math> 1-CL=P(\eta \leq \eta _{L})=\int_{0}^{\eta _{L}}f(\eta |Data)d\eta \,\!</math>

The two-sided bounds of <math>\eta\,\!</math> is:

::<math> CL=P(\eta _{L}\leq \eta \leq \eta _{U})=\int_{\eta _{L}}^{\eta _{U}}f(\eta |Data)d\eta \,\!</math>

Same method is used to obtain the bounds of <math>\beta\,\!</math>.

=== Bounds on Reliability ===
::<math> CL=\Pr (R\leq R_{U})=\Pr (\eta \leq T\exp (-\frac{\ln (-\ln R_{U})}{\beta })) \,\!</math>

From the posterior distribution of <math>\eta\,\!</math> we have:

::<math> CL=\dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{T\exp (-\dfrac{\ln (-\ln R_{U})}{\beta })}L(\beta ,\eta )\frac{1}{\beta }\frac{1}{\eta }d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\frac{1}{\beta }\frac{1}{\eta }d\eta d\beta } \,\!</math>

The above equation is solved numerically for <math>{{R}_{U}}\,\!</math>. The same method can be used to calculate the one sided lower bounds and two-sided bounds on reliability.

=== Bounds on Time ===
From [[Confidence Bounds]], we know that:

::<math> CL=\Pr (T\leq T_{U})=\Pr (\eta \leq T_{U}\exp (-\frac{\ln (-\ln R)}{\beta })) \,\!</math>

From the posterior distribution of <math>\eta\,\!</math>, we have:

::<math> CL=\dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{T_{U}\exp (-\dfrac{ \ln (-\ln R)}{\beta })}L(\beta ,\eta )\frac{1}{\beta }\frac{1}{\eta }d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\frac{1}{\beta }\frac{1}{\eta }d\eta d\beta } \,\!</math>

The above equation is solved numerically for <math>{{T}_{U}}\,\!</math>. The same method can be applied to calculate one sided lower bounds and two-sided bounds on time.

Lognormal Confidence Bounds

2023-03-11T00:29:10Z

M Spivey: /* Bounds on Time (Type 1) */

{{template:LDABOOK|10.2|Lognormal Confidence Bounds}}
==Confidence Bounds==
The method used by the application in estimating the different types of confidence bounds for lognormally distributed data is presented in this section. Note that there are closed-form solutions for both the normal and lognormal reliability that can be obtained without the use of the Fisher information matrix. However, these closed-form solutions only apply to complete data. To achieve consistent application across all possible data types, Weibull++ always uses the Fisher matrix in computing confidence intervals. The complete derivations were presented in detail for a general function in [[Confidence Bounds]]. For a discussion on exact confidence bounds for the normal and lognormal, see [[The Normal Distribution]].

===Fisher Matrix Bounds===
====Bounds on the Parameters====
The lower and upper bounds on the mean, <math>{\mu }'\,\!</math>, are estimated from:

::<math>\begin{align}
& \mu _{U}^{\prime }= & {{\widehat{\mu }}^{\prime }}+{{K}_{\alpha }}\sqrt{Var({{\widehat{\mu }}^{\prime }})}\text{ (upper bound),} \\
& \mu _{L}^{\prime }= & {{\widehat{\mu }}^{\prime }}-{{K}_{\alpha }}\sqrt{Var({{\widehat{\mu }}^{\prime }})}\text{ (lower bound)}\text{.}
\end{align}\,\!</math>

For the standard deviation, <math>{\widehat{\sigma}'}\,\!</math>, <math>\ln ({{\widehat{\sigma'}}})\,\!</math> is treated as normally distributed, and the bounds are estimated from:

::<math>\begin{align}
& {{\sigma}_{U}}= & {{\widehat{\sigma'}}}\cdot {{e}^{\tfrac{{{K}_{\alpha }}\sqrt{Var({{\widehat{\sigma'}}})}}{{{\widehat{\sigma'}}}}}}\text{ (upper bound),} \\
& {{\sigma }_{L}}= & \frac{{{\widehat{\sigma'}}}}{{{e}^{\tfrac{{{K}_{\alpha }}\sqrt{Var({{\widehat{\sigma' }}})}}{{{\widehat{\sigma'}}}}}}}\text{ (lower bound),}
\end{align}\,\!</math>

where <math>{{K}_{\alpha }}\,\!</math> is defined by:

::<math>\alpha =\frac{1}{\sqrt{2\pi }}\int_{{{K}_{\alpha }}}^{\infty }{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt=1-\Phi ({{K}_{\alpha }})\,\!</math>

If <math>\delta \,\!</math> is the confidence level, then <math>\alpha =\tfrac{1-\delta }{2}\,\!</math> for the two-sided bounds and <math>\alpha =1-\delta \,\!</math> for the one-sided bounds.

The variances and covariances of <math>{{\widehat{\mu }}^{\prime }}\,\!</math> and <math>{{\widehat{\sigma'}}}\,\!</math> are estimated as follows:

::<math>\left( \begin{matrix}
\widehat{Var}\left( {{\widehat{\mu }}^{\prime }} \right) & \widehat{Cov}\left( {{\widehat{\mu }}^{\prime }},{{\widehat{\sigma'}}} \right) \\
\widehat{Cov}\left( {{\widehat{\mu }}^{\prime }},{{\widehat{\sigma'}}} \right) & \widehat{Var}\left( {{\widehat{\sigma'}}} \right) \\
\end{matrix} \right)=\left( \begin{matrix}
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {{({\mu }')}^{2}}} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial {\mu }'\partial {{\sigma'}}} \\
{} & {} \\
-\tfrac{{{\partial }^{2}}\Lambda }{\partial {\mu }'\partial {{\sigma'}}} & -\tfrac{{{\partial }^{2}}\Lambda }{\partial \sigma'^{2}} \\
\end{matrix} \right)_{{\mu }'={{\widehat{\mu }}^{\prime }},{{\sigma'}}={{\widehat{\sigma'}}}}^{-1}\,\!</math>

where <math>\Lambda \,\!</math> is the log-likelihood function of the lognormal distribution.

====Bounds on Time (Type 1)====
The bounds around time for a given lognormal percentile, or unreliability, are estimated by first solving the reliability equation with respect to time, as follows:

::<math>{t}'({{\widehat{\mu }}^{\prime }},{{\widehat{\sigma' }}})={{\widehat{\mu }}^{\prime }}+z\cdot {{\widehat{\sigma' }}}\,\!</math>

where:

::<math>z={{\Phi }^{-1}}\left[ F({t}') \right]\,\!</math>

and:

::<math>\Phi (z)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{z({t}')}{{e}^{-\tfrac{1}{2}{{z}^{2}}}}dz\,\!</math>

The next step is to calculate the variance of <math>{T}'({{\widehat{\mu }}^{\prime }},{{\widehat{\sigma }}}):\,\!</math>

::<math>\begin{align}
& Var({{{\hat{t}}}^{\prime }})= & {{\left( \frac{\partial {t}'}{\partial {\mu }'} \right)}^{2}}Var({{\widehat{\mu }}^{\prime }})+{{\left( \frac{\partial {t}'}{\partial {{\sigma' }}} \right)}^{2}}Var({{\widehat{\sigma' }}}) \\
& & +2\left( \frac{\partial {t}'}{\partial {\mu }'} \right)\left( \frac{\partial {t}'}{\partial {{\sigma' }}} \right)Cov\left( {{\widehat{\mu }}^{\prime }},{{\widehat{\sigma' }}} \right) \\
& & \\
& Var({{{\hat{t}}}^{\prime }})= & Var({{\widehat{\mu }}^{\prime }})+{{\widehat{z}}^{2}}Var({{\widehat{\sigma' }}})+2\cdot \widehat{z}\cdot Cov\left( {{\widehat{\mu }}^{\prime }},{{\widehat{\sigma' }}} \right)
\end{align}\,\!</math>

The upper and lower bounds are then found by:

::<math>\begin{align}
& t_{U}^{\prime }= & \ln {{t}_{U}}={{{\hat{t}}}^{\prime }}+{{K}_{\alpha }}\sqrt{Var({{{\hat{t}}}^{\prime }})} \\
& t_{L}^{\prime }= & \ln {{t}_{L}}={{{\hat{t}}}^{\prime }}-{{K}_{\alpha }}\sqrt{Var({{{\hat{t}}}^{\prime }})}
\end{align}\,\!</math>

Solving for <math>{{t}_{U}}\,\!</math> and <math>{{t}_{L}}\,\!</math> we get:

::<math>\begin{align}
& {{t}_{U}}= & {{e}^{t_{U}^{\prime }}}\text{ (upper bound),} \\
& {{t}_{L}}= & {{e}^{t_{L}^{\prime }}}\text{ (lower bound)}\text{.}
\end{align}\,\!</math>

====Bounds on Reliability (Type 2)====
The reliability of the lognormal distribution is:

::<math>\hat{R}(t;{{\hat{\mu }}^{'}},{{\hat{\sigma }}^{'}})=\int_{t'}^{\infty }{\frac{1}{{{{\hat{\sigma }}}^{'}}\sqrt{2\pi }}}{{e}^{-\frac{1}{2}{{\left( \frac{x-{{{\hat{\mu }}}^{'}}}{{{{\hat{\sigma }}}^{'}}} \right)}^{2}}}}dx\,\!</math>

where <math>t'=\ln (t)\,\!</math>. Let <math>\hat{z}(x)=\frac{x-{{{\hat{\mu }}}^{'}}}{{{\sigma }^{'}}}\,\!</math>, the above equation then becomes:

::<math>\hat{R}\left( \hat{z}(t') \right)=\int_{\hat{z}(t')}^{\infty }{\frac{1}{\sqrt{2\pi }}}{{e}^{-\frac{1}{2}{{z}^{2}}}}dz\,\!</math>

The bounds on <math>z\,\!</math> are estimated from:

::<math>\begin{align}
& {{z}_{U}}= & \widehat{z}+{{K}_{\alpha }}\sqrt{Var(\widehat{z})} \\
& {{z}_{L}}= & \widehat{z}-{{K}_{\alpha }}\sqrt{Var(\widehat{z})}
\end{align}\,\!</math>

where:

::<math>\begin{align}
& Var(\hat{z})=\left( \frac{\partial {z}}{\partial \mu '} \right)_{\hat{\mu }'}^{2}Var\left( \hat{\mu }' \right)+\left( \frac{\partial {z}}{\partial \sigma '} \right)_{\hat{\sigma }'}^{2}Var\left( \hat{\sigma }' \right) \\
& +2\left( \frac{\partial{z}}{\partial \mu '} \right)_{\hat{\mu }'}^{{}}\left( \frac{\partial {z}}{\partial \sigma '} \right)_{\hat{\sigma }'}^{{}}Cov\left( \hat{\mu }',\hat{\sigma }' \right)
\end{align}\,\!</math>

or:

::<math>Var(\hat{z})=\frac{1}{{{{\hat{\sigma }}}^{'2}}}\left[ Var\left( \hat{\mu }' \right)+{{{\hat{z}}}^{2}}Var\left( \sigma ' \right)+2\cdot \hat{z}\cdot Cov\left( \hat{\mu }',\hat{\sigma }' \right) \right]\,\!</math>

The upper and lower bounds on reliability are:

::<math>\begin{align}
& {{R}_{U}}= & \int_{{{z}_{L}}}^{\infty }\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{z}^{2}}}}dz\text{ (Upper bound)} \\
& {{R}_{L}}= & \int_{{{z}_{U}}}^{\infty }\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{z}^{2}}}}dz\text{ (Lower bound)}
\end{align}\,\!</math>

===Likelihood Ratio Confidence Bounds===
====Bounds on Parameters====
As covered in [[Parameter Estimation]], the likelihood confidence bounds are calculated by finding values for <math>{{\theta }_{1}}\,\!</math> and <math>{{\theta }_{2}}\,\!</math> that satisfy:

::<math>-2\cdot \text{ln}\left( \frac{L({{\theta }_{1}},{{\theta }_{2}})}{L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})} \right)=\chi _{\alpha ;1}^{2}\,\!</math>

This equation can be rewritten as:

::<math>L({{\theta }_{1}},{{\theta }_{2}})=L({{\widehat{\theta }}_{1}},{{\widehat{\theta }}_{2}})\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}\,\!</math>

For complete data, the likelihood formula for the normal distribution is given by:

::<math>L({\mu }',{{\sigma' }})=\underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};{\mu }',{{\sigma' }})=\underset{i=1}{\overset{N}{\mathop \prod }}\,\frac{1}{{{x}_{i}}\cdot {{\sigma' }}\cdot \sqrt{2\pi }}\cdot {{e}^{-\tfrac{1}{2}{{\left( \tfrac{\text{ln}({{x}_{i}})-{\mu }'}{{{\sigma'}}} \right)}^{2}}}}\,\!</math>

where the <math>{{x}_{i}}\,\!</math> values represent the original time-to-failure data. For a given value of <math>\alpha \,\!</math>, values for <math>{\mu }'\,\!</math> and <math>{{\sigma' }}\,\!</math> can be found which represent the maximum and minimum values that satisfy likelihood ratio equation. These represent the confidence bounds for the parameters at a confidence level <math>\delta ,\,\!</math> where <math>\alpha =\delta \,\!</math> for two-sided bounds and <math>\alpha =2\delta -1\,\!</math> for one-sided.
=====Example: LR Bounds on Parameters=====
'''Lognormal Distribution Likelihood Ratio Bound Example (Parameters)'''

Five units are put on a reliability test and experience failures at 45, 60, 75, 90, and 115 hours. Assuming a lognormal distribution, the MLE parameter estimates are calculated to be <math>{{\widehat{\mu }}^{\prime }}=4.2926\,\!</math> and <math>{{\widehat{\sigma'}}}=0.32361.\,\!</math> Calculate the two-sided 75% confidence bounds on these parameters using the likelihood ratio method.

'''Solution'''

The first step is to calculate the likelihood function for the parameter estimates:

::<math>\begin{align}
L({{\widehat{\mu }}^{\prime }},{{\widehat{\sigma' }}})= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};{{\widehat{\mu }}^{\prime }},{{\widehat{\sigma' }}}), \\
= & \underset{i=1}{\overset{N}{\mathop \prod }}\,\frac{1}{{{x}_{i}}\cdot {{\widehat{\sigma' }}}\cdot \sqrt{2\pi }}\cdot {{e}^{-\tfrac{1}{2}{{\left( \tfrac{\text{ln}({{x}_{i}})-{{\widehat{\mu }}^{\prime }}}{{{\widehat{\sigma' }}}} \right)}^{2}}}} \\
L({{\widehat{\mu }}^{\prime }},{{\widehat{\sigma'}}})= & \underset{i=1}{\overset{5}{\mathop \prod }}\,\frac{1}{{{x}_{i}}\cdot 0.32361\cdot \sqrt{2\pi }}\cdot {{e}^{-\tfrac{1}{2}{{\left( \tfrac{\text{ln}({{x}_{i}})-4.2926}{0.32361} \right)}^{2}}}} \\
L({{\widehat{\mu }}^{\prime }},{{\widehat{\sigma'}}})= & 1.115256\times {{10}^{-10}}
\end{align}\,\!</math>

where <math>{{x}_{i}}\,\!</math> are the original time-to-failure data points. We can now rearrange the likelihod ratio equation to the form:

::<math>L({\mu }',{{\sigma' }})-L({{\widehat{\mu }}^{\prime }},{{\widehat{\sigma' }}})\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0\,\!</math>

Since our specified confidence level, <math>\delta \,\!</math>, is 75%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.75;1}^{2}=1.323303.\,\!</math> We can now substitute this information into the equation:

::<math>\begin{align}
& L({\mu }',{{\sigma' }})-L({{\widehat{\mu }}^{\prime }},{{\widehat{\sigma' }}})\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0 \\
& L({\mu }',{{\sigma'}})-1.115256\times {{10}^{-10}}\cdot {{e}^{\tfrac{-1.323303}{2}}}= & 0 \\
& L({\mu }',{{\sigma'}})-5.754703\times {{10}^{-11}}= & 0
\end{align}\,\!</math>

It now remains to find the values of <math>{\mu }'\,\!</math> and <math>{{\sigma'}}\,\!</math> which satisfy this equation. This is an iterative process that requires setting the value of <math>{{\sigma'}}\,\!</math> and finding the appropriate values of <math>{\mu }'\,\!</math>, and vice versa.

The following table gives the values of <math>{\mu }'\,\!</math> based on given values of <math>{{\sigma'}}\,\!</math>.

<center><math>\begin{matrix}
{{\sigma' }} & \mu _{1}^{\prime } & \mu _{2}^{\prime } & {{\sigma' }} & \mu _{1}^{\prime } & \mu _{2}^{\prime } \\
0.24 & 4.2421 & 4.3432 & 0.37 & 4.1145 & 4.4708 \\
0.25 & 4.2115 & 4.3738 & 0.38 & 4.1152 & 4.4701 \\
0.26 & 4.1909 & 4.3944 & 0.39 & 4.1170 & 4.4683 \\
0.27 & 4.1748 & 4.4105 & 0.40 & 4.1200 & 4.4653 \\
0.28 & 4.1618 & 4.4235 & 0.41 & 4.1244 & 4.4609 \\
0.29 & 4.1509 & 4.4344 & 0.42 & 4.1302 & 4.4551 \\
0.30 & 4.1419 & 4.4434 & 0.43 & 4.1377 & 4.4476 \\
0.31 & 4.1343 & 4.4510 & 0.44 & 4.1472 & 4.4381 \\
0.32 & 4.1281 & 4.4572 & 0.45 & 4.1591 & 4.4262 \\
0.33 & 4.1231 & 4.4622 & 0.46 & 4.1742 & 4.4111 \\
0.34 & 4.1193 & 4.4660 & 0.47 & 4.1939 & 4.3914 \\
0.35 & 4.1166 & 4.4687 & 0.48 & 4.2221 & 4.3632 \\
0.36 & 4.1150 & 4.4703 & {} & {} & {} \\
\end{matrix}\,\!</math></center>

These points are represented graphically in the following contour plot:

[[Image:WB.10 lognormal contour plot.png|center|450px| ]]

(Note that this plot is generated with degrees of freedom <math>k=1\,\!</math>, as we are only determining bounds on one parameter. The contour plots generated in Weibull++ are done with degrees of freedom <math>k=2\,\!</math>, for use in comparing both parameters simultaneously.) As can be determined from the table the lowest calculated value for <math>{\mu }'\,\!</math> is 4.1145, while the highest is 4.4708. These represent the two-sided 75% confidence limits on this parameter. Since solutions for the equation do not exist for values of <math>{{\sigma'
}}\,\!</math> below 0.24 or above 0.48, these can be considered the two-sided 75% confidence limits for this parameter. In order to obtain more accurate values for the confidence limits on <math>{{\sigma'}}\,\!</math>, we can perform the same procedure as before, but finding the two values of <math>\sigma \,\!</math> that correspond with a given value of <math>{\mu }'.\,\!</math> Using this method, we find that the 75% confidence limits on <math>{{\sigma'}}\,\!</math> are 0.23405 and 0.48936, which are close to the initial estimates of 0.24 and 0.48.

====Bounds on Time and Reliability====
In order to calculate the bounds on a time estimate for a given reliability, or on a reliability estimate for a given time, the likelihood function needs to be rewritten in terms of one parameter and time/reliability, so that the maximum and minimum values of the time can be observed as the parameter is varied. This can be accomplished by substituting a form of the normal reliability equation into the likelihood function. The normal reliability equation can be written as:

::<math>R=1-\Phi \left( \frac{\text{ln}(t)-{\mu }'}{{{\sigma'}}} \right)\,\!</math>

This can be rearranged to the form:

::<math>{\mu }'=\text{ln}(t)-{{\sigma'}}\cdot {{\Phi }^{-1}}(1-R)\,\!</math>

where <math>{{\Phi }^{-1}}\,\!</math> is the inverse standard normal. This equation can now be substituted into likelihood function to produce a likelihood equation in terms of <math>{{\sigma'}},\,\!</math> <math>t\,\!</math> and <math>R\,\!</math>:

::<math>L({{\sigma'}},t/R)=\underset{i=1}{\overset{N}{\mathop \prod }}\,\frac{1}{{{x}_{i}}\cdot {{\sigma'}}\cdot \sqrt{2\pi }}\cdot {{e}^{-\tfrac{1}{2}{{\left( \tfrac{\text{ln}({{x}_{i}})-\left( \text{ln}(t)-{{\sigma'}}\cdot {{\Phi }^{-1}}(1-R) \right)}{{{\sigma'}}} \right)}^{2}}}}\,\!</math>

The unknown variable <math>t/R\,\!</math> depends on what type of bounds are being determined. If one is trying to determine the bounds on time for a given reliability, then <math>R\,\!</math> is a known constant and <math>t\,\!</math> is the unknown variable. Conversely, if one is trying to determine the bounds on reliability for a given time, then <math>t\,\!</math> is a known constant and <math>R\,\!</math> is the unknown variable. Either way, the above equation can be used to solve the likelihood ratio equation for the values of interest.

=====Example: LR Bounds on Time=====
'''Lognormal Distribution Likelihood Ratio Bound Example (Time)'''

For the same data set given for the [[The_Lognormal_Distribution#Example:_LR_Bounds_on_Parameters|parameter bounds example]], determine the two-sided 75% confidence bounds on the time estimate for a reliability of 80%. The ML estimate for the time at <math>R(t)=80%\,\!</math> is 55.718.

'''Solution'''

In this example, we are trying to determine the two-sided 75% confidence bounds on the time estimate of 55.718. This is accomplished by substituting <math>R=0.80\,\!</math> and <math>\alpha =0.75\,\!</math> into the likelihood function, and varying <math>{{\sigma' }}\,\!</math> until the maximum and minimum values of <math>t\,\!</math> are found. The following table gives the values of <math>t\,\!</math> based on given values of <math>{{\sigma' }}\,\!</math>.

<center><math>\begin{matrix}
{{\sigma' }} & {{t}_{1}} & {{t}_{2}} & {{\sigma' }} & {{t}_{1}} & {{t}_{2}} \\
0.24 & 56.832 & 62.879 & 0.37 & 44.841 & 64.031 \\
0.25 & 54.660 & 64.287 & 0.38 & 44.494 & 63.454 \\
0.26 & 53.093 & 65.079 & 0.39 & 44.200 & 62.809 \\
0.27 & 51.811 & 65.576 & 0.40 & 43.963 & 62.093 \\
0.28 & 50.711 & 65.881 & 0.41 & 43.786 & 61.304 \\
0.29 & 49.743 & 66.041 & 0.42 & 43.674 & 60.436 \\
0.30 & 48.881 & 66.085 & 0.43 & 43.634 & 59.481 \\
0.31 & 48.106 & 66.028 & 0.44 & 43.681 & 58.426 \\
0.32 & 47.408 & 65.883 & 0.45 & 43.832 & 57.252 \\
0.33 & 46.777 & 65.657 & 0.46 & 44.124 & 55.924 \\
0.34 & 46.208 & 65.355 & 0.47 & 44.625 & 54.373 \\
0.35 & 45.697 & 64.983 & 0.48 & 45.517 & 52.418 \\
0.36 & 45.242 & 64.541 & {} & {} & {} \\
\end{matrix}\,\!</math></center>

This data set is represented graphically in the following contour plot:

[[Image:WB.10 time vs sigma.png|center|450px| ]]

As can be determined from the table, the lowest calculated value for <math>t\,\!</math> is 43.634, while the highest is 66.085. These represent the two-sided 75% confidence limits on the time at which reliability is equal to 80%.

=====Example: LR Bounds on Reliability=====
'''Lognormal Distribution Likelihood Ratio Bound Example (Reliability)'''

For the same data set given above for the [[The_Lognormal_Distribution#Example:_LR_Bounds_on_Parameters|parameter bounds example]], determine the two-sided 75% confidence bounds on the reliability estimate for <math>t=65\,\!</math>. The ML estimate for the reliability at <math>t=65\,\!</math> is 64.261%.

'''Solution'''

In this example, we are trying to determine the two-sided 75% confidence bounds on the reliability estimate of 64.261%. This is accomplished by substituting <math>t=65\,\!</math> and <math>\alpha =0.75\,\!</math> into the likelihood function, and varying <math>{{\sigma'}}\,\!</math> until the maximum and minimum values of <math>R\,\!</math> are found. The following table gives the values of <math>R\,\!</math> based on given values of <math>{{\sigma' }}\,\!</math>.

<center><math>\begin{matrix}
{{\sigma'}} & {{R}_{1}} & {{R}_{2}} & {{\sigma'}} & {{R}_{1}} & {{R}_{2}} \\
0.24 & 61.107% & 75.910% & 0.37 & 43.573% & 78.845% \\
0.25 & 55.906% & 78.742% & 0.38 & 43.807% & 78.180% \\
0.26 & 55.528% & 80.131% & 0.39 & 44.147% & 77.448% \\
0.27 & 50.067% & 80.903% & 0.40 & 44.593% & 76.646% \\
0.28 & 48.206% & 81.319% & 0.41 & 45.146% & 75.767% \\
0.29 & 46.779% & 81.499% & 0.42 & 45.813% & 74.802% \\
0.30 & 45.685% & 81.508% & 0.43 & 46.604% & 73.737% \\
0.31 & 44.857% & 81.387% & 0.44 & 47.538% & 72.551% \\
0.32 & 44.250% & 81.159% & 0.45 & 48.645% & 71.212% \\
0.33 & 43.827% & 80.842% & 0.46 & 49.980% & 69.661% \\
0.34 & 43.565% & 80.446% & 0.47 & 51.652% & 67.789% \\
0.35 & 43.444% & 79.979% & 0.48 & 53.956% & 65.299% \\
0.36 & 43.450% & 79.444% & {} & {} & {} \\
\end{matrix}\,\!</math></center>

This data set is represented graphically in the following contour plot:

[[Image:WB.10 reliability v sigma.png|center|450px| ]]

As can be determined from the table, the lowest calculated value for <math>R\,\!</math> is 43.444%, while the highest is 81.508%. These represent the two-sided 75% confidence limits on the reliability at <math>t=65\,\!</math>.

===Bayesian Confidence Bounds===
====Bounds on Parameters====
From [[Parameter Estimation]], we know that the marginal distribution of parameter <math>{\mu }'\,\!</math> is:

::<math>\begin{align}
f({\mu }'|Data)= & \int_{0}^{\infty }f({\mu }',{{\sigma'}}|Data)d{{\sigma'}} \\
= & \frac{\int_{0}^{\infty }L(Data|{\mu }',{{\sigma'}})\varphi ({\mu }')\varphi ({{\sigma'}})d{{\sigma'}}}{\int_{0}^{\infty }\int_{-\infty }^{\infty }L(Data|{\mu }',{{\sigma'}})\varphi ({\mu }')\varphi ({{\sigma'}})d{\mu }'d{{\sigma'}}}
\end{align}\,\!</math>

where:
::<math>\varphi ({{\sigma '}})\,\!</math> is <math>\tfrac{1}{{{\sigma '}}}\,\!</math>, non-informative prior of <math>{{\sigma '}}\,\!</math>.

<math>\varphi ({\mu }')\,\!</math> is an uniform distribution from - <math>\infty \,\!</math> to + <math>\infty \,\!</math>, non-informative prior of <math>{\mu }'\,\!</math>.
With the above prior distributions, <math>f({\mu }'|Data)\,\!</math> can be rewritten as:

::<math>f({\mu }'|Data)=\frac{\int_{0}^{\infty }L(Data|{\mu }',{{\sigma '}})\tfrac{1}{{{\sigma '}}}d{{\sigma '}}}{\int_{0}^{\infty }\int_{-\infty }^{\infty }L(Data|{\mu }',{{\sigma '}})\tfrac{1}{{{\sigma '}}}d{\mu }'d{{\sigma '}}}\,\!</math>

The one-sided upper bound of <math>{\mu }'\,\!</math> is:

::<math>CL=P({\mu }'\le \mu _{U}^{\prime })=\int_{-\infty }^{\mu _{U}^{\prime }}f({\mu }'|Data)d{\mu }'\,\!</math>

The one-sided lower bound of <math>{\mu }'\,\!</math> is:

::<math>1-CL=P({\mu }'\le \mu _{L}^{\prime })=\int_{-\infty }^{\mu _{L}^{\prime }}f({\mu }'|Data)d{\mu }'\,\!</math>

The two-sided bounds of <math>{\mu }'\,\!</math> is:

::<math>CL=P(\mu _{L}^{\prime }\le {\mu }'\le \mu _{U}^{\prime })=\int_{\mu _{L}^{\prime }}^{\mu _{U}^{\prime }}f({\mu }'|Data)d{\mu }'\,\!</math>

The same method can be used to obtained the bounds of <math>{{\sigma '}}\,\!</math>.

====Bounds on Time (Type 1)====
The reliable life of the lognormal distribution is:

::<math>\begin{align}
\ln T={\mu }'+{{\sigma '}}{{\Phi }^{-1}}(1-R)
\end{align}\,\!</math>

The one-sided upper on time bound is given by:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\ln t\le \ln {{t}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,({\mu }'+{{\sigma '}}{{\Phi }^{-1}}(1-R)\le \ln {{t}_{U}})\,\!</math>

The above equation can be rewritten in terms of <math>{\mu }'\,\!</math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,({\mu }'\le \ln {{t}_{U}}-{{\sigma '}}{{\Phi }^{-1}}(1-R)\,\!</math>

From the posterior distribution of <math>{\mu }'\,\!</math> get:

::<math>CL=\frac{\int_{0}^{\infty }\int_{-\infty }^{\ln {{t}_{U}}-{{\sigma'}}{{\Phi }^{-1}}(1-R)}L({{\sigma '}},{\mu }')\tfrac{1}{{{\sigma '}}}d{\mu }'d{{\sigma '}}}{\int_{0}^{\infty }\int_{-\infty }^{\infty }L({{\sigma '}},{\mu }')\tfrac{1}{{{\sigma '}}}d{\mu }'d{{\sigma '}}}\,\!</math>

The above equation is solved w.r.t. <math>{{t}_{U}}.\,\!</math> The same method can be applied for one-sided lower bounds and two-sided bounds on Time.

====Bounds on Reliability (Type 2)====

The one-sided upper bound on reliability is given by:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(R\le {{R}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,({\mu }'\le \ln t-{{\sigma '}}{{\Phi }^{-1}}(1-{{R}_{U}}))\,\!</math>

From the posterior distribution of <math>{\mu }'\,\!</math> is:

::<math>CL=\frac{\int_{0}^{\infty }\int_{-\infty }^{\ln t-{{\sigma '}}{{\Phi }^{-1}}(1-{{R}_{U}})}L({{\sigma'}},{\mu }')\tfrac{1}{{{\sigma'}}}d{\mu }'d{{\sigma '}}}{\int_{0}^{\infty }\int_{-\infty }^{\infty }L({{\sigma '}},{\mu }')\tfrac{1}{{{\sigma '}}}d{\mu }'d{{\sigma '}}}\,\!</math>

The above equation is solved w.r.t. <math>{{R}_{U}}.\,\!</math> The same method is used to calculate the one-sided lower bounds and two-sided bounds on Reliability.

====Example: Bayesian Bounds====
'''Lognormal Distribution Bayesian Bound Example (Parameters)'''

Determine the two-sided 90% Bayesian confidence bounds on the lognormal parameter estimates for the data given next:

{| border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|- align="center"
!Data Point Index
!State End Time
|- align="center"
|1 ||2
|- align="center"
|2 ||5
|- align="center"
|3 ||11
|- align="center"
|4 ||23
|- align="center"
|5 ||29
|- align="center"
|6 ||37
|- align="center"
|7 ||43
|- align="center"
|8 ||59
|}

'''Solution'''

The data points are entered into a times-to-failure data sheet. The lognormal distribution is selected under Distributions. The Bayesian confidence bounds method only applies for the MLE analysis method, therefore, Maximum Likelihood (MLE) is selected under Analysis Method and Use Bayesian is selected under the Confidence Bounds Method in the Analysis tab.

The two-sided 90% Bayesian confidence bounds on the lognormal parameter are obtained using the QCP and clicking on the Calculate Bounds button in the Parameter Bounds tab as follows:

[[Image:Lognormal Distribution Example 8 QCP.png|center|650px| ]]

[[Image:Lognormal Distribution Example 8 Parameter Bounds.png|center|500px| ]]

Parameter Estimation

2022-12-17T00:07:10Z

M Spivey: /* Linearizing the Unreliability Function */

{{template:LDABOOK|4|Parameter Estimation}}
The term ''parameter estimation'' refers to the process of using sample data (in reliability engineering, usually times-to-failure or success data) to estimate the parameters of the selected distribution. Several parameter estimation methods are available. This section presents an overview of the available methods used in life data analysis. More specifically, we start with the relatively simple method of Probability Plotting and continue with the more sophisticated methods of Rank Regression (or Least Squares), Maximum Likelihood Estimation and Bayesian Estimation Methods.

=Probability Plotting=
The least mathematically intensive method for parameter estimation is the method of probability plotting. As the term implies, probability plotting involves a physical plot of the data on specially constructed ''probability plotting paper''. This method is easily implemented by hand, given that one can obtain the appropriate probability plotting paper.

The method of probability plotting takes the ''cdf'' of the distribution and attempts to linearize it by employing a specially constructed paper. The following sections illustrate the steps in this method using the 2-parameter Weibull distribution as an example. This includes:

*Linearize the unreliability function
*Construct the probability plotting paper
*Determine the X and Y positions of the plot points

And then using the plot to read any particular time or reliability/unreliability value of interest.

==Linearizing the Unreliability Function==

In the case of the 2-parameter Weibull, the ''cdf'' (also the unreliability <math>Q(t)\,\!</math>) is given by:

::<math>F(t)=Q(t)=1-{e^{-\left(\tfrac{t}{\eta}\right)^{\beta}}}\,\!</math>

This function can then be linearized (i.e., put in the common form of <math>y = m'x + b\,\!</math> format) as follows''':'''

::<math>\begin{align}
Q(t)= & 1-{e^{-\left(\tfrac{t}{\eta}\right)^{\beta}}} \\
\ln (1-Q(t))= & \ln \left[ {e^{-\left(\tfrac{t}{\eta}\right)^{\beta}}} \right] \\
\ln (1-Q(t))=& -\left(\tfrac{t}{\eta}\right)^{\beta} \\
\ln ( -\ln (1-Q(t)))= & \beta \left(\ln \left( \frac{t}{\eta }\right)\right) \\
\ln \left( \ln \left( \frac{1}{1-Q(t)}\right) \right) = & \beta\ln{ t} -\beta\ln{\eta} \\
\end{align}\,\!</math>

Then by setting:

::<math>y=\ln \left( \ln \left( \frac{1}{1-Q(t)} \right) \right)\,\!</math>

and:

::<math>x=\ln \left( t \right)\,\!</math>

the equation can then be rewritten as:

::<math>y=\beta x-\beta \ln \left( \eta \right)\,\!</math>

which is now a linear equation with a slope of:

::<math>\begin{align}
m = \beta
\end{align}\,\!</math>

and an intercept of:

::<math>b=-\beta \cdot \ln(\eta)\,\!</math>

==Constructing the Paper==
The next task is to construct the Weibull probability plotting paper with the appropriate y and x axes. The x-axis transformation is simply logarithmic. The y-axis is a bit more complex, requiring a double log reciprocal transformation, or:

::<math>y=\ln \left(\ln \left( \frac{1}{1-Q(t)} ) \right) \right)\,\!</math>

where <math>Q(t)\,\!</math> is the unreliability.

Such papers have been created by different vendors and are called ''probability plotting papers''. ReliaSoft's reliability engineering resource website at www.weibull.com has different plotting papers available for [http://www.weibull.com/GPaper/index.htm download].

[[Image:WeibullPaper2C.png|center|450px]]

To illustrate, consider the following probability plot on a slightly different type of Weibull probability paper.

[[Image:different_weibull_paper.png|center|450px]]

This paper is constructed based on the mentioned y and x transformations, where the y-axis represents unreliability and the x-axis represents time. Both of these values must be known for each time-to-failure point we want to plot.

Then, given the <math>y\,\!</math> and <math>x\,\!</math> value for each point, the points can easily be put on the plot. Once the points have been placed on the plot, the best possible straight line is drawn through these points. Once the line has been drawn, the slope of the line can be obtained (some probability papers include a slope indicator to simplify this calculation). This is the parameter <math>\beta\,\!</math>, which is the value of the slope. To determine the scale parameter, <math>\eta\,\!</math> (also called the ''characteristic life''), one reads the time from the x-axis corresponding to <math>Q(t)=63.2%\,\!</math>.

Note that at:

::<math>\begin{align}
Q(t=\eta)= & 1-{{e}^{-{{\left( \tfrac{t}{\eta } \right)}^{\beta }}}} \\
= & 1-{{e}^{-1}} \\
= & 0.632 \\
= & 63.2%
\end{align}\,\!</math>

Thus, if we enter the ''y'' axis at <math>Q(t)=63.2%\,\!</math>, the corresponding value of <math>t\,\!</math> will be equal to <math>\eta\,\!</math>. Thus, using this simple methodology, the parameters of the Weibull distribution can be estimated.

==Determining the X and Y Position of the Plot Points==
The points on the plot represent our data or, more specifically, our times-to-failure data. If, for example, we tested four units that failed at 10, 20, 30 and 40 hours, then we would use these times as our ''x'' values or time values.

Determining the appropriate ''y'' plotting positions, or the unreliability values, is a little more complex. To determine the ''y'' plotting positions, we must first determine a value indicating the corresponding unreliability for that failure. In other words, we need to obtain the cumulative percent failed for each time-to-failure. For example, the cumulative percent failed by 10 hours may be 25%, by 20 hours 50%, and so forth. This is a simple method illustrating the idea. The problem with this simple method is the fact that the 100% point is not defined on most probability plots; thus, an alternative and more robust approach must be used. The most widely used method of determining this value is the method of obtaining the ''median rank'' for each failure, as discussed next.

===Median Ranks ===
The Median Ranks method is used to obtain an estimate of the unreliability for each failure. The median rank is the value that the true probability of failure, <math>Q({{T}_{j}})\,\!</math>, should have at the <math>{{j}^{th}}\,\!</math> failure out of a sample of <math>N\,\!</math> units at the 50% confidence level.

The rank can be found for any percentage point, <math>P\,\!</math>, greater than zero and less than one, by solving the cumulative binomial equation for <math>Z\,\!</math>. This represents the rank, or unreliability estimate, for the <math>{{j}^{th}}\,\!</math> failure in the following equation for the cumulative binomial:

::<math>P=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}\,\!</math>

where <math>N\,\!</math> is the sample size and <math>j\,\!</math> the order number.

The median rank is obtained by solving this equation for <math>Z\,\!</math> at <math>P = 0.50\,\!</math>:

::<math>0.50=\underset{k=j}{\overset{N}{\mathop \sum }}\,\left( \begin{matrix}
N \\
k \\
\end{matrix} \right){{Z}^{k}}{{\left( 1-Z \right)}^{N-k}}\,\!</math>

For example, if <math>N=4\,\!</math> and we have four failures, we would solve the median rank equation for the value of <math>Z\,\!</math> four times; once for each failure with <math>j= 1, 2, 3 \text{ and }4\,\!</math>. This result can then be used as the unreliability estimate for each failure or the <math>y\,\!</math> plotting position. (See also [[The Weibull Distribution|The Weibull Distribution]] for a step-by-step example of this method.) The solution of cumulative binomial equation for <math>Z\,\!</math> requires the use of numerical methods.

===Beta and F Distributions Approach===
A more straightforward and easier method of estimating median ranks is by applying two transformations to the cumulative binomial equation, first to the beta distribution and then to the F distribution, resulting in [[Appendix:_Life_Data_Analysis_References|[12, 13]]]:

::<math>\begin{array}{*{35}{l}}
MR & = & \tfrac{1}{1+\tfrac{N-j+1}{j}{{F}_{0.50;m;n}}} \\
m & = & 2(N-j+1) \\
n & = & 2j \\
\end{array}\,\!</math>

where <math>{{F}_{0.50;m;n}}\,\!</math> denotes the <math>F\,\!</math> distribution at the 0.50 point, with <math>m\,\!</math> and <math>n\,\!</math> degrees of freedom, for failure <math>j\,\!</math> out of <math>N\,\!</math> units.

=== Benard's Approximation for Median Ranks ===
Another quick, and less accurate, approximation of the median ranks is also given by:

::<math>MR = \frac{{j - 0.3}}{{N + 0.4}}\,\!</math>

This approximation of the median ranks is also known as ''Benard's approximation''.

===Kaplan-Meier===
The Kaplan-Meier estimator (also known as the ''product limit estimator'') is used as an alternative to the median ranks method for calculating the estimates of the unreliability for probability plotting purposes. The equation of the estimator is given by:

::<math>\widehat{F}({{t}_{i}})=1-\underset{j=1}{\overset{i}{\mathop \prod }}\,\frac{{{n}_{j}}-{{r}_{j}}}{{{n}_{j}}},\text{ }i=1,...,m\,\!</math>

where:

::<math>\begin{align}
m = & {\text{total number of data points}} \\
n = & {\text{the total number of units}} \\
{n_i} = & n - \sum_{j = 0}^{i - 1}{s_j} - \sum_{j = 0}^{i - 1}{r_j}, \text{i = 1,...,m }\\
{r_j} = & {\text{ number of failures in the }}{j^{th}}{\text{ data group, and}} \\
{s_j} = & {\text{number of surviving units in the }}{j^{th}}{\text{ data group}} \\
\end{align}
\,\!</math>

== Probability Plotting Example ==
This same methodology can be applied to other distributions with ''cdf'' equations that can be linearized. Different probability papers exist for each distribution, because different distributions have different ''cdf'' equations. ReliaSoft's software tools automatically create these plots for you. Special scales on these plots allow you to derive the parameter estimates directly from the plots, similar to the way <math>\beta\,\!</math> and <math>\eta\,\!</math> were obtained from the Weibull probability plot. The following example demonstrates the method again, this time using the 1-parameter exponential distribution.

{{:Probability Plotting Example}}

== Comments on the Probability Plotting Method ==
Besides the most obvious drawback to probability plotting, which is the amount of effort required, manual probability plotting is not always consistent in the results. Two people plotting a straight line through a set of points will not always draw this line the same way, and thus will come up with slightly different results. This method was used primarily before the widespread use of computers that could easily perform the calculations for more complicated parameter estimation methods, such as the least squares and maximum likelihood methods.

= Least Squares (Rank Regression) =
Using the idea of probability plotting, regression analysis mathematically fits the best straight line to a set of points, in an attempt to estimate the parameters. Essentially, this is a mathematically based version of the probability plotting method discussed previously.

The method of linear least squares is used for all regression analysis performed by Weibull++, except for the cases of the 3-parameter Weibull, mixed Weibull, gamma and generalized gamma distributions, where a non-linear regression technique is employed. The terms ''linear regression'' and ''least squares'' are used synonymously in this reference. In Weibull++, the term ''rank regression'' is used instead of least squares, or linear regression, because the regression is performed on the rank values, more specifically, the median rank values (represented on the y-axis). The method of least squares requires that a straight line be fitted to a set of data points, such that the sum of the squares of the distance of the points to the fitted line is minimized. This minimization can be performed in either the vertical or horizontal direction. If the regression is on <math>X\,\!</math>, then the line is fitted so that the horizontal deviations from the points to the line are minimized. If the regression is on Y, then this means that the distance of the vertical deviations from the points to the line is minimized. This is illustrated in the following figure.

[[Image:minimizingdistance.png|center|500px]]

=== Rank Regression on Y ===
Assume that a set of data pairs <math>({{x}_{1}},{{y}_{1}})\,\!</math>, <math>({{x}_{2}},{{y}_{2}})\,\!</math>,..., <math>({{x}_{N}},{{y}_{N}})\,\!</math> were obtained and plotted, and that the <math>x\,\!</math>-values are known exactly. Then, according to the ''least squares principle,'' which minimizes the vertical distance between the data points and the straight line fitted to the data, the best fitting straight line to these data is the straight line <math>y=\hat{a}+\hat{b}x\,\!</math> (where the recently introduced (<math>\hat{ }\,\!</math>) symbol indicates that this value is an estimate) such that:

::<math>\sum\limits_{i=1}^{N}{{{\left( \hat{a}+\hat{b}{{x}_{i}}-{{y}_{i}} \right)}^{2}}=\min \sum\limits_{i=1}^{N}{{{\left( a+b{{x}_{i}}-{{y}_{i}} \right)}^{2}}}}\,\!</math>

and where <math>\hat{a}\,\!</math> and <math>\hat b\,\!</math> are the ''least squares estimates'' of <math>a\,\!</math> and <math>b\,\!</math>, and <math>N\,\!</math> is the number of data points. These equations are minimized by estimates of <math>\widehat a\,\!</math> and <math>\widehat{b}\,\!</math> such that:

::<math>\hat{a}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}-\hat{b}\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}}{N}=\bar{y}-\hat{b}\bar{x}\,\!</math>

and:

::<math>\hat{b}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,x_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}} \right)}^{2}}}{N}}\,\!</math>

=== Rank Regression on X ===
Assume that a set of data pairs .., <math>({{x}_{2}},{{y}_{2}})\,\!</math>,..., <math>({{x}_{N}},{{y}_{N}})\,\!</math> were obtained and plotted, and that the y-values are known exactly. The same least squares principle is applied, but this time, minimizing the horizontal distance between the data points and the straight line fitted to the data. The best fitting straight line to these data is the straight line <math>x=\widehat{a}+\widehat{b}y\,\!</math> such that:

::<math>\underset{i=1}{\overset{N}{\mathop \sum }}\,{{(\widehat{a}+\widehat{b}{{y}_{i}}-{{x}_{i}})}^{2}}=min(a,b)\underset{i=1}{\overset{N}{\mathop \sum }}\,{{(a+b{{y}_{i}}-{{x}_{i}})}^{2}}\,\!</math>

Again, <math>\widehat{a}\,\!</math> and <math>\widehat b\,\!</math> are the least squares estimates of and <math>b\,\!</math>, and <math>N\,\!</math> is the number of data points. These equations are minimized by estimates of <math>\widehat a\,\!</math> and <math>\widehat{b}\,\!</math> such that:

::<math>\hat{a}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}}{N}-\hat{b}\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}=\bar{x}-\hat{b}\bar{y}\,\!</math>

:and:

::<math>\widehat{b}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,y_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}} \right)}^{2}}}{N}}\,\!</math>

The corresponding relations for determining the parameters for specific distributions (i.e., Weibull, exponential, etc.), are presented in the chapters covering that distribution.

=== Correlation Coefficient ===
The correlation coefficient is a measure of how well the linear regression model fits the data and is usually denoted by <math>\rho\,\!</math>. In the case of life data analysis, it is a measure for the strength of the linear relation (correlation) between the median ranks and the data. The population correlation coefficient is defined as follows:

::<math>\rho =\frac{{{\sigma }_{xy}}}{{{\sigma }_{x}}{{\sigma }_{y}}}\,\!</math>

where <math>{{\sigma}_{xy}} = \,\!</math> covariance of <math>x\,\!</math> and <math>y\,\!</math>, <math>{{\sigma}_{x}} = \,\!</math> standard deviation of <math>x\,\!</math>, and <math>{{\sigma}_{y}} = \,\!</math> standard deviation of <math>y\,\!</math>.

The estimator of <math>\rho\,\!</math> is the sample correlation coefficient, <math>\hat{\rho }\,\!</math>, given by:

::<math>\hat{\rho }=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}}{\sqrt{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,x_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}} \right)}^{2}}}{N} \right)\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,y_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}} \right)}^{2}}}{N} \right)}}\,\!</math>

The range of <math>\hat \rho \,\!</math> is <math>-1\le \hat{\rho }\le 1\,\!</math>.

[[Image:correlationcoeffficient.png|center|600px]]

The closer the value is to <math>\pm 1\,\!</math>, the better the linear fit. Note that +1 indicates a perfect fit (the paired values (<math>{{x}_{i}},{{y}_{i}}\,\!</math>) lie on a straight line) with a positive slope, while -1 indicates a perfect fit with a negative slope. A correlation coefficient value of zero would indicate that the data are randomly scattered and have no pattern or correlation in relation to the regression line model.

===Comments on the Least Squares Method===
The least squares estimation method is quite good for functions that can be linearized. For these distributions, the calculations are relatively easy and straightforward, having closed-form solutions that can readily yield an answer without having to resort to numerical techniques or tables. Furthermore, this technique provides a good measure of the goodness-of-fit of the chosen distribution in the correlation coefficient. Least squares is generally best used with data sets containing complete data, that is, data consisting only of single times-to-failure with no censored or interval data. (See [[Life Data Classification]] for information about the different data types, including complete, left censored, right censored (or suspended) and interval data.)

''See also:''

*[[Least Squares/Rank Regression Equations]]
*[[Appendix:_Special_Analysis_Methods|Grouped Data Analysis]]

=Rank Methods for Censored Data=
All available data should be considered in the analysis of times-to-failure data. This includes the case when a particular unit in a sample has been removed from the test prior to failure. An item, or unit, which is removed from a reliability test prior to failure, or a unit which is in the field and is still operating at the time the reliability of these units is to be determined, is called a ''suspended item ''or ''right censored observation ''or ''right censored'' data point''. ''Suspended items analysis would also be considered when:

#We need to make an analysis of the available results before test completion.
#The failure modes which are occurring are different than those anticipated and such units are withdrawn from the test.
#We need to analyze a single mode and the actual data set comprises multiple modes.
#A ''warranty analysis'' is to be made of all units in the field (non-failed and failed units). The non-failed units are considered to be suspended items (or right censored).

This section describes the rank methods that are used in both probability plotting and least squares (rank regression) to handle censored data. This includes:

*The rank adjustment method for right censored (suspension) data.
*ReliaSoft's alternative ranking method for censored data including left censored, right censored, and interval data.
=== Rank Adjustment Method for Right Censored Data ===
When using the probability plotting or least squares (rank regression) method for data sets where some of the units did not fail, or were suspended, we need to adjust their probability of failure, or unreliability. As discussed before, estimates of the unreliability for complete data are obtained using the median ranks approach. The following methodology illustrates how adjusted median ranks are computed to account for right censored data. To better illustrate the methodology, consider the following example in Kececioglu [[Appendix:_Life_Data_Analysis_References| [20]]] where five items are tested resulting in three failures and two suspensions.

{| border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|-
! Item Number (Position)
! Failure (F) or Suspension (S)
! Life of item, hr
|- align="center"
| 1
| <math>{{F}_{1}}\,\!</math>
| 5,100
|- align="center"
| 2
| <math>{{S}_{1}}\,\!</math>
| 9,500
|- align="center"
| 3
| <math>{{F}_{2}}\,\!</math>
| 15,000
|- align="center"
| 4
| <math>{{S}_{2}}\,\!</math>
| 22,000
|- align="center"
| 5
| <math>{{F}_{3}}\,\!</math>
| 40,000
|}

 

The methodology for plotting suspended items involves adjusting the rank positions and plotting the data based on new positions, determined by the location of the suspensions. If we consider these five units, the following methodology would be used: The first item must be the first failure; hence, it is assigned failure order number <math>j = 1\,\!</math>. The actual failure order number (or position) of the second failure, <math>{{F}_{2}}\,\!</math> is in doubt. It could either be in position 2 or in position 3. Had <math>{{S}_{1}}\,\!</math> not been withdrawn from the test at 9,500 hours, it could have operated successfully past 15,000 hours, thus placing <math>{{F}_{2}}\,\!</math> in position 2. Alternatively, <math>{{S}_{1}}\,\!</math> could also have failed before 15,000 hours, thus placing <math>{{F}_{2}}\,\!</math> in position 3. In this case, the failure order number for <math>{{F}_{2}}\,\!</math> will be some number between 2 and 3. To determine this number, consider the following:

We can find the number of ways the second failure can occur in either order number 2 (position 2) or order number 3 (position 3). The possible ways are listed next.

 

{| border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|-
! style="text-align: center" colspan="6" | <math>{{F}_{2}}\,\!</math> in Position 2
| style="text: align:center" rowspan="7" | OR
! style="text-align: center" colspan="2" | <math>{{F}_{2}}\,\!</math> in Position 3
|- align="center"
| 1
| 2
| 3
| 4
| 5
| 6
| 1
| 2
|- align="center"
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>
|- align="center"
| <math>{{F}_{2}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
|- align="center"
| <math>{{S}_{1}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
|- align="center"
| <math>{{S}_{2}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
|- align="center"
| <math>{{F}_{3}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
|}

 

It can be seen that <math>{{F}_{2}}\,\!</math> can occur in the second position six ways and in the third position two ways. The most probable position is the average of these possible ways, or the ''mean order number'' ( MON ), given by:

::<math>{{F}_{2}}=MO{{N}_{2}}=\frac{(6\times 2)+(2\times 3)}{6+2}=2.25\,\!</math>

 Using the same logic on the third failure, it can be located in position numbers 3, 4 and 5 in the possible ways listed next.

{| border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|-
! style="text-align: center" colspan="2" | <math>{{F}_{3}}\,\!</math> in Position 3
| style="text-align: center" rowspan="7" | OR
! style="text-align: center" colspan="3" | <math>{{F}_{3}}\,\!</math> in Position 4
| style="text-align: center" rowspan="7" | OR
! style="text-align: center" colspan="3" | <math>{{F}_{3}}\,\!</math> in Position 5
|-
| 1
| 2
| 1
| 2
| 3
| 1
| 2
| 3
|-
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>>
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>
| <math>{{F}_{1}}\,\!</math>
|-
| <math>{{F}_{2}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
|-
| <math>{{F}_{3}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
| <math>{{F}_{2}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
|-
| <math>{{S}_{1}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
|-
| <math>{{S}_{2}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
| <math>{{S}_{2}}\,\!</math>
| <math>{{S}_{1}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
| <math>{{F}_{3}}\,\!</math>
|}

 Then, the mean order number for the third failure, (item 5) is:
 

::<math>MO{{N}_{3}}=\frac{(2\times 3)+(3\times 4)+(3\times 5)}{2+3+3}=4.125\,\!</math>

 Once the mean order number for each failure has been established, we obtain the median rank positions for these failures at their mean order number. Specifically, we obtain the median rank of the order numbers 1, 2.25 and 4.125 out of a sample size of 5, as given next.

{| border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|-
! style="text-align: center" colspan="3" | Plotting Positions for the Failures (Sample Size=5)
|- align="center"
! Failure Number
! MON
! Median Rank Position(%)
|- align="center"
| 1:<math>{{F}_{1}}\,\!</math>
| 1
| 13%
|- align="center"
| 2:<math>{{F}_{2}}\,\!</math>
| 2.25
| 36%
|- align="center"
| 3:<math>{{F}_{3}}\,\!</math>
| 4.125
| 71%
|}

Once the median rank values have been obtained, the probability plotting analysis is identical to that presented before. As you might have noticed, this methodology is rather laborious. Other techniques and shortcuts have been developed over the years to streamline this procedure. For more details on this method, see Kececioglu [[Appendix:_Life_Data_Analysis_References|[20]]]. Here, we will introduce one of these methods. This method calculates MON using an increment, ''I'', which is defined by:

:: <math>{{I}_{i}}=\frac{N+1-PMON}{1+NIBPSS}</math>

Where
* ''N''= the sample size, or total number of items in the test
* ''PMON'' = previous mean order number
* ''NIBPSS'' = the number of items beyond the present suspended set. It is the number of units (including all the failures and suspensions) at the current failure time.
* ''i'' = the ith failure item

MON is given as:

:: <math>MO{{N}_{i}}=MO{{N}_{i-1}}+{{I}_{i}}</math>

Let's calculate the previous example using the method.

For F1:

::<math>MO{{N}_{1}}=MO{{N}_{0}}+{{I}_{1}}=\frac{5+1-0}{1+5}=1</math>

For F2:
::<math>MO{{N}_{2}}=MO{{N}_{1}}+{{I}_{2}}=1+\frac{5+1-1}{1+3}=2.25</math>

For F3:
::<math>MO{{N}_{3}}=MO{{N}_{2}}+{{I}_{3}}=2.25+\frac{5+1-2.25}{1+1}=4.125</math>

The MON obtained for each failure item via this method is same as from the first method, so the median rank values will also be the same.

For Grouped data, the increment <math>{{I}_{i}}</math> at each failure group will be multiplied by the number of failures in that group.

==== Shortfalls of the Rank Adjustment Method ====
Even though the rank adjustment method is the most widely used method for performing analysis for analysis of suspended items, we would like to point out the following shortcoming. As you may have noticed, only the position where the failure occurred is taken into account, and not the exact time-to-suspension. For example, this methodology would yield the exact same results for the next two cases.

{| border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|-
! style="text-align: center" colspan="3" | Case 1
! style="text-align: center" colspan="3" | Case 2
|- align="center"
! Item Number
! State*"F" or "S"
! Life of an item, hr
! Item number
! State*,"F" or "S"
! Life of item, hr
|- align="center"
| 1
| <math>{{F}_{1}}\,\!</math>
| 1,000
| 1
| <math>{{F}_{1}}\,\!</math>
| 1,000
|- align="center"
| 2
| <math>{{S}_{1}}\,\!</math>
| 1,100
| 2
| <math>{{S}_{1}}\,\!</math>
| 9,700
|- align="center"
| 3
| <math>{{S}_{2}}\,\!</math>
| 1,200
| 3
| <math>{{S}_{2}}\,\!</math>
| 9,800
|- align="center"
| 4
| <math>{{S}_{3}}\,\!</math>
| 1,300
| 4
| <math>{{S}_{3}}\,\!</math>
| 9,900
|- align="center"
| 5
| <math>{{F}_{2}}\,\!</math>
| 10,000
| 5
| <math>{{F}_{2}}\,\!</math>
| 10,000
|- align="center"
| style="text-align: center" colspan="3" | * ''F'' - ''Failed, S'' - ''Suspended''
| style="text-align: center" colspan="3" | * ''F'' - ''Failed, S'' - ''Suspended''
|}

This shortfall is significant when the number of failures is small and the number of suspensions is large and not spread uniformly between failures, as with these data. In cases like this, it is highly recommended to use maximum likelihood estimation (MLE) to estimate the parameters instead of using least squares, because MLE does not look at ranks or plotting positions, but rather considers each unique time-to-failure or suspension. For the data given above, the results are as follows. The estimated parameters using the method just described are the same for both cases (1 and 2):

::<math>\begin{array}{*{35}{l}}
\widehat{\beta }= & \text{0}\text{.81} \\
\widehat{\eta }= & \text{11,400 hr} \\
\end{array}
\,\!</math>

However, the MLE results for Case 1 are:

::<math>\begin{array}{*{35}{l}}
\widehat{\beta }= & \text{1.33} \\
\widehat{\eta }= & \text{6,920 hr} \\
\end{array}\,\!</math>

And the MLE results for Case 2 are:

::<math>\begin{array}{*{35}{l}}
\widehat{\beta }= & \text{0}\text{.93} \\
\widehat{\eta }= & \text{21,300 hr} \\
\end{array}\,\!</math>

As we can see, there is a sizable difference in the results of the two sets calculated using MLE and the results using regression with the SRM. The results for both cases are identical when using the regression estimation technique with SRM, as SRM considers only the positions of the suspensions. The MLE results are quite different for the two cases, with the second case having a much larger value of <math>\eta \,\!</math>, which is due to the higher values of the suspension times in Case 2. This is because the maximum likelihood technique, unlike rank regression with SRM, considers the values of the suspensions when estimating the parameters. This is illustrated in the [[Parameter_Estimation#Maximum_Likelihood_Estimation_.28MLE.29|discussion of MLE]] given below.

One alternative to improve the regression method is to use the following ReliaSoft Ranking Method (RRM) to calculate the rank. RRM does consider the effect of the censoring time.

== ReliaSoft's Ranking Method (RRM) for Interval Censored Data==
When analyzing interval data, it is commonplace to assume that the actual failure time occurred at the midpoint of the interval. To be more conservative, you can use the starting point of the interval or you can use the end point of the interval to be most optimistic. Weibull++ allows you to employ ReliaSoft's ranking method (RRM) when analyzing interval data. Using an iterative process, this ranking method is an improvement over the standard ranking method (SRM).

When analyzing left or right censored data, RRM also considers the effect of the actual censoring time. Therefore, the resulted rank will be more accurate than the SRM where only the position not the exact time of censoring is used.

For more details on this method see [[Appendix:_Special_Analysis_Methods#ReliaSoft_Ranking_Method|ReliaSoft's Ranking Method]].

= Maximum Likelihood Estimation (MLE) = 
From a statistical point of view, the method of maximum likelihood estimation method is, with some exceptions, considered to be the most robust of the parameter estimation techniques discussed here. The method presented in this section is for complete data (i.e., data consisting only of times-to-failure). The analysis for [[Parameter_Estimation#MLE_for_Right_Censored_Data|right censored (suspension) data]], and for [[Parameter_Estimation#MLE_for_Interval_and_Left_Censored_Data|interval or left censored data]], are then discussed in the following sections.

The basic idea behind MLE is to obtain the most likely values of the parameters, for a given distribution, that will best describe the data. As an example, consider the following data (-3, 0, 4) and assume that you are trying to estimate the mean of the data. Now, if you have to choose the most likely value for the mean from -5, 1 and 10, which one would you choose? In this case, the most likely value is 1 (given your limit on choices). Similarly, under MLE, one determines the most likely values for the parameters of the assumed distribution. It is mathematically formulated as follows.

If <math>x\,\!</math> is a continuous random variable with ''pdf'':
::<math>\begin{align}
& f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}) \\
\end{align}
\,\!</math>

where <math>{{\theta}_{1}},{{\theta}_{2}},...,{{\theta}_{k}}\,\!</math> are <math>k\,\!</math> unknown parameters which need to be estimated, with R independent observations,<math>{{x}_{1,}}{{x}_{2}},\cdots ,{{x}_{R}}\,\!</math>, which correspond in the case of life data analysis to failure times. The likelihood function is given by:

::<math>L({{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}|{{x}_{1}},{{x}_{2}},...,{{x}_{R}})=L=\underset{i=1}{\overset{R}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})
\,\!</math>

::<math>i = 1,2,...,R\,\!</math>

The logarithmic likelihood function is given by:

::<math>\Lambda = \ln L =\sum_{i = 1}^R \ln f({x_i};{\theta _1},{\theta _2},...,{\theta _k})\,\!</math>

The maximum likelihood estimators (or parameter values) of <math>{{\theta}_{1}},{{\theta}_{2}},...,{{\theta}_{k}}\,\!</math> are obtained by maximizing <math>L\,\!</math> or <math>\Lambda\,\!</math>.

By maximizing <math>\Lambda\,\!</math> which is much easier to work with than <math>L\,\!</math>, the maximum likelihood estimators (MLE) of <math>{{\theta}_{1}},{{\theta}_{2}},...,{{\theta}_{k}}\,\!</math> are the simultaneous solutions of <math>k\,\!</math> equations such that:

::<math>\frac{\partial{\Lambda}}{\partial{\theta_j}}=0, \text{ j=1,2...,k}\,\!</math>

Even though it is common practice to plot the MLE solutions using median ranks (points are plotted according to median ranks and the line according to the MLE solutions), this is not completely representative. As can be seen from the equations above, the MLE method is independent of any kind of ranks. For this reason, the MLE solution often appears not to track the data on the probability plot. This is perfectly acceptable because the two methods are independent of each other, and in no way suggests that the solution is wrong.

=== MLE for Right Censored Data ===
When performing maximum likelihood analysis on data with suspended items, the likelihood function needs to be expanded to take into account the suspended items. The overall estimation technique does not change, but another term is added to the likelihood function to account for the suspended items. Beyond that, the method of solving for the parameter estimates remains the same. For example, consider a distribution where <math>x\,\!</math> is a continuous random variable with ''pdf'' and ''cdf'':

::<math>\begin{align}
& f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}) \\
& F(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})
\end{align}
\,\!</math>

where <math>{{\theta}_{1}},{{\theta}_{2}},...,{{\theta}_{k}}\,\!</math> are the unknown parameters which need to be estimated from <math>R\,\!</math> observed failures at <math>{{T}_{1}},{{T}_{2}},...,{{T}_{R}}\,\!</math>, and <math>M\,\!</math> observed suspensions at <math>{{S}_{1}},{{S}_{2}},...,{{S}_{M}}\,\!</math> then the likelihood function is formulated as follows:

::<math>\begin{align}
L({{\theta }_{1}},...,{{\theta }_{k}}|{{T}_{1}},...,{{T}_{R,}}{{S}_{1}},...,{{S}_{M}})= & \underset{i=1}{\overset{R}{\mathop \prod }}\,f({{T}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}) \\
& \cdot \underset{j=1}{\overset{M}{\mathop \prod }}\,[1-F({{S}_{j}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})]
\end{align}\,\!</math>

The parameters are solved by maximizing this equation. In most cases, no closed-form solution exists for this maximum or for the parameters. Solutions specific to each distribution utilizing MLE are presented in [[Appendix:_Log-Likelihood_Equations|Appendix D]].

=== MLE for Interval and Left Censored Data ===
The inclusion of left and interval censored data in an MLE solution for parameter estimates involves adding a term to the likelihood equation to account for the data types in question. When using interval data, it is assumed that the failures occurred in an interval; i.e., in the interval from time <math>A\,\!</math> to time <math>B\,\!</math> (or from time 0 to time <math>B\,\!</math> if left censored), where <math>A < B\,\!</math>. In the case of interval data, and given <math>P\,\!</math> interval observations, the likelihood function is modified by multiplying the likelihood function with an additional term as follows:

::<math>\begin{align}
L({{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}|{{x}_{1}},{{x}_{2}},...,{{x}_{P}})= & \underset{i=1}{\overset{P}{\mathop \prod }}\,\{F({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}) \\
& \ \ -F({{x}_{i-1}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})\}
\end{align}\,\!</math>

Note that if only interval data are present, this term will represent the entire likelihood function for the MLE solution. The next section gives a formulation of the complete likelihood function for all possible censoring schemes.

=== The Complete Likelihood Function ===
We have now seen that obtaining MLE parameter estimates for different types of data involves incorporating different terms in the likelihood function to account for complete data, right censored data, and left, interval censored data. After including the terms for the different types of data, the likelihood function can now be expressed in its complete form or:

::<math>\begin{array}{*{35}{l}}
L= & \underset{i=1}{\mathop{\overset{R}{\mathop{\prod }}\,}}\,f({{T}_{i}};{{\theta }_{1}},...,{{\theta }_{k}})\cdot \underset{j=1}{\mathop{\overset{M}{\mathop{\prod }}\,}}\,[1-F({{S}_{j}};{{\theta }_{1}},...,{{\theta }_{k}})] \\
& \cdot \underset{l=1}{\mathop{\overset{P}{\mathop{\prod }}\,}}\,\left\{ F({{I}_{{{l}_{U}}}};{{\theta }_{1}},...,{{\theta }_{k}})-F({{I}_{{{l}_{L}}}};{{\theta }_{1}},...,{{\theta }_{k}}) \right\} \\
\end{array}
\,\!</math>

where:

::<math> L\to L({{\theta }_{1}},...,{{\theta }_{k}}|{{T}_{1}},...,{{T}_{R}},{{S}_{1}},...,{{S}_{M}},{{I}_{1}},...{{I}_{P}})\,\!</math>

and:
*<math>R\,\!</math> is the number of units with exact failures
*<math>M\,\!</math> is the number of suspended units
*<math>P\,\!</math> is the number of units with left censored or interval times-to-failure
*<math>{{\theta}_{k}}\,\!</math> are the parameters of the distribution
*<math>{{T}_{i}}\,\!</math> is the <math>{{i}^{th}}\,\!</math> time to failure
*<math>{{S}_{j}}\,\!</math> is the <math>{{j}^{th}}\,\!</math> time of suspension
*<math>{{I}_{{{l}_{U}}}}\,\!</math> is the ending of the time interval of the <math>{{l}^{th}}\,\!</math> group
*<math>{{I}_{{{l}_{L}}}}\,\!</math> is the beginning of the time interval of the <math>{{l}^{th}}\,\!</math> group

 The total number of units is <math>N = R + M + P\,\!</math>. It should be noted that in this formulation, if either <math>R\,\!</math>, <math>M\,\!</math> or <math>P\,\!</math> is zero then the product term associated with them is assumed to be one and not zero.

== Comments on the MLE Method ==
The MLE method has many large sample properties that make it attractive for use. It is asymptotically consistent, which means that as the sample size gets larger, the estimates converge to the right values. It is asymptotically efficient, which means that for large samples, it produces the most precise estimates. It is asymptotically unbiased, which means that for large samples, one expects to get the right value on average. The distribution of the estimates themselves is normal, if the sample is large enough, and this is the basis for the usual [[Confidence_Bounds#Fisher_Matrix_Confidence_Bounds|Fisher Matrix Confidence Bounds]] discussed later. These are all excellent large sample properties.

Unfortunately, the size of the sample necessary to achieve these properties can be quite large: thirty to fifty to more than a hundred exact failure times, depending on the application. With fewer points, the methods can be badly biased. It is known, for example, that MLE estimates of the shape parameter for the Weibull distribution are badly biased for small sample sizes, and the effect can be increased depending on the amount of censoring. This bias can cause major discrepancies in analysis. There are also pathological situations when the asymptotic properties of the MLE do not apply. One of these is estimating the location parameter for the three-parameter Weibull distribution when the shape parameter has a value close to 1. These problems, too, can cause major discrepancies.

However, MLE can handle suspensions and interval data better than rank regression, particularly when dealing with a heavily censored data set with few exact failure times or when the censoring times are unevenly distributed. It can also provide estimates with one or no observed failures, which rank regression cannot do. As a rule of thumb, our recommendation is to use rank regression techniques when the sample sizes are small and without heavy censoring (censoring is discussed in [[Life Data Classification|Life Data Classifications]]). When heavy or uneven censoring is present, when a high proportion of interval data is present and/or when the sample size is sufficient, MLE should be preferred.

''See also:''

*[[Appendix:_Maximum_Likelihood_Estimation_Example|Maximum Likelihood Parameter Estimation Example]]
*[[Appendix:_Special_Analysis_Methods|Grouped Data Analysis]]

=Bayesian Parameter Estimation Methods=
Up to this point, we have dealt exclusively with what is commonly referred to as classical statistics. In this section, another school of thought in statistical analysis will be introduced, namely Bayesian statistics. The premise of Bayesian statistics (within the context of life data analysis) is to incorporate prior knowledge, along with a given set of current observations, in order to make statistical inferences. The prior information could come from operational or observational data, from previous comparable experiments or from engineering knowledge. This type of analysis can be particularly useful when there is limited test data for a given design or failure mode but there is a strong prior understanding of the failure rate behavior for that design or mode. By incorporating prior information about the parameter(s), a posterior distribution for the parameter(s) can be obtained and inferences on the model parameters and their functions can be made. This section is intended to give a quick and elementary overview of Bayesian methods, focused primarily on the material necessary for understanding the Bayesian analysis methods available in Weibull++. Extensive coverage of the subject can be found in numerous books dealing with Bayesian statistics.

===Bayes’s Rule===
Bayes’s rule provides the framework for combining prior information with sample data. In this reference, we apply Bayes’s rule for combining prior information on the assumed distribution's parameter(s) with sample data in order to make inferences based on the model. The prior knowledge about the parameter(s) is expressed in terms of a <math>\varphi (\theta ),\,\!</math> called the ''prior distribution''. The ''posterior'' distribution of <math>\theta \,\!</math> given the sample data, using Bayes's rule, provides the updated information about the parameters <math>\theta \,\!</math>. This is expressed with the following posterior ''pdf'':

::<math> f(\theta |Data) = \frac{L(Data|\theta )\varphi (\theta )}{\int_{\zeta}^{} L(Data|\theta )\varphi(\theta )d (\theta)}
\,\!</math>

where:

*<math>\theta \,\!</math> is a vector of the parameters of the chosen distribution
*<math>\zeta\,\!</math> is the range of <math>\theta\,\!</math>
*<math> L(Data|\theta)\,\!</math> is the likelihood function based on the chosen distribution and data
*<math>\varphi(\theta )\,\!</math> is the prior distribution for each of the parameters

The integral in the Bayes's rule equation is often referred to as the marginal probability, which is a constant number that can be interpreted as the probability of obtaining the sample data given a prior distribution. Generally, the integral in the Bayes's rule equation does not have a closed form solution and numerical methods are needed for its solution.

As can be seen from the Bayes's rule equation, there is a significant difference between classical and Bayesian statistics. First, the idea of prior information does not exist in classical statistics. All inferences in classical statistics are based on the sample data. On the other hand, in the Bayesian framework, prior information constitutes the basis of the theory. Another difference is in the overall approach of making inferences and their interpretation. For example, in Bayesian analysis, the parameters of the distribution to be fitted are the random variables. In reality, there is no distribution fitted to the data in the Bayesian case.

For instance, consider the case where data is obtained from a reliability test. Based on prior experience on a similar product, the analyst believes that the shape parameter of the Weibull distribution has a value between <math>{\beta _1}\,\!</math> and <math>{{\beta }_{2}}\,\!</math> and wants to utilize this information. This can be achieved by using the Bayes theorem. At this point, the analyst is automatically forcing the Weibull distribution as a model for the data and with a shape parameter between <math>{\beta _1}\,\!</math> and <math>{\beta _2}\,\!</math>. In this example, the range of values for the shape parameter is the prior distribution, which in this case is Uniform. By applying Bayes's rule, the posterior distribution of the shape parameter will be obtained. Thus, we end up with a distribution for the parameter rather than an estimate of the parameter, as in classical statistics.

To better illustrate the example, assume that a set of failure data was provided along with a distribution for the shape parameter (i.e., uniform prior) of the Weibull (automatically assuming that the data are Weibull distributed). Based on that, a new distribution (the posterior) for that parameter is then obtained using Bayes's rule. This posterior distribution of the parameter may or may not resemble in form the assumed prior distribution. In other words, in this example the prior distribution of <math>\beta \,\!</math> was assumed to be uniform but the posterior is most likely not a uniform distribution.

The question now becomes: what is the value of the shape parameter? What about the reliability and other results of interest? In order to answer these questions, we have to remember that in the Bayesian framework all of these metrics are random variables. Therefore, in order to obtain an estimate, a probability needs to be specified or we can use the expected value of the posterior distribution.

In order to demonstrate the procedure of obtaining results from the posterior distribution, we will rewrite the Bayes's rule equation for a single parameter <math>{\theta _1}\,\!</math>:

::<math> f(\theta |Data) = \frac{L(Data|\theta_1 )\varphi (\theta_1 )}{\int_{\zeta}^{} L(Data|\theta_1 )\varphi(\theta_1 )d (\theta)}
\,\!</math>

The expected value (or mean value) of the parameter <math>{{\theta }_{1}}\,\!</math> can be obtained using the equation for the mean and the Bayes's rule equation for single parameter:

::<math>E({\theta _1}) = {m_{{\theta _1}}} = \int_{\zeta}^{}{\theta _1} \cdot f({\theta _1}|Data)d{\theta _1}\,\!</math>

An alternative result for <math>{\theta _1}\,\!</math> would be the median value. Using the equation for the median and the Bayes's rule equation for a single parameter:

::<math>\int_{-\infty ,0}^{{\theta }_{0.5}}f({{\theta }_{1}}|Data)d{{\theta }_{1}}=0.5\,\!</math>

The equation for the median is solved for <math>{\theta _{0.5}}\,\!</math> the median value of <math>{\theta _1}\,\!</math>

Similarly, any other percentile of the posterior ''pdf'' can be calculated and reported. For example, one could calculate the 90th percentile of <math>{\theta _1}\,\!</math>’s posterior ''pdf'':

::<math>\int_{-\infty ,0}^{{{\theta }_{0.9}}}f({{\theta }_{1}}|Data)d{{\theta }_{1}}=0.9\,\!</math>

This calculation will be used in [[Confidence Bounds]] and [[The Weibull Distribution]] for obtaining confidence bounds on the parameter(s).

The next step will be to make inferences on the reliability. Since the parameter <math>{\theta _1}\,\!</math> is a random variable described by the posterior ''pdf,'' all subsequent functions of <math>{{\theta }_{1}}\,\!</math> are distributed random variables as well and are entirely based on the posterior ''pdf'' of <math>{{\theta }_{1}}\,\!</math>. Therefore, expected value, median or other percentile values will also need to be calculated. For example, the expected reliability at time <math>T\,\!</math> is:

::<math>E[R(T|Data)] = \int_{\varsigma}^{} R(T)f(\theta |Data)d{\theta}\,\!</math>

In other words, at a given time <math>T\,\!</math>, there is a distribution that governs the reliability value at that time, <math>T\,\!</math>, and by using Bayes's rule, the expected (or mean) value of the reliability is obtained. Other percentiles of this distribution can also be obtained.
A similar procedure is followed for other functions of <math>{\theta _1}\,\!</math>, such as failure rate, reliable life, etc.

===Prior Distributions===
Prior distributions play a very important role in Bayesian Statistics. They are essentially the basis in Bayesian analysis. Different types of prior distributions exist, namely ''informative'' and ''non-informative''. Non-informative prior distributions (a.k.a. ''vague'', ''flat'' and ''diffuse'') are distributions that have no population basis and play a minimal role in the posterior distribution. The idea behind the use of non-informative prior distributions is to make inferences that are not greatly affected by external information or when external information is not available. The uniform distribution is frequently used as a non-informative prior.

On the other hand, informative priors have a stronger influence on the posterior distribution. The influence of the prior distribution on the posterior is related to the sample size of the data and the form of the prior. Generally speaking, large sample sizes are required to modify strong priors, where weak priors are overwhelmed by even relatively small sample sizes. Informative priors are typically obtained from past data.

The Weibull Distribution

2022-12-16T23:22:01Z

M Spivey: /* Rank Regression on Y */

{{template:LDABOOK|8|The Weibull Distribution}}
The Weibull distribution is one of the most widely used lifetime distributions in reliability engineering. It is a versatile distribution that can take on the characteristics of other types of distributions, based on the value of the shape parameter, <math> {\beta} \,\!</math>. This chapter provides a brief background on the Weibull distribution, presents and derives most of the applicable equations and presents examples calculated both manually and by using ReliaSoft's [https://koi-3QN72QORVC.marketingautomation.services/net/m?md=Rw01CJDOxn%2FabhkPlZsy6DwBQ%2BaCXsGR Weibull++ software].

== Weibull Probability Density Function ==
===The 3-Parameter Weibull===
{{three-parameter weibull distribution}}

===The 2-Parameter Weibull ===
The 2-parameter Weibull ''pdf'' is obtained by setting
<math> \gamma=0 \,\!</math>, and is given by:

:<math> f(t)={ \frac{\beta }{\eta }}\left( {\frac{t}{\eta }}\right) ^{\beta -1}e^{-\left( { \frac{t}{\eta }}\right) ^{\beta }} \,\!</math>

=== The 1-Parameter Weibull===
The 1-parameter Weibull ''pdf'' is obtained by again setting
<math>\gamma=0 \,\!</math> and assuming <math>\beta=C=Constant \,\!</math> assumed value or:

::<math> f(t)={ \frac{C}{\eta }}\left( {\frac{t}{\eta }}\right) ^{C-1}e^{-\left( {\frac{t}{ \eta }}\right) ^{C}} \,\!</math>

where the only unknown parameter is the scale parameter, <math>\eta\,\!</math>.

Note that in the formulation of the 1-parameter Weibull, we assume that the shape parameter <math>\beta \,\!</math> is known ''a priori'' from past experience with identical or similar products. The advantage of doing this is that data sets with few or no failures can be analyzed.

==Weibull Distribution Functions==
{{:Weibull Distribution Functions}}

== Characteristics of the Weibull Distribution ==
{{:Weibull Distribution Characteristics}}

== Estimation of the Weibull Parameters ==
The estimates of the parameters of the Weibull distribution can be found graphically via probability plotting paper, or analytically, using either least squares (rank regression) or maximum likelihood estimation (MLE).

=== Probability Plotting ===
One method of calculating the parameters of the Weibull distribution is by using probability plotting. To better illustrate this procedure, consider the following example from Kececioglu [[Appendix:_Life_Data_Analysis_References|[20]]].

Assume that six identical units are being reliability tested at the same application and operation stress levels. All of these units fail during the test after operating the following number of hours: 93, 34, 16, 120, 53 and 75. Estimate the values of the parameters for a 2-parameter Weibull distribution and determine the reliability of the units at a time of 15 hours.

'''Solution'''

The steps for determining the parameters of the Weibull representing the data, using probability plotting, are outlined in the following instructions. First, rank the times-to-failure in ascending order as shown next.

{| border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|-
! valign="middle" scope="col" align="center" | Time-to-failure, hours
! valign="middle" scope="col" align="center" | Failure Order Number out of Sample Size of 6
|-
| valign="middle" align="center" | 16
| valign="middle" align="center" | 1
|-
| valign="middle" align="center" | 34
| valign="middle" align="center" | 2
|-
| valign="middle" align="center" | 53
| valign="middle" align="center" | 3
|-
| valign="middle" align="center" | 75
| valign="middle" align="center" | 4
|-
| valign="middle" align="center" | 93
| valign="middle" align="center" | 5
|-
| valign="middle" align="center" | 120
| valign="middle" align="center" | 6
|}

Obtain their median rank plotting positions. Median rank positions are used instead of other ranking methods because median ranks are at a specific confidence level (50%). Median ranks can be found tabulated in many reliability books. They can also be estimated using the following equation:

::<math> MR \sim { \frac{i-0.3}{N+0.4}}\cdot 100 \,\!</math>

where <math>i\,\!</math> is the failure order number and <math>N\,\!</math> is the total sample size. The exact median ranks are found in Weibull++ by solving:

::<math>\sum_{k=i}^N{\binom{N}{k}}{MR^k}{(1-MR)^{N-k}}=0.5=50%
\,\!</math>

for <math>MR\,\!</math>, where <math>N\,\!</math> is the sample size and <math>i\,\!</math> the order number. The times-to-failure, with their corresponding median ranks, are shown next.

{|border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|-
! Time-to-failure, hours
! Median Rank,%
|-
| 16
| 10.91
|-
| 34
| 26.44
|-
| 53
| 42.14
|-
| 75
| 57.86
|-
| 93
| 73.56
|-
| 120
| 89.1
|}

On a Weibull probability paper, plot the times and their corresponding ranks. A sample of a Weibull probability paper is given in the following figure.

[[Image:WB.8 example of paper.png|center|450px| Example of Weibull probability plotting paper. ]]

The points of the data in the example are shown in the figure below. Draw the best possible straight line through these points, as shown below, then obtain the slope of this line by drawing a line, parallel to the one just obtained, through the slope indicator. This value is the estimate of the shape parameter <math> \hat{\beta } \,\!</math>, in this case <math> \hat{\beta }=1.4 \,\!</math>.

[[Image:WB.8 probability plotting.png|center|350px| Probability plot of data in Example 1.]]

At the <math> Q(t)=63.2%\,\!</math> ordinate point, draw a straight horizontal line until this line intersects the fitted straight line. Draw a vertical line through this intersection until it crosses the abscissa. The value at the intersection of the abscissa is the estimate of <math> \hat{\eta } \,\!</math>. For this case, <math> \hat{\eta }=76 \,\!</math> hours. This is always at 63.2% since:

::<math> Q(t)=1-e^{-(\frac{t}{\eta })^{\beta }}=1-e^{-1}=0.632=63.2% \,\!</math>

Now any reliability value for any mission time <math>t\,\!</math> can be obtained. For example, the reliability for a mission of 15 hours, or any other time, can now be obtained either from the plot or analytically. To obtain the value from the plot, draw a vertical line from the abscissa, at hours, to the fitted line. Draw a horizontal line from this intersection to the ordinate and read <math> Q(t)\,\!</math>, in this case <math> Q(t)=9.8%\,\!</math>. Thus, <math> R(t)=1-Q(t)=90.2%\,\!</math>. This can also be obtained analytically from the Weibull reliability function since the estimates of both of the parameters are known or:

::<math> R(t=15)=e^{-\left( \frac{15}{\eta }\right) ^{\beta }}=e^{-\left( \frac{15}{76 }\right) ^{1.4}}=90.2% \,\!</math>

====Probability Plotting for the Location Parameter, Gamma====

The third parameter of the Weibull distribution is utilized when the data do not fall on a straight line, but fall on either a concave up or down curve. The following statements can be made regarding the value of <math>\gamma \,\!</math>:

*'''Case 1:''' If the curve for MR versus <math>{{t}_{j}}\,\!</math> is concave down and the curve for MR versus <math>{({t}_{j}-{t}_{1})}\,\!</math> is concave up, then there exists a <math>\gamma \,\!</math> such that <math>0< \gamma < t_{1}\,\!</math>, or <math>\gamma \,\!</math> has a positive value.

*'''Case 2''': If the curves for MR versus <math>{{t}_{j}}\,\!</math> and MR versus <math>{({t}_{j}-{t}_{1})}\,\!</math> are both concave up, then there exists a negative <math>\gamma \,\!</math> which will straighten out the curve of MR versus <math>{{t}_{j}}\,\!</math>.

*'''Case 3''': If neither one of the previous two cases prevails, then either reject the Weibull as one capable of representing the data, or proceed with the multiple population (mixed Weibull) analysis. To obtain the location parameter, <math>\gamma \,\!</math>:

::*Subtract the same arbitrary value, <math>\gamma \,\!</math>, from all the times to failure and replot the data.
::*If the initial curve is concave up, subtract a negative <math>\gamma \,\!</math> from each failure time.
::*If the initial curve is concave down, subtract a positive <math>\gamma \,\!</math> from each failure time.
::*Repeat until the data plots on an acceptable straight line.
::*The value of <math>\gamma \,\!</math> is the subtracted (positive or negative) value that places the points in an acceptable straight line.

The other two parameters are then obtained using the techniques previously described. Also, it is important to note that we used the term subtract a positive or negative gamma, where subtracting a negative gamma is equivalent to adding it. Note that when adjusting for gamma, the x-axis scale for the straight line becomes <math>{({t}-\gamma)}\,\!</math>.

=== Rank Regression on Y ===
Performing rank regression on Y requires that a straight line mathematically be fitted to a set of data points such that the sum of the squares of the vertical deviations from the points to the line is minimized. This is in essence the same methodology as the probability plotting method, except that we use the principle of least squares to determine the line through the points, as opposed to just eyeballing it. The first step is to bring our function into a linear form. For the two-parameter Weibull distribution, the (cumulative density function) is:

::<math> F(t)=1-e^{-\left( \frac{t}{\eta }\right) ^{\beta }} \,\!</math>

Taking the natural logarithm of both sides of the equation yields:

::<math>\ln[ 1-F(t)] =-( \frac{t}{\eta }) ^{\beta } \,\!</math>

::<math> \ln{ -\ln[ 1-F(t)]} =\beta \ln ( \frac{t}{ \eta }) \,\!</math>

or:

::<math>\begin{align}
\ln \{ -\ln[ 1-F(t)]\} =-\beta \ln (\eta )+\beta \ln (t)
\end{align}\,\!</math>

Now let:

::<math>\begin{align}
y = \ln \{ -\ln[ 1-F(t)]\}
\end{align}\,\!</math>

::<math>\begin{align}
a = - \beta \ln(\eta)
\end{align}\,\!</math>

and:

::<math>\begin{align}
b= \beta
\end{align}\,\!</math>

which results in the linear equation of:

::<math>\begin{align}
y=a+bx
\end{align}\,\!</math>

The least squares parameter estimation method (also known as ''regression analysis'') was discussed in [[Parameter Estimation]], and the following equations for regression on Y were derived:

::<math> \hat{a}=\frac{\sum\limits_{i=1}^{N}y_{i}}{N}-\hat{b}\frac{ \sum\limits_{i=1}^{N}x_{i}}{N}=\bar{y}-\hat{b}\bar{x} \,\!</math>

and:

::<math> \hat{b}={\frac{\sum\limits_{i=1}^{N}x_{i}y_{i}-\frac{\sum \limits_{i=1}^{N}x_{i}\sum\limits_{i=1}^{N}y_{i}}{N}}{\sum \limits_{i=1}^{N}x_{i}^{2}-\frac{\left( \sum\limits_{i=1}^{N}x_{i}\right) ^{2}}{N}}} \,\!</math>

In this case the equations for <math>{{y}_{i}}\,\!</math> and <math>{{x}_{i}}\,\!</math> are:

::<math> y_{i}=\ln \left\{ -\ln [1-F(t_{i})]\right\} \,\!</math>

and:
::<math>\begin{align}
x_{i}=\ln(t_{i})
\end{align}\,\!</math>

The <math> F(t_{i})\,\!</math> values are estimated from the median ranks.

Once <math> \hat{a} \,\!</math> and <math> \hat{b} \,\!</math> are obtained, then <math> \hat{\beta } \,\!</math> and <math> \hat{\eta } \,\!</math> can easily be obtained from previous equations.

'''The Correlation Coefficient'''

The correlation coefficient is defined as follows:

::<math> \rho ={\frac{\sigma _{xy}}{\sigma _{x}\sigma _{y}}} \,\!</math>

where <math>\sigma_{xy}\,\!</math> = covariance of <math>x\,\!</math> and <math>y\,\!</math>, <math>\sigma_{x}\,\!</math> = standard deviation of <math>x\,\!</math>, and <math>\sigma_{y}\,\!</math> = standard deviation of <math>y\,\!</math>. The estimator of <math>\rho\,\!</math> is the sample correlation coefficient, <math> \hat{\rho} \,\!</math>, given by:

::<math> \hat{\rho}=\frac{\sum\limits_{i=1}^{N}(x_{i}-\overline{x})(y_{i}-\overline{y} )}{\sqrt{\sum\limits_{i=1}^{N}(x_{i}-\overline{x})^{2}\cdot \sum\limits_{i=1}^{N}(y_{i}-\overline{y})^{2}}}\,\!</math>

====RRY Example====

Consider the same data set from the [[The_Weibull_Distribution#Probability_Plotting|probability plotting example]] given above (with six failures at 16, 34, 53, 75, 93 and 120 hours). Estimate the parameters and the correlation coefficient using rank regression on Y, assuming that the data follow the 2-parameter Weibull distribution.

'''Solution'''

Construct a table as shown next.

{|border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|-
!colspan="8" style="text-align:center"| Least Squares Analysis
|-
!<math>N\,\!</math>
!<math>T_{i}\,\!</math>
!<math>ln(T_{i})\,\!</math>
!<math>F(T_i)\,\!</math>
!<math>y_{i}\,\!</math>
!<math>(ln{T_i})^2\,\!</math>
!<math>{y_i}^2\,\!</math>
!<math>(ln{T_i})y_i\,\!</math>
|-
|1 ||16||2.7726||0.1091||-2.1583||7.6873||4.6582||-5.9840
|-
|2 ||34||3.5264||0.2645||-1.1802||12.4352||1.393||-4.1620
|-
|3 ||53||3.9703||0.4214||-0.6030||15.7632||0.3637||-2.3943
|-
|4 ||75||4.3175||0.5786||-0.146||18.6407||0.0213||-0.6303
|-
|5 ||93||4.5326||0.7355||0.2851||20.5445||0.0813||1.2923
|-
|6 ||120||4.7875||0.8909||0.7955||22.9201||0.6328||3.8083
|-
|<math>\sum\,\!</math>|| ||23.9068|| ||-3.007||97.9909||7.1502||-8.0699
|}

Utilizing the values from the table, calculate <math> \hat{a} \,\!</math> and <math> \hat{b} \,\!</math> using the following equations:
::<math> \hat{b} =\frac{\sum\limits_{i=1}^{6}(\ln t_{i})y_{i}-(\sum\limits_{i=1}^{6}\ln t_{i})(\sum\limits_{i=1}^{6}y_{i})/6}{ \sum\limits_{i=1}^{6}(\ln t_{i})^{2}-(\sum\limits_{i=1}^{6}\ln t_{i})^{2}/6}
\,\!</math>

::<math> \hat{b}=\frac{-8.0699-(23.9068)(-3.0070)/6}{97.9909-(23.9068)^{2}/6} \,\!</math>

or:

::<math> \hat{b}=1.4301 \,\!</math>

and:

::<math> \hat{a}=\overline{y}-\hat{b}\overline{T}=\frac{\sum \limits_{i=1}^{N}y_{i}}{N}-\hat{b}\frac{\sum\limits_{i=1}^{N}\ln t_{i}}{N } \,\!</math>

or:

::<math> \hat{a}=\frac{(-3.0070)}{6}-(1.4301)\frac{23.9068}{6}=-6.19935 \,\!</math>

Therefore:

::<math> \hat{\beta }=\hat{b}=1.4301 \,\!</math>

and:

::<math> \hat{\eta }=e^{-\frac{\hat{a}}{\hat{b}}}=e^{-\frac{(-6.19935)}{ 1.4301}} \,\!</math>

or:

::<math> \hat{\eta }=76.318\text{ hr} \,\!</math>

The correlation coefficient can be estimated as:

::<math> \hat{\rho }=0.9956 \,\!</math>
 
This example can be repeated in the Weibull++ software. The following plot shows the Weibull probability plot for the data set (with 90% two-sided confidence bounds).

[[Image:Weibull Distribution Example 3 RRY Confidence Plot.png|center|450px| ]]

If desired, the Weibull ''pdf'' representing the data set can be written as:

::<math> f(t)={\frac{\beta }{\eta }}\left( {\frac{t}{\eta }}\right) ^{\beta -1}e^{-\left( {\frac{t}{\eta }}\right) ^{\beta }} \,\!</math>

or:

::<math> f(t)={\frac{1.4302}{76.317}}\left( {\frac{t}{76.317}}\right) ^{0.4302}e^{-\left( {\frac{t}{76.317}}\right) ^{1.4302}} \,\!</math>

You can also plot this result in Weibull++, as shown next. From this point on, different results, reports and plots can be obtained.

[[Image:Weibull Distribution Example 3 pdf Plot.png|center|450px]]

=== Rank Regression on X ===
Performing a rank regression on X is similar to the process for rank regression on Y, with the difference being that the ''horizontal'' deviations from the points to the line are minimized rather than the vertical. Again, the first task is to bring the reliability function into a linear form. This step is exactly the same as in the regression on Y analysis and all the equations apply in this case too. The derivation from the previous analysis begins on the least squares fit part, where in this case we treat as the dependent variable and as the independent variable. The best-fitting straight line to the data, for regression on X (see [[Parameter Estimation|Parameter Estimation]]), is the straight line:

::<math> x= \hat{a}+\hat{b}y \,\!</math>

The corresponding equations for <math> \hat{a} \,\!</math> and <math> \hat{b} \,\!</math> are:

::<math> \hat{a}=\overline{x}-\hat{b}\overline{y}=\frac{\sum\limits_{i=1}^{N}x_{i}}{N} -\hat{b}\frac{\sum\limits_{i=1}^{N}y_{i}}{N} \,\!</math>

and:

::<math> \hat{b}={\frac{\sum\limits_{i=1}^{N}x_{i}y_{i}-\frac{\sum \limits_{i=1}^{N}x_{i}\sum\limits_{i=1}^{N}y_{i}}{N}}{\sum \limits_{i=1}^{N}y_{i}^{2}-\frac{\left( \sum\limits_{i=1}^{N}y_{i}\right) ^{2}}{N}}} \,\!</math>

where:

::<math> y_{i}=\ln \left\{ -\ln [1-F(t_{i})]\right\} \,\!</math>

and:

::<math>\begin{align}
x_{i}=\ln (t_{i})
\end{align}\,\!</math>

and the <math>F({{t}_{i}})\,\!</math> values are again obtained from the median ranks.

Once <math> \hat{a} \,\!</math> and <math> \hat{b} \,\!</math> are obtained, solve the linear equation for <math>y\,\!</math>, which corresponds to:

::<math> y=-\frac{\hat{a}}{\hat{b}}+\frac{1}{\hat{b}}x \,\!</math> Solving for the parameters from above equations, we get:

::<math> a=-\frac{\hat{a}}{\hat{b}}=-\beta \ln (\eta )\,\!</math>

and

::<math> b=\frac{1}{\hat{b}}=\beta\,\!</math>

The correlation coefficient is evaluated as before.
====RRX Example====
Again using the same data set from the [[The_Weibull_Distribution#Probability_Plotting|probability plotting]] and [[The_Weibull_Distribution#RRY_Example|RRY]] examples (with six failures at 16, 34, 53, 75, 93 and 120 hours), calculate the parameters using rank regression on X.

'''Solution'''

The same table constructed above for the [[The_Weibull_Distribution#RRY_Example|RRY example]] can also be applied for RRX.

Using the values from this table we get:

::<math> \hat{b} ={\frac{\sum\limits_{i=1}^{6}(\ln T_{i})y_{i}-\frac{ \sum\limits_{i=1}^{6}\ln T_{i}\sum\limits_{i=1}^{6}y_{i}}{6}}{ \sum\limits_{i=1}^{6}y_{i}^{2}-\frac{\left( \sum\limits_{i=1}^{6}y_{i}\right) ^{2}}{6}}}
\,\!</math>

::<math>\hat{b} =\frac{-8.0699-(23.9068)(-3.0070)/6}{7.1502-(-3.0070)^{2}/6} \,\!</math>

or:

::<math> \hat{b}=0.6931 \,\!</math>

and:

::<math> \hat{a}=\overline{x}-\hat{b}\overline{y}=\frac{\sum\limits_{i=1}^{6}\ln T_{i} }{6}-\hat{b}\frac{\sum\limits_{i=1}^{6}y_{i}}{6} \,\!</math>

or:

::<math> \hat{a}=\frac{23.9068}{6}-(0.6931)\frac{(-3.0070)}{6}=4.3318 \,\!</math>

Therefore:

::<math> \hat{\beta }=\frac{1}{\hat{b}}=\frac{1}{0.6931}=1.4428 \,\!</math>

and:

::<math> \hat{\eta }=e^{\frac{\hat{a}}{\hat{b}}\cdot \frac{1}{\hat{ \beta }}}=e^{\frac{4.3318}{0.6931}\cdot \frac{1}{1.4428}}=76.0811\text{ hr} \,\!</math>

The correlation coefficient is:

::<math> \hat{\rho }=0.9956 \,\!</math>

The results and the associated graph using Weibull++ are shown next. Note that the slight variation in the results is due to the number of significant figures used in the estimation of the median ranks. Weibull++ by default uses double precision accuracy when computing the median ranks.

[[Image:Weibull Distribution Example 4 RRX Plot.png|center|450px| ]]

 

=== 3-Parameter Weibull Regression ===
When the MR versus <math>{{t}_{j}}\,\!</math> points plotted on the Weibull probability paper do not fall on a satisfactory straight line and the points fall on a curve, then a location parameter, <math>\gamma\,\!</math>, might exist which may straighten out these points. The goal in this case is to fit a curve, instead of a line, through the data points using nonlinear regression. The Gauss-Newton method can be used to solve for the parameters, <math>\beta\,\!</math>, <math>\eta\,\!</math> and <math>\gamma\,\!</math>, by performing a Taylor series expansion on <math>F(t{_{i}};\beta ,\eta, \gamma )\,\!</math>. Then the nonlinear model is approximated with linear terms and ordinary least squares are employed to estimate the parameters. This procedure is iterated until a satisfactory solution is reached.

(Note that other shapes, particularly ''S'' shapes, might suggest the existence of more than one population. In these cases, the multiple population [[The Mixed Weibull Distribution|mixed Weibull distribution]], may be more appropriate.)

When you use the 3-parameter Weibull distribution, Weibull++ calculates the value of <math>\gamma\,\!</math> by utilizing an optimized Nelder-Mead algorithm and adjusts the points by this value of <math>\gamma\,\!</math> such that they fall on a straight line, and then plots both the adjusted and the original unadjusted points. To draw a curve through the original unadjusted points, if so desired, select Weibull 3P Line Unadjusted for Gamma from the ''Show Plot Line'' submenu under the ''Plot Options'' menu. The returned estimations of the parameters are the same when selecting RRX or RRY. To display the unadjusted data points and line along with the adjusted data points and line, select ''Show/Hide Items'' under the ''Plot Options ''menu and include the unadjusted data points and line as follows:

[[Image:showhideplotitems.png|center]]

[[Image:Weibull Distribution Example 4 Show Hide Items.png|center|450px]]

The results and the associated graph for the previous example using the 3-parameter Weibull case are shown next:

[[Image:Weibull Distribution Example 4 Plot.png|center|450px| ]]

=== Maximum Likelihood Estimation ===
As outlined in [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function, but this can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood function with respect to the parameters, setting the resulting equations equal to zero and solving simultaneously to determine the values of the parameter estimates. ( Note that MLE asymptotic properties do not hold when estimating <math>\gamma\,\!</math> using MLE, as discussed in Meeker and Escobar [[Appendix:_Life_Data_Analysis_References|[27]]].) The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the Weibull distribution are covered in [[Appendix:_Log-Likelihood_Equations|Appendix D]].
====MLE Example====
One last time, use the same data set from the [[The_Weibull_Distribution#Probability_Plotting|probability plotting]], [[The_Weibull_Distribution#RRY_Example|RRY]] and [[The_Weibull_Distribution#RRY_Example|RRX]] examples (with six failures at 16, 34, 53, 75, 93 and 120 hours) and calculate the parameters using MLE.
 

'''Solution'''

In this case, we have non-grouped data with no suspensions or intervals, (i.e., complete data). The equations for the partial derivatives of the log-likelihood function are derived in [[Appendix:_Log-Likelihood_Equations|an appendix]] and given next:
::<math> \frac{\partial \Lambda }{\partial \beta }=\frac{6}{\beta } +\sum_{i=1}^{6}\ln \left( \frac{T_{i}}{\eta }\right) -\sum_{i=1}^{6}\left( \frac{T_{i}}{\eta }\right) ^{\beta }\ln \left( \frac{T_{i}}{\eta }\right) =0
\,\!</math>

And:

::<math> \frac{\partial \Lambda }{\partial \eta }=\frac{-\beta }{\eta }\cdot 6+\frac{ \beta }{\eta }\sum\limits_{i=1}^{6}\left( \frac{T_{i}}{\eta }\right) ^{\beta }=0 \,\!</math>

Solving the above equations simultaneously we get:

::<math> \hat{\beta }=1.933,\,\!</math> <math>\hat{\eta }=73.526 \,\!</math>

The variance/covariance matrix is found to be:

::<math> \left[ \begin{array}{ccc} \hat{Var}\left( \hat{\beta }\right) =0.4211 & \hat{Cov}( \hat{\beta },\hat{\eta })=3.272 \\

\hat{Cov}(\hat{\beta },\hat{\eta })=3.272 & \hat{Var} \left( \hat{\eta }\right) =266.646 \end{array} \right] \,\!</math>

The results and the associated plot using Weibull++ (MLE) are shown next.

[[Image:Weibull Distribution Example 5 Plot.png|center|450px| ]]

You can view the variance/covariance matrix directly by clicking the '''Analysis Summary''' table in the control panel. Note that the decimal accuracy displayed and used is based on your individual Application Setup.

[[Image: Weibull Distribution Example 5 Variance Matrix.png|center|450px| ]]

====Unbiased MLE <math>\beta \,\!</math> ====
It is well known that the MLE <math>\beta \,\!</math> is biased. The biasness will affect the accuracy of reliability prediction, especially when the number of failures are small. Weibull++ provides a simple way to correct the bias of MLE <math>\beta \,\!</math>.

When there are no right censored observations in the data, the following equation provided by Hirose [[Appendix:_Life_Data_Analysis_References|[39]]] is used to calculated the unbiased <math>\beta \,\!</math>.

<math>{{\beta }_{U}}=\frac{\beta }{1.0115+\frac{1.278}{r}+\frac{2.001}{{{r}^{2}}}+\frac{20.35}{{{r}^{3}}}-\frac{46.98}{{{r}^{4}}}}</math>

where <math>r\,\!</math> is the number of failures.

When there are right censored observations in the data, the following equation provided by Ross [[Appendix:_Life_Data_Analysis_References|[40]]] is used to calculated the unbiased <math>\beta\,\!</math>.

<math>{{\beta }_{U}}=\frac{\beta }{1+\frac{1.37}{r-1.92}\sqrt{\frac{n}{r}}}</math>

where <math>n\,\!</math> is the number of observations.

The software will use the above equations only when there are more than two failures in the data set.
<div class="noprint">
{{Examples Box|http://www.weibull.com/hotwire/issue109/relbasics109.htm|For an example on how you might correct biased estimates, see also:
{{Examples Link External|http://www.weibull.com/hotwire/issue109/relbasics109.htm|Unbiasing Parameters in Weibull++}}<nowiki/>
}}
</div>

== Fisher Matrix Confidence Bounds ==
One of the methods used by the application in estimating the different types of confidence bounds for Weibull data, the Fisher matrix method, is presented in this section. The complete derivations were presented in detail (for a general function) in [[Confidence Bounds]].

=== Bounds on the Parameters ===
One of the properties of maximum likelihood estimators is that they are asymptotically normal, meaning that for large samples they are normally distributed. Additionally, since both the shape parameter estimate, <math> \hat{\beta } \,\!</math>, and the scale parameter estimate, <math> \hat{\eta }, \,\!</math> must be positive, thus <math>ln\beta \,\!</math> and <math>ln\eta \,\!</math> are treated as being normally distributed as well. The lower and upper bounds on the parameters are estimated from Nelson [[Appendix:_Life_Data_Analysis_References|[30]]]:

::<math> \beta _{U} =\hat{\beta }\cdot e^{\frac{K_{\alpha }\sqrt{Var(\hat{ \beta })}}{\hat{\beta }}}\text{ (upper bound)} \,\!</math>

::<math> \beta _{L} =\frac{\hat{\beta }}{e^{\frac{K_{\alpha }\sqrt{Var(\hat{ \beta })}}{\hat{\beta }}}} \text{ (lower bound)}
\,\!</math>

and:

::<math> \eta _{U} =\hat{\eta }\cdot e^{\frac{K_{\alpha }\sqrt{Var(\hat{ \eta })}}{\hat{\eta }}}\text{ (upper bound)}
\,\!</math>

::<math> \eta _{L} =\frac{\hat{\eta }}{e^{\frac{K_{\alpha }\sqrt{Var(\hat{ \eta })}}{\hat{\eta }}}}\text{ (lower bound)} \,\!</math>

where <math> K_{\alpha}\,\!</math> is defined by:

::<math> \alpha =\frac{1}{\sqrt{2\pi }}\int_{K_{\alpha }}^{\infty }e^{-\frac{t^{2}}{2} }dt=1-\Phi (K_{\alpha }) \,\!</math>

If <math>d\,\!</math> is the confidence level, then <math> \alpha =\frac{1-\delta }{2} \,\!</math> for the two-sided bounds and <math>a = 1 - d\,\!</math> for the one-sided bounds. The variances and covariances of <math> \hat{\beta }\,\!</math> and <math> \hat{\eta }\,\!</math> are estimated from the inverse local Fisher matrix, as follows:

::<math> \left( \begin{array}{cc} \hat{Var}\left( \hat{\beta }\right) & \hat{Cov}\left( \hat{ \beta },\hat{\eta }\right)
\\
\hat{Cov}\left( \hat{\beta },\hat{\eta }\right) & \hat{Var} \left( \hat{\eta }\right) \end{array} \right) =\left( \begin{array}{cc} -\frac{\partial ^{2}\Lambda }{\partial \beta ^{2}} & -\frac{\partial ^{2}\Lambda }{\partial \beta \partial \eta }
\\

-\frac{\partial ^{2}\Lambda }{\partial \beta \partial \eta } & -\frac{ \partial ^{2}\Lambda }{\partial \eta ^{2}} \end{array} \right) _{\beta =\hat{\beta },\text{ }\eta =\hat{\eta }}^{-1} \,\!</math>

'''Fisher Matrix Confidence Bounds and Regression Analysis'''

Note that the variance and covariance of the parameters are obtained from the inverse Fisher information matrix as described in this section. The local Fisher information matrix is obtained from the second partials of the likelihood function, by substituting the solved parameter estimates into the particular functions. This method is based on maximum likelihood theory and is derived from the fact that the parameter estimates were computed using maximum likelihood estimation methods. When one uses least squares or regression analysis for the parameter estimates, this methodology is theoretically then not applicable. However, if one assumes that the variance and covariance of the parameters will be similar ( One also assumes similar properties for both estimators.) regardless of the underlying solution method, then the above methodology can also be used in regression analysis.

The Fisher matrix is one of the methodologies that Weibull++ uses for both MLE and regression analysis. Specifically, Weibull++ uses the likelihood function and computes the local Fisher information matrix based on the estimates of the parameters and the current data. This gives consistent confidence bounds regardless of the underlying method of solution, (i.e., MLE or regression). In addition, Weibull++ checks this assumption and proceeds with it if it considers it to be acceptable. In some instances, Weibull++ will prompt you with an "Unable to Compute Confidence Bounds" message when using regression analysis. This is an indication that these assumptions were violated.

=== Bounds on Reliability ===
The bounds on reliability can easily be derived by first looking at the general extreme value distribution (EVD). Its reliability function is given by:

::<math> R(t)=e^{-e^{\left( \frac{t-p_{1}}{p_{2}}\right) }} \,\!</math>

By transforming <math>t = \ln t\,\!</math> and converting <math> p_{1}=\ln({\eta})\,\!</math>, <math> p_{2}=\frac{1}{ \beta } \,\!</math>, the above equation becomes the Weibull reliability function:

::<math> R(t)=e^{-e^{\beta \left( \ln t-\ln \eta \right) }}=e^{-e^{\ln \left( \frac{t }{\eta }\right) ^{\beta }}}=e^{-\left( \frac{t}{\eta }\right) ^{\beta }} \,\!</math>

with:

::<math> R(T)=e^{-e^{\beta \left( \ln t-\ln \eta \right) }}\,\!</math>

set:

:: <math> u=\beta \left( \ln t-\ln \eta \right) \,\!</math>

The reliability function now becomes:

::<math> R(T)=e^{-e^{u}} \,\!</math>

The next step is to find the upper and lower bounds on <math>u\,\!</math>. Using the equations derived in [[Confidence Bounds]], the bounds on are then estimated from Nelson [[Appendix:_Life_Data_Analysis_References|[30]]]:

::<math> u_{U} =\hat{u}+K_{\alpha }\sqrt{Var(\hat{u})}
\,\!</math>

::<math> u_{L} =\hat{u}-K_{\alpha }\sqrt{Var(\hat{u})}
\,\!</math>

where:

::<math> Var(\hat{u}) =\left( \frac{\partial u}{\partial \beta }\right) ^{2}Var( \hat{\beta })+\left( \frac{\partial u}{\partial \eta }\right) ^{2}Var( \hat{\eta }) +2\left( \frac{\partial u}{\partial \beta }\right) \left( \frac{\partial u }{\partial \eta }\right) Cov\left( \hat{\beta },\hat{\eta }\right) \,\!</math>

or:

::<math> Var(\hat{u}) =\frac{\hat{u}^{2}}{\hat{\beta }^{2}}Var(\hat{ \beta })+\frac{\hat{\beta }^{2}}{\hat{\eta }^{2}}Var(\hat{\eta }) -\left( \frac{2\hat{u}}{\hat{\eta }}\right) Cov\left( \hat{\beta }, \hat{\eta }\right). \,\!</math>

The upper and lower bounds on reliability are:

::<math> R_{U} =e^{-e^{u_{L}}}\text{ (upper bound)}\,\!</math>

::<math> R_{L} =e^{-e^{u_{U}}}\text{ (lower bound)}\,\!</math>

'''Other Weibull Forms'''

Weibull++ makes the following assumptions/substitutions when using the three-parameter or one-parameter forms:

*For the 3-parameter case, substitute <math> t=\ln (t-\hat{\gamma }) \,\!</math> (and by definition <math>\gamma\, < t\!</math>), instead of <math>\ln t\,\!</math>. (Note that this is an approximation since it eliminates the third parameter and assumes that <math> Var( \hat{\gamma })=0. \,\!</math>)
*For the 1-parameter, <math> Var(\hat{\beta })=0, \,\!</math> thus:

::<math> Var(\hat{u})=\left( \frac{\partial u}{\partial \eta }\right) ^{2}Var( \hat{\eta })=\left( \frac{\hat{\beta }}{\hat{\eta }}\right) ^{2}Var(\hat{\eta }) \,\!</math>

Also note that the time axis (x-axis) in the three-parameter Weibull plot in Weibull++ is not <math>{t}\,\!</math> but <math>t - \gamma\,\!</math>. This means that one must be cautious when obtaining confidence bounds from the plot. If one desires to estimate the confidence bounds on reliability for a given time <math>{{t}_{0}}\,\!</math> from the adjusted plotted line, then these bounds should be obtained for a <math>{{t}_{0}} - \gamma\,\!</math> entry on the time axis.

=== Bounds on Time ===
The bounds around the time estimate or reliable life estimate, for a given Weibull percentile (unreliability), are estimated by first solving the reliability equation with respect to time, as discussed in Lloyd and Lipow [[Appendix:_Life_Data_Analysis_References|[24]]] and in Nelson [[Appendix:_Life_Data_Analysis_References|[30]]]:

::<math> \ln R =-\left( \frac{t}{\eta }\right) ^{\beta }
\,\!</math>

::<math> \ln (-\ln R) =\beta \ln \left( \frac{t}{\eta }\right) \,\!</math>

::<math>\begin{align}
\ln (-\ln R) =\beta (\ln t-\ln \eta )
\end{align}\,\!</math>

or:

::<math> u=\frac{1}{\beta }\ln (-\ln R)+\ln \eta \,\!</math>

where <math>u = \ln t\,\!</math> .

The upper and lower bounds on are estimated from:

::<math> u_{U} =\hat{u}+K_{\alpha }\sqrt{Var(\hat{u})} \,\!</math>

::<math> u_{L} =\hat{u}-K_{\alpha }\sqrt{Var(\hat{u})} \,\!</math>

where:

::<math> Var(\hat{u})=\left( \frac{\partial u}{\partial \beta }\right) ^{2}Var( \hat{\beta })+\left( \frac{\partial u}{\partial \eta }\right) ^{2}Var( \hat{\eta })+2\left( \frac{\partial u}{\partial \beta }\right) \left( \frac{\partial u}{\partial \eta }\right) Cov\left( \hat{\beta },\hat{ \eta }\right) \,\!</math>

or:
::<math> Var(\hat{u}) =\frac{1}{\hat{\beta }^{4}}\left[ \ln (-\ln R)\right] ^{2}Var(\hat{\beta })+\frac{1}{\hat{\eta }^{2}}Var(\hat{\eta })+2\left( -\frac{\ln (-\ln R)}{\hat{\beta }^{2}}\right) \left( \frac{1}{ \hat{\eta }}\right) Cov\left( \hat{\beta },\hat{\eta }\right) \,\!</math>

The upper and lower bounds are then found by:

::<math> T_{U} =e^{u_{U}}\text{ (upper bound)} \,\!</math>

::<math> T_{L} =e^{u_{L}}\text{ (lower bound)} \,\!</math>

== Likelihood Ratio Confidence Bounds ==
As covered in [[Confidence Bounds]], the likelihood confidence bounds are calculated by finding values for <math>{{\theta}_{1}}\,\!</math> and <math>{{\theta}_{2}}\,\!</math> that satisfy:

::<math> -2\cdot \text{ln}\left( \frac{L(\theta _{1},\theta _{2})}{L(\hat{\theta }_{1}, \hat{\theta }_{2})}\right) =\chi _{\alpha ;1}^{2} \,\!</math>

This equation can be rewritten as:

::<math> L(\theta _{1},\theta _{2})=L(\hat{\theta }_{1},\hat{\theta } _{2})\cdot e^{\frac{-\chi _{\alpha ;1}^{2}}{2}} \,\!</math>

For complete data, the likelihood function for the Weibull distribution is given by:

::<math> L(\beta ,\eta )=\prod_{i=1}^{N}f(x_{i};\beta ,\eta )=\prod_{i=1}^{N}\frac{ \beta }{\eta }\cdot \left( \frac{x_{i}}{\eta }\right) ^{\beta -1}\cdot e^{-\left( \frac{x_{i}}{\eta }\right) ^{\beta }} \,\!</math>

For a given value of <math>\alpha\,\!</math>, values for <math>\beta\,\!</math> and <math>\eta\,\!</math> can be found which represent the maximum and minimum values that satisfy the above equation. These represent the confidence bounds for the parameters at a confidence level <math>\delta\,\!</math>, where <math>\alpha = \delta\,\!</math> for two-sided bounds and <math>\alpha = 2\delta - 1\,\!</math> for one-sided.

Similarly, the bounds on time and reliability can be found by substituting the Weibull reliability equation into the likelihood function so that it is in terms of <math>\beta\,\!</math> and time or reliability, as discussed in [[Confidence Bounds]]. The likelihood ratio equation used to solve for bounds on time (Type 1) is:

::<math> L(\beta ,t)=\prod_{i=1}^{N}\frac{\beta }{\left( \frac{t}{(-\text{ln}(R))^{ \frac{1}{\beta }}}\right) }\cdot \left( \frac{x_{i}}{\left( \frac{t}{(-\text{ ln}(R))^{\frac{1}{\beta }}}\right) }\right) ^{\beta -1}\cdot \text{exp}\left[ -\left( \frac{x_{i}}{\left( \frac{t}{(-\text{ln}(R))^{\frac{1}{\beta }}} \right) }\right) ^{\beta }\right] \,\!</math>

The likelihood ratio equation used to solve for bounds on reliability (Type 2) is:

::<math> L(\beta ,R)=\prod_{i=1}^{N}\frac{\beta }{\left( \frac{t}{(-\text{ln}(R))^{ \frac{1}{\beta }}}\right) }\cdot \left( \frac{x_{i}}{\left( \frac{t}{(-\text{ ln}(R))^{\frac{1}{\beta }}}\right) }\right) ^{\beta -1}\cdot \text{exp}\left[ -\left( \frac{x_{i}}{\left( \frac{t}{(-\text{ln}(R))^{\frac{1}{\beta }}} \right) }\right) ^{\beta }\right] \,\!</math>

== Bayesian Confidence Bounds ==
=== Bounds on Parameters ===
Bayesian Bounds use non-informative prior distributions for both parameters. From [[Confidence Bounds]], we know that if the prior distribution of <math>\eta\,\!</math> and <math>\beta\,\!</math> are independent, the posterior joint distribution of <math>\eta\,\!</math> and <math>\beta\,\!</math> can be written as:

::<math> f(\eta ,\beta |Data)= \dfrac{L(Data|\eta ,\beta )\varphi (\eta )\varphi (\beta )}{\int_{0}^{\infty }\int_{0}^{\infty }L(Data|\eta ,\beta )\varphi (\eta )\varphi (\beta )d\eta d\beta } \,\!</math>

The marginal distribution of <math>\eta\,\!</math> is:

::<math> f(\eta |Data) =\int_{0}^{\infty }f(\eta ,\beta |Data)d\beta =
\dfrac{\int_{0}^{\infty }L(Data|\eta ,\beta )\varphi (\eta )\varphi (\beta )d\beta }{\int_{0}^{\infty }\int_{-\infty }^{\infty }L(Data|\eta ,\beta )\varphi (\eta )\varphi (\beta )d\eta d\beta }
\,\!</math>

where: <math> \varphi (\beta )=\frac{1}{\beta } \,\!</math> is the non-informative prior of <math>\beta\,\!</math>. <math> \varphi (\eta )=\frac{1}{\eta } \,\!</math> is the non-informative prior of <math>\eta\,\!</math>. Using these non-informative prior distributions, <math>f(\eta|Data)\,\!</math> can be rewritten as:

::<math> f(\eta |Data)=\dfrac{\int_{0}^{\infty }L(Data|\eta ,\beta )\frac{1}{\beta } \frac{1}{\eta }d\beta }{\int_{0}^{\infty }\int_{0}^{\infty }L(Data|\eta ,\beta )\frac{1}{\beta }\frac{1}{\eta }d\eta d\beta } \,\!</math>

The one-sided upper bounds of <math>\eta\,\!</math> is:

::<math> CL=P(\eta \leq \eta _{U})=\int_{0}^{\eta _{U}}f(\eta |Data)d\eta \,\!</math>

The one-sided lower bounds of <math>\eta\,\!</math> is:

::<math> 1-CL=P(\eta \leq \eta _{L})=\int_{0}^{\eta _{L}}f(\eta |Data)d\eta \,\!</math>

The two-sided bounds of <math>\eta\,\!</math> is:

::<math> CL=P(\eta _{L}\leq \eta \leq \eta _{U})=\int_{\eta _{L}}^{\eta _{U}}f(\eta |Data)d\eta \,\!</math>

Same method is used to obtain the bounds of <math>\beta\,\!</math>.

=== Bounds on Reliability ===
::<math> CL=\Pr (R\leq R_{U})=\Pr (\eta \leq T\exp (-\frac{\ln (-\ln R_{U})}{\beta })) \,\!</math>

From the posterior distribution of <math>\eta\,\!</math> we have:

::<math> CL=\dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{T\exp (-\dfrac{\ln (-\ln R_{U})}{\beta })}L(\beta ,\eta )\frac{1}{\beta }\frac{1}{\eta }d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\frac{1}{\beta }\frac{1}{\eta }d\eta d\beta } \,\!</math>

The above equation is solved numerically for <math>{{R}_{U}}\,\!</math>. The same method can be used to calculate the one sided lower bounds and two-sided bounds on reliability.

=== Bounds on Time ===
From [[Confidence Bounds]], we know that:

::<math> CL=\Pr (T\leq T_{U})=\Pr (\eta \leq T_{U}\exp (-\frac{\ln (-\ln R)}{\beta })) \,\!</math>

From the posterior distribution of <math>\eta\,\!</math>, we have:

::<math> CL=\dfrac{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{T_{U}\exp (-\dfrac{ \ln (-\ln R)}{\beta })}L(\beta ,\eta )\frac{1}{\beta }\frac{1}{\eta }d\eta d\beta }{\int\nolimits_{0}^{\infty }\int\nolimits_{0}^{\infty }L(\beta ,\eta )\frac{1}{\beta }\frac{1}{\eta }d\eta d\beta } \,\!</math>

The above equation is solved numerically for <math>{{T}_{U}}\,\!</math>. The same method can be applied to calculate one sided lower bounds and two-sided bounds on time.

== Bayesian-Weibull Analysis ==
{{:Bayesian-Weibull_Analysis}}

==Weibull Distribution Examples==
{{:Weibull Distribution Examples}}

ReliaSoft's Alternate Ranking Method

2022-12-15T16:44:51Z

M Spivey: Fixed subscript typo.

<noinclude>{{Banner Weibull Articles}}
''This article appears in the [[Appendix:_Special_Analysis_Methods#ReliaSoft_Ranking_Method|Life Data Analysis Reference book]].''

</noinclude>
In probability plotting or rank regression analysis of '''interval''' or '''left censored''' data, difficulties arise when attempting to estimate the exact time within the interval when the failure actually occurs, especially when an overlap on the intervals is present. In this case, the ''standard ranking method'' (SRM) is not applicable when dealing with interval data; thus, ReliaSoft has formulated a more sophisticated methodology to allow for more accurate probability plotting and regression analysis of data sets with interval or left censored data. This method utilizes the traditional rank regression method and iteratively improves upon the computed ranks by parametrically recomputing new ranks and the most probable failure time for interval data.

In the traditional method for plotting or rank regression analysis of '''right censored''' data, the effect of the exact censoring time is not considered. One example of this can be seen at the [[Parameter_Estimation#Shortfalls_of_the_Rank_Adjustment_Method|parameter estimation]] chapter. The ReliaSoft ranking method also can be used to overcome this shortfall of the standard ranking method.

The following step-by-step example illustrates the ReliaSoft ranking method (RRM), which is an iterative improvement on the standard ranking method (SRM). Although this method is illustrated by the use of the two-parameter Weibull distribution, it can be easily generalized for other models.

Consider the following test data:

{|border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|-
|colspan="4" style="text-align:center"|Table B.1- The Test Data
|-
!Number of Items
!Type
!Last Inspection
!Time
|-align="center"
|1||Exact Failure|| ||10
|-align="center"
|1||Right Censored|| ||20
|- align="center"
|2||Left Censored||0||30
|-align="center"
|2||Exact Failure|| ||40
|-align="center"
|1||Exact Failure|| ||50
|- align="center"
|1||Right Censored|| ||60
|- align="center"
|1||Left Censored||0||70
|- align="center"
|2||Interval Failure||20||80
|- align="center"
|1||Interval Failure||10||85
|- align="center"
|1||Left Censored||0||100
|}

=== Initial Parameter Estimation===
As a preliminary step, we need to provide a crude estimate of the Weibull parameters for this data. To begin, we will extract the exact times-to-failure (10, 40, and 50) and the midpoints of the interval failures. The midpoints are 50 (for the interval of 20 to 80) and 47.5 (for the interval of 10 to 85). Next, we group together the items that have the same failure times, as shown in Table B.2.

Using the traditional rank regression, we obtain the first initial estimates:

::<math>\begin{align}
& {{\widehat{\beta }}_{0}}= & 1.91367089 \\
& {{\widehat{\eta }}_{0}}= & 43.91657736
\end{align}\,\!</math>

{|border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|colspan="4" style="text-align:center"|Table B.2- The Union of Exact Times-to-Failure with the "Midpoint" of the Interval Failures
|-
!Number of Items
!Type
!Last Inspection
!Time
|-
|1||Exact Failure|| ||10
|-
|2||Exact Failure|| ||40
|-
|1||Exact Failure|| ||47.5
|-
|3||Exact Failure|| ||50
|}

'''Step 1'''

For all intervals, we obtain a weighted ''midpoint'' using:

::<math>\begin{align}
{{{\hat{t}}}_{m}}\left( \hat{\beta },\hat{\eta } \right)= & \frac{\int_{LI}^{TF}t\text{ }f(t;\hat{\beta },\hat{\eta })dt}{\int_{LI}^{TF}f(t;\hat{\beta },\hat{\eta })dt}, \\
= & \frac{\int_{LI}^{TF}t\tfrac{{\hat{\beta }}}{{\hat{\eta }}}{{\left( \tfrac{t}{{\hat{\eta }}} \right)}^{\hat{\beta }-1}}{{e}^{-{{\left( \tfrac{t}{{\hat{\eta }}} \right)}^{{\hat{\beta }}}}}}dt}{\int_{LI}^{TF}\tfrac{{\hat{\beta }}}{{\hat{\eta }}}{{\left( \tfrac{t}{{\hat{\eta }}} \right)}^{\hat{\beta }-1}}{{e}^{-{{\left( \tfrac{t}{{\hat{\eta }}} \right)}^{{\hat{\beta }}}}}}dt}
\end{align}\,\!</math>

This transforms our data into the format in Table B.3.

{|border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|colspan="5" style="text-align:center"|Table B.3- The Union of Exact Times-to-Failure with the "Midpoint" of the Interval Failures, Based upon the Parameters <math>\beta\,\!</math> and <math>\eta\,\!</math>.
|-
!Number of Items
!Type
!Last Inspection
!Time
!Weighted "Midpoint"
|- align="center"
|1||Exact Failure|| ||10 ||
|- align="center"
|2||Exact Failure|| ||40||
|-align="center"
|1||Exact Failure|| || 50||
|-align="center"
|2||Interval Failure||20||80||42.837
|-align="center"
|1||Interval Failure||10||85||39.169
|}

'''Step 2'''

Now we arrange the data as in Table B.4.

{|border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|colspan="2"|Table B.4- The Union of Exact Times-to-Failure with the "Midpoint" of the Interval Failures, in Ascending Order.
|-align="center"
!Number of Items
!Time
|- align="center"
|1||10
|- align="center"
|1||39.169
|- align="center"
|2||40
|- align="center"
|2||42.837
|- align="center"
|1||50
|}

'''Step 3'''

We now consider the left and right censored data, as in Table B.5.

{|border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|-
|colspan="7" style="text-align:center"|Table B.5- Computation of Increments in a Matrix Format for Computing a Revised Mean Order Number.
|-
!Number of items
!Time of Failure
!2 Left Censored ''t'' = 30
!1 Left Censored ''t'' = 70
!1 Left Censored ''t'' = 100
!1 Right Censored ''t'' = 20
!1 Right Censored ''t'' = 60
|-
|1||10||<math>2 \frac{\int_0^{10} f_0(t)dt}{F_0 (30)-F_0 (0)}\,\!</math> ||<math>\frac{\int_0^{10} f_0 (t)dt}{F_0(70)-F_0(0)}\,\!</math> || <math>\frac{\int_0^{10} f_0(t)dt}{F_0(100)-F_0(0)}\,\!</math> || 0||0
|-
|1||39.169||<math>2 \frac{\int_{10}^{30} f_0(t)dt}{F_0(30)-F_0(0)}\,\!</math> ||<math>\frac{\int_{10}^{39.169} f_0(t)dt}{F_0(70)-F_0(0)}\,\!</math> ||<math>\frac{\int_{10}^{39.169} f_0(t)dt}{F_0(100)-F_0(0)}\,\!</math> || <math>\frac{\int_{20}^{39.169} f_0(t)dt}{F_0(\infty)-F_0(20)}\,\!</math>||0
|-
|2||40||0||<math>\frac{\int_{39.169}^{40} f_0(t)dt}{F_0(70)-F_0(0)}\,\!</math> || <math>\frac{\int_{39.169}^{40} f_0(t)dt}{F_0(100)-F_0(0)}\,\!</math> ||<math>\frac{\int_{39.169}^{40} f_0(t)dt}{F_0(\infty)-F_0(20)}\,\!</math> ||0
|-
|2||42.837||0|| <math>\frac{\int_{40}^{42.837} f_0(t)dt}{F_0(70)-F_0(0)}\,\!</math> || <math>\frac{\int_{40}^{42.837} f_0(t)dt}{F_0(100)-F_0(0)}\,\!</math>|| <math>\frac{\int_{40}^{42.837} f_0(t)dt}{F_0(\infty)-F_0(0)}\,\!</math>||0
|-
|1||50||0||<math>\frac{\int_{42.837}^{50} f_0(t)dt}{F_0(70)-F_0(0)}\,\!</math> ||<math>\frac{\int_{42.837}^{50} f_0(t)dt}{F_0(100)-F_0(0)}\,\!</math> || <math>\frac{\int_{42.837}^{50} f_0(t)dt}{F_0(\infty)-F_0(0)}\,\!</math>||0
|}

In general, for left censored data:

:• The increment term for <math>n\,\!</math> left censored items at time <math>={{t}_{0}},\,\!</math> with a time-to-failure of <math>{{t}_{i}}\,\!</math> when <math>{{t}_{0}}\le {{t}_{i-1}}\,\!</math> is zero.
:• When <math>{{t}_{0}}>{{t}_{i-1}},\,\!</math> the contribution is:

::<math>\frac{n}{{{F}_{0}}({{t}_{0}})-{{F}_{0}}(0)}\underset{{{t}_{i-1}}}{\overset{MIN({{t}_{i}},{{t}_{0}})}{\mathop \int }}\,{{f}_{0}}\left( t \right)dt\,\!</math>

:or:

::<math>n\frac{{{F}_{0}}(MIN({{t}_{i}},{{t}_{0}}))-{{F}_{0}}({{t}_{i-1}})}{{{F}_{0}}({{t}_{0}})-{{F}_{0}}(0)}\,\!</math>

where <math>{{t}_{i-1}}\,\!</math> is the time-to-failure previous to the <math>{{t}_{i}}\,\!</math> time-to-failure and <math>n\,\!</math> is the number of units associated with that time-to-failure (or units in the group).

In general, for right censored data:
:• The increment term for <math>n\,\!</math> right censored at time <math>={{t}_{0}},\,\!</math> with a time-to-failure of <math>{{t}_{i}}\,\!</math>, when <math>{{t}_{0}}\ge {{t}_{i}}\,\!</math> is zero.
:• When <math>{{t}_{0}}<{{t}_{i}},\,\!</math> the contribution is:

::<math>\frac{n}{{{F}_{0}}(\infty )-{{F}_{0}}({{t}_{0}})}\underset{MAX({{t}_{0}},{{t}_{i-1}})}{\overset{{{t}_{i}}}{\mathop \int }}\,{{f}_{0}}\left( t \right)dt\,\!</math>

:or:

::<math>n\frac{{{F}_{0}}({{t}_{i}})-{{F}_{0}}(MAX({{t}_{0}},{{t}_{i-1}}))}{{{F}_{0}}(\infty )-{{F}_{0}}({{t}_{0}})}\,\!</math>

where <math>{{t}_{i-1}}\,\!</math> is the time-to-failure previous to the <math>{{t}_{i}}\,\!</math> time-to-failure and <math>n\,\!</math> is the number of units associated with that time-to-failure (or units in the group).

'''Step 4'''

Sum up the increments (horizontally in rows), as in Table B.6.

{|border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|-
|colspan="8" style="text-align:center"|Table B.6- Increments Solved Numerically, Along with the Sum of Each Row.
|-
!Number of items
!Time of Failure
!2 Left Censored ''t''=30
!1 Left Censored ''t''=70
!1 Left Censored ''t''=100
!1 Right Censored ''t''=20
!1 Right Censored ''t''=60
!Sum of row(increment)
|-
|1||10||0.299065||0.062673||0.057673||0||0||0.419411
|-
|1||39.169||1.700935||0.542213||0.498959||0.440887||0||3.182994
|-
|2||40||0||0.015892||0.014625||0.018113||0||0.048630
|-
|2||42.831||0||0.052486||0.048299||0.059821||0||0.160606
|-
|1||50||0||0.118151||0.108726||0.134663||0||0.361540
|}

'''Step 5'''

Compute new mean order numbers (MON), as shown Table B.7, utilizing the increments obtained in Table B.6, by adding the ''number of items'' plus the ''previous MON'' plus the current increment.

{|border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|-
|colspan="4" style="text-align:center"|Table B.7- Mean Order Numbers (MON)
|-
!Number of items
!Time of Failure
!Sum of row(increment)
!Mean Order Number
|-
|1||10||0.419411||1.419411
|-
|1||39.169||3.182994||5.602405
|-
|2||40||0.048630||7.651035
|-
|2||42.837||0.160606||9.811641
|-
|1||50||0.361540||11.173181
|}

'''Step 6'''

Compute the median ranks based on these new MONs as shown in Table B.8.

{|border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|colspan="3" style="text-align:center"|Table B.8- Mean Order Numbers with Their Ranks for a Sample Size of 13 Units.
|-
!Time
!MON
!Ranks
|-
|10||1.419411||0.0825889
|-
|39.169||5.602405||0.3952894
|-
|40||7.651035||0.5487781
|-
|42.837||9.811641||0.7106217
|-
|50||11.173181||0.8124983
|}

'''Step 7'''

Compute new <math>\beta \,\!</math> and <math>\eta ,\,\!</math> using standard rank regression and based upon the data as shown in Table B.9.
 
{|border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
!Time
!Ranks
|-
|10||0.0826889
|-
|39.169||0.3952894
|-
|40||0.5487781
|-
|42.837||0.7106217
|-
|50||0.8124983
|}

'''Step 8'''
Return and repeat the process from Step 1 until an acceptable convergence is reached on the parameters (i.e., the parameter values stabilize).

===Results===
The results of the first five iterations are shown in Table B.10.
Using Weibull++ with rank regression on X yields:
 
{|border="1" align="center" style="border-collapse: collapse;" cellpadding="5" cellspacing="5"
|colspan="3" style="text-align:center;"|Table B.10-The parameters after the first five iterations
|-align="center"
!''Iteration''
!<math>\beta\,\!</math>
!<math>\eta\,\!</math>
|-align="center"
|1||1.845638||42.576422
|-align="center"
|2||1.830621 ||42.039743
|-align="center"
|3||1.828010 ||41.830615
|-align="center"
|4||1.828030 ||41.749708
|-align="center"
|5||1.828383 ||41.717990
|}

::<math>{{\widehat{\beta }}_{RRX}}=1.82890,\text{ }{{\widehat{\eta }}_{RRX}}=41.69774\,\!</math>

The direct MLE solution yields:

::<math>{{\widehat{\beta }}_{MLE}}=2.10432,\text{ }{{\widehat{\eta }}_{MLE}}=42.31535\,\!</math>

The Exponential Distribution

2022-11-14T20:38:30Z

M Spivey: /* Example: LR Bounds on Reliability */

{{template:LDABOOK|7|The Exponential Distribution}}
The exponential distribution is a commonly used distribution in reliability engineering. Mathematically, it is a fairly simple distribution, which many times leads to its use in inappropriate situations. It is, in fact, a special case of the Weibull distribution where <math>\beta =1\,\!</math>. The exponential distribution is used to model the behavior of units that have a constant failure rate (or units that do not degrade with time or wear out).

==Exponential Probability Density Function==
===The 2-Parameter Exponential Distribution===
The 2-parameter exponential ''pdf'' is given by:

::<math>f(t)=\lambda {{e}^{-\lambda (t-\gamma )}},f(t)\ge 0,\lambda >0,t\ge \gamma \,\!</math>

where <math>\gamma \,\!</math> is the location parameter.
Some of the characteristics of the 2-parameter exponential distribution are discussed in Kececioglu [[Appendix:_Life_Data_Analysis_References|[19]]]:
*The location parameter, <math>\gamma \,\!</math>, if positive, shifts the beginning of the distribution by a distance of <math>\gamma \,\!</math> to the right of the origin, signifying that the chance failures start to occur only after <math>\gamma \,\!</math> hours of operation, and cannot occur before.
*The scale parameter is <math>\tfrac{1}{\lambda }=\bar{t}-\gamma =m-\gamma \,\!</math>.
*The exponential ''pdf'' has no shape parameter, as it has only one shape.
*The distribution starts at <math>t=\gamma \,\!</math> at the level of <math>f(t=\gamma )=\lambda \,\!</math> and decreases thereafter exponentially and monotonically as <math>t\,\!</math> increases beyond <math>\gamma \,\!</math> and is convex.
*As <math>t\to \infty \,\!</math>, <math>f(t)\to 0\,\!</math>.

===The 1-Parameter Exponential Distribution===
The 1-parameter exponential ''pdf'' is obtained by setting <math>\gamma =0\,\!</math>, and is given by:

::<math> \begin{align}f(t)= & \lambda {{e}^{-\lambda t}}=\frac{1}{m}{{e}^{-\tfrac{1}{m}t}},
& t\ge 0, \lambda >0,m>0
\end{align}
\,\!</math>

where:

::<math>\lambda \,\!</math> = constant rate, in failures per unit of measurement, (e.g., failures per hour, per cycle, etc.)

::<math>\lambda =\frac{1}{m}\,\!</math>
::<math>m\,\!</math> = mean time between failures, or to failure
::<math>t\,\!</math> = operating time, life, or age, in hours, cycles, miles, actuations, etc.

This distribution requires the knowledge of only one parameter, <math>\lambda \,\!</math>, for its application. Some of the characteristics of the 1-parameter exponential distribution are discussed in Kececioglu [[Appendix:_Life_Data_Analysis_References| [19]]]:
*The location parameter, <math>\gamma \,\!</math>, is zero.
*The scale parameter is <math>\tfrac{1}{\lambda }=m\,\!</math>.
*As <math>\lambda \,\!</math> is decreased in value, the distribution is stretched out to the right, and as <math>\lambda \,\!</math> is increased, the distribution is pushed toward the origin.
*This distribution has no shape parameter as it has only one shape, (i.e., the exponential, and the only parameter it has is the failure rate, <math>\lambda \,\!</math>).
*The distribution starts at <math>t=0\,\!</math> at the level of <math>f(t=0)=\lambda \,\!</math> and decreases thereafter exponentially and monotonically as <math>t\,\!</math> increases, and is convex.
*As <math>t\to \infty \,\!</math>, <math>f(t)\to 0\,\!</math>.
*The ''pdf'' can be thought of as a special case of the Weibull ''pdf'' with <math>\gamma =0\,\!</math> and <math>\beta =1\,\!</math>.

==Exponential Distribution Functions==
{{:Exponential Distribution Functions}}

==Characteristics of the Exponential Distribution==
{{:Exponential Distribution Characteristics}}

==Estimation of the Exponential Parameters==
===Probability Plotting===
Estimation of the parameters for the exponential distribution via probability plotting is very similar to the process used when dealing with the Weibull distribution. Recall, however, that the appearance of the probability plotting paper and the methods by which the parameters are estimated vary from distribution to distribution, so there will be some noticeable differences. In fact, due to the nature of the exponential ''cdf'', the exponential probability plot is the only one with a negative slope. This is because the y-axis of the exponential probability plotting paper represents the reliability, whereas the y-axis for most of the other life distributions represents the unreliability.

This is illustrated in the process of linearizing the ''cdf'', which is necessary to construct the exponential probability plotting paper. For the two-parameter exponential distribution the cumulative density function is given by:

::<math>\begin{align}
F(t)=1-{{e}^{-\lambda (t-\gamma )}}
\end{align}\,\!</math>

Taking the natural logarithm of both sides of the above equation yields:

::<math>\ln \left[ 1-F(t) \right]=-\lambda (t-\gamma )\,\!</math>

or:

::<math>\begin{align}
\ln [1-F(t)]=\lambda \gamma -\lambda t
\end{align}\,\!</math>

Now, let:

::<math>\begin{align}
y=\ln [1-F(t)]
\end{align}\,\!</math>

::<math>\begin{align}
a=\lambda \gamma
\end{align}\,\!</math>

and:

::<math>\begin{align}
b=-\lambda
\end{align}\,\!</math>

which results in the linear equation of:

::<math>\begin{align}
y=a+bt
\end{align}\,\!</math>

Note that with the exponential probability plotting paper, the y-axis scale is logarithmic and the x-axis scale is linear. This means that the zero value is present only on the x-axis. For <math>t=0\,\!</math>, <math>R=1\,\!</math> and <math>F(t)=0\,\!</math>. So if we were to use <math>F(t)\,\!</math> for the y-axis, we would have to plot the point <math>(0,0)\,\!</math>. However, since the y-axis is logarithmic, there is no place to plot this on the exponential paper. Also, the failure rate, <math>\lambda \,\!</math>, is the negative of the slope of the line, but there is an easier way to determine the value of <math>\lambda \,\!</math> from the probability plot, as will be illustrated in the following example.
====Plotting Example====
{{:1P Exponential Example}}

===Rank Regression on Y===
Performing a rank regression on Y requires that a straight line be fitted to the set of available data points such that the sum of the squares of the vertical deviations from the points to the line is minimized.
The least squares parameter estimation method (regression analysis) was discussed in [[Parameter Estimation]], and the following equations for rank regression on Y (RRY) were derived:

::<math>\hat{a}=\bar{y}-\hat{b}\bar{x}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}-\hat{b}\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}}{N}\,\!</math>

and:

::<math>\hat{b}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,x_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}} \right)}^{2}}}{N}}\,\!</math>

In our case, the equations for <math>{{y}_{i}}\,\!</math> and <math>{{x}_{i}}\,\!</math> are:

::<math>\begin{align}
{{y}_{i}}=\ln [1-F({{t}_{i}})]
\end{align}\,\!</math>

and:

::<math>\begin{align}
{{x}_{i}}={{t}_{i}}
\end{align}\,\!</math>

and the <math>F({{t}_{i}})\,\!</math> is estimated from the median ranks. Once <math>\hat{a}\,\!</math> and <math>\hat{b}\,\!</math> are obtained, then <math>\hat{\lambda }\,\!</math> and <math>\hat{\gamma }\,\!</math> can easily be obtained from above equations.
For the one-parameter exponential, equations for estimating ''a'' and ''b'' become:

::<math>\begin{align}
\hat{a}= & 0, \\
\hat{b}= & \frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,x_{i}^{2}}
\end{align}\,\!</math>

'''The Correlation Coefficient'''

The estimator of <math>\rho \,\!</math> is the sample correlation coefficient, <math>\hat{\rho }\,\!</math>, given by:

::<math>\hat{\rho }=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,({{x}_{i}}-\overline{x})({{y}_{i}}-\overline{y})}{\sqrt{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{({{x}_{i}}-\overline{x})}^{2}}\cdot \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{({{y}_{i}}-\overline{y})}^{2}}}}\,\!</math>

====RRY Example==== 
{{:2P_Exponential_Example}}

===Rank Regression on X===
Similar to rank regression on Y, performing a rank regression on X requires that a straight line be fitted to a set of data points such that the sum of the squares of the horizontal deviations from the points to the line is minimized.

Again the first task is to bring our exponential ''cdf'' function into a linear form. This step is exactly the same as in regression on Y analysis. The deviation from the previous analysis begins on the least squares fit step, since in this case we treat <math>x\,\!</math> as the dependent variable and <math>y\,\!</math> as the independent variable. The best-fitting straight line to the data, for regression on X (see [[Parameter Estimation]]), is the straight line:

::<math>x=\hat{a}+\hat{b}y\,\!</math>

The corresponding equations for <math>\hat{a}\,\!</math> and <math>\hat{b}\,\!</math> are:

::<math>\hat{a}=\overline{x}-\hat{b}\overline{y}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}}{N}-\hat{b}\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}\,\!</math>

and:

::<math>\hat{b}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,y_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}} \right)}^{2}}}{N}}\,\!</math>

where:

::<math>\begin{align}
{{y}_{i}}=\ln [1-F({{t}_{i}})]
\end{align}\,\!</math>

and:

::<math>\begin{align}
{{x}_{i}}={{t}_{i}}
\end{align}\,\!</math>

The values of <math>F({{t}_{i}})\,\!</math> are estimated from the median ranks. Once <math>\hat{a}\,\!</math> and <math>\hat{b}\,\!</math> are obtained, solve for the unknown <math>y\,\!</math> value, which corresponds to:

::<math>y=-\frac{\hat{a}}{\hat{b}}+\frac{1}{\hat{b}}x\,\!</math>

Solving for the parameters from above equations we get:

::<math>a=-\frac{\hat{a}}{\hat{b}}=\lambda \gamma \Rightarrow \gamma =\hat{a}\,\!</math>

and:

::<math>b=\frac{1}{\hat{b}}=-\lambda \Rightarrow \lambda =-\frac{1}{\hat{b}}\,\!</math>

For the one-parameter exponential case, equations for estimating a and b become:

::<math>\begin{align}
\hat{a}= & 0 \\
\hat{b}= & \frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,y_{i}^{2}}
\end{align}\,\!</math>

The correlation coefficient is evaluated as before.

====RRX Example====
'''2-Parameter Exponential RRX Example'''

Using the same data set from the [[The_Exponential_Distribution#RRY_Example|RRY example above]] and assuming a 2-parameter exponential distribution, estimate the parameters and determine the correlation coefficient estimate, <math>\hat{\rho }\,\!</math>, using rank regression on X.

''' Solution'''

The table constructed for the RRY analysis applies to this example also. Using the values from this table, we get:

::<math>\begin{align}
\hat{b}= & \frac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{t}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{t}_{i}}\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{y}_{i}}}{14}}{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,y_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{y}_{i}} \right)}^{2}}}{14}} \\
\\
\hat{b}= & \frac{-927.4899-(630)(-13.2315)/14}{22.1148-{{(-13.2315)}^{2}}/14}
\end{align}\,\!</math>

or:

::<math>\hat{b}=-34.5563\,\!</math>

and:

::<math>\hat{a}=\overline{x}-\hat{b}\overline{y}=\frac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{t}_{i}}}{14}-\hat{b}\frac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{y}_{i}}}{14}\,\!</math>

or:

::<math>\hat{a}=\frac{630}{14}-(-34.5563)\frac{(-13.2315)}{14}=12.3406\,\!</math>

Therefore:

::<math>\hat{\lambda }=-\frac{1}{\hat{b}}=-\frac{1}{(-34.5563)}=0.0289\text{ failures/hour}\,\!</math>

and:

::<math>\hat{\gamma }=\hat{a}=12.3406\,\!</math>

The correlation coefficient is found to be:

::<math>\hat{\rho }=-0.9679\,\!</math>

Note that the equation for regression on Y is not necessarily the same as that for the regression on X. The only time when the two regression methods yield identical results is when the data lie perfectly on a line. If this were the case, the correlation coefficient would be <math>-1\,\!</math>. The negative value of the correlation coefficient is due to the fact that the slope of the exponential probability plot is negative.

This example can be repeated using Weibull++, choosing two-parameter exponential and rank regression on X (RRX) methods for analysis, as shown below.
The estimated parameters and the correlation coefficient using Weibull++ were found to be:

::<math>\begin{array}{*{35}{l}}
\hat{\lambda }= &0.0289 \text{failures/hour} \\
\hat{\gamma}= & 12.3395 \text{hours} \\
\hat{\rho} = &-0.9679 \\
\end{array}\,\!</math>

[[Image:Exponential Distribution Example 3 Data Folio.png|center|700px|]]

The probability plot can be obtained simply by clicking the '''Plot''' icon.

[[Image:Exponential Distribution Example 3 Plot.png|center|600px|]]

===Maximum Likelihood Estimation===
As outlined in [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood equation with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the exponential distribution are covered in [[Appendix:_Log-Likelihood_Equations|Appendix D]].

====MLE Example====
'''MLE for the Exponential Distribution'''

Using the same data set from the [[The_Exponential_Distribution#RRY_Example|RRY and RRX examples above]] and assuming a 2-parameter exponential distribution, estimate the parameters using the MLE method.

'''Solution'''

In this example, we have complete data only. The partial derivative of the log-likelihood function, <math>\Lambda ,\,\!</math> is given by:

::<math>\frac{\partial \Lambda }{\partial \lambda }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right]=\underset{i=1}{\overset{14}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right]=0\,\!</math>

Complete descriptions of the partial derivatives can be found in [[Appendix:_Log-Likelihood_Equations|Appendix D]]. Recall that when using the MLE method for the exponential distribution, the value of <math>\gamma \,\!</math> is equal to that of the first failure time. The first failure occurred at 5 hours, thus <math>\gamma =5\,\!</math> hours<math>.\,\!</math> Substituting the values for <math>T\,\!</math> and <math>\gamma \,\!</math> we get:

::<math>\frac{14}{\hat{\lambda }}=560\,\!</math>

or:

::<math>\hat{\lambda }=0.025\text{ failures/hour}\,\!</math>

Using Weibull++:

[[Image:Exponential Distribution Example 4 Data.png|center|700px|]]

The probability plot is:

[[Image:Exponential Distribution Example 4 Plot.png|center|650px|]]

==Confidence Bounds==
In this section, we present the methods used in the application to estimate the different types of confidence bounds for exponentially distributed data. The complete derivations were presented in detail (for a general function) in the chapter for [[Confidence Bounds]].
At this time we should point out that exact confidence bounds for the exponential distribution have been derived, and exist in a closed form, utilizing the <math>{{\chi }^{2}}\,\!</math> distribution. These are described in detail in Kececioglu [[Appendix:_Life_Data_Analysis_References|[20]]], and are covered in the section in the [[Reliability Test Design|test design chapter]]. For most exponential data analyses, Weibull++ will use the approximate confidence bounds, provided from the Fisher information matrix or the likelihood ratio, in order to stay consistent with all of the other available distributions in the application. The <math>{{\chi }^{2}}\,\!</math> confidence bounds for the exponential distribution are discussed in more detail in the [[Reliability Test Design|test design chapter]].

===Fisher Matrix Bounds===
====Bounds on the Parameters====
For the failure rate <math>\hat{\lambda }\,\!</math> the upper (<math>{{\lambda }_{U}}\,\!</math>) and lower (<math>{{\lambda }_{L}}\,\!</math>) bounds are estimated by Nelson [[Appendix:_Life_Data_Analysis_References|[30]]]:

::<math>\begin{align}
& {{\lambda }_{U}}= & \hat{\lambda }\cdot {{e}^{\left[ \tfrac{{{K}_{\alpha }}\sqrt{Var(\hat{\lambda })}}{\hat{\lambda }} \right]}} \\
& & \\
& {{\lambda }_{L}}= & \frac{\hat{\lambda }}{{{e}^{\left[ \tfrac{{{K}_{\alpha }}\sqrt{Var(\hat{\lambda })}}{\hat{\lambda }} \right]}}}
\end{align}\,\!</math>

where <math>{{K}_{\alpha }}\,\!</math> is defined by:

::<math>\alpha =\frac{1}{\sqrt{2\pi }}\int_{{{K}_{\alpha }}}^{\infty }{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt=1-\Phi ({{K}_{\alpha }})\,\!</math>

If <math>\delta \,\!</math> is the confidence level, then <math>\alpha =\tfrac{1-\delta }{2}\,\!</math> for the two-sided bounds, and <math>\alpha =1-\delta \,\!</math> for the one-sided bounds.

The variance of <math>\hat{\lambda },\,\!</math> <math>Var(\hat{\lambda }),\,\!</math> is estimated from the Fisher matrix, as follows:

::<math>Var(\hat{\lambda })={{\left( -\frac{{{\partial }^{2}}\Lambda }{\partial {{\lambda }^{2}}} \right)}^{-1}}\,\!</math>

where <math>\Lambda \,\!</math> is the log-likelihood function of the exponential distribution, described in [[Appendix:_Log-Likelihood_Equations|Appendix D]].

Note that, for fixed <math>\lambda \,\!</math>, the log-likelihood function is increasing in <math>\gamma</math>. This means that the MLE solution for <math>\gamma</math> cannot be found by setting <math>\tfrac{\partial \Lambda}{\partial \gamma}</math> to zero. (Since <math>\Lambda</math> is increasing in <math>\gamma</math>, <math>\tfrac{\partial \Lambda}{\partial \gamma}</math> can never be zero.) Instead, the MLE solution for <math>\gamma</math> is simply its largest possible value allowed by the sample; namely, <math>\gamma = t_1</math>, the first failure time. The MLE solution for <math>\lambda</math> is found in the usual fashion by setting <math>\tfrac{\partial \Lambda}{\partial \lambda}</math> to zero and solving. Weibull++ treats <math>\gamma</math> as a constant when computing bounds; i.e., <math>Var(\hat{\gamma}) = 0</math>. (See the discussion in Appendix D for more information.)

====Bounds on Reliability====
The reliability of the two-parameter exponential distribution is:

::<math>\hat{R}(t;\hat{\lambda })={{e}^{-\hat{\lambda }(t-\hat{\gamma })}}\,\!</math>

The corresponding confidence bounds are estimated from:

::<math>\begin{align}
& {{R}_{L}}= & {{e}^{-{{\lambda }_{U}}(t-\hat{\gamma })}} \\
& {{R}_{U}}= & {{e}^{-{{\lambda }_{L}}(t-\hat{\gamma })}}
\end{align}\,\!</math>

These equations hold true for the 1-parameter exponential distribution, with <math>\gamma =0\,\!</math>.

====Bounds on Time====
The bounds around time for a given exponential percentile, or reliability value, are estimated by first solving the reliability equation with respect to time, or reliable life:

::<math>\hat{t}=-\frac{1}{{\hat{\lambda }}}\cdot \ln (R)+\hat{\gamma }\,\!</math>

The corresponding confidence bounds are estimated from:

::<math>\begin{align}
& {{t}_{U}}= & -\frac{1}{{{\lambda }_{L}}}\cdot \ln (R)+\hat{\gamma } \\
& {{t}_{L}}= & -\frac{1}{{{\lambda }_{U}}}\cdot \ln (R)+\hat{\gamma }
\end{align}\,\!</math>

The same equations apply for the one-parameter exponential with <math>\gamma =0.\,\!</math>

===Likelihood Ratio Confidence Bounds===
====Bounds on Parameters====
For one-parameter distributions such as the exponential, the likelihood confidence bounds are calculated by finding values for <math>\theta \,\!</math> that satisfy:

::<math>-2\cdot \text{ln}\left( \frac{L(\theta )}{L(\hat{\theta })} \right)=\chi _{\alpha ;1}^{2}\,\!</math>

This equation can be rewritten as:

::<math>L(\theta )=L(\hat{\theta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}\,\!</math>

For complete data, the likelihood function for the exponential distribution is given by:

::<math>L(\lambda )=\underset{i=1}{\overset{N}{\mathop \prod }}\,f({{t}_{i}};\lambda )=\underset{i=1}{\overset{N}{\mathop \prod }}\,\lambda \cdot {{e}^{-\lambda \cdot {{t}_{i}}}}\,\!</math>

where the <math>{{t}_{i}}\,\!</math> values represent the original time-to-failure data. For a given value of <math>\alpha \,\!</math>, values for <math>\lambda \,\!</math> can be found which represent the maximum and minimum values that satisfy the above likelihood ratio equation. These represent the confidence bounds for the parameters at a confidence level <math>\delta ,\,\!</math> where <math>\alpha =\delta \,\!</math> for two-sided bounds and <math>\alpha =2\delta -1\,\!</math> for one-sided.

=====Example: LR Bounds for Lambda=====
Five units are put on a reliability test and experience failures at 20, 40, 60, 100, and 150 hours. Assuming an exponential distribution, the MLE parameter estimate is calculated to be <math>\hat{\lambda }=0.013514\,\!</math>. Calculate the 85% two-sided confidence bounds on these parameters using the likelihood ratio method.

'''Solution'''

The first step is to calculate the likelihood function for the parameter estimates:

::<math>\begin{align}
L(\hat{\lambda })= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\hat{\lambda })=\underset{i=1}{\overset{N}{\mathop \prod }}\,\hat{\lambda }\cdot {{e}^{-\hat{\lambda }\cdot {{x}_{i}}}} \\
L(\hat{\lambda })= & \underset{i=1}{\overset{5}{\mathop \prod }}\,0.013514\cdot {{e}^{-0.013514\cdot {{x}_{i}}}} \\
L(\hat{\lambda })= & 3.03647\times {{10}^{-12}}
\end{align}\,\!</math>

where <math>{{x}_{i}}\,\!</math> are the original time-to-failure data points. We can now rearrange the likelihood ratio equation to the form:

::<math>L(\lambda )-L(\hat{\lambda })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0\,\!</math>

Since our specified confidence level, <math>\delta \,\!</math>, is 85%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.85;1}^{2}=2.072251.\,\!</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\lambda )-L(\hat{\lambda })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0, \\
L(\lambda )-3.03647\times {{10}^{-12}}\cdot {{e}^{\tfrac{-2.072251}{2}}}= & 0, \\
L(\lambda )-1.07742\times {{10}^{-12}}= & 0.
\end{align}\,\!</math>

It now remains to find the values of <math>\lambda \,\!</math> which satisfy this equation. Since there is only one parameter, there are only two values of <math>\lambda \,\!</math> that will satisfy the equation. These values represent the <math>\delta =85%\,\!</math> two-sided confidence limits of the parameter estimate <math>\hat{\lambda }\,\!</math>. For our problem, the confidence limits are:

::<math>\begin{align}
{{\lambda }_{0.85}}=(0.006572,0.024172)
\end{align}\,\!</math>

====Bounds on Time and Reliability====
In order to calculate the bounds on a time estimate for a given reliability, or on a reliability estimate for a given time, the likelihood function needs to be rewritten in terms of one parameter and time/reliability, so that the maximum and minimum values of the time can be observed as the parameter is varied. This can be accomplished by substituting a form of the exponential reliability equation into the likelihood function. The exponential reliability equation can be written as:

::<math>R={{e}^{-\lambda \cdot t}}\,\!</math>

This can be rearranged to the form:

::<math>\lambda =\frac{-\text{ln}(R)}{t}\,\!</math>

This equation can now be substituted into the likelihood ratio equation to produce a likelihood equation in terms of <math>t\,\!</math> and <math>R:\,\!</math>

::<math>L(t/R)=\underset{i=1}{\overset{N}{\mathop \prod }}\,\left( \frac{-\text{ln}(R)}{t} \right)\cdot {{e}^{\left( \tfrac{\text{ln}(R)}{t} \right)\cdot {{x}_{i}}}}\,\!</math>

The unknown parameter <math>t/R\,\!</math> depends on what type of bounds are being determined. If one is trying to determine the bounds on time for the equation for the mean and the Bayes's rule equation for single parametera given reliability, then <math>R\,\!</math> is a known constant and <math>t\,\!</math> is the unknown parameter. Conversely, if one is trying to determine the bounds on reliability for a given time, then <math>t\,\!</math> is a known constant and <math>R\,\!</math> is the unknown parameter. Either way, the likelihood ratio function can be solved for the values of interest.

=====Example: LR Bounds on Time =====
For the data given above for the [[The_Exponential_Distribution#Example:_LR_Bounds_for_Lambda|LR Bounds on Lambda example]] (five failures at 20, 40, 60, 100 and 150 hours), determine the 85% two-sided confidence bounds on the time estimate for a reliability of 90%. The ML estimate for the time at <math>R(t)=90%\,\!</math> is <math>\hat{t}=7.797\,\!</math>.

'''Solution'''

In this example, we are trying to determine the 85% two-sided confidence bounds on the time estimate of 7.797. This is accomplished by substituting <math>R=0.90\,\!</math> and <math>\alpha =0.85\,\!</math> into the likelihood ratio bound equation. It now remains to find the values of <math>t\,\!</math> which satisfy this equation. Since there is only one parameter, there are only two values of <math>t\,\!</math> that will satisfy the equation. These values represent the <math>\delta =85%\,\!</math> two-sided confidence limits of the time estimate <math>\hat{t}\,\!</math>. For our problem, the confidence limits are:

::<math>{{\hat{t}}_{R=0.9}}=(4.359,16.033)\,\!</math>

=====Example: LR Bounds on Reliability=====
Again using the data given above for the [[The_Exponential_Distribution#Example:_LR_Bounds_for_Lambda|LR Bounds on Lambda example]] (five failures at 20, 40, 60, 100 and 150 hours), determine the 85% two-sided confidence bounds on the reliability estimate for a <math>t=50\,\!</math>. The ML estimate for the time at <math>t=50\,\!</math> is <math>\hat{R}=50.881%\,\!</math>.

'''Solution'''

In this example, we are trying to determine the 85% two-sided confidence bounds on the reliability estimate of 50.881%. This is accomplished by substituting <math>t=50\,\!</math> and <math>\alpha =0.85\,\!</math> into the likelihood ratio bound equation. It now remains to find the values of <math>R\,\!</math> which satisfy this equation. Since there is only one parameter, there are only two values of <math>R\,\!</math> that will satisfy the equation. These values represent the <math>\delta =85%\,\!</math> two-sided confidence limits of the reliability estimate <math>\hat{R}\,\!</math>. For our problem, the confidence limits are:

::<math>{{\hat{R}}_{t=50}}=(29.861\%,71.794\%)\,\!</math>.

===Bayesian Confidence Bounds===
====Bounds on Parameters====
From [[Confidence Bounds]], we know that the posterior distribution of <math>\lambda \,\!</math> can be written as:

::<math>f(\lambda |Data)=\frac{L(Data|\lambda )\varphi (\lambda )}{\int_{0}^{\infty }L(Data|\lambda )\varphi (\lambda )d\lambda }\,\!</math>

where <math>\varphi (\lambda )=\tfrac{1}{\lambda }\,\!</math>, is the non-informative prior of <math>\lambda \,\!</math>.

With the above prior distribution, <math>f(\lambda |Data)\,\!</math> can be rewritten as:

::<math>f(\lambda |Data)=\frac{L(Data|\lambda )\tfrac{1}{\lambda }}{\int_{0}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }\,\!</math>

The one-sided upper bound of <math>\lambda \,\!</math> is:

::<math>CL=P(\lambda \le {{\lambda }_{U}})=\int_{0}^{{{\lambda }_{U}}}f(\lambda |Data)d\lambda \,\!</math>

The one-sided lower bound of <math>\lambda \,\!</math> is:

::<math>1-CL=P(\lambda \le {{\lambda }_{L}})=\int_{0}^{{{\lambda }_{L}}}f(\lambda |Data)d\lambda \,\!</math>

The two-sided bounds of <math>\lambda \,\!</math> are:

::<math>CL=P({{\lambda }_{L}}\le \lambda \le {{\lambda }_{U}})=\int_{{{\lambda }_{L}}}^{{{\lambda }_{U}}}f(\lambda |Data)d\lambda \,\!</math>

====Bounds on Time (Type 1)====
The reliable life equation is:

::<math>t=\frac{-\ln R}{\lambda }\,\!</math>

For the one-sided upper bound on time we have:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(t\le {{T}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\frac{-\ln R}{\lambda }\le {{T}_{U}})\,\!</math>

The above equation can be rewritten in terms of <math>\lambda \,\!</math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\frac{-\ln R}{{{t}_{U}}}\le \lambda )\,\!</math>

From the above posterior distribuiton equation, we have:

::<math>CL=\frac{\int_{\tfrac{-\ln R}{{{t}_{U}}}}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }{\int_{0}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }\,\!</math>

The above equation is solved w.r.t. <math>{{t}_{U}}.\,\!</math> The same method is applied for one-sided lower and two-sided bounds on time.

====Bounds on Reliability (Type 2)====
The one-sided upper bound on reliability is given by:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(R\le {{R}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\exp (-\lambda t)\le {{R}_{U}})\,\!</math>

The above equaation can be rewritten in terms of <math>\lambda \,\!</math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\frac{-\ln {{R}_{U}}}{t}\le \lambda )\,\!</math>

From the equation for posterior distribution we have:

::<math>CL=\frac{\int_{\tfrac{-\ln {{R}_{U}}}{t}}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }{\int_{0}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }\,\!</math>

The above equation is solved w.r.t. <math>{{R}_{U}}.\,\!</math> The same method can be used to calculate one-sided lower and two sided bounds on reliability.

==Exponential Distribution Examples==
{{:Exponential Distribution Examples}}

Appendix: Log-Likelihood Equations

2022-11-14T20:27:18Z

M Spivey: /* The Two-Parameter Exponential */

{{Template:LDABOOK_SUB|Appendix D|Log-Likelihood Equations}}

This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed in more detail in the chapter for each distribution.
===Weibull Log-Likelihood Functions and their Partials===
====The Two-Parameter Weibull====
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}} \right]
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\beta \,\!</math> is the Weibull shape parameter (unknown a priori, the first of two parameters to be found)
:*<math>\eta \,\!</math> is the Weibull scale parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval failure data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.\,\!</math>

The solution will be found by solving for a pair of parameters <math>\left( \widehat{\beta },\widehat{\eta } \right)\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0\,\!</math> and <math>\tfrac{\partial \Lambda }{\partial \eta }=0.\,\!</math> It should be noted that other methods can also be used, such as direct maximization of the likelihood function, without having to compute the derivatives.

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}}{\eta } \right) \\
& -\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}}{\eta } \right)-\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }}{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}+{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }} \\
& +\frac{\beta }{\eta }\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}\,\!</math>

==== The Three-Parameter Weibull====
This log-likelihood function is again composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }} \\
& \\
& +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}} \right]
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\beta \,\!</math> is the Weibull shape parameter (unknown a priori, the first of three parameters to be found)
:*<math>\eta \,\!</math> is the Weibull scale parameter (unknown a priori, the second of three parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>\gamma \,\!</math> is the Weibull location parameter (unknown a priori, the third of three parameters to be found)
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in the <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*and <math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

The solution is found by solving for <math>\left( \widehat{\beta },\widehat{\eta },\widehat{\gamma } \right)\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0,\,\!</math> <math>\tfrac{\partial \Lambda }{\partial \eta }=0,\,\!</math> and <math>\tfrac{\partial \Lambda }{\partial \gamma }=0.\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right)-\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right) \\
& -\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}+\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& -\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \gamma }= & \left( 1-\beta \right)\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,\left( \frac{{{N}_{i}}}{{{T}_{i}}-\gamma } \right)+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{{{T}_{i}}-\gamma } \right) \\
& +\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{T_{i}^{\prime }-\gamma } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{T_{Li}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-\tfrac{\beta }{T_{Ri}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}\,\!</math>

It should be pointed out that the solution to the three-parameter Weibull via MLE is not always stable and can collapse if <math>\beta \sim 1.\,\!</math> In estimating the true MLE of the three-parameter Weibull distribution, two difficulties arise. The first is a problem of non-regularity and the second is the parameter divergence problem, as discussed in Hirose [[Appendix:_Life_Data_Analysis_References|[14]]].

Non-regularity occurs when <math>\beta \le 2.\,\!</math> In general, there are no MLE solutions in the region of <math>0<\beta <1.\,\!</math> When <math>1<\beta <2,\,\!</math> MLE solutions exist but are not asymptotically normal, as discussed in Hirose [[Appendix:_Life_Data_Analysis_References|[14]]]. In the case of non-regularity, the solution is treated anomalously.

Weibull++ attempts to find a solution in all of the regions using a variety of methods, but the user should be forewarned that not all possible data can be addressed. Thus, some solutions using MLE for the three-parameter Weibull will fail when the algorithm has reached predefined limits or fails to converge. In these cases, the user can change to the non-true MLE approach (in Weibull++ Application Setup), where <math>\gamma \,\!</math> is estimated using non-linear regression. Once <math>\gamma \,\!</math> is obtained, the MLE estimates of <math>\widehat{\beta }\,\!</math> and <math>\widehat{\eta }\,\!</math> are computed using the transformation <math>T_{i}^{\prime }=({{T}_{i}}-\gamma ).\,\!</math>

=== Exponential Log-Likelihood Functions and their Partials===
==== The One-Parameter Exponential====
This log-likelihood function is composed of three summation portions:

::<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right]\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\lambda \,\!</math> is the failure rate parameter (unknown a priori, the only parameter to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in the <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in the <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

The solution will be found by solving for a parameter <math>\widehat{\lambda }\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.\,\!</math> Note that for <math>FI=0\,\!</math> there exists a closed form solution.

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right]
\end{align}\,\!</math>

==== The Two-Parameter Exponential====
This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:

::<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma \right) \\
& & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}} \right],
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\lambda \,\!</math> is the failure rate parameter (unknown a priori, the first of two parameters to be found)
:*<math>\gamma \,\!</math> is the location parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in the <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in the <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

To find the two-parameter solution, look at the partial derivatives <math>\tfrac{\partial \Lambda }{\partial \lambda }</math> and <math>\tfrac{\partial \Lambda }{\partial \gamma}</math>:

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right] \\
& -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}} \right]
\end{align}\,\!</math>

and

::<math>\begin{align}\frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda \,\!\end{align}</math>.

From here we see that <math>\frac{\partial \Lambda }{\partial \gamma}</math> is a positive, constant function of <math>\gamma</math>. As alluded to in the chapter on the exponential distribution, this implies that the log-likelihood function <math>\Lambda</math> is, for fixed <math>\lambda</math>, an increasing function of <math>\gamma</math>. Thus the MLE for <math>\gamma</math> is its largest possible value <math>T_1</math>. Therefore, to find the full MLE solution <math>(\widehat{\lambda },\widehat{\gamma})</math> for the two-parameter exponential distribution, one should set <math>\gamma</math> equal to the first failure time and then find (numerically) a <math>\lambda</math> such that <math>\tfrac{\partial \Lambda}{\partial \lambda} = 0</math>.

The 3D Plot utility in Weibull++ further illustrates this behavior of the log-likelihood function, as shown next:

[[image: appendixc__127.gif|center|350px]]

=== Normal Log-Likelihood Functions and their Partials===
The complete normal likelihood function (without the constant) is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{\sigma }\phi \left( \frac{{{T}_{i}}-\mu }{\sigma } \right) \right] \\
& +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{^{\prime }}\ln \left[ 1-\Phi \left( \frac{T_{i}^{^{\prime }}-\mu }{\sigma } \right) \right] \\
& \text{ }+\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{^{\prime \prime }}\ln \left[ \Phi \left( \frac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right)-\Phi \left( \frac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right) \right]
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\mu \,\!</math> is the mean parameter (unknown a priori, the first of two parameters to be found)
:*<math>\sigma \,\!</math> is the standard deviation parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in the <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>{{F}_{i}}\,\!</math> is the number of interval data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in the <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {{\mu }_{0}},{{\sigma }_{0}} \right)\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0\,\!</math> and <math>\tfrac{\partial \Lambda }{\partial \sigma }=0.\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & \frac{1}{{{\sigma }^{2}}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}({{T}_{i}}-\mu ) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( {{T}_{i}}-\mu \right)}^{2}}}{{{\sigma }^{3}}}-\frac{1}{\sigma } \right) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}\,\!</math>

where:

::<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{{{x}^{2}}}{2}}}\,\!</math>

and:

::<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt\,\!</math>

==== Complete Data====
Note that for the normal distribution, and in the case of complete data only (as was shown in [[Basic Statistical Background]]), there exists a closed-form solution for both of the parameters or:

::<math>\widehat{\mu }=\widehat{{\bar{T}}}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}\,\!</math>

and:

::<math>\begin{align}
\hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\
{{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}}
\end{align}\,\!</math>

=== Lognormal Log-Likelihood Functions and their Partials===
The general log-likelihood function (without the constant) for the lognormal distribution is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{{{T}_{i}} {{\sigma }_{{{T}'}}}}\phi \left( \frac{\ln \left( {{T}_{i}} \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ 1-\Phi \left( \frac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \Phi \left( \frac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \frac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right]
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>{\mu }'\,\!</math> is the mean of the natural logarithms of the times-to-failure (unknown a priori, the first of two parameters to be found)
:*<math>{{\sigma }_{{{T}'}}}\,\!</math> is the standard deviation of the natural logarithms of the times-to-failure (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in the <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in the <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {\mu }',{{\sigma }_{{{T}'}}} \right)\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial {\mu }'}=0\,\!</math> and <math>\tfrac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}=0\,\!</math>:

::<math>\begin{align}
\frac{\partial \Lambda }{\partial {\mu }'}= & \frac{1}{\sigma _{{{T}'}}^{2}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,N_{i}(\ln ({{T}_{i}})-{\mu }') \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& \ \ -\underset{i=1}{\overset{FI}{\mathop \sum }}\,\frac{N_{i}^{\prime \prime }}{\sigma }\frac{\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,N_{i}\left( \frac{{{\left( \ln ({{T}_{i}})-{\mu }' \right)}^{2}}}{\sigma _{{{T}'}}^{3}}-\frac{1}{{{\sigma }_{{{T}'}}}} \right) \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& -\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}\,\!</math>

where:

::<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}\cdot {{e}^{-\tfrac{{{x}^{2}}}{2}}}\,\!</math>

and:

::<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt\,\!</math>

=== Mixed Weibull Log-Likelihood Functions and their Partials===
The log-likelihood function (without the constant) is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}{{e}^{-{{\left( \tfrac{T_{i}^{\prime }}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right]
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>Q\,\!</math> is the number of subpopulations
:*<math>{{\rho }_{k}}\,\!</math> is the proportionality of the <math>{{k}^{th}}\,\!</math> subpopulation (unknown a priori, the first set of three sets of parameters to be found)
:*<math>{{\beta }_{k}}\,\!</math> is the Weibull shape parameter of the <math>{{k}^{th}}\,\!</math> subpopulation (unknown a priori, the second set of three sets of parameters to be found)
:*<math>{{\eta }_{k}}\,\!</math> is the Weibull scale parameter (unknown a priori, the third set of three sets of parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of groups of interval data points
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

The solution will be found by solving for a group of parameters:

::<math>\left( \widehat{{{\rho }_{1,}}}\widehat{{{\beta }_{1}}},\widehat{{{\eta }_{1}}},\widehat{{{\rho }_{2,}}}\widehat{{{\beta }_{2}}},\widehat{{{\eta }_{2}}},...,\widehat{{{\rho }_{Q,}}}\widehat{{{\beta }_{Q}}},\widehat{{{\eta }_{Q}}} \right)\,\!</math>

so that:

::<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\rho }_{1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\rho }_{2}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{2}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{2}}}=0 \\
\vdots \\
\frac{\partial \Lambda }{\partial {{\rho }_{Q-1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{Q-1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{Q-1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\beta }_{Q}}}= & 0,\text{ and }\frac{\partial \Lambda }{\partial {{\eta }_{Q}}}=0
\end{align}\,\!</math>

=== Logistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{\sigma {{(1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}})}^{2}}} \right)-\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\mu \,\!</math> is the logistic shape parameter (unknown a priori, the first of two parameters to be found)
:*<math>\eta \,\!</math> is the logistic scale parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval failure data group
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.\,\!</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& -\frac{\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\mathop{}_{}^{}}}\,}}\,N_{i}^{^{\prime \prime }}}{\sigma }+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \\
& -\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}})
\end{align}\,\!</math>

=== The Loglogistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{\sigma t{{(1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}})}^{2}}} \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\mu \,\!</math> is the loglogistic shape parameter (unknown a priori, the first of two parameters to be found)
:*<math>\sigma \,\!</math> is the loglogistic scale parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval failure data groups,
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.\,\!</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\sigma }+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}-\frac{{{F}_{I}}}{\sigma } \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\ln ({{T}_{i}})-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \\
& -\frac{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}})
\end{align}\,\!</math>

=== The Gumbel Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}}}{\sigma } \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}} \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}\,\!</math>

or:

::<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\left( \frac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \right)-\ln (\sigma )\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\mu \,\!</math> is the Gumbel shape parameter (unknown a priori, the first of two parameters to be found)
:*<math>\sigma \,\!</math> is the Gumbel scale parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval failure data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.\,\!</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })\,\!</math> so that:

::<math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \\
& -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }} \\
& \left( \frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}\,\!</math>

=== The Gamma Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{k(\ln ({{T}_{i}})-\mu )-{{e}^{{{e}^{\ln ({{T}_{i}})-\mu }}}}}}}{{{T}_{i}}\Gamma (k)} \right) \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-\Gamma \left( _{1}k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right) \right)
\end{align}\,\!</math>

or:

::<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln ({{T}_{i}})\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln (\Gamma (k))+k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu ) \\
& \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu )}} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu )}} \right) \right)
\end{align}\,\!</math>

where:
:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\mu \,\!</math> is the gamma shape parameter (unknown a priori, the first of two parameters to be found)
:*<math>k\,\!</math> is the gamma scale parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval failure data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*and <math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.\,\!</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial k}=0.\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{k\left( \ln (T_{i}^{^{\prime }})-\mu )-{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right)}}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\{\frac{{{e}^{k{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}-{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)} \\
& -\frac{{{e}^{k{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}-{{e}^{{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}\}
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial k}= & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu )-\frac{{{\Gamma }^{^{\prime }}}(k)\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\Gamma (k)} \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\partial {{\Gamma }_{1}}(k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }})}{\partial k}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}-\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}) \right)} \right)
\end{align}\,\!</math>

Appendix: Log-Likelihood Equations

2022-11-12T01:14:16Z

M Spivey: /* The Two-Parameter Exponential */

{{Template:LDABOOK_SUB|Appendix D|Log-Likelihood Equations}}

This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed in more detail in the chapter for each distribution.
===Weibull Log-Likelihood Functions and their Partials===
====The Two-Parameter Weibull====
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}} \right]
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\beta \,\!</math> is the Weibull shape parameter (unknown a priori, the first of two parameters to be found)
:*<math>\eta \,\!</math> is the Weibull scale parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval failure data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.\,\!</math>

The solution will be found by solving for a pair of parameters <math>\left( \widehat{\beta },\widehat{\eta } \right)\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0\,\!</math> and <math>\tfrac{\partial \Lambda }{\partial \eta }=0.\,\!</math> It should be noted that other methods can also be used, such as direct maximization of the likelihood function, without having to compute the derivatives.

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}}{\eta } \right) \\
& -\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}}{\eta } \right)-\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }}{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}+{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }} \\
& +\frac{\beta }{\eta }\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}
\end{align}\,\!</math>

==== The Three-Parameter Weibull====
This log-likelihood function is again composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }} \\
& \\
& +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}} \right]
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\beta \,\!</math> is the Weibull shape parameter (unknown a priori, the first of three parameters to be found)
:*<math>\eta \,\!</math> is the Weibull scale parameter (unknown a priori, the second of three parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>\gamma \,\!</math> is the Weibull location parameter (unknown a priori, the third of three parameters to be found)
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in the <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*and <math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

The solution is found by solving for <math>\left( \widehat{\beta },\widehat{\eta },\widehat{\gamma } \right)\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \beta }=0,\,\!</math> <math>\tfrac{\partial \Lambda }{\partial \eta }=0,\,\!</math> and <math>\tfrac{\partial \Lambda }{\partial \gamma }=0.\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right)-\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right) \\
& -\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}+\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{\eta } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\
& -\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \gamma }= & \left( 1-\beta \right)\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,\left( \frac{{{N}_{i}}}{{{T}_{i}}-\gamma } \right)+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{{{T}_{i}}-\gamma } \right) \\
& +\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{T_{i}^{\prime }-\gamma } \right) \\
& +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{T_{Li}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-\tfrac{\beta }{T_{Ri}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}
\end{align}\,\!</math>

It should be pointed out that the solution to the three-parameter Weibull via MLE is not always stable and can collapse if <math>\beta \sim 1.\,\!</math> In estimating the true MLE of the three-parameter Weibull distribution, two difficulties arise. The first is a problem of non-regularity and the second is the parameter divergence problem, as discussed in Hirose [[Appendix:_Life_Data_Analysis_References|[14]]].

Non-regularity occurs when <math>\beta \le 2.\,\!</math> In general, there are no MLE solutions in the region of <math>0<\beta <1.\,\!</math> When <math>1<\beta <2,\,\!</math> MLE solutions exist but are not asymptotically normal, as discussed in Hirose [[Appendix:_Life_Data_Analysis_References|[14]]]. In the case of non-regularity, the solution is treated anomalously.

Weibull++ attempts to find a solution in all of the regions using a variety of methods, but the user should be forewarned that not all possible data can be addressed. Thus, some solutions using MLE for the three-parameter Weibull will fail when the algorithm has reached predefined limits or fails to converge. In these cases, the user can change to the non-true MLE approach (in Weibull++ Application Setup), where <math>\gamma \,\!</math> is estimated using non-linear regression. Once <math>\gamma \,\!</math> is obtained, the MLE estimates of <math>\widehat{\beta }\,\!</math> and <math>\widehat{\eta }\,\!</math> are computed using the transformation <math>T_{i}^{\prime }=({{T}_{i}}-\gamma ).\,\!</math>

=== Exponential Log-Likelihood Functions and their Partials===
==== The One-Parameter Exponential====
This log-likelihood function is composed of three summation portions:

::<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right]\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\lambda \,\!</math> is the failure rate parameter (unknown a priori, the only parameter to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in the <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in the <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

The solution will be found by solving for a parameter <math>\widehat{\lambda }\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.\,\!</math> Note that for <math>FI=0\,\!</math> there exists a closed form solution.

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right]
\end{align}\,\!</math>

==== The Two-Parameter Exponential====
This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:

::<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma \right) \\
& & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}} \right],
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\lambda \,\!</math> is the failure rate parameter (unknown a priori, the first of two parameters to be found)
:*<math>\gamma \,\!</math> is the location parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in the <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in the <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

To find the two-parameter solution, look at the partial derivatives <math>\tfrac{\partial \Lambda }{\partial \lambda }</math> and <math>\tfrac{\partial \Lambda }{\partial \gamma}</math>:

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right] \\
& -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}} \right]
\end{align}\,\!</math>

and

::<math>\frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda \,\!</math>.

From here we see that <math>\frac{\partial \Lambda }{\partial \gamma}</math> is a positive, constant function of <math>\gamma</math>. As alluded to in the section on the exponential distribution, this implies that the log-likelihood function <math>\Lambda</math> is, for fixed <math>\lambda</math>, an increasing function of <math>\gamma</math>. Thus the MLE for <math>\gamma</math> is its largest possible value <math>T_1</math>. Therefore, to find the full MLE solution <math>(\widehat{\lambda },\widehat{\gamma})</math> for the two-parameter exponential distribution, one should set <math>\gamma</math> equal to the first failure time and then find a <math>\lambda</math> such that <math>\tfrac{\partial \Lambda}{\partial \lambda} = 0</math>.

The 3D Plot utility in Weibull++ further illustrates this behavior of the log-likelihood function, as shown next:

[[image: appendixc__127.gif|center|350px]]

=== Normal Log-Likelihood Functions and their Partials===
The complete normal likelihood function (without the constant) is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{\sigma }\phi \left( \frac{{{T}_{i}}-\mu }{\sigma } \right) \right] \\
& +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{^{\prime }}\ln \left[ 1-\Phi \left( \frac{T_{i}^{^{\prime }}-\mu }{\sigma } \right) \right] \\
& \text{ }+\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{^{\prime \prime }}\ln \left[ \Phi \left( \frac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right)-\Phi \left( \frac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right) \right]
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\mu \,\!</math> is the mean parameter (unknown a priori, the first of two parameters to be found)
:*<math>\sigma \,\!</math> is the standard deviation parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in the <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>{{F}_{i}}\,\!</math> is the number of interval data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in the <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {{\mu }_{0}},{{\sigma }_{0}} \right)\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0\,\!</math> and <math>\tfrac{\partial \Lambda }{\partial \sigma }=0.\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & \frac{1}{{{\sigma }^{2}}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}({{T}_{i}}-\mu ) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( {{T}_{i}}-\mu \right)}^{2}}}{{{\sigma }^{3}}}-\frac{1}{\sigma } \right) \\
& +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\
& -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}
\end{align}\,\!</math>

where:

::<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{{{x}^{2}}}{2}}}\,\!</math>

and:

::<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt\,\!</math>

==== Complete Data====
Note that for the normal distribution, and in the case of complete data only (as was shown in [[Basic Statistical Background]]), there exists a closed-form solution for both of the parameters or:

::<math>\widehat{\mu }=\widehat{{\bar{T}}}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}}\,\!</math>

and:

::<math>\begin{align}
\hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\
{{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}}
\end{align}\,\!</math>

=== Lognormal Log-Likelihood Functions and their Partials===
The general log-likelihood function (without the constant) for the lognormal distribution is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{{{T}_{i}} {{\sigma }_{{{T}'}}}}\phi \left( \frac{\ln \left( {{T}_{i}} \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ 1-\Phi \left( \frac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \Phi \left( \frac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \frac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right]
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>{\mu }'\,\!</math> is the mean of the natural logarithms of the times-to-failure (unknown a priori, the first of two parameters to be found)
:*<math>{{\sigma }_{{{T}'}}}\,\!</math> is the standard deviation of the natural logarithms of the times-to-failure (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in the <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in the <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

The solution will be found by solving for a pair of parameters <math>\left( {\mu }',{{\sigma }_{{{T}'}}} \right)\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial {\mu }'}=0\,\!</math> and <math>\tfrac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}=0\,\!</math>:

::<math>\begin{align}
\frac{\partial \Lambda }{\partial {\mu }'}= & \frac{1}{\sigma _{{{T}'}}^{2}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,N_{i}(\ln ({{T}_{i}})-{\mu }') \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& \ \ -\underset{i=1}{\overset{FI}{\mathop \sum }}\,\frac{N_{i}^{\prime \prime }}{\sigma }\frac{\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,N_{i}\left( \frac{{{\left( \ln ({{T}_{i}})-{\mu }' \right)}^{2}}}{\sigma _{{{T}'}}^{3}}-\frac{1}{{{\sigma }_{{{T}'}}}} \right) \\
& +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\
& -\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}
\end{align}\,\!</math>

where:

::<math>\phi \left( x \right)=\frac{1}{\sqrt{2\pi }}\cdot {{e}^{-\tfrac{{{x}^{2}}}{2}}}\,\!</math>

and:

::<math>\Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt\,\!</math>

=== Mixed Weibull Log-Likelihood Functions and their Partials===
The log-likelihood function (without the constant) is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}{{e}^{-{{\left( \tfrac{T_{i}^{\prime }}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\
& \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right]
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>Q\,\!</math> is the number of subpopulations
:*<math>{{\rho }_{k}}\,\!</math> is the proportionality of the <math>{{k}^{th}}\,\!</math> subpopulation (unknown a priori, the first set of three sets of parameters to be found)
:*<math>{{\beta }_{k}}\,\!</math> is the Weibull shape parameter of the <math>{{k}^{th}}\,\!</math> subpopulation (unknown a priori, the second set of three sets of parameters to be found)
:*<math>{{\eta }_{k}}\,\!</math> is the Weibull scale parameter (unknown a priori, the third set of three sets of parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of groups of interval data points
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

The solution will be found by solving for a group of parameters:

::<math>\left( \widehat{{{\rho }_{1,}}}\widehat{{{\beta }_{1}}},\widehat{{{\eta }_{1}}},\widehat{{{\rho }_{2,}}}\widehat{{{\beta }_{2}}},\widehat{{{\eta }_{2}}},...,\widehat{{{\rho }_{Q,}}}\widehat{{{\beta }_{Q}}},\widehat{{{\eta }_{Q}}} \right)\,\!</math>

so that:

::<math>\begin{align}
\frac{\partial \Lambda }{\partial {{\rho }_{1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\rho }_{2}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{2}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{2}}}=0 \\
\vdots \\
\frac{\partial \Lambda }{\partial {{\rho }_{Q-1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{Q-1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{Q-1}}}=0 \\
\frac{\partial \Lambda }{\partial {{\beta }_{Q}}}= & 0,\text{ and }\frac{\partial \Lambda }{\partial {{\eta }_{Q}}}=0
\end{align}\,\!</math>

=== Logistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{\sigma {{(1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}})}^{2}}} \right)-\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\mu \,\!</math> is the logistic shape parameter (unknown a priori, the first of two parameters to be found)
:*<math>\eta \,\!</math> is the logistic scale parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval failure data group
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.\,\!</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& -\frac{\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\mathop{}_{}^{}}}\,}}\,N_{i}^{^{\prime \prime }}}{\sigma }+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right)
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \\
& -\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}})
\end{align}\,\!</math>

=== The Loglogistic Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{\sigma t{{(1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}})}^{2}}} \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\mu \,\!</math> is the loglogistic shape parameter (unknown a priori, the first of two parameters to be found)
:*<math>\sigma \,\!</math> is the loglogistic scale parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval failure data groups,
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.\,\!</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\sigma }+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}-\frac{{{F}_{I}}}{\sigma } \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right)
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\ln ({{T}_{i}})-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}} \\
& \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \\
& -\frac{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}})
\end{align}\,\!</math>

=== The Gumbel Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}}}{\sigma } \right) \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}} \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}\,\!</math>

or:

::<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\left( \frac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \right)-\ln (\sigma )\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right)
\end{align}\,\!</math>

where:

:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\mu \,\!</math> is the Gumbel shape parameter (unknown a priori, the first of two parameters to be found)
:*<math>\sigma \,\!</math> is the Gumbel scale parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval failure data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*<math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.\,\!</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })\,\!</math> so that:

::<math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0.\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\
& +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \\
& -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }} \\
& \left( \frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right)
\end{align}\,\!</math>

=== The Gamma Log-Likelihood Functions and their Partials===
This log-likelihood function is composed of three summation portions:

::<math>\begin{align}
\ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{k(\ln ({{T}_{i}})-\mu )-{{e}^{{{e}^{\ln ({{T}_{i}})-\mu }}}}}}}{{{T}_{i}}\Gamma (k)} \right) \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-\Gamma \left( _{1}k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right) \right)
\end{align}\,\!</math>

or:

::<math>\begin{align}
\Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln ({{T}_{i}})\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln (\Gamma (k))+k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu ) \\
& \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right) \right) \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu )}} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu )}} \right) \right)
\end{align}\,\!</math>

where:
:*<math>{{F}_{e}}\,\!</math> is the number of groups of times-to-failure data points
:*<math>{{N}_{i}}\,\!</math> is the number of times-to-failure in the <math>{{i}^{th}}\,\!</math> time-to-failure data group
:*<math>\mu \,\!</math> is the gamma shape parameter (unknown a priori, the first of two parameters to be found)
:*<math>k\,\!</math> is the gamma scale parameter (unknown a priori, the second of two parameters to be found)
:*<math>{{T}_{i}}\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> group of time-to-failure data
:*<math>S\,\!</math> is the number of groups of suspension data points
:*<math>N_{i}^{\prime }\,\!</math> is the number of suspensions in <math>{{i}^{th}}\,\!</math> group of suspension data points
:*<math>T_{i}^{\prime }\,\!</math> is the time of the <math>{{i}^{th}}\,\!</math> suspension data group
:*<math>FI\,\!</math> is the number of interval failure data groups
:*<math>N_{i}^{\prime \prime }\,\!</math> is the number of intervals in <math>{{i}^{th}}\,\!</math> group of data intervals
:*<math>T_{Li}^{\prime \prime }\,\!</math> is the beginning of the <math>{{i}^{th}}\,\!</math> interval
:*and <math>T_{Ri}^{\prime \prime }\,\!</math> is the ending of the <math>{{i}^{th}}\,\!</math> interval

For the purposes of MLE, left censored data will be considered to be intervals with <math>T_{Li}^{\prime \prime }=0.\,\!</math>

The solution of the maximum log-likelihood function is found by solving for (<math>\widehat{\mu },\widehat{\sigma })\,\!</math> so that <math>\tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial k}=0.\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \mu }= & -k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{k\left( \ln (T_{i}^{^{\prime }})-\mu )-{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right)}}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\{\frac{{{e}^{k{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}-{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)} \\
& -\frac{{{e}^{k{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}-{{e}^{{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}\}
\end{align}\,\!</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial k}= & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu )-\frac{{{\Gamma }^{^{\prime }}}(k)\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\Gamma (k)} \\
& -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\partial {{\Gamma }_{1}}(k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }})}{\partial k}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\
& +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}-\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}) \right)} \right)
\end{align}\,\!</math>

The Exponential Distribution

2022-11-12T00:19:52Z

M Spivey: /* Bounds on the Parameters */

{{template:LDABOOK|7|The Exponential Distribution}}
The exponential distribution is a commonly used distribution in reliability engineering. Mathematically, it is a fairly simple distribution, which many times leads to its use in inappropriate situations. It is, in fact, a special case of the Weibull distribution where <math>\beta =1\,\!</math>. The exponential distribution is used to model the behavior of units that have a constant failure rate (or units that do not degrade with time or wear out).

==Exponential Probability Density Function==
===The 2-Parameter Exponential Distribution===
The 2-parameter exponential ''pdf'' is given by:

::<math>f(t)=\lambda {{e}^{-\lambda (t-\gamma )}},f(t)\ge 0,\lambda >0,t\ge \gamma \,\!</math>

where <math>\gamma \,\!</math> is the location parameter.
Some of the characteristics of the 2-parameter exponential distribution are discussed in Kececioglu [[Appendix:_Life_Data_Analysis_References|[19]]]:
*The location parameter, <math>\gamma \,\!</math>, if positive, shifts the beginning of the distribution by a distance of <math>\gamma \,\!</math> to the right of the origin, signifying that the chance failures start to occur only after <math>\gamma \,\!</math> hours of operation, and cannot occur before.
*The scale parameter is <math>\tfrac{1}{\lambda }=\bar{t}-\gamma =m-\gamma \,\!</math>.
*The exponential ''pdf'' has no shape parameter, as it has only one shape.
*The distribution starts at <math>t=\gamma \,\!</math> at the level of <math>f(t=\gamma )=\lambda \,\!</math> and decreases thereafter exponentially and monotonically as <math>t\,\!</math> increases beyond <math>\gamma \,\!</math> and is convex.
*As <math>t\to \infty \,\!</math>, <math>f(t)\to 0\,\!</math>.

===The 1-Parameter Exponential Distribution===
The 1-parameter exponential ''pdf'' is obtained by setting <math>\gamma =0\,\!</math>, and is given by:

::<math> \begin{align}f(t)= & \lambda {{e}^{-\lambda t}}=\frac{1}{m}{{e}^{-\tfrac{1}{m}t}},
& t\ge 0, \lambda >0,m>0
\end{align}
\,\!</math>

where:

::<math>\lambda \,\!</math> = constant rate, in failures per unit of measurement, (e.g., failures per hour, per cycle, etc.)

::<math>\lambda =\frac{1}{m}\,\!</math>
::<math>m\,\!</math> = mean time between failures, or to failure
::<math>t\,\!</math> = operating time, life, or age, in hours, cycles, miles, actuations, etc.

This distribution requires the knowledge of only one parameter, <math>\lambda \,\!</math>, for its application. Some of the characteristics of the 1-parameter exponential distribution are discussed in Kececioglu [[Appendix:_Life_Data_Analysis_References| [19]]]:
*The location parameter, <math>\gamma \,\!</math>, is zero.
*The scale parameter is <math>\tfrac{1}{\lambda }=m\,\!</math>.
*As <math>\lambda \,\!</math> is decreased in value, the distribution is stretched out to the right, and as <math>\lambda \,\!</math> is increased, the distribution is pushed toward the origin.
*This distribution has no shape parameter as it has only one shape, (i.e., the exponential, and the only parameter it has is the failure rate, <math>\lambda \,\!</math>).
*The distribution starts at <math>t=0\,\!</math> at the level of <math>f(t=0)=\lambda \,\!</math> and decreases thereafter exponentially and monotonically as <math>t\,\!</math> increases, and is convex.
*As <math>t\to \infty \,\!</math>, <math>f(t)\to 0\,\!</math>.
*The ''pdf'' can be thought of as a special case of the Weibull ''pdf'' with <math>\gamma =0\,\!</math> and <math>\beta =1\,\!</math>.

==Exponential Distribution Functions==
{{:Exponential Distribution Functions}}

==Characteristics of the Exponential Distribution==
{{:Exponential Distribution Characteristics}}

==Estimation of the Exponential Parameters==
===Probability Plotting===
Estimation of the parameters for the exponential distribution via probability plotting is very similar to the process used when dealing with the Weibull distribution. Recall, however, that the appearance of the probability plotting paper and the methods by which the parameters are estimated vary from distribution to distribution, so there will be some noticeable differences. In fact, due to the nature of the exponential ''cdf'', the exponential probability plot is the only one with a negative slope. This is because the y-axis of the exponential probability plotting paper represents the reliability, whereas the y-axis for most of the other life distributions represents the unreliability.

This is illustrated in the process of linearizing the ''cdf'', which is necessary to construct the exponential probability plotting paper. For the two-parameter exponential distribution the cumulative density function is given by:

::<math>\begin{align}
F(t)=1-{{e}^{-\lambda (t-\gamma )}}
\end{align}\,\!</math>

Taking the natural logarithm of both sides of the above equation yields:

::<math>\ln \left[ 1-F(t) \right]=-\lambda (t-\gamma )\,\!</math>

or:

::<math>\begin{align}
\ln [1-F(t)]=\lambda \gamma -\lambda t
\end{align}\,\!</math>

Now, let:

::<math>\begin{align}
y=\ln [1-F(t)]
\end{align}\,\!</math>

::<math>\begin{align}
a=\lambda \gamma
\end{align}\,\!</math>

and:

::<math>\begin{align}
b=-\lambda
\end{align}\,\!</math>

which results in the linear equation of:

::<math>\begin{align}
y=a+bt
\end{align}\,\!</math>

Note that with the exponential probability plotting paper, the y-axis scale is logarithmic and the x-axis scale is linear. This means that the zero value is present only on the x-axis. For <math>t=0\,\!</math>, <math>R=1\,\!</math> and <math>F(t)=0\,\!</math>. So if we were to use <math>F(t)\,\!</math> for the y-axis, we would have to plot the point <math>(0,0)\,\!</math>. However, since the y-axis is logarithmic, there is no place to plot this on the exponential paper. Also, the failure rate, <math>\lambda \,\!</math>, is the negative of the slope of the line, but there is an easier way to determine the value of <math>\lambda \,\!</math> from the probability plot, as will be illustrated in the following example.
====Plotting Example====
{{:1P Exponential Example}}

===Rank Regression on Y===
Performing a rank regression on Y requires that a straight line be fitted to the set of available data points such that the sum of the squares of the vertical deviations from the points to the line is minimized.
The least squares parameter estimation method (regression analysis) was discussed in [[Parameter Estimation]], and the following equations for rank regression on Y (RRY) were derived:

::<math>\hat{a}=\bar{y}-\hat{b}\bar{x}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}-\hat{b}\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}}{N}\,\!</math>

and:

::<math>\hat{b}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,x_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}} \right)}^{2}}}{N}}\,\!</math>

In our case, the equations for <math>{{y}_{i}}\,\!</math> and <math>{{x}_{i}}\,\!</math> are:

::<math>\begin{align}
{{y}_{i}}=\ln [1-F({{t}_{i}})]
\end{align}\,\!</math>

and:

::<math>\begin{align}
{{x}_{i}}={{t}_{i}}
\end{align}\,\!</math>

and the <math>F({{t}_{i}})\,\!</math> is estimated from the median ranks. Once <math>\hat{a}\,\!</math> and <math>\hat{b}\,\!</math> are obtained, then <math>\hat{\lambda }\,\!</math> and <math>\hat{\gamma }\,\!</math> can easily be obtained from above equations.
For the one-parameter exponential, equations for estimating ''a'' and ''b'' become:

::<math>\begin{align}
\hat{a}= & 0, \\
\hat{b}= & \frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,x_{i}^{2}}
\end{align}\,\!</math>

'''The Correlation Coefficient'''

The estimator of <math>\rho \,\!</math> is the sample correlation coefficient, <math>\hat{\rho }\,\!</math>, given by:

::<math>\hat{\rho }=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,({{x}_{i}}-\overline{x})({{y}_{i}}-\overline{y})}{\sqrt{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{({{x}_{i}}-\overline{x})}^{2}}\cdot \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{({{y}_{i}}-\overline{y})}^{2}}}}\,\!</math>

====RRY Example==== 
{{:2P_Exponential_Example}}

===Rank Regression on X===
Similar to rank regression on Y, performing a rank regression on X requires that a straight line be fitted to a set of data points such that the sum of the squares of the horizontal deviations from the points to the line is minimized.

Again the first task is to bring our exponential ''cdf'' function into a linear form. This step is exactly the same as in regression on Y analysis. The deviation from the previous analysis begins on the least squares fit step, since in this case we treat <math>x\,\!</math> as the dependent variable and <math>y\,\!</math> as the independent variable. The best-fitting straight line to the data, for regression on X (see [[Parameter Estimation]]), is the straight line:

::<math>x=\hat{a}+\hat{b}y\,\!</math>

The corresponding equations for <math>\hat{a}\,\!</math> and <math>\hat{b}\,\!</math> are:

::<math>\hat{a}=\overline{x}-\hat{b}\overline{y}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}}{N}-\hat{b}\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}\,\!</math>

and:

::<math>\hat{b}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,y_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}} \right)}^{2}}}{N}}\,\!</math>

where:

::<math>\begin{align}
{{y}_{i}}=\ln [1-F({{t}_{i}})]
\end{align}\,\!</math>

and:

::<math>\begin{align}
{{x}_{i}}={{t}_{i}}
\end{align}\,\!</math>

The values of <math>F({{t}_{i}})\,\!</math> are estimated from the median ranks. Once <math>\hat{a}\,\!</math> and <math>\hat{b}\,\!</math> are obtained, solve for the unknown <math>y\,\!</math> value, which corresponds to:

::<math>y=-\frac{\hat{a}}{\hat{b}}+\frac{1}{\hat{b}}x\,\!</math>

Solving for the parameters from above equations we get:

::<math>a=-\frac{\hat{a}}{\hat{b}}=\lambda \gamma \Rightarrow \gamma =\hat{a}\,\!</math>

and:

::<math>b=\frac{1}{\hat{b}}=-\lambda \Rightarrow \lambda =-\frac{1}{\hat{b}}\,\!</math>

For the one-parameter exponential case, equations for estimating a and b become:

::<math>\begin{align}
\hat{a}= & 0 \\
\hat{b}= & \frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,y_{i}^{2}}
\end{align}\,\!</math>

The correlation coefficient is evaluated as before.

====RRX Example====
'''2-Parameter Exponential RRX Example'''

Using the same data set from the [[The_Exponential_Distribution#RRY_Example|RRY example above]] and assuming a 2-parameter exponential distribution, estimate the parameters and determine the correlation coefficient estimate, <math>\hat{\rho }\,\!</math>, using rank regression on X.

''' Solution'''

The table constructed for the RRY analysis applies to this example also. Using the values from this table, we get:

::<math>\begin{align}
\hat{b}= & \frac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{t}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{t}_{i}}\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{y}_{i}}}{14}}{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,y_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{y}_{i}} \right)}^{2}}}{14}} \\
\\
\hat{b}= & \frac{-927.4899-(630)(-13.2315)/14}{22.1148-{{(-13.2315)}^{2}}/14}
\end{align}\,\!</math>

or:

::<math>\hat{b}=-34.5563\,\!</math>

and:

::<math>\hat{a}=\overline{x}-\hat{b}\overline{y}=\frac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{t}_{i}}}{14}-\hat{b}\frac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{y}_{i}}}{14}\,\!</math>

or:

::<math>\hat{a}=\frac{630}{14}-(-34.5563)\frac{(-13.2315)}{14}=12.3406\,\!</math>

Therefore:

::<math>\hat{\lambda }=-\frac{1}{\hat{b}}=-\frac{1}{(-34.5563)}=0.0289\text{ failures/hour}\,\!</math>

and:

::<math>\hat{\gamma }=\hat{a}=12.3406\,\!</math>

The correlation coefficient is found to be:

::<math>\hat{\rho }=-0.9679\,\!</math>

Note that the equation for regression on Y is not necessarily the same as that for the regression on X. The only time when the two regression methods yield identical results is when the data lie perfectly on a line. If this were the case, the correlation coefficient would be <math>-1\,\!</math>. The negative value of the correlation coefficient is due to the fact that the slope of the exponential probability plot is negative.

This example can be repeated using Weibull++, choosing two-parameter exponential and rank regression on X (RRX) methods for analysis, as shown below.
The estimated parameters and the correlation coefficient using Weibull++ were found to be:

::<math>\begin{array}{*{35}{l}}
\hat{\lambda }= &0.0289 \text{failures/hour} \\
\hat{\gamma}= & 12.3395 \text{hours} \\
\hat{\rho} = &-0.9679 \\
\end{array}\,\!</math>

[[Image:Exponential Distribution Example 3 Data Folio.png|center|700px|]]

The probability plot can be obtained simply by clicking the '''Plot''' icon.

[[Image:Exponential Distribution Example 3 Plot.png|center|600px|]]

===Maximum Likelihood Estimation===
As outlined in [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood equation with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the exponential distribution are covered in [[Appendix:_Log-Likelihood_Equations|Appendix D]].

====MLE Example====
'''MLE for the Exponential Distribution'''

Using the same data set from the [[The_Exponential_Distribution#RRY_Example|RRY and RRX examples above]] and assuming a 2-parameter exponential distribution, estimate the parameters using the MLE method.

'''Solution'''

In this example, we have complete data only. The partial derivative of the log-likelihood function, <math>\Lambda ,\,\!</math> is given by:

::<math>\frac{\partial \Lambda }{\partial \lambda }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right]=\underset{i=1}{\overset{14}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right]=0\,\!</math>

Complete descriptions of the partial derivatives can be found in [[Appendix:_Log-Likelihood_Equations|Appendix D]]. Recall that when using the MLE method for the exponential distribution, the value of <math>\gamma \,\!</math> is equal to that of the first failure time. The first failure occurred at 5 hours, thus <math>\gamma =5\,\!</math> hours<math>.\,\!</math> Substituting the values for <math>T\,\!</math> and <math>\gamma \,\!</math> we get:

::<math>\frac{14}{\hat{\lambda }}=560\,\!</math>

or:

::<math>\hat{\lambda }=0.025\text{ failures/hour}\,\!</math>

Using Weibull++:

[[Image:Exponential Distribution Example 4 Data.png|center|700px|]]

The probability plot is:

[[Image:Exponential Distribution Example 4 Plot.png|center|650px|]]

==Confidence Bounds==
In this section, we present the methods used in the application to estimate the different types of confidence bounds for exponentially distributed data. The complete derivations were presented in detail (for a general function) in the chapter for [[Confidence Bounds]].
At this time we should point out that exact confidence bounds for the exponential distribution have been derived, and exist in a closed form, utilizing the <math>{{\chi }^{2}}\,\!</math> distribution. These are described in detail in Kececioglu [[Appendix:_Life_Data_Analysis_References|[20]]], and are covered in the section in the [[Reliability Test Design|test design chapter]]. For most exponential data analyses, Weibull++ will use the approximate confidence bounds, provided from the Fisher information matrix or the likelihood ratio, in order to stay consistent with all of the other available distributions in the application. The <math>{{\chi }^{2}}\,\!</math> confidence bounds for the exponential distribution are discussed in more detail in the [[Reliability Test Design|test design chapter]].

===Fisher Matrix Bounds===
====Bounds on the Parameters====
For the failure rate <math>\hat{\lambda }\,\!</math> the upper (<math>{{\lambda }_{U}}\,\!</math>) and lower (<math>{{\lambda }_{L}}\,\!</math>) bounds are estimated by Nelson [[Appendix:_Life_Data_Analysis_References|[30]]]:

::<math>\begin{align}
& {{\lambda }_{U}}= & \hat{\lambda }\cdot {{e}^{\left[ \tfrac{{{K}_{\alpha }}\sqrt{Var(\hat{\lambda })}}{\hat{\lambda }} \right]}} \\
& & \\
& {{\lambda }_{L}}= & \frac{\hat{\lambda }}{{{e}^{\left[ \tfrac{{{K}_{\alpha }}\sqrt{Var(\hat{\lambda })}}{\hat{\lambda }} \right]}}}
\end{align}\,\!</math>

where <math>{{K}_{\alpha }}\,\!</math> is defined by:

::<math>\alpha =\frac{1}{\sqrt{2\pi }}\int_{{{K}_{\alpha }}}^{\infty }{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt=1-\Phi ({{K}_{\alpha }})\,\!</math>

If <math>\delta \,\!</math> is the confidence level, then <math>\alpha =\tfrac{1-\delta }{2}\,\!</math> for the two-sided bounds, and <math>\alpha =1-\delta \,\!</math> for the one-sided bounds.

The variance of <math>\hat{\lambda },\,\!</math> <math>Var(\hat{\lambda }),\,\!</math> is estimated from the Fisher matrix, as follows:

::<math>Var(\hat{\lambda })={{\left( -\frac{{{\partial }^{2}}\Lambda }{\partial {{\lambda }^{2}}} \right)}^{-1}}\,\!</math>

where <math>\Lambda \,\!</math> is the log-likelihood function of the exponential distribution, described in [[Appendix:_Log-Likelihood_Equations|Appendix D]].

Note that, for fixed <math>\lambda \,\!</math>, the log-likelihood function is increasing in <math>\gamma</math>. This means that the MLE solution for <math>\gamma</math> cannot be found by setting <math>\tfrac{\partial \Lambda}{\partial \gamma}</math> to zero. (Since <math>\Lambda</math> is increasing in <math>\gamma</math>, <math>\tfrac{\partial \Lambda}{\partial \gamma}</math> can never be zero.) Instead, the MLE solution for <math>\gamma</math> is simply its largest possible value allowed by the sample; namely, <math>\gamma = t_1</math>, the first failure time. The MLE solution for <math>\lambda</math> is found in the usual fashion by setting <math>\tfrac{\partial \Lambda}{\partial \lambda}</math> to zero and solving. Weibull++ treats <math>\gamma</math> as a constant when computing bounds; i.e., <math>Var(\hat{\gamma}) = 0</math>. (See the discussion in Appendix D for more information.)

====Bounds on Reliability====
The reliability of the two-parameter exponential distribution is:

::<math>\hat{R}(t;\hat{\lambda })={{e}^{-\hat{\lambda }(t-\hat{\gamma })}}\,\!</math>

The corresponding confidence bounds are estimated from:

::<math>\begin{align}
& {{R}_{L}}= & {{e}^{-{{\lambda }_{U}}(t-\hat{\gamma })}} \\
& {{R}_{U}}= & {{e}^{-{{\lambda }_{L}}(t-\hat{\gamma })}}
\end{align}\,\!</math>

These equations hold true for the 1-parameter exponential distribution, with <math>\gamma =0\,\!</math>.

====Bounds on Time====
The bounds around time for a given exponential percentile, or reliability value, are estimated by first solving the reliability equation with respect to time, or reliable life:

::<math>\hat{t}=-\frac{1}{{\hat{\lambda }}}\cdot \ln (R)+\hat{\gamma }\,\!</math>

The corresponding confidence bounds are estimated from:

::<math>\begin{align}
& {{t}_{U}}= & -\frac{1}{{{\lambda }_{L}}}\cdot \ln (R)+\hat{\gamma } \\
& {{t}_{L}}= & -\frac{1}{{{\lambda }_{U}}}\cdot \ln (R)+\hat{\gamma }
\end{align}\,\!</math>

The same equations apply for the one-parameter exponential with <math>\gamma =0.\,\!</math>

===Likelihood Ratio Confidence Bounds===
====Bounds on Parameters====
For one-parameter distributions such as the exponential, the likelihood confidence bounds are calculated by finding values for <math>\theta \,\!</math> that satisfy:

::<math>-2\cdot \text{ln}\left( \frac{L(\theta )}{L(\hat{\theta })} \right)=\chi _{\alpha ;1}^{2}\,\!</math>

This equation can be rewritten as:

::<math>L(\theta )=L(\hat{\theta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}\,\!</math>

For complete data, the likelihood function for the exponential distribution is given by:

::<math>L(\lambda )=\underset{i=1}{\overset{N}{\mathop \prod }}\,f({{t}_{i}};\lambda )=\underset{i=1}{\overset{N}{\mathop \prod }}\,\lambda \cdot {{e}^{-\lambda \cdot {{t}_{i}}}}\,\!</math>

where the <math>{{t}_{i}}\,\!</math> values represent the original time-to-failure data. For a given value of <math>\alpha \,\!</math>, values for <math>\lambda \,\!</math> can be found which represent the maximum and minimum values that satisfy the above likelihood ratio equation. These represent the confidence bounds for the parameters at a confidence level <math>\delta ,\,\!</math> where <math>\alpha =\delta \,\!</math> for two-sided bounds and <math>\alpha =2\delta -1\,\!</math> for one-sided.

=====Example: LR Bounds for Lambda=====
Five units are put on a reliability test and experience failures at 20, 40, 60, 100, and 150 hours. Assuming an exponential distribution, the MLE parameter estimate is calculated to be <math>\hat{\lambda }=0.013514\,\!</math>. Calculate the 85% two-sided confidence bounds on these parameters using the likelihood ratio method.

'''Solution'''

The first step is to calculate the likelihood function for the parameter estimates:

::<math>\begin{align}
L(\hat{\lambda })= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\hat{\lambda })=\underset{i=1}{\overset{N}{\mathop \prod }}\,\hat{\lambda }\cdot {{e}^{-\hat{\lambda }\cdot {{x}_{i}}}} \\
L(\hat{\lambda })= & \underset{i=1}{\overset{5}{\mathop \prod }}\,0.013514\cdot {{e}^{-0.013514\cdot {{x}_{i}}}} \\
L(\hat{\lambda })= & 3.03647\times {{10}^{-12}}
\end{align}\,\!</math>

where <math>{{x}_{i}}\,\!</math> are the original time-to-failure data points. We can now rearrange the likelihood ratio equation to the form:

::<math>L(\lambda )-L(\hat{\lambda })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0\,\!</math>

Since our specified confidence level, <math>\delta \,\!</math>, is 85%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.85;1}^{2}=2.072251.\,\!</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\lambda )-L(\hat{\lambda })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0, \\
L(\lambda )-3.03647\times {{10}^{-12}}\cdot {{e}^{\tfrac{-2.072251}{2}}}= & 0, \\
L(\lambda )-1.07742\times {{10}^{-12}}= & 0.
\end{align}\,\!</math>

It now remains to find the values of <math>\lambda \,\!</math> which satisfy this equation. Since there is only one parameter, there are only two values of <math>\lambda \,\!</math> that will satisfy the equation. These values represent the <math>\delta =85%\,\!</math> two-sided confidence limits of the parameter estimate <math>\hat{\lambda }\,\!</math>. For our problem, the confidence limits are:

::<math>\begin{align}
{{\lambda }_{0.85}}=(0.006572,0.024172)
\end{align}\,\!</math>

====Bounds on Time and Reliability====
In order to calculate the bounds on a time estimate for a given reliability, or on a reliability estimate for a given time, the likelihood function needs to be rewritten in terms of one parameter and time/reliability, so that the maximum and minimum values of the time can be observed as the parameter is varied. This can be accomplished by substituting a form of the exponential reliability equation into the likelihood function. The exponential reliability equation can be written as:

::<math>R={{e}^{-\lambda \cdot t}}\,\!</math>

This can be rearranged to the form:

::<math>\lambda =\frac{-\text{ln}(R)}{t}\,\!</math>

This equation can now be substituted into the likelihood ratio equation to produce a likelihood equation in terms of <math>t\,\!</math> and <math>R:\,\!</math>

::<math>L(t/R)=\underset{i=1}{\overset{N}{\mathop \prod }}\,\left( \frac{-\text{ln}(R)}{t} \right)\cdot {{e}^{\left( \tfrac{\text{ln}(R)}{t} \right)\cdot {{x}_{i}}}}\,\!</math>

The unknown parameter <math>t/R\,\!</math> depends on what type of bounds are being determined. If one is trying to determine the bounds on time for the equation for the mean and the Bayes's rule equation for single parametera given reliability, then <math>R\,\!</math> is a known constant and <math>t\,\!</math> is the unknown parameter. Conversely, if one is trying to determine the bounds on reliability for a given time, then <math>t\,\!</math> is a known constant and <math>R\,\!</math> is the unknown parameter. Either way, the likelihood ratio function can be solved for the values of interest.

=====Example: LR Bounds on Time =====
For the data given above for the [[The_Exponential_Distribution#Example:_LR_Bounds_for_Lambda|LR Bounds on Lambda example]] (five failures at 20, 40, 60, 100 and 150 hours), determine the 85% two-sided confidence bounds on the time estimate for a reliability of 90%. The ML estimate for the time at <math>R(t)=90%\,\!</math> is <math>\hat{t}=7.797\,\!</math>.

'''Solution'''

In this example, we are trying to determine the 85% two-sided confidence bounds on the time estimate of 7.797. This is accomplished by substituting <math>R=0.90\,\!</math> and <math>\alpha =0.85\,\!</math> into the likelihood ratio bound equation. It now remains to find the values of <math>t\,\!</math> which satisfy this equation. Since there is only one parameter, there are only two values of <math>t\,\!</math> that will satisfy the equation. These values represent the <math>\delta =85%\,\!</math> two-sided confidence limits of the time estimate <math>\hat{t}\,\!</math>. For our problem, the confidence limits are:

::<math>{{\hat{t}}_{R=0.9}}=(4.359,16.033)\,\!</math>

=====Example: LR Bounds on Reliability=====
Again using the data given above for the [[The_Exponential_Distribution#Example:_LR_Bounds_for_Lambda|LR Bounds on Lambda example]] (five failures at 20, 40, 60, 100 and 150 hours), determine the 85% two-sided confidence bounds on the reliability estimate for a <math>t=50\,\!</math>. The ML estimate for the time at <math>t=50\,\!</math> is <math>\hat{R}=50.881%\,\!</math>.

'''Solution'''

In this example, we are trying to determine the 85% two-sided confidence bounds on the reliability estimate of 50.881%. This is accomplished by substituting <math>t=50\,\!</math> and <math>\alpha =0.85\,\!</math> into the likelihood ratio bound equation. It now remains to find the values of <math>R\,\!</math> which satisfy this equation. Since there is only one parameter, there are only two values of <math>t\,\!</math> that will satisfy the equation. These values represent the <math>\delta =85%\,\!</math> two-sided confidence limits of the reliability estimate <math>\hat{R}\,\!</math>. For our problem, the confidence limits are:

::<math>{{\hat{R}}_{t=50}}=(29.861%,71.794%)\,\!</math>

===Bayesian Confidence Bounds===
====Bounds on Parameters====
From [[Confidence Bounds]], we know that the posterior distribution of <math>\lambda \,\!</math> can be written as:

::<math>f(\lambda |Data)=\frac{L(Data|\lambda )\varphi (\lambda )}{\int_{0}^{\infty }L(Data|\lambda )\varphi (\lambda )d\lambda }\,\!</math>

where <math>\varphi (\lambda )=\tfrac{1}{\lambda }\,\!</math>, is the non-informative prior of <math>\lambda \,\!</math>.

With the above prior distribution, <math>f(\lambda |Data)\,\!</math> can be rewritten as:

::<math>f(\lambda |Data)=\frac{L(Data|\lambda )\tfrac{1}{\lambda }}{\int_{0}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }\,\!</math>

The one-sided upper bound of <math>\lambda \,\!</math> is:

::<math>CL=P(\lambda \le {{\lambda }_{U}})=\int_{0}^{{{\lambda }_{U}}}f(\lambda |Data)d\lambda \,\!</math>

The one-sided lower bound of <math>\lambda \,\!</math> is:

::<math>1-CL=P(\lambda \le {{\lambda }_{L}})=\int_{0}^{{{\lambda }_{L}}}f(\lambda |Data)d\lambda \,\!</math>

The two-sided bounds of <math>\lambda \,\!</math> are:

::<math>CL=P({{\lambda }_{L}}\le \lambda \le {{\lambda }_{U}})=\int_{{{\lambda }_{L}}}^{{{\lambda }_{U}}}f(\lambda |Data)d\lambda \,\!</math>

====Bounds on Time (Type 1)====
The reliable life equation is:

::<math>t=\frac{-\ln R}{\lambda }\,\!</math>

For the one-sided upper bound on time we have:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(t\le {{T}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\frac{-\ln R}{\lambda }\le {{T}_{U}})\,\!</math>

The above equation can be rewritten in terms of <math>\lambda \,\!</math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\frac{-\ln R}{{{t}_{U}}}\le \lambda )\,\!</math>

From the above posterior distribuiton equation, we have:

::<math>CL=\frac{\int_{\tfrac{-\ln R}{{{t}_{U}}}}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }{\int_{0}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }\,\!</math>

The above equation is solved w.r.t. <math>{{t}_{U}}.\,\!</math> The same method is applied for one-sided lower and two-sided bounds on time.

====Bounds on Reliability (Type 2)====
The one-sided upper bound on reliability is given by:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(R\le {{R}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\exp (-\lambda t)\le {{R}_{U}})\,\!</math>

The above equaation can be rewritten in terms of <math>\lambda \,\!</math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\frac{-\ln {{R}_{U}}}{t}\le \lambda )\,\!</math>

From the equation for posterior distribution we have:

::<math>CL=\frac{\int_{\tfrac{-\ln {{R}_{U}}}{t}}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }{\int_{0}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }\,\!</math>

The above equation is solved w.r.t. <math>{{R}_{U}}.\,\!</math> The same method can be used to calculate one-sided lower and two sided bounds on reliability.

==Exponential Distribution Examples==
{{:Exponential Distribution Examples}}

The Exponential Distribution

2022-11-12T00:14:40Z

M Spivey: /* Bounds on the Parameters */

{{template:LDABOOK|7|The Exponential Distribution}}
The exponential distribution is a commonly used distribution in reliability engineering. Mathematically, it is a fairly simple distribution, which many times leads to its use in inappropriate situations. It is, in fact, a special case of the Weibull distribution where <math>\beta =1\,\!</math>. The exponential distribution is used to model the behavior of units that have a constant failure rate (or units that do not degrade with time or wear out).

==Exponential Probability Density Function==
===The 2-Parameter Exponential Distribution===
The 2-parameter exponential ''pdf'' is given by:

::<math>f(t)=\lambda {{e}^{-\lambda (t-\gamma )}},f(t)\ge 0,\lambda >0,t\ge \gamma \,\!</math>

where <math>\gamma \,\!</math> is the location parameter.
Some of the characteristics of the 2-parameter exponential distribution are discussed in Kececioglu [[Appendix:_Life_Data_Analysis_References|[19]]]:
*The location parameter, <math>\gamma \,\!</math>, if positive, shifts the beginning of the distribution by a distance of <math>\gamma \,\!</math> to the right of the origin, signifying that the chance failures start to occur only after <math>\gamma \,\!</math> hours of operation, and cannot occur before.
*The scale parameter is <math>\tfrac{1}{\lambda }=\bar{t}-\gamma =m-\gamma \,\!</math>.
*The exponential ''pdf'' has no shape parameter, as it has only one shape.
*The distribution starts at <math>t=\gamma \,\!</math> at the level of <math>f(t=\gamma )=\lambda \,\!</math> and decreases thereafter exponentially and monotonically as <math>t\,\!</math> increases beyond <math>\gamma \,\!</math> and is convex.
*As <math>t\to \infty \,\!</math>, <math>f(t)\to 0\,\!</math>.

===The 1-Parameter Exponential Distribution===
The 1-parameter exponential ''pdf'' is obtained by setting <math>\gamma =0\,\!</math>, and is given by:

::<math> \begin{align}f(t)= & \lambda {{e}^{-\lambda t}}=\frac{1}{m}{{e}^{-\tfrac{1}{m}t}},
& t\ge 0, \lambda >0,m>0
\end{align}
\,\!</math>

where:

::<math>\lambda \,\!</math> = constant rate, in failures per unit of measurement, (e.g., failures per hour, per cycle, etc.)

::<math>\lambda =\frac{1}{m}\,\!</math>
::<math>m\,\!</math> = mean time between failures, or to failure
::<math>t\,\!</math> = operating time, life, or age, in hours, cycles, miles, actuations, etc.

This distribution requires the knowledge of only one parameter, <math>\lambda \,\!</math>, for its application. Some of the characteristics of the 1-parameter exponential distribution are discussed in Kececioglu [[Appendix:_Life_Data_Analysis_References| [19]]]:
*The location parameter, <math>\gamma \,\!</math>, is zero.
*The scale parameter is <math>\tfrac{1}{\lambda }=m\,\!</math>.
*As <math>\lambda \,\!</math> is decreased in value, the distribution is stretched out to the right, and as <math>\lambda \,\!</math> is increased, the distribution is pushed toward the origin.
*This distribution has no shape parameter as it has only one shape, (i.e., the exponential, and the only parameter it has is the failure rate, <math>\lambda \,\!</math>).
*The distribution starts at <math>t=0\,\!</math> at the level of <math>f(t=0)=\lambda \,\!</math> and decreases thereafter exponentially and monotonically as <math>t\,\!</math> increases, and is convex.
*As <math>t\to \infty \,\!</math>, <math>f(t)\to 0\,\!</math>.
*The ''pdf'' can be thought of as a special case of the Weibull ''pdf'' with <math>\gamma =0\,\!</math> and <math>\beta =1\,\!</math>.

==Exponential Distribution Functions==
{{:Exponential Distribution Functions}}

==Characteristics of the Exponential Distribution==
{{:Exponential Distribution Characteristics}}

==Estimation of the Exponential Parameters==
===Probability Plotting===
Estimation of the parameters for the exponential distribution via probability plotting is very similar to the process used when dealing with the Weibull distribution. Recall, however, that the appearance of the probability plotting paper and the methods by which the parameters are estimated vary from distribution to distribution, so there will be some noticeable differences. In fact, due to the nature of the exponential ''cdf'', the exponential probability plot is the only one with a negative slope. This is because the y-axis of the exponential probability plotting paper represents the reliability, whereas the y-axis for most of the other life distributions represents the unreliability.

This is illustrated in the process of linearizing the ''cdf'', which is necessary to construct the exponential probability plotting paper. For the two-parameter exponential distribution the cumulative density function is given by:

::<math>\begin{align}
F(t)=1-{{e}^{-\lambda (t-\gamma )}}
\end{align}\,\!</math>

Taking the natural logarithm of both sides of the above equation yields:

::<math>\ln \left[ 1-F(t) \right]=-\lambda (t-\gamma )\,\!</math>

or:

::<math>\begin{align}
\ln [1-F(t)]=\lambda \gamma -\lambda t
\end{align}\,\!</math>

Now, let:

::<math>\begin{align}
y=\ln [1-F(t)]
\end{align}\,\!</math>

::<math>\begin{align}
a=\lambda \gamma
\end{align}\,\!</math>

and:

::<math>\begin{align}
b=-\lambda
\end{align}\,\!</math>

which results in the linear equation of:

::<math>\begin{align}
y=a+bt
\end{align}\,\!</math>

Note that with the exponential probability plotting paper, the y-axis scale is logarithmic and the x-axis scale is linear. This means that the zero value is present only on the x-axis. For <math>t=0\,\!</math>, <math>R=1\,\!</math> and <math>F(t)=0\,\!</math>. So if we were to use <math>F(t)\,\!</math> for the y-axis, we would have to plot the point <math>(0,0)\,\!</math>. However, since the y-axis is logarithmic, there is no place to plot this on the exponential paper. Also, the failure rate, <math>\lambda \,\!</math>, is the negative of the slope of the line, but there is an easier way to determine the value of <math>\lambda \,\!</math> from the probability plot, as will be illustrated in the following example.
====Plotting Example====
{{:1P Exponential Example}}

===Rank Regression on Y===
Performing a rank regression on Y requires that a straight line be fitted to the set of available data points such that the sum of the squares of the vertical deviations from the points to the line is minimized.
The least squares parameter estimation method (regression analysis) was discussed in [[Parameter Estimation]], and the following equations for rank regression on Y (RRY) were derived:

::<math>\hat{a}=\bar{y}-\hat{b}\bar{x}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}-\hat{b}\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}}{N}\,\!</math>

and:

::<math>\hat{b}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,x_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}} \right)}^{2}}}{N}}\,\!</math>

In our case, the equations for <math>{{y}_{i}}\,\!</math> and <math>{{x}_{i}}\,\!</math> are:

::<math>\begin{align}
{{y}_{i}}=\ln [1-F({{t}_{i}})]
\end{align}\,\!</math>

and:

::<math>\begin{align}
{{x}_{i}}={{t}_{i}}
\end{align}\,\!</math>

and the <math>F({{t}_{i}})\,\!</math> is estimated from the median ranks. Once <math>\hat{a}\,\!</math> and <math>\hat{b}\,\!</math> are obtained, then <math>\hat{\lambda }\,\!</math> and <math>\hat{\gamma }\,\!</math> can easily be obtained from above equations.
For the one-parameter exponential, equations for estimating ''a'' and ''b'' become:

::<math>\begin{align}
\hat{a}= & 0, \\
\hat{b}= & \frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,x_{i}^{2}}
\end{align}\,\!</math>

'''The Correlation Coefficient'''

The estimator of <math>\rho \,\!</math> is the sample correlation coefficient, <math>\hat{\rho }\,\!</math>, given by:

::<math>\hat{\rho }=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,({{x}_{i}}-\overline{x})({{y}_{i}}-\overline{y})}{\sqrt{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{({{x}_{i}}-\overline{x})}^{2}}\cdot \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{({{y}_{i}}-\overline{y})}^{2}}}}\,\!</math>

====RRY Example==== 
{{:2P_Exponential_Example}}

===Rank Regression on X===
Similar to rank regression on Y, performing a rank regression on X requires that a straight line be fitted to a set of data points such that the sum of the squares of the horizontal deviations from the points to the line is minimized.

Again the first task is to bring our exponential ''cdf'' function into a linear form. This step is exactly the same as in regression on Y analysis. The deviation from the previous analysis begins on the least squares fit step, since in this case we treat <math>x\,\!</math> as the dependent variable and <math>y\,\!</math> as the independent variable. The best-fitting straight line to the data, for regression on X (see [[Parameter Estimation]]), is the straight line:

::<math>x=\hat{a}+\hat{b}y\,\!</math>

The corresponding equations for <math>\hat{a}\,\!</math> and <math>\hat{b}\,\!</math> are:

::<math>\hat{a}=\overline{x}-\hat{b}\overline{y}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}}{N}-\hat{b}\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}\,\!</math>

and:

::<math>\hat{b}=\frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}}}{N}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,y_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{y}_{i}} \right)}^{2}}}{N}}\,\!</math>

where:

::<math>\begin{align}
{{y}_{i}}=\ln [1-F({{t}_{i}})]
\end{align}\,\!</math>

and:

::<math>\begin{align}
{{x}_{i}}={{t}_{i}}
\end{align}\,\!</math>

The values of <math>F({{t}_{i}})\,\!</math> are estimated from the median ranks. Once <math>\hat{a}\,\!</math> and <math>\hat{b}\,\!</math> are obtained, solve for the unknown <math>y\,\!</math> value, which corresponds to:

::<math>y=-\frac{\hat{a}}{\hat{b}}+\frac{1}{\hat{b}}x\,\!</math>

Solving for the parameters from above equations we get:

::<math>a=-\frac{\hat{a}}{\hat{b}}=\lambda \gamma \Rightarrow \gamma =\hat{a}\,\!</math>

and:

::<math>b=\frac{1}{\hat{b}}=-\lambda \Rightarrow \lambda =-\frac{1}{\hat{b}}\,\!</math>

For the one-parameter exponential case, equations for estimating a and b become:

::<math>\begin{align}
\hat{a}= & 0 \\
\hat{b}= & \frac{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{x}_{i}}{{y}_{i}}}{\underset{i=1}{\overset{N}{\mathop{\sum }}}\,y_{i}^{2}}
\end{align}\,\!</math>

The correlation coefficient is evaluated as before.

====RRX Example====
'''2-Parameter Exponential RRX Example'''

Using the same data set from the [[The_Exponential_Distribution#RRY_Example|RRY example above]] and assuming a 2-parameter exponential distribution, estimate the parameters and determine the correlation coefficient estimate, <math>\hat{\rho }\,\!</math>, using rank regression on X.

''' Solution'''

The table constructed for the RRY analysis applies to this example also. Using the values from this table, we get:

::<math>\begin{align}
\hat{b}= & \frac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{t}_{i}}{{y}_{i}}-\tfrac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{t}_{i}}\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{y}_{i}}}{14}}{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,y_{i}^{2}-\tfrac{{{\left( \underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{y}_{i}} \right)}^{2}}}{14}} \\
\\
\hat{b}= & \frac{-927.4899-(630)(-13.2315)/14}{22.1148-{{(-13.2315)}^{2}}/14}
\end{align}\,\!</math>

or:

::<math>\hat{b}=-34.5563\,\!</math>

and:

::<math>\hat{a}=\overline{x}-\hat{b}\overline{y}=\frac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{t}_{i}}}{14}-\hat{b}\frac{\underset{i=1}{\overset{14}{\mathop{\sum }}}\,{{y}_{i}}}{14}\,\!</math>

or:

::<math>\hat{a}=\frac{630}{14}-(-34.5563)\frac{(-13.2315)}{14}=12.3406\,\!</math>

Therefore:

::<math>\hat{\lambda }=-\frac{1}{\hat{b}}=-\frac{1}{(-34.5563)}=0.0289\text{ failures/hour}\,\!</math>

and:

::<math>\hat{\gamma }=\hat{a}=12.3406\,\!</math>

The correlation coefficient is found to be:

::<math>\hat{\rho }=-0.9679\,\!</math>

Note that the equation for regression on Y is not necessarily the same as that for the regression on X. The only time when the two regression methods yield identical results is when the data lie perfectly on a line. If this were the case, the correlation coefficient would be <math>-1\,\!</math>. The negative value of the correlation coefficient is due to the fact that the slope of the exponential probability plot is negative.

This example can be repeated using Weibull++, choosing two-parameter exponential and rank regression on X (RRX) methods for analysis, as shown below.
The estimated parameters and the correlation coefficient using Weibull++ were found to be:

::<math>\begin{array}{*{35}{l}}
\hat{\lambda }= &0.0289 \text{failures/hour} \\
\hat{\gamma}= & 12.3395 \text{hours} \\
\hat{\rho} = &-0.9679 \\
\end{array}\,\!</math>

[[Image:Exponential Distribution Example 3 Data Folio.png|center|700px|]]

The probability plot can be obtained simply by clicking the '''Plot''' icon.

[[Image:Exponential Distribution Example 3 Plot.png|center|600px|]]

===Maximum Likelihood Estimation===
As outlined in [[Parameter Estimation]], maximum likelihood estimation works by developing a likelihood function based on the available data and finding the values of the parameter estimates that maximize the likelihood function. This can be achieved by using iterative methods to determine the parameter estimate values that maximize the likelihood function. This can be rather difficult and time-consuming, particularly when dealing with the three-parameter distribution. Another method of finding the parameter estimates involves taking the partial derivatives of the likelihood equation with respect to the parameters, setting the resulting equations equal to zero, and solving simultaneously to determine the values of the parameter estimates. The log-likelihood functions and associated partial derivatives used to determine maximum likelihood estimates for the exponential distribution are covered in [[Appendix:_Log-Likelihood_Equations|Appendix D]].

====MLE Example====
'''MLE for the Exponential Distribution'''

Using the same data set from the [[The_Exponential_Distribution#RRY_Example|RRY and RRX examples above]] and assuming a 2-parameter exponential distribution, estimate the parameters using the MLE method.

'''Solution'''

In this example, we have complete data only. The partial derivative of the log-likelihood function, <math>\Lambda ,\,\!</math> is given by:

::<math>\frac{\partial \Lambda }{\partial \lambda }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right]=\underset{i=1}{\overset{14}{\mathop \sum }}\,\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right]=0\,\!</math>

Complete descriptions of the partial derivatives can be found in [[Appendix:_Log-Likelihood_Equations|Appendix D]]. Recall that when using the MLE method for the exponential distribution, the value of <math>\gamma \,\!</math> is equal to that of the first failure time. The first failure occurred at 5 hours, thus <math>\gamma =5\,\!</math> hours<math>.\,\!</math> Substituting the values for <math>T\,\!</math> and <math>\gamma \,\!</math> we get:

::<math>\frac{14}{\hat{\lambda }}=560\,\!</math>

or:

::<math>\hat{\lambda }=0.025\text{ failures/hour}\,\!</math>

Using Weibull++:

[[Image:Exponential Distribution Example 4 Data.png|center|700px|]]

The probability plot is:

[[Image:Exponential Distribution Example 4 Plot.png|center|650px|]]

==Confidence Bounds==
In this section, we present the methods used in the application to estimate the different types of confidence bounds for exponentially distributed data. The complete derivations were presented in detail (for a general function) in the chapter for [[Confidence Bounds]].
At this time we should point out that exact confidence bounds for the exponential distribution have been derived, and exist in a closed form, utilizing the <math>{{\chi }^{2}}\,\!</math> distribution. These are described in detail in Kececioglu [[Appendix:_Life_Data_Analysis_References|[20]]], and are covered in the section in the [[Reliability Test Design|test design chapter]]. For most exponential data analyses, Weibull++ will use the approximate confidence bounds, provided from the Fisher information matrix or the likelihood ratio, in order to stay consistent with all of the other available distributions in the application. The <math>{{\chi }^{2}}\,\!</math> confidence bounds for the exponential distribution are discussed in more detail in the [[Reliability Test Design|test design chapter]].

===Fisher Matrix Bounds===
====Bounds on the Parameters====
For the failure rate <math>\hat{\lambda }\,\!</math> the upper (<math>{{\lambda }_{U}}\,\!</math>) and lower (<math>{{\lambda }_{L}}\,\!</math>) bounds are estimated by Nelson [[Appendix:_Life_Data_Analysis_References|[30]]]:

::<math>\begin{align}
& {{\lambda }_{U}}= & \hat{\lambda }\cdot {{e}^{\left[ \tfrac{{{K}_{\alpha }}\sqrt{Var(\hat{\lambda })}}{\hat{\lambda }} \right]}} \\
& & \\
& {{\lambda }_{L}}= & \frac{\hat{\lambda }}{{{e}^{\left[ \tfrac{{{K}_{\alpha }}\sqrt{Var(\hat{\lambda })}}{\hat{\lambda }} \right]}}}
\end{align}\,\!</math>

where <math>{{K}_{\alpha }}\,\!</math> is defined by:

::<math>\alpha =\frac{1}{\sqrt{2\pi }}\int_{{{K}_{\alpha }}}^{\infty }{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt=1-\Phi ({{K}_{\alpha }})\,\!</math>

If <math>\delta \,\!</math> is the confidence level, then <math>\alpha =\tfrac{1-\delta }{2}\,\!</math> for the two-sided bounds, and <math>\alpha =1-\delta \,\!</math> for the one-sided bounds.

The variance of <math>\hat{\lambda },\,\!</math> <math>Var(\hat{\lambda }),\,\!</math> is estimated from the Fisher matrix, as follows:

::<math>Var(\hat{\lambda })={{\left( -\frac{{{\partial }^{2}}\Lambda }{\partial {{\lambda }^{2}}} \right)}^{-1}}\,\!</math>

where <math>\Lambda \,\!</math> is the log-likelihood function of the exponential distribution, described in [[Appendix:_Log-Likelihood_Equations|Appendix D]].

Note that, for fixed <math>\lambda \,\!</math>, the log-likelihood function is increasing in <math>\gamma</math>. This means that the MLE solution for <math>\gamma</math> cannot be found by setting <math>\tfrac{d \Lambda}{d \gamma}</math> to zero. (Since <math>\Lambda</math> is increasing in <math>\gamma</math>, <math>\tfrac{d \Lambda}{d \gamma}</math> can never be zero.) Instead, the MLE solution for <math>\gamma</math> is simply its largest possible value allowed by the sample; namely, <math>\gamma = t_1</math>, the first failure time. The MLE solution for <math>\lambda</math> is found in the usual fashion by setting <math>\tfrac{d \Lambda}{d \lambda}</math> to zero and solving. Weibull++ treats <math>\gamma</math> as a constant when computing bounds; i.e., <math>Var(\hat{\gamma}) = 0</math>. (See the discussion in Appendix D for more information.)

====Bounds on Reliability====
The reliability of the two-parameter exponential distribution is:

::<math>\hat{R}(t;\hat{\lambda })={{e}^{-\hat{\lambda }(t-\hat{\gamma })}}\,\!</math>

The corresponding confidence bounds are estimated from:

::<math>\begin{align}
& {{R}_{L}}= & {{e}^{-{{\lambda }_{U}}(t-\hat{\gamma })}} \\
& {{R}_{U}}= & {{e}^{-{{\lambda }_{L}}(t-\hat{\gamma })}}
\end{align}\,\!</math>

These equations hold true for the 1-parameter exponential distribution, with <math>\gamma =0\,\!</math>.

====Bounds on Time====
The bounds around time for a given exponential percentile, or reliability value, are estimated by first solving the reliability equation with respect to time, or reliable life:

::<math>\hat{t}=-\frac{1}{{\hat{\lambda }}}\cdot \ln (R)+\hat{\gamma }\,\!</math>

The corresponding confidence bounds are estimated from:

::<math>\begin{align}
& {{t}_{U}}= & -\frac{1}{{{\lambda }_{L}}}\cdot \ln (R)+\hat{\gamma } \\
& {{t}_{L}}= & -\frac{1}{{{\lambda }_{U}}}\cdot \ln (R)+\hat{\gamma }
\end{align}\,\!</math>

The same equations apply for the one-parameter exponential with <math>\gamma =0.\,\!</math>

===Likelihood Ratio Confidence Bounds===
====Bounds on Parameters====
For one-parameter distributions such as the exponential, the likelihood confidence bounds are calculated by finding values for <math>\theta \,\!</math> that satisfy:

::<math>-2\cdot \text{ln}\left( \frac{L(\theta )}{L(\hat{\theta })} \right)=\chi _{\alpha ;1}^{2}\,\!</math>

This equation can be rewritten as:

::<math>L(\theta )=L(\hat{\theta })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}\,\!</math>

For complete data, the likelihood function for the exponential distribution is given by:

::<math>L(\lambda )=\underset{i=1}{\overset{N}{\mathop \prod }}\,f({{t}_{i}};\lambda )=\underset{i=1}{\overset{N}{\mathop \prod }}\,\lambda \cdot {{e}^{-\lambda \cdot {{t}_{i}}}}\,\!</math>

where the <math>{{t}_{i}}\,\!</math> values represent the original time-to-failure data. For a given value of <math>\alpha \,\!</math>, values for <math>\lambda \,\!</math> can be found which represent the maximum and minimum values that satisfy the above likelihood ratio equation. These represent the confidence bounds for the parameters at a confidence level <math>\delta ,\,\!</math> where <math>\alpha =\delta \,\!</math> for two-sided bounds and <math>\alpha =2\delta -1\,\!</math> for one-sided.

=====Example: LR Bounds for Lambda=====
Five units are put on a reliability test and experience failures at 20, 40, 60, 100, and 150 hours. Assuming an exponential distribution, the MLE parameter estimate is calculated to be <math>\hat{\lambda }=0.013514\,\!</math>. Calculate the 85% two-sided confidence bounds on these parameters using the likelihood ratio method.

'''Solution'''

The first step is to calculate the likelihood function for the parameter estimates:

::<math>\begin{align}
L(\hat{\lambda })= & \underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};\hat{\lambda })=\underset{i=1}{\overset{N}{\mathop \prod }}\,\hat{\lambda }\cdot {{e}^{-\hat{\lambda }\cdot {{x}_{i}}}} \\
L(\hat{\lambda })= & \underset{i=1}{\overset{5}{\mathop \prod }}\,0.013514\cdot {{e}^{-0.013514\cdot {{x}_{i}}}} \\
L(\hat{\lambda })= & 3.03647\times {{10}^{-12}}
\end{align}\,\!</math>

where <math>{{x}_{i}}\,\!</math> are the original time-to-failure data points. We can now rearrange the likelihood ratio equation to the form:

::<math>L(\lambda )-L(\hat{\lambda })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}=0\,\!</math>

Since our specified confidence level, <math>\delta \,\!</math>, is 85%, we can calculate the value of the chi-squared statistic, <math>\chi _{0.85;1}^{2}=2.072251.\,\!</math> We can now substitute this information into the equation:

::<math>\begin{align}
L(\lambda )-L(\hat{\lambda })\cdot {{e}^{\tfrac{-\chi _{\alpha ;1}^{2}}{2}}}= & 0, \\
L(\lambda )-3.03647\times {{10}^{-12}}\cdot {{e}^{\tfrac{-2.072251}{2}}}= & 0, \\
L(\lambda )-1.07742\times {{10}^{-12}}= & 0.
\end{align}\,\!</math>

It now remains to find the values of <math>\lambda \,\!</math> which satisfy this equation. Since there is only one parameter, there are only two values of <math>\lambda \,\!</math> that will satisfy the equation. These values represent the <math>\delta =85%\,\!</math> two-sided confidence limits of the parameter estimate <math>\hat{\lambda }\,\!</math>. For our problem, the confidence limits are:

::<math>\begin{align}
{{\lambda }_{0.85}}=(0.006572,0.024172)
\end{align}\,\!</math>

====Bounds on Time and Reliability====
In order to calculate the bounds on a time estimate for a given reliability, or on a reliability estimate for a given time, the likelihood function needs to be rewritten in terms of one parameter and time/reliability, so that the maximum and minimum values of the time can be observed as the parameter is varied. This can be accomplished by substituting a form of the exponential reliability equation into the likelihood function. The exponential reliability equation can be written as:

::<math>R={{e}^{-\lambda \cdot t}}\,\!</math>

This can be rearranged to the form:

::<math>\lambda =\frac{-\text{ln}(R)}{t}\,\!</math>

This equation can now be substituted into the likelihood ratio equation to produce a likelihood equation in terms of <math>t\,\!</math> and <math>R:\,\!</math>

::<math>L(t/R)=\underset{i=1}{\overset{N}{\mathop \prod }}\,\left( \frac{-\text{ln}(R)}{t} \right)\cdot {{e}^{\left( \tfrac{\text{ln}(R)}{t} \right)\cdot {{x}_{i}}}}\,\!</math>

The unknown parameter <math>t/R\,\!</math> depends on what type of bounds are being determined. If one is trying to determine the bounds on time for the equation for the mean and the Bayes's rule equation for single parametera given reliability, then <math>R\,\!</math> is a known constant and <math>t\,\!</math> is the unknown parameter. Conversely, if one is trying to determine the bounds on reliability for a given time, then <math>t\,\!</math> is a known constant and <math>R\,\!</math> is the unknown parameter. Either way, the likelihood ratio function can be solved for the values of interest.

=====Example: LR Bounds on Time =====
For the data given above for the [[The_Exponential_Distribution#Example:_LR_Bounds_for_Lambda|LR Bounds on Lambda example]] (five failures at 20, 40, 60, 100 and 150 hours), determine the 85% two-sided confidence bounds on the time estimate for a reliability of 90%. The ML estimate for the time at <math>R(t)=90%\,\!</math> is <math>\hat{t}=7.797\,\!</math>.

'''Solution'''

In this example, we are trying to determine the 85% two-sided confidence bounds on the time estimate of 7.797. This is accomplished by substituting <math>R=0.90\,\!</math> and <math>\alpha =0.85\,\!</math> into the likelihood ratio bound equation. It now remains to find the values of <math>t\,\!</math> which satisfy this equation. Since there is only one parameter, there are only two values of <math>t\,\!</math> that will satisfy the equation. These values represent the <math>\delta =85%\,\!</math> two-sided confidence limits of the time estimate <math>\hat{t}\,\!</math>. For our problem, the confidence limits are:

::<math>{{\hat{t}}_{R=0.9}}=(4.359,16.033)\,\!</math>

=====Example: LR Bounds on Reliability=====
Again using the data given above for the [[The_Exponential_Distribution#Example:_LR_Bounds_for_Lambda|LR Bounds on Lambda example]] (five failures at 20, 40, 60, 100 and 150 hours), determine the 85% two-sided confidence bounds on the reliability estimate for a <math>t=50\,\!</math>. The ML estimate for the time at <math>t=50\,\!</math> is <math>\hat{R}=50.881%\,\!</math>.

'''Solution'''

In this example, we are trying to determine the 85% two-sided confidence bounds on the reliability estimate of 50.881%. This is accomplished by substituting <math>t=50\,\!</math> and <math>\alpha =0.85\,\!</math> into the likelihood ratio bound equation. It now remains to find the values of <math>R\,\!</math> which satisfy this equation. Since there is only one parameter, there are only two values of <math>t\,\!</math> that will satisfy the equation. These values represent the <math>\delta =85%\,\!</math> two-sided confidence limits of the reliability estimate <math>\hat{R}\,\!</math>. For our problem, the confidence limits are:

::<math>{{\hat{R}}_{t=50}}=(29.861%,71.794%)\,\!</math>

===Bayesian Confidence Bounds===
====Bounds on Parameters====
From [[Confidence Bounds]], we know that the posterior distribution of <math>\lambda \,\!</math> can be written as:

::<math>f(\lambda |Data)=\frac{L(Data|\lambda )\varphi (\lambda )}{\int_{0}^{\infty }L(Data|\lambda )\varphi (\lambda )d\lambda }\,\!</math>

where <math>\varphi (\lambda )=\tfrac{1}{\lambda }\,\!</math>, is the non-informative prior of <math>\lambda \,\!</math>.

With the above prior distribution, <math>f(\lambda |Data)\,\!</math> can be rewritten as:

::<math>f(\lambda |Data)=\frac{L(Data|\lambda )\tfrac{1}{\lambda }}{\int_{0}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }\,\!</math>

The one-sided upper bound of <math>\lambda \,\!</math> is:

::<math>CL=P(\lambda \le {{\lambda }_{U}})=\int_{0}^{{{\lambda }_{U}}}f(\lambda |Data)d\lambda \,\!</math>

The one-sided lower bound of <math>\lambda \,\!</math> is:

::<math>1-CL=P(\lambda \le {{\lambda }_{L}})=\int_{0}^{{{\lambda }_{L}}}f(\lambda |Data)d\lambda \,\!</math>

The two-sided bounds of <math>\lambda \,\!</math> are:

::<math>CL=P({{\lambda }_{L}}\le \lambda \le {{\lambda }_{U}})=\int_{{{\lambda }_{L}}}^{{{\lambda }_{U}}}f(\lambda |Data)d\lambda \,\!</math>

====Bounds on Time (Type 1)====
The reliable life equation is:

::<math>t=\frac{-\ln R}{\lambda }\,\!</math>

For the one-sided upper bound on time we have:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(t\le {{T}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\frac{-\ln R}{\lambda }\le {{T}_{U}})\,\!</math>

The above equation can be rewritten in terms of <math>\lambda \,\!</math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\frac{-\ln R}{{{t}_{U}}}\le \lambda )\,\!</math>

From the above posterior distribuiton equation, we have:

::<math>CL=\frac{\int_{\tfrac{-\ln R}{{{t}_{U}}}}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }{\int_{0}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }\,\!</math>

The above equation is solved w.r.t. <math>{{t}_{U}}.\,\!</math> The same method is applied for one-sided lower and two-sided bounds on time.

====Bounds on Reliability (Type 2)====
The one-sided upper bound on reliability is given by:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(R\le {{R}_{U}})=\underset{}{\overset{}{\mathop{\Pr }}}\,(\exp (-\lambda t)\le {{R}_{U}})\,\!</math>

The above equaation can be rewritten in terms of <math>\lambda \,\!</math> as:

::<math>CL=\underset{}{\overset{}{\mathop{\Pr }}}\,(\frac{-\ln {{R}_{U}}}{t}\le \lambda )\,\!</math>

From the equation for posterior distribution we have:

::<math>CL=\frac{\int_{\tfrac{-\ln {{R}_{U}}}{t}}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }{\int_{0}^{\infty }L(Data|\lambda )\tfrac{1}{\lambda }d\lambda }\,\!</math>

The above equation is solved w.r.t. <math>{{R}_{U}}.\,\!</math> The same method can be used to calculate one-sided lower and two sided bounds on reliability.

==Exponential Distribution Examples==
{{:Exponential Distribution Examples}}