Template:Maximum Likelihood Parameter Estimation

Maximum Likelihood Estimation

If [math]\displaystyle{ x }[/math] is a continuous random variable with [math]\displaystyle{ pdf\ \ : }[/math]

[math]\displaystyle{ f(x;{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}), }[/math]

where [math]\displaystyle{ {{\theta }_{1}}, }[/math] [math]\displaystyle{ {{\theta }_{2}}, }[/math][math]\displaystyle{ ..., }[/math] [math]\displaystyle{ {{\theta }_{k}} }[/math] are [math]\displaystyle{ k }[/math] unknown constant parameters that need to be estimated, conduct an experiment and obtain [math]\displaystyle{ N }[/math] independent observations, [math]\displaystyle{ {{x}_{1}}, }[/math] [math]\displaystyle{ {{x}_{2}}, }[/math][math]\displaystyle{ ..., }[/math] [math]\displaystyle{ {{x}_{N}} }[/math], which correspond in the case of life data analysis to failure times. The likelihood function (for complete data) is given by:

[math]\displaystyle{ L({{x}_{1}},{{x}_{2}},...,{{x}_{N}}|{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}) }[/math]

[math]\displaystyle{ i=1,2,...,N }[/math]

The logarithmic likelihood function is:

[math]\displaystyle{ \Lambda =\ln L=\underset{i=1}{\overset{N}{\mathop \sum }}\,\ln f({{x}_{i}};{{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}) }[/math]

The maximum likelihood estimators (MLE) of [math]\displaystyle{ {{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}}, }[/math] are obtained by maximizing [math]\displaystyle{ L }[/math] or [math]\displaystyle{ \Lambda . }[/math]
By maximizing [math]\displaystyle{ \Lambda , }[/math] which is much easier to work with than [math]\displaystyle{ L }[/math], the maximum likelihood estimators (MLE) of [math]\displaystyle{ {{\theta }_{1}},{{\theta }_{2}},...,{{\theta }_{k}} }[/math] are the simultaneous solutions of [math]\displaystyle{ k }[/math] equations such that:

[math]\displaystyle{ \frac{\partial (\Lambda )}{\partial {{\theta }_{j}}}=0,j=1,2,...,k }[/math]

Even though it is common practice to plot the MLE solutions using median ranks (points are plotted according to median ranks and the line according to the MLE solutions), this is not completely accurate. As it can be seen from the equations above, the MLE method is independent of any kind of ranks. For this reason, many times the MLE solution appears not to track the data on the probability plot. This is perfectly acceptable since the two methods are independent of each other, and in no way suggests that the solution is wrong.

Illustrating the MLE Method Using the Exponential Distribution

• To estimate [math]\displaystyle{ \widehat{\lambda } }[/math] for a sample of [math]\displaystyle{ n }[/math] units (all tested to failure), first obtain the likelihood function:

[math]\displaystyle{ \begin{align} L(\lambda |{{t}_{1}},{{t}_{2}},...,{{t}_{n}})= & \underset{i=1}{\overset{n}{\mathop \prod }}\,f({{t}_{i}}) \\ = & \underset{i=1}{\overset{n}{\mathop \prod }}\,\lambda {{e}^{-\lambda {{t}_{i}}}} \\ = & {{\lambda }^{n}}\cdot {{e}^{-\lambda \underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{t}_{i}}}} \end{align} }[/math]

• Take the natural log of both sides:

[math]\displaystyle{ \Lambda =\ln (L)=n\ln (\lambda )-\lambda \underset{i=1}{\overset{n}{\mathop \sum }}\,{{t}_{i}}. }[/math]

• Obtain [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \lambda } }[/math], and set it equal to zero:

[math]\displaystyle{ \frac{\partial \Lambda }{\partial \lambda }=\frac{n}{\lambda }-\underset{i=1}{\overset{n}{\mathop \sum }}\,{{t}_{i}}=0 }[/math]

• Solve for [math]\displaystyle{ \widehat{\lambda } }[/math] or:

[math]\displaystyle{ \hat{\lambda }=\frac{n}{\underset{i=1}{\overset{n}{\mathop{\sum }}}\,{{t}_{i}}} }[/math]

Notes About [math]\displaystyle{ \widehat{\lambda } }[/math]

Note that the value of [math]\displaystyle{ \widehat{\lambda } }[/math] is an estimate because if we obtain another sample from the same population and re-estimate [math]\displaystyle{ \lambda }[/math], the new value would differ from the one previously calculated. In plain language, [math]\displaystyle{ \hat{\lambda } }[/math] is an estimate of the true value of ... How close is the value of our estimate to the true value? To answer this question, one must first determine the distribution of the parameter, in this case [math]\displaystyle{ \lambda }[/math]. This methodology introduces a new term, confidence bound, which allows us to specify a range for our estimate with a certain confidence level. The treatment of confidence bounds is integral to reliability engineering, and to all of statistics. (Confidence bounds are covered in Chapter 5.)

Illustrating the MLE Method Using Normal Distribution

To obtain the MLE estimates for the mean, [math]\displaystyle{ \bar{T}, }[/math] and standard deviation, [math]\displaystyle{ {{\sigma }_{T}}, }[/math] for the normal distribution, start with the [math]\displaystyle{ pdf }[/math] of the normal distribution which is given by:

[math]\displaystyle{ f(T)=\frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{T-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}} }[/math]

If [math]\displaystyle{ {{T}_{1}},{{T}_{2}},...,{{T}_{N}} }[/math] are known times-to-failure (and with no suspensions), then the likelihood function is given by:

[math]\displaystyle{ L({{T}_{1}},{{T}_{2}},...,{{T}_{N}}|\bar{T},{{\sigma }_{T}})=L=\underset{i=1}{\overset{N}{\mathop \prod }}\,\left[ \frac{1}{{{\sigma }_{T}}\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( \tfrac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}} \right] }[/math]

[math]\displaystyle{ L=\frac{1}{{{({{\sigma }_{T}}\sqrt{2\pi })}^{N}}}{{e}^{-\tfrac{1}{2}\underset{i=1}{\overset{N}{\mathop{\sum }}}\,{{\left( \tfrac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)}^{2}}}} }[/math]

then:

[math]\displaystyle{ \Lambda =\ln L=-\frac{N}{2}\ln (2\pi )-N\ln {{\sigma }_{T}}-\frac{1}{2}\underset{i=1}{\overset{N}{\mathop \sum }}\,\left( \frac{{{T}_{i}}-\bar{T}}{{{\sigma }_{T}}} \right)_{}^{2} }[/math]

Then taking the partial derivatives of [math]\displaystyle{ \Lambda }[/math] with respect to each one of the parameters and setting them equal to zero yields:

[math]\displaystyle{ \frac{\partial (\Lambda )}{\partial \bar{T}}=\frac{1}{\sigma _{T}^{2}}\underset{i=1}{\overset{N}{\mathop \sum }}\,({{T}_{i}}-\bar{T})=0 }[/math]

and:

[math]\displaystyle{ \frac{\partial (\Lambda )}{\partial {{\sigma }_{T}}}=-\frac{N}{{{\sigma }_{T}}}+\frac{1}{\sigma _{T}^{3}}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}=0 }[/math]

Solving Eqns. (dldt) and (dlds) simultaneously yields:

[math]\displaystyle{ \bar{T}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}} }[/math]

and:

[math]\displaystyle{ \begin{align} & \hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\ & & \\ & {{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}} \end{align} }[/math]

It should be noted that these solutions are only valid for data with no suspensions, i.e. all units are tested to failure. In the case where suspensions are present or all units are not tested to failure, the methodology changes and the problem becomes much more complicated.

Illustrating with an Example of the Normal Distribution

If we had five units that failed at 10, 20, 30, 40 and 50 hours, the mean would be:

[math]\displaystyle{ \begin{align} \bar{T}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}} \\ = & \frac{10+20+30+40+50}{5} \\ = & 30 \end{align} }[/math]

The standard deviation estimate then would be:

[math]\displaystyle{ \begin{align} {{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}} \\ = & \sqrt{\frac{{{(10-30)}^{2}}+{{(20-30)}^{2}}+{{(30-30)}^{2}}+{{(40-30)}^{2}}+{{(50-30)}^{2}}}{5}}, \\ = & 14.1421 \end{align} }[/math]

A look at the likelihood function surface plot in Figure A-1 reveals that both of these values are the maximum values of the function.

This three-dimensional plot represents the likelihood function. As can be seen from the plot, the maximum likelihood estimates for the two parameters correspond with the peak or maximum of the likelihood function surface.

Distribution Log-Likelihood Equations

This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed in more detail in Chapters 6 through 10.

Weibull Log-Likelihood Functions and their Partials

The Two-Parameter Weibull

This log-likelihood function is composed of three summation portions:

[math]\displaystyle{ \begin{align} & \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\ & & \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}} \right] \end{align} }[/math]

where:

• [math]\displaystyle{ {{F}_{e}} }[/math] is the number of groups of times-to-failure data points

• [math]\displaystyle{ {{N}_{i}} }[/math] is the number of times-to-failure in the [math]\displaystyle{ {{i}^{th}} }[/math] time-to-failure data group

• [math]\displaystyle{ \beta }[/math] is the Weibull shape parameter (unknown a priori, the first of two parameters to be found)

• [math]\displaystyle{ \eta }[/math] is the Weibull scale parameter (unknown a priori, the second of two parameters to be found)

• [math]\displaystyle{ {{T}_{i}} }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] group of time-to-failure data

• [math]\displaystyle{ S }[/math] is the number of groups of suspension data points

• [math]\displaystyle{ N_{i}^{\prime } }[/math] is the number of suspensions in [math]\displaystyle{ {{i}^{th}} }[/math] group of suspension data points

• [math]\displaystyle{ T_{i}^{\prime } }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] suspension data group

• [math]\displaystyle{ FI }[/math] is the number of interval failure data groups

• [math]\displaystyle{ N_{i}^{\prime \prime } }[/math] is the number of intervals in [math]\displaystyle{ {{i}^{th}} }[/math] group of data intervals

• [math]\displaystyle{ T_{Li}^{\prime \prime } }[/math] is the beginning of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

• and [math]\displaystyle{ T_{Ri}^{\prime \prime } }[/math] is the ending of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

For the purposes of MLE, left censored data will be considered to be intervals with [math]\displaystyle{ T_{Li}^{\prime \prime }=0. }[/math]

The solution will be found by solving for a pair of parameters [math]\displaystyle{ \left( \widehat{\beta },\widehat{\eta } \right) }[/math] so that [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \beta }=0 }[/math] and [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \eta }=0. }[/math] It should be noted that other methods can also be used, such as direct maximization of the likelihood function, without having to compute the derivatives.

[math]\displaystyle{ \begin{align} & \frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}}{\eta } \right) \\ & & -\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}}{\eta } \right)-\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }}{\eta } \right) \\ & & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}+{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}} \end{align} }[/math]

[math]\displaystyle{ \begin{align} & \frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}}{\eta } \right)}^{\beta }} \\ & & +\frac{\beta }{\eta }\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }}{\eta } \right)}^{\beta }} \\ & & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-\left( \tfrac{\beta }{\eta } \right){{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }}{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }}{\eta } \right)}^{\beta }}}}} \end{align} }[/math]

The Three-Parameter Weibull

This log-likelihood function is again composed of three summation portions:

[math]\displaystyle{ \begin{align} & \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{\beta }{\eta }{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta -1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }} \\ & & \\ & & +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}} \right] \end{align} }[/math]

where,

• [math]\displaystyle{ {{F}_{e}} }[/math] is the number of groups of times-to-failure data points

• [math]\displaystyle{ {{N}_{i}} }[/math] is the number of times-to-failure in the [math]\displaystyle{ {{i}^{th}} }[/math] time-to-failure data group

• [math]\displaystyle{ \beta }[/math] is the Weibull shape parameter (unknown a priori, the first of three parameters to be found)

• [math]\displaystyle{ \eta }[/math] is the Weibull scale parameter (unknown a priori, the second of three parameters to be found)

• [math]\displaystyle{ {{T}_{i}} }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] group of time-to-failure data

• [math]\displaystyle{ \gamma }[/math] is the Weibull location parameter (unknown a priori, the third of three parameters to be found)

• [math]\displaystyle{ S }[/math] is the number of groups of suspension data points

• [math]\displaystyle{ N_{i}^{\prime } }[/math] is the number of suspensions in [math]\displaystyle{ {{i}^{th}} }[/math] group of suspension data points

• [math]\displaystyle{ T_{i}^{\prime } }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] suspension data group

• [math]\displaystyle{ FI }[/math] is the number of interval data groups

• [math]\displaystyle{ N_{i}^{\prime \prime } }[/math] is the number of intervals in the [math]\displaystyle{ {{i}^{th}} }[/math] group of data intervals

• [math]\displaystyle{ T_{Li}^{\prime \prime } }[/math] is the beginning of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

• and [math]\displaystyle{ T_{Ri}^{\prime \prime } }[/math] is the ending of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

The solution is found by solving for [math]\displaystyle{ \left( \widehat{\beta },\widehat{\eta },\widehat{\gamma } \right) }[/math] so that [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \beta }=0, }[/math]

[math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \eta }=0, }[/math] and [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \gamma }=0. }[/math]

[math]\displaystyle{ \begin{align} & \frac{\partial \Lambda }{\partial \beta }= & \frac{1}{\beta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right)-\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{{{T}_{i}}-\gamma }{\eta } \right) \\ & & -\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right) \\ & & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\ & & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \end{align} }[/math]

[math]\displaystyle{ \begin{align} & \frac{\partial \Lambda }{\partial \eta }= & \frac{-\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}+\frac{\beta }{\eta }\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}+\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{\eta } \right) \\ & & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \\ & & -\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{\eta }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}\ln \left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right){{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \end{align} }[/math]

[math]\displaystyle{ \begin{align} & \frac{\partial \Lambda }{\partial \gamma }= & \left( 1-\beta \right)\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,\left( \frac{{{N}_{i}}}{{{T}_{i}}-\gamma } \right)+\underset{i=1}{\overset{{{F}_{e}}}{\mathop{\sum }}}\,{{N}_{i}}{{\left( \frac{{{T}_{i}}-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{{{T}_{i}}-\gamma } \right) \\ & & +\underset{i=1}{\overset{S}{\mathop{\sum }}}\,N_{i}^{\prime }{{\left( \frac{T_{i}^{\prime }-\gamma }{\eta } \right)}^{\beta }}\left( \frac{\beta }{T_{i}^{\prime }-\gamma } \right) \\ & & +\underset{i=1}{\overset{FI}{\mathop{\sum }}}\,N_{i}^{\prime \prime }\frac{\tfrac{\beta }{T_{Li}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-\tfrac{\beta }{T_{Ri}^{\prime \prime }-\gamma }{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}}{{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}-{{e}^{-{{\left( \tfrac{T_{Ri}^{\prime \prime }-\gamma }{\eta } \right)}^{\beta }}}}} \end{align} }[/math]

It should be pointed out that the solution to the three-parameter Weibull via MLE is not always stable and can collapse if [math]\displaystyle{ \beta \sim 1. }[/math] In estimating the true MLE of the three-parameter Weibull distribution, two difficulties arise. The first is a problem of non-regularity and the second is the parameter divergence problem [14].

Non-regularity occurs when [math]\displaystyle{ \beta \le 2. }[/math] In general, there are no MLE solutions in the region of [math]\displaystyle{ 0\lt \beta \lt 1. }[/math] When [math]\displaystyle{ 1\lt \beta \lt 2, }[/math] MLE solutions exist but are not asymptotically normal [14]. In the case of non-regularity, the solution is treated anomalously.

Weibull++ attempts to find a solution in all of the regions using a variety of methods, but the user should be forewarned that not all possible data can be addressed. Thus, some solutions using MLE for the three-parameter Weibull will fail when the algorithm has reached predefined limits or fails to converge. In these cases, the user can change to the non-true MLE approach (in Weibull++ User Setup), where [math]\displaystyle{ \gamma }[/math] is estimated using non-linear regression. Once [math]\displaystyle{ \gamma }[/math] is obtained, the MLE estimates of [math]\displaystyle{ \widehat{\beta } }[/math] and [math]\displaystyle{ \widehat{\eta } }[/math] are computed using the transformation [math]\displaystyle{ T_{i}^{\prime }=({{T}_{i}}-\gamma ). }[/math]

Exponential Log-Likelihood Functions and their Partials

The One-Parameter Exponential

This log-likelihood function is composed of three summation portions:

[math]\displaystyle{ \ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right] }[/math]

where:

• [math]\displaystyle{ {{F}_{e}} }[/math] is the number of groups of times-to-failure data points

• [math]\displaystyle{ {{N}_{i}} }[/math] is the number of times-to-failure in the [math]\displaystyle{ {{i}^{th}} }[/math] time-to-failure data group

• [math]\displaystyle{ \lambda }[/math] is the failure rate parameter (unknown a priori, the only parameter to be found)

• [math]\displaystyle{ {{T}_{i}} }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] group of time-to-failure data

• [math]\displaystyle{ S }[/math] is the number of groups of suspension data points

• [math]\displaystyle{ N_{i}^{\prime } }[/math] is the number of suspensions in the [math]\displaystyle{ {{i}^{th}} }[/math] group of suspension data points

• [math]\displaystyle{ T_{i}^{\prime } }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] suspension data group

• [math]\displaystyle{ FI }[/math] is the number of interval data groups

• [math]\displaystyle{ N_{i}^{\prime \prime } }[/math] is the number of intervals in the [math]\displaystyle{ {{i}^{th}} }[/math] group of data intervals

• [math]\displaystyle{ T_{Li}^{\prime \prime } }[/math] is the beginning of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

• and [math]\displaystyle{ T_{Ri}^{\prime \prime } }[/math] is the ending of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

The solution will be found by solving for a parameter [math]\displaystyle{ \widehat{\lambda } }[/math] so that [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \lambda }=0. }[/math] Note that for [math]\displaystyle{ FI=0 }[/math] there exists a closed form solution.

[math]\displaystyle{ \begin{align} & \frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\ & & -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right] \end{align} }[/math]

The Two-Parameter Exponential

This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:

[math]\displaystyle{ \begin{align} & \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma \right) \\ & & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}} \right], \end{align} }[/math]

where,

• [math]\displaystyle{ {{F}_{e}} }[/math] is the number of groups of times-to-failure data points

• [math]\displaystyle{ {{N}_{i}} }[/math] is the number of times-to-failure in the [math]\displaystyle{ {{i}^{th}} }[/math] time-to-failure data group

• [math]\displaystyle{ \lambda }[/math] is the failure rate parameter (unknown a priori, the first of two parameters to be found)

• [math]\displaystyle{ \gamma }[/math] is the location parameter (unknown a priori, the second of two parameters to be found)

• [math]\displaystyle{ {{T}_{i}} }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] group of time-to-failure data

• [math]\displaystyle{ S }[/math] is the number of groups of suspension data points

• [math]\displaystyle{ N_{i}^{\prime } }[/math] is the number of suspensions in the [math]\displaystyle{ {{i}^{th}} }[/math] group of suspension data points

• [math]\displaystyle{ T_{i}^{\prime } }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] suspension data group

• [math]\displaystyle{ FI }[/math] is the number of interval data groups

• [math]\displaystyle{ N_{i}^{\prime \prime } }[/math] is the number of intervals in the [math]\displaystyle{ {{i}^{th}} }[/math] group of data intervals

• [math]\displaystyle{ T_{Li}^{\prime \prime } }[/math] is the beginning of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

• and [math]\displaystyle{ T_{Ri}^{\prime \prime } }[/math] is the ending of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

The two-parameter solution will be found by solving for a pair of parameters ([math]\displaystyle{ \widehat{\lambda },\widehat{\gamma }), }[/math] such that [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0. }[/math] For the one-parameter case, solve for [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \lambda }=0. }[/math]

[math]\displaystyle{ \begin{align} \frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right] \\ & -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\ & -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}} \right] \end{align} }[/math]

and:

[math]\displaystyle{ \frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda }[/math]

Examination of Eqn. (expll1) will reveal that:

[math]\displaystyle{ \frac{\partial \Lambda }{\partial \gamma }=\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)\lambda \equiv 0 }[/math]

or Eqn. (expll2) will be equal to zero only if either:

[math]\displaystyle{ \lambda =0 }[/math]

or:

[math]\displaystyle{ \left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)=0 }[/math]

This is an unwelcome fact, alluded to earlier in the chapter, that essentially indicates that there is no realistic solution for the two-parameter MLE for exponential. The above equations indicate that there is no non-trivial MLE solution that satisfies both [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0. }[/math] It can be shown that the best solution for [math]\displaystyle{ \gamma , }[/math] satisfying the constraint that [math]\displaystyle{ \gamma \le {{T}_{1}} }[/math] is [math]\displaystyle{ \gamma ={{T}_{1}}. }[/math] To then solve for the two-parameter exponential distribution via MLE, one can set equal to the first time-to-failure, and then find a [math]\displaystyle{ \lambda }[/math] such that [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \lambda }=0. }[/math]

Using this methodology, a maximum can be achieved along the [math]\displaystyle{ \lambda }[/math]-axis, and a local maximum along the [math]\displaystyle{ \gamma }[/math]-axis at [math]\displaystyle{ \gamma ={{T}_{1}} }[/math], constrained by the fact that [math]\displaystyle{ \gamma \le {{T}_{1}} }[/math]. The 3D Plot utility in Weibull++ illustrates this behavior of the log-likelihood function, as shown next:

[math]\displaystyle{ }[/math]

Normal Log-Likelihood Functions and their Partials

The complete normal likelihood function (without the constant) is composed of three summation portions:

[math]\displaystyle{ \begin{align} \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{\sigma }\phi \left( \frac{{{T}_{i}}-\mu }{\sigma } \right) \right] \\ & +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{^{\prime }}\ln \left[ 1-\Phi \left( \frac{T_{i}^{^{\prime }}-\mu }{\sigma } \right) \right] \\ & \text{ }+\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{^{\prime \prime }}\ln \left[ \Phi \left( \frac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right)-\Phi \left( \frac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma } \right) \right] \end{align} }[/math]

where:

• [math]\displaystyle{ {{F}_{e}} }[/math] is the number of groups of times-to-failure data points

• [math]\displaystyle{ {{N}_{i}} }[/math] is the number of times-to-failure in the [math]\displaystyle{ {{i}^{th}} }[/math] time-to-failure data group

• [math]\displaystyle{ \mu }[/math] is the mean parameter (unknown a priori, the first of two parameters to be found)

• [math]\displaystyle{ \sigma }[/math] is the standard deviation parameter (unknown a priori, the second of two parameters to be found)

• [math]\displaystyle{ {{T}_{i}} }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] group of time-to-failure data

• [math]\displaystyle{ S }[/math] is the number of groups of suspension data points

• [math]\displaystyle{ N_{i}^{\prime } }[/math] is the number of suspensions in the [math]\displaystyle{ {{i}^{th}} }[/math] group of suspension data points

• [math]\displaystyle{ T_{i}^{\prime } }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] suspension data group

• [math]\displaystyle{ {{F}_{i}} }[/math] is the number of interval data groups

• [math]\displaystyle{ N_{i}^{\prime \prime } }[/math] is the number of intervals in the [math]\displaystyle{ {{i}^{th}} }[/math] group of data intervals

• [math]\displaystyle{ T_{Li}^{\prime \prime } }[/math] is the beginning of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

• and [math]\displaystyle{ T_{Ri}^{\prime \prime } }[/math] is the ending of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

The solution will be found by solving for a pair of parameters [math]\displaystyle{ \left( {{\mu }_{0}},{{\sigma }_{0}} \right) }[/math] so that [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \mu }=0 }[/math] and [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \sigma }=0. }[/math]

[math]\displaystyle{ \begin{align} \frac{\partial \Lambda }{\partial \mu }= & \frac{1}{{{\sigma }^{2}}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}({{T}_{i}}-\mu ) \\ & +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\ & -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)} \end{align} }[/math]

[math]\displaystyle{ \begin{align} \frac{\partial \Lambda }{\partial \sigma }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( {{T}_{i}}-\mu \right)}^{2}}}{{{\sigma }^{3}}}-\frac{1}{\sigma } \right) \\ & +\frac{1}{\sigma }\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)}{1-\Phi \left( \tfrac{T_{i}^{\prime }-\mu }{\sigma } \right)} \\ & -\frac{1}{\sigma }\underset{i=1}{\overset{{{F}_{i}}}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)\phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)}{\Phi \left( \tfrac{T_{Ri}^{\prime \prime }-\mu }{\sigma } \right)-\Phi \left( \tfrac{T_{Li}^{\prime \prime }-\mu }{\sigma } \right)} \end{align} }[/math]

where:

[math]\displaystyle{ \phi \left( x \right)=\frac{1}{\sqrt{2\pi }}{{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}} }[/math]

and:

[math]\displaystyle{ \Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt }[/math]

Complete Data

Note that for the normal distribution, and in the case of complete data only (as was shown in Chapter 3), there exists a closed-form solution for both of the parameters or:

[math]\displaystyle{ \widehat{\mu }=\widehat{{\bar{T}}}=\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{T}_{i}} }[/math]

and:

[math]\displaystyle{ \begin{align} \hat{\sigma }_{T}^{2}= & \frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}} \\ {{{\hat{\sigma }}}_{T}}= & \sqrt{\frac{1}{N}\underset{i=1}{\overset{N}{\mathop \sum }}\,{{({{T}_{i}}-\bar{T})}^{2}}} \end{align} }[/math]

Lognormal Log-Likelihood Functions and their Partials

The general log-likelihood function (without the constant) for the lognormal distribution is composed of three summation portions:

[math]\displaystyle{ \begin{align} \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \frac{1}{{{\sigma }_{{{T}'}}}}\phi \left( \frac{\ln \left( {{T}_{i}} \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\ & \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ 1-\Phi \left( \frac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \\ & \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \Phi \left( \frac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \frac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right) \right] \end{align} }[/math]

where:

• [math]\displaystyle{ {{F}_{e}} }[/math] is the number of groups of times-to-failure data points

• [math]\displaystyle{ {{N}_{i}} }[/math] is the number of times-to-failure in the [math]\displaystyle{ {{i}^{th}} }[/math] time-to-failure data group

• [math]\displaystyle{ {\mu }' }[/math] is the mean of the natural logarithms of the times-to-failure (unknown a priori, the first of two parameters to be found)

• [math]\displaystyle{ {{\sigma }_{{{T}'}}} }[/math] is the standard deviation of the natural logarithms of the times-to-failure (unknown a priori, the second of two parameters to be found)

• [math]\displaystyle{ {{T}_{i}} }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] group of time-to-failure data

• [math]\displaystyle{ S }[/math] is the number of groups of suspension data points

• [math]\displaystyle{ N_{i}^{\prime } }[/math] is the number of suspensions in the [math]\displaystyle{ {{i}^{th}} }[/math] group of suspension data points

• [math]\displaystyle{ T_{i}^{\prime } }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] suspension data group

• [math]\displaystyle{ FI }[/math] is the number of interval data groups

• [math]\displaystyle{ N_{i}^{\prime \prime } }[/math] is the number of intervals in the [math]\displaystyle{ {{i}^{th}} }[/math] group of data intervals

• [math]\displaystyle{ T_{Li}^{\prime \prime } }[/math] is the beginning of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

• and [math]\displaystyle{ T_{Ri}^{\prime \prime } }[/math] is the ending of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

The solution will be found by solving for a pair of parameters [math]\displaystyle{ \left( {\mu }',{{\sigma }_{{{T}'}}} \right) }[/math] so that [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial {\mu }'}=0 }[/math] and [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}=0 }[/math]:

[math]\displaystyle{ \begin{align} \frac{\partial \Lambda }{\partial {\mu }'}= & \frac{1}{\sigma _{{{T}'}}^{2}}\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}(\ln ({{T}_{i}})-{\mu }') \\ & +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\ & \ \ -\underset{i=1}{\overset{FI}{\mathop \sum }}\,\frac{N_{i}^{\prime \prime }}{\sigma }\frac{\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \end{align} }[/math]

[math]\displaystyle{ }[/math]

where:

[math]\displaystyle{ \phi \left( x \right)=\frac{1}{\sqrt{2\pi }}\cdot {{e}^{-\tfrac{1}{2}{{\left( x \right)}^{2}}}} }[/math]

and:

[math]\displaystyle{ \Phi (x)=\frac{1}{\sqrt{2\pi }}\int_{-\infty }^{x}{{e}^{-\tfrac{{{t}^{2}}}{2}}}dt }[/math]

Mixed Weibull Log-Likelihood Functions and their Partials

The log-likelihood function (without the constant) is composed of three summation portions: [math]\displaystyle{ \begin{align} \frac{\partial \Lambda }{\partial {{\sigma }_{{{T}'}}}}= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{{{\left( \ln ({{T}_{i}})-{\mu }' \right)}^{2}}}{\sigma _{{{T}'}}^{3}}-\frac{1}{{{\sigma }_{{{T}'}}}} \right) \\ & +\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\frac{\left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{1-\Phi \left( \tfrac{\ln \left( T_{i}^{\prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \\ & -\frac{1}{{{\sigma }_{{{T}'}}}}\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\frac{\left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)\phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)}{\Phi \left( \tfrac{\ln \left( T_{Ri}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)-\Phi \left( \tfrac{\ln \left( T_{Li}^{\prime \prime } \right)-{\mu }'}{{{\sigma }_{{{T}'}}}} \right)} \end{align} }[/math]

[math]\displaystyle{ \begin{align} \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{{{T}_{i}}}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\ & \text{ }+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}{{e}^{-{{\left( \tfrac{T_{i}^{\prime }}{{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \\ & \text{ }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ \underset{k=1}{\overset{Q}{\mathop \sum }}\,{{\rho }_{k}}\frac{{{\beta }_{k}}}{{{\eta }_{k}}}{{\left( \frac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left( \tfrac{T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}} \right)}^{{{\beta }_{k}}}}}} \right] \end{align} }[/math]

where:

• [math]\displaystyle{ {{F}_{e}} }[/math] is the number of groups of times-to-failure data points

• [math]\displaystyle{ {{N}_{i}} }[/math] is the number of times-to-failure in the [math]\displaystyle{ {{i}^{th}} }[/math] time-to-failure data group

• [math]\displaystyle{ Q }[/math] is the number of subpopulations

• [math]\displaystyle{ {{\rho }_{k}} }[/math] is the proportionality of the [math]\displaystyle{ {{k}^{th}} }[/math] subpopulation (unknown a priori, the first set of three sets of parameters to be found)

• [math]\displaystyle{ {{\beta }_{k}} }[/math] is the Weibull shape parameter of the [math]\displaystyle{ {{k}^{th}} }[/math] subpopulation (unknown a priori, the second set of three sets of parameters to be found)

• [math]\displaystyle{ {{\eta }_{k}} }[/math] is the Weibull scale parameter (unknown a priori, the third set of three sets of parameters to be found)

• [math]\displaystyle{ {{T}_{i}} }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] group of time-to-failure data

• [math]\displaystyle{ S }[/math] is the number of groups of suspension data points

• [math]\displaystyle{ N_{i}^{\prime } }[/math] is the number of suspensions in [math]\displaystyle{ {{i}^{th}} }[/math] group of suspension data points

• [math]\displaystyle{ T_{i}^{\prime } }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] suspension data group

• [math]\displaystyle{ FI }[/math] is the number of groups of interval data points

• [math]\displaystyle{ N_{i}^{\prime \prime } }[/math] is the number of intervals in [math]\displaystyle{ {{i}^{th}} }[/math] group of data intervals

• [math]\displaystyle{ T_{Li}^{\prime \prime } }[/math] is the beginning of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

• and [math]\displaystyle{ T_{Ri}^{\prime \prime } }[/math] is the ending of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

The solution will be found by solving for a group of parameters:

[math]\displaystyle{ \left( \widehat{{{\rho }_{1,}}}\widehat{{{\beta }_{1}}},\widehat{{{\eta }_{1}}},\widehat{{{\rho }_{2,}}}\widehat{{{\beta }_{2}}},\widehat{{{\eta }_{2}}},...,\widehat{{{\rho }_{Q,}}}\widehat{{{\beta }_{Q}}},\widehat{{{\eta }_{Q}}} \right) }[/math]

so that:

[math]\displaystyle{ \begin{align} \frac{\partial \Lambda }{\partial {{\rho }_{1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{1}}}=0 \\ \frac{\partial \Lambda }{\partial {{\rho }_{2}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{2}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{2}}}=0 \\ \vdots \\ \frac{\partial \Lambda }{\partial {{\rho }_{Q-1}}}= & 0,\frac{\partial \Lambda }{\partial {{\beta }_{Q-1}}}=0,\frac{\partial \Lambda }{\partial {{\eta }_{Q-1}}}=0 \\ \frac{\partial \Lambda }{\partial {{\beta }_{Q}}}= & 0,\text{ and }\frac{\partial \Lambda }{\partial {{\eta }_{Q}}}=0 \end{align} }[/math]

Logistic Log-Likelihood Functions and their Partials

This log-likelihood function is composed of three summation portions:

[math]\displaystyle{ \begin{align} & \ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{\sigma {{(1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}})}^{2}}} \right)-\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}) \\ & & +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right) \end{align} }[/math]

where:

• [math]\displaystyle{ {{F}_{e}} }[/math] is the number of groups of times-to-failure data points

• [math]\displaystyle{ {{N}_{i}} }[/math] is the number of times-to-failure in the [math]\displaystyle{ {{i}^{th}} }[/math] time-to-failure data group

• [math]\displaystyle{ \mu }[/math] is the logistic shape parameter (unknown a priori, the first of two parameters to be found)

• [math]\displaystyle{ \eta }[/math] is the logistic scale parameter (unknown a priori, the second of two parameters to be found)

• [math]\displaystyle{ {{T}_{i}} }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] group of time-to-failure data

• [math]\displaystyle{ S }[/math] is the number of groups of suspension data points

• [math]\displaystyle{ N_{i}^{\prime } }[/math] is the number of suspensions in [math]\displaystyle{ {{i}^{th}} }[/math] group of suspension data points

• [math]\displaystyle{ T_{i}^{\prime } }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] suspension data group

• [math]\displaystyle{ FI }[/math] is the number of interval failure data group

• [math]\displaystyle{ N_{i}^{\prime \prime } }[/math] is the number of intervals in [math]\displaystyle{ {{i}^{th}} }[/math] group of data intervals

• [math]\displaystyle{ T_{Li}^{\prime \prime } }[/math] is the beginning of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

• and [math]\displaystyle{ T_{Ri}^{\prime \prime } }[/math] is the ending of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

For the purposes of MLE, left censored data will be considered to be intervals with [math]\displaystyle{ T_{Li}^{\prime \prime }=0. }[/math]

The solution of the maximum log-likelihood function is found by solving for ([math]\displaystyle{ \widehat{\mu },\widehat{\sigma }) }[/math] so that [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0. }[/math]

[math]\displaystyle{ \begin{align} & \frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\ & & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\mathop{}_{}^{}}}\,}}\,N_{i}^{^{\prime \prime }}}{\sigma }+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \right) \end{align} }[/math]

[math]\displaystyle{ \begin{align} & \frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}} \\ & & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}} \\ & & \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{1+{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}} \\ & & -\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}{{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}) \end{align} }[/math]

The Loglogistic Log-Likelihood Functions and their Partials

This log-likelihood function is composed of three summation portions:

[math]\displaystyle{ \begin{align} \ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{\sigma t{{(1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}})}^{2}}} \right) \\ & -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln (1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}) \\ & +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( \frac{1}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}-\frac{1}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right) \end{align} }[/math]

where:

• [math]\displaystyle{ {{F}_{e}} }[/math] is the number of groups of times-to-failure data points

• [math]\displaystyle{ {{N}_{i}} }[/math] is the number of times-to-failure in the [math]\displaystyle{ {{i}^{th}} }[/math] time-to-failure data group

• [math]\displaystyle{ \mu }[/math] is the loglogistic shape parameter (unknown a priori, the first of two parameters to be found)

• [math]\displaystyle{ \sigma }[/math] is the loglogistic scale parameter (unknown a priori, the second of two parameters to be found)

• [math]\displaystyle{ {{T}_{i}} }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] group of time-to-failure data

• [math]\displaystyle{ S }[/math] is the number of groups of suspension data points

• [math]\displaystyle{ N_{i}^{\prime } }[/math] is the number of suspensions in [math]\displaystyle{ {{i}^{th}} }[/math] group of suspension data points

• [math]\displaystyle{ T_{i}^{\prime } }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] suspension data group

• [math]\displaystyle{ FI }[/math] is the number of interval failure data groups,

• [math]\displaystyle{ N_{i}^{\prime \prime } }[/math] is the number of intervals in [math]\displaystyle{ {{i}^{th}} }[/math] group of data intervals

• [math]\displaystyle{ T_{Li}^{\prime \prime } }[/math] is the beginning of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

• and [math]\displaystyle{ T_{Ri}^{\prime \prime } }[/math] is the ending of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

For the purposes of MLE, left censored data will be considered to be intervals with [math]\displaystyle{ T_{Li}^{\prime \prime }=0. }[/math]

The solution of the maximum log-likelihood function is found by solving for ([math]\displaystyle{ \widehat{\mu },\widehat{\sigma }) }[/math] so that [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0. }[/math]

[math]\displaystyle{ \begin{align} \frac{\partial \Lambda }{\partial \mu }= & -\frac{\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\sigma }+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\ & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}-\frac{{{F}_{I}}}{\sigma } \\ & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \right) \end{align} }[/math]

[math]\displaystyle{ \begin{align} \frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\ln ({{T}_{i}})-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{2}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln ({{T}_{i}})-\mu }{\sigma }}}} \\ & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{i}^{^{\prime }})-\mu }{\sigma }}}} \\ & \frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}(\frac{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}+\frac{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{1+{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}} \\ & -\frac{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}{{{e}^{\tfrac{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}-{{e}^{\tfrac{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }{\sigma }}}}) \end{align} }[/math]

The Gumbel Log-Likelihood Functions and their Partials

This log-likelihood function is composed of three summation portions:

[math]\displaystyle{ \begin{align} \ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}}}}{\sigma } \right) \\ & -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}}} \right) \\ & +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right) \end{align} }[/math]

or

[math]\displaystyle{ \begin{align} \Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\left( \frac{{{T}_{i}}-\mu }{\sigma }-{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \right)-\ln (\sigma )\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}} \\ & +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\ & +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}} \right) \end{align} }[/math]

where:

• [math]\displaystyle{ {{F}_{e}} }[/math] is the number of groups of times-to-failure data points

• [math]\displaystyle{ {{N}_{i}} }[/math] is the number of times-to-failure in the [math]\displaystyle{ {{i}^{th}} }[/math] time-to-failure data group

• [math]\displaystyle{ \mu }[/math] is the Gumbel shape parameter (unknown a priori, the first of two parameters to be found)

• [math]\displaystyle{ \sigma }[/math] is the Gumbel scale parameter (unknown a priori, the second of two parameters to be found)

• [math]\displaystyle{ {{T}_{i}} }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] group of time-to-failure data

• [math]\displaystyle{ S }[/math] is the number of groups of suspension data points

• [math]\displaystyle{ N_{i}^{\prime } }[/math] is the number of suspensions in [math]\displaystyle{ {{i}^{th}} }[/math] group of suspension data points

• [math]\displaystyle{ T_{i}^{\prime } }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] suspension data group

• [math]\displaystyle{ FI }[/math] is the number of interval failure data groups

• [math]\displaystyle{ N_{i}^{\prime \prime } }[/math] is the number of intervals in [math]\displaystyle{ {{i}^{th}} }[/math] group of data intervals

• [math]\displaystyle{ T_{Li}^{\prime \prime } }[/math] is the beginning of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

• and [math]\displaystyle{ T_{Ri}^{\prime \prime } }[/math] is the ending of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

For the purposes of MLE, left censored data will be considered to be intervals with [math]\displaystyle{ T_{Li}^{\prime \prime }=0. }[/math]

The solution of the maximum log-likelihood function is found by solving for ([math]\displaystyle{ \widehat{\mu },\widehat{\sigma }) }[/math] so that:

[math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial \sigma }=0. }[/math]

[math]\displaystyle{ \begin{align} \frac{\partial \Lambda }{\partial \mu }= & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}} \\ & +\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right) \end{align} }[/math]

[math]\displaystyle{ \begin{align} \frac{\partial \Lambda }{\partial \sigma }= & -\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{{{\sigma }^{2}}}-\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\frac{{{T}_{i}}-\mu }{\sigma }{{e}^{\tfrac{{{T}_{i}}-\mu }{\sigma }}} \\ & -\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{T_{i}^{^{\prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{i}^{^{\prime }}-\mu }{\sigma }}}+\frac{1}{\sigma }\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }} \\ & \left( \frac{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}}{{{e}^{-{{e}^{\tfrac{T_{{{L}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}-{{e}^{-{{e}^{\tfrac{T_{{{R}_{i}}}^{^{\prime \prime }}-\mu }{\sigma }}}}}} \right) \end{align} }[/math]

The Gamma Log-Likelihood Functions and their Partials

This log-likelihood function is composed of three summation portions:

[math]\displaystyle{ \begin{align} \ln (L)= & \Lambda =\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln \left( \frac{{{e}^{k(\ln ({{T}_{i}})-\mu )-{{e}^{{{e}^{\ln ({{T}_{i}})-\mu }}}}}}}{{{T}_{i}}\Gamma (k)} \right) \\ & +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-\Gamma \left( _{1}k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right) \right) \\ & +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right) \right) \end{align} }[/math]

or:

[math]\displaystyle{ \begin{align} \Lambda = & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln ({{T}_{i}})\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}\ln (\Gamma (k))+k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu ) \\ & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{-\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\ & +\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\ln \left( 1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right) \right) \\ & +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\ln \left( {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu )}} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu )}} \right) \right) \end{align} }[/math]

where: • [math]\displaystyle{ {{F}_{e}} }[/math] is the number of groups of times-to-failure data points

• [math]\displaystyle{ {{N}_{i}} }[/math] is the number of times-to-failure in the [math]\displaystyle{ {{i}^{th}} }[/math] time-to-failure data group

• [math]\displaystyle{ \mu }[/math] is the gamma shape parameter (unknown a priori, the first of two parameters to be found)

• [math]\displaystyle{ k }[/math] is the gamma scale parameter (unknown a priori, the second of two parameters to be found)

• [math]\displaystyle{ {{T}_{i}} }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] group of time-to-failure data

• [math]\displaystyle{ S }[/math] is the number of groups of suspension data points

• .. is the number of suspensions in [math]\displaystyle{ {{i}^{th}} }[/math] group of suspension data points

• [math]\displaystyle{ T_{i}^{\prime } }[/math] is the time of the [math]\displaystyle{ {{i}^{th}} }[/math] suspension data group

• [math]\displaystyle{ FI }[/math] is the number of interval failure data groups

• [math]\displaystyle{ N_{i}^{\prime \prime } }[/math] is the number of intervals in [math]\displaystyle{ {{i}^{th}} }[/math] group of data intervals

• [math]\displaystyle{ T_{Li}^{\prime \prime } }[/math] is the beginning of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

• and [math]\displaystyle{ T_{Ri}^{\prime \prime } }[/math] is the ending of the [math]\displaystyle{ {{i}^{th}} }[/math] interval

For the purposes of MLE, left censored data will be considered to be intervals with [math]\displaystyle{ T_{Li}^{\prime \prime }=0. }[/math]

The solution of the maximum log-likelihood function is found by solving for ([math]\displaystyle{ \widehat{\mu },\widehat{\sigma }) }[/math] so that [math]\displaystyle{ \tfrac{\partial \Lambda }{\partial \mu }=0,\tfrac{\partial \Lambda }{\partial k}=0. }[/math]

[math]\displaystyle{ \begin{align} \frac{\partial \Lambda }{\partial \mu }= & -k\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}+\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}{{e}^{\ln ({{T}_{i}})-\mu }} \\ & +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{{{e}^{k\left( \ln (T_{i}^{^{\prime }})-\mu )-{{e}^{\ln (T_{i}^{^{\prime }})-\mu )}} \right)}}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\ & +\frac{1}{\Gamma (k)}\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\{\frac{{{e}^{k{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}-{{e}^{{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)} \\ & -\frac{{{e}^{k{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}-{{e}^{{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}}}}}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}\} \end{align} }[/math]

[math]\displaystyle{ \begin{align} \frac{\partial \Lambda }{\partial k}= & \underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,{{N}_{i}}(\ln ({{T}_{i}})-\mu )-\frac{{{\Gamma }^{^{\prime }}}(k)\underset{i=1}{\mathop{\overset{{{F}_{e}}}{\mathop{\mathop{}_{}^{}}}\,}}\,{{N}_{i}}}{\Gamma (k)} \\ & -\underset{i=1}{\mathop{\overset{S}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime }}\frac{\tfrac{\partial {{\Gamma }_{1}}(k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }})}{\partial k}}{1-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{i}^{^{\prime }})-\mu }} \right)} \\ & +\underset{i=1}{\mathop{\overset{{{F}_{I}}}{\mathop{\underset{}{\overset{}{\mathop \sum }}\,}}\,}}\,N_{i}^{^{\prime \prime }}\left( \frac{\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}-\tfrac{\partial {{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)}{\partial k}}{{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{R}_{i}}}^{^{\prime \prime }})-\mu }} \right)-{{\Gamma }_{1}}\left( k;{{e}^{\ln (T_{{{L}_{i}}}^{^{\prime \prime }})-\mu }}) \right)} \right) \end{align} }[/math]