Exponential Log-Likelihood Functions and their Partials - Revision history

Lisa Hacker: Redirected page to Appendix: Log-Likelihood Equations

2015-06-25T20:08:03Z

Redirected page to Appendix: Log-Likelihood Equations

← Older revision		Revision as of 20:08, 25 June 2015
Line 1:		Line 1:
	~~=== Exponential Log-Likelihood Functions and their Partials===~~		#REDIRECT [[Appendix:_Log-Likelihood_Equations]]
	~~==== The One-Parameter Exponential====~~
	~~This log-likelihood function is composed of three summation portions:~~

	::<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right]</math>

	~~:where:~~
	~~::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points~~
	~~::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group~~
	~~::• <math>\lambda </math> is the failure rate parameter (unknown a priori, the only parameter to be found)~~
	~~::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data~~
	~~::• <math>S</math> is the number of groups of suspension data points~~
	~~::• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points~~
	~~::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group~~
	~~::• <math>FI</math> is the number of interval data groups~~
	~~::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals~~
	~~::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval~~
	~~::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval~~

	The solution will be found by solving for a parameter <math>\widehat{\lambda }</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math> Note that for <math>FI=0</math> there exists a closed form solution.

	~~::<math>\begin{align}~~
	& \frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\
	~~& & -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left~~[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right]
	~~\end{align}</math>~~

	~~==== The Two-Parameter Exponential====~~
	~~This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:~~


	~~::<math>\begin{align}~~
	~~& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left~~[ ~~\lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma \right) \\~~
	& & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}} \right],
	~~\end{align}</math>~~

	~~:where,~~
	~~::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points~~
	~~::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group~~
	~~::• <math>\lambda </math> is the failure rate parameter (unknown a priori, the first of two parameters to be found)~~
	~~::• <math>\gamma </math> is the location parameter (unknown a priori, the second of two parameters to be found)~~
	::~~• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time~~-~~to-failure data~~
	~~::• <math>S</math> is the number of groups of suspension data points~~
	~~::• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points~~
	~~::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group~~
	~~::• <math>FI</math> is the number of interval data groups~~
	~~::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals~~
	~~::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval~~
	~~::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval~~


	The two-parameter solution will be found by solving for a pair of parameters (<math>\widehat{\lambda },\widehat{\gamma }),</math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math> For the one-parameter case, solve for <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

	~~::<math>\begin{align}~~
	~~\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right~~] \\
	~~& -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\~~
	& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}} \right]
	~~\end{align}</math>~~

	~~:and:~~

	::<math>\frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda </math>

	~~Examination of Eqn. (expll1) will reveal that:~~

	::<math>\frac{\partial \Lambda }{\partial \gamma }=\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)\lambda \equiv 0</math>

	~~:or Eqn. (expll2) will be equal to zero only if either:~~

	~~::<math>\lambda =0</math>~~

	~~:or:~~

	::<math>\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)=0</math>

	This is an unwelcome fact, alluded to earlier in the chapter, that essentially indicates that there is no realistic solution for the two-parameter MLE for exponential. The above equations indicate that there is no non-trivial MLE solution that satisfies both <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>
	It can be shown that the best solution for <math>\gamma ,</math> satisfying the constraint that <math>\gamma \le {{T}_{1}}</math> is <math>\gamma ={{T}_{1}}.</math> To then solve for the two-parameter exponential distribution via MLE, one can set equal to the first time-to-failure, and then find a <math>\lambda </math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

	Using this methodology, a maximum can be achieved along the <math>\lambda </math>-axis, and a local maximum along the <math>\gamma </math>-axis at <math>\gamma ={{T}_{1}}</math>, constrained by the fact that <math>\gamma \le {{T}_{1}}</math>. The 3D Plot utility in Weibull++ illustrates this behavior of the log-likelihood function, as shown next:

	~~<math></math>~~

Pantelis: Created page with '=== Exponential Log-Likelihood Functions and their Partials=== ==== The One-Parameter Exponential==== This log-likelihood function is composed of three summation portions: ::
2011-10-29T13:15:53Z

Created page with '=== Exponential Log-Likelihood Functions and their Partials=== ==== The One-Parameter Exponential==== This log-likelihood function is composed of three summation portions: ::<m…'

New page
=== Exponential Log-Likelihood Functions and their Partials===
==== The One-Parameter Exponential====
This log-likelihood function is composed of three summation portions:

::<math>\ln (L)=\Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda {{T}_{i}}}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}} \right]</math>

:where:
::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>\lambda </math> is the failure rate parameter (unknown a priori, the only parameter to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>S</math> is the number of groups of suspension data points
::• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>FI</math> is the number of interval data groups
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The solution will be found by solving for a parameter <math>\widehat{\lambda }</math> so that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math> Note that for <math>FI=0</math> there exists a closed form solution.

::<math>\begin{align}
& \frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left( \frac{1}{\lambda }-{{T}_{i}} \right)-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }T_{i}^{\prime } \\
& & -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}} \right]
\end{align}</math>

==== The Two-Parameter Exponential====
This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:

::<math>\begin{align}
& \ln (L)= & \Lambda =\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\ln \left[ \lambda {{e}^{-\lambda \left( {{T}_{i}}-\gamma \right)}} \right]-\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda \left( T_{i}^{\prime }-\gamma \right) \\
& & \ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\ln \left[ {{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}} \right],
\end{align}</math>

:where,
::• <math>{{F}_{e}}</math> is the number of groups of times-to-failure data points
::• <math>{{N}_{i}}</math> is the number of times-to-failure in the <math>{{i}^{th}}</math> time-to-failure data group
::• <math>\lambda </math> is the failure rate parameter (unknown a priori, the first of two parameters to be found)
::• <math>\gamma </math> is the location parameter (unknown a priori, the second of two parameters to be found)
::• <math>{{T}_{i}}</math> is the time of the <math>{{i}^{th}}</math> group of time-to-failure data
::• <math>S</math> is the number of groups of suspension data points
::• <math>N_{i}^{\prime }</math> is the number of suspensions in the <math>{{i}^{th}}</math> group of suspension data points
::• <math>T_{i}^{\prime }</math> is the time of the <math>{{i}^{th}}</math> suspension data group
::• <math>FI</math> is the number of interval data groups
::• <math>N_{i}^{\prime \prime }</math> is the number of intervals in the <math>{{i}^{th}}</math> group of data intervals
::• <math>T_{Li}^{\prime \prime }</math> is the beginning of the <math>{{i}^{th}}</math> interval
::• and <math>T_{Ri}^{\prime \prime }</math> is the ending of the <math>{{i}^{th}}</math> interval

The two-parameter solution will be found by solving for a pair of parameters (<math>\widehat{\lambda },\widehat{\gamma }),</math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math> For the one-parameter case, solve for <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

::<math>\begin{align}
\frac{\partial \Lambda }{\partial \lambda }= & \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\left[ \frac{1}{\lambda }-\left( {{T}_{i}}-\gamma \right) \right] \\
& -\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\left( T_{i}^{\prime }-\gamma \right) \\
& -\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\left[ \frac{\left( T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Li}^{\prime \prime }-{{\gamma }_{0}} \right)}}-\left( T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left( T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left( T_{Ri}^{\prime \prime }-\gamma \right)}}} \right]
\end{align}</math>

:and:

::<math>\frac{\partial \Lambda }{\partial \gamma }=\underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}\lambda +\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\lambda +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime }\lambda </math>

Examination of Eqn. (expll1) will reveal that:

::<math>\frac{\partial \Lambda }{\partial \gamma }=\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)\lambda \equiv 0</math>

:or Eqn. (expll2) will be equal to zero only if either:

::<math>\lambda =0</math>

:or:

::<math>\left( \underset{i=1}{\overset{{{F}_{e}}}{\mathop \sum }}\,{{N}_{i}}+\underset{i=1}{\overset{S}{\mathop \sum }}\,N_{i}^{\prime }\ \ +\underset{i=1}{\overset{FI}{\mathop \sum }}\,N_{i}^{\prime \prime } \right)=0</math>

This is an unwelcome fact, alluded to earlier in the chapter, that essentially indicates that there is no realistic solution for the two-parameter MLE for exponential. The above equations indicate that there is no non-trivial MLE solution that satisfies both <math>\tfrac{\partial \Lambda }{\partial \lambda }=0,\tfrac{\partial \Lambda }{\partial \gamma }=0.</math>
It can be shown that the best solution for <math>\gamma ,</math> satisfying the constraint that <math>\gamma \le {{T}_{1}}</math> is <math>\gamma ={{T}_{1}}.</math> To then solve for the two-parameter exponential distribution via MLE, one can set equal to the first time-to-failure, and then find a <math>\lambda </math> such that <math>\tfrac{\partial \Lambda }{\partial \lambda }=0.</math>

Using this methodology, a maximum can be achieved along the <math>\lambda </math>-axis, and a local maximum along the <math>\gamma </math>-axis at <math>\gamma ={{T}_{1}}</math>, constrained by the fact that <math>\gamma \le {{T}_{1}}</math>. The 3D Plot utility in Weibull++ illustrates this behavior of the log-likelihood function, as shown next:

<math></math>