Age Distributions in Continuous Markov Models


In another note we considered a generalized form of Markov models in which the transition rate parameters may be functions of the ages of the respective components. This was modeled in discrete time, and each state with n operational components consisted of an n-dimensional lattice space, with suitable aging transitions. In particular, we focused on a system with two components and three macro-states as shown below.



The transitions with rate parameters l1 and l2 leading back to State 0 result in the renewal of both components, whereas the transitions with rate parameters m1 and m2 represent repairs of just the single failed unit, while the unfailed unit is not renewed.


We denote by amn, bn, and cn the probabilities of the microstates in the macrostates 0, 1, and 2 respectively, with the understanding that the subscripts designate the (mean) age of the component(s) in a given microstate, measured in discrete units of the time increment Dt. Although the original motivation for considering these age sub-spaces was to allow for the transition rate parameters to be functions of the ages of the components, we sought to demonstrate the soundness of such models by first considering the special case with all constant transition rate parameters. In terms of the nomenclature defined above, and putting q = 1/ Dt, we found the following steady-state solution





We then defined the variables bn = bn/a00 and g0 = c0/a00, and found that



and for brevity we have put





In these terms the value of a00 was found to be given by





We now wish to determine the solution in the continuous limit, i.e., as the size of the time increments goes to zero. Thus for any given age t = n Dt we let Dt goes to zero and n go to infinity. The characteristic roots go to



so equation (1) becomes



Recalling the limiting identity



we can express (2) for sufficiently small Dt as



In the limit as Dt goes to zero the quantities in the square brackets approach unity, so this can be written as



The other auxiliary parameters approach





Inserting these into the expressions for b0 and g0, we get



Making these substitutions into the expression for a00 (for sufficiently small Dt), and simplifying, we get



Inserting these values of b0 and a00 into equation (3) and simplifying, we arrive at



The represents the incremental probability b for an incremental age range, so if we divide through by Dt and convert to differentials, this gives the continuous density distribution for the ages of the components in State 1 of the model (at steady-state)



To check this, we can integrate this density for component ages ranging from t = 0 to infinity, which gives the total probability



This is consistent with the steady-state solution presented in the earlier note. By the same analysis we find that the density function for the component ages in State 2 is



A plot of the age density for State 1 is shown below, based on the example specified in the earlier note, i.e., we have constant failure rate parameters of l1 = (50)10-6 per hour and l2 = (20)10-6 per hour, and repair rate parameters of m1 = 1/100 and m2 = 2/1000.



The density is zero at t = 0, but increases rapidly to a maximum at about 400 hours, and on the larger scale the density decays approximately exponentially. Notice that, since States 1 and 2 have only a single operational component, they are represented by one-dimensional spaces. In contrast, State 0 has two operational components, so it is represented by a two-dimensional space. From the earlier note we have the relations



and we also know that consecutive values along any diagonal are reduced by the factor



Consequently we have



Replacing k+m with n, and choosing Dt to be arbitrarily small, this leads to



Hence for arbitrarily small increments we have



Inserting the previous expression for Db, we get for the result





The age density in State 0 with t2 greater than t1 is therefore



The symmetrical formula applies for the region where t1 is greater than t2. Note that both of these density functions vanish identically on the diagonal t1 = t2.


We must now consider the diagonal points in the State 0 space, i.e., the conditions for which t1 equals t2, which signifies that the components are of exactly the same age. It might seem as if this should not contribute any probability in the continuous limit, because the ďareaĒ of the diagonal in the age space is zero. Hence for any finite density it would contribute no probability. However, our expression for a00 above contains only a single factor of† Dt, so the density is infinite at that point, and hence at every point on the diagonal. Itís true that the probability contributed by each of those diagonal points is zero, but when we integrate along the entire diagonal we get a finite contribution. The discrete sum of diagonal terms is



so, inserting the previous expression for a00, we find that the total probability along the continuous diagonal is



This shouldnít be surprising, because the double-fault transitions return the system to the fully-renewed microstate (i.e., the microstate a00), so there is always a significant portion of the population with both components of exactly the same age. The density along this singular diagonal is



To confirm these results we can integrate the expression for the age density of State 0 for the region with t1 less than t2 as follows



Similarly for the other region, i.e., with t2 less than t1, we get the integrated probability



Summing these two quantities and adding the contribution of the diagonal, we get (focusing on just the numerators, since they all have the same denominator)



Hence the sum of the three contributions equals the steady-state value of P0 determined in the previous note. As an aside, note that the double integral of either one of the density distributions (for the region above or below the diagonal) evaluated over the entire space also equals the correct steady-state value of P0, although apparently just by coincidence. This concordance applies not just to the integral over the entire age space of State 0, but even over each slice of constant t1 or t2, a fact which is significant for evaluating the continuous form of the age distribution equations.


Notice that the general case of equation (9) in the previous note can be expressed in the form



for any k greater than zero. (Boundary states are exceptions, because they lack one or the other of the aging transitions, but those will be treated appropriately when we sum these microstates to determine the probabilities of the macrostates.) To go over to the continuous case, we first define three density functions f, y1, and y2 for microstates of incremental size by the following relations



Substituting these into the preceding equation (and noting that Dt = dt1 = dt2) gives



Dividing through by dt2 and converting the summation to an integral, we get



By symmetry we also have the equation



Similarly, taking equations (8) from the previous note to the continuous limit gives



for† t1 , t2 > 0. The restriction to positive arguments is necessary due to the fact that equation (4) doesnít apply for the singular boundary case d0(0,0). Note that the solution derived above satisfies these relations. The continuous limit of equations (9) of the previous note gives the governing equation for the age density function of State 0



and the preceding equations (6) give the boundary conditions for this equation in terms of the density functions of the single-fault states. By direct substitution of our previous solution for the age density of State 0 into this equation we can show that it satisfies the equation for every point other than the diagonal, where the function is discontinuous. The previous two equations, (4) and (5) relate the full-up and single-fault density functions, and together with the continuous gauge equation



they enable us to solve for the steady-state density functions for the three system states. Of course, the age densities are functions of time, so the overall probabilities for the three macrostates at any time t are



To show that the above equations reduce to the usual Markov model equations when the transition rate parameters l1 and l2 are constant, we can integrate equation (4) for t2 = 0 to infinity, giving the result



The first integral in this equation vanishes because



so we have the expected relation



which is the same as the first of the steady-state equations for the time-independent model. Likewise integrating equation (5) leads to the second of the steady-state time-independent equations. Of course, the gauge condition (8) reduces to P0 + P1 + P2 = 1, so we have recovered all three of the steady-state equations with constant transition rate parameters.


Itís also interesting to see how our solution satisfies equation (4) directly, as opposed to the integral of equation (4). To show this, itís convenient to define the constants





Then our solution for State 0 can be expressed as



and for States 1 and 2 we have



Now, by direct substitution and differentiation into (4) we find that the left hand side is



The right hand side of (4) is more difficult to evaluate, because the integration extends over all three of the regions of State 0. Thus the integral consists of the three terms



The third term on the right side might be surprising, since each point on the diagonal contributes zero probability. However, the terms of this equation are not probabilities, they are one-dimensional densities, and the diagonal contributes a positive density at each point. Evaluating each of these three terms, we get



The first two terms of the first integral equal the left side of equation (4), so all the remaining terms must sum to zero. Hence we must have



The constants C1 and C2 each have a factor of l1l2/D, so we can factor that out of all the terms, and we get (after canceling terms)



This does indeed equal zero, confirming that the solution satisfies equation (4). We also find that equation (4) is satisfied even if we evaluate the entire right hand integral (from zero to infinity) using just the d0 function that strictly applies only for t1<t2.


Return to MathPages Main Menu