Age Distributions in Continuous Markov Models 

In another note we considered a generalized form of Markov models in which the transition rate parameters may be functions of the ages of the respective components. This was modeled in discrete time, and each state with n operational components consisted of an ndimensional lattice space, with suitable aging transitions. In particular, we focused on a system with two components and three macrostates as shown below. 


The transitions with rate parameters l_{1} and l_{2} leading back to State 0 result in the renewal of both components, whereas the transitions with rate parameters m_{1} and m_{2} represent repairs of just the single failed unit, while the unfailed unit is not renewed. 

We denote by a_{mn}, b_{n}, and c_{n} the probabilities of the microstates in the macrostates 0, 1, and 2 respectively, with the understanding that the subscripts designate the (mean) age of the component(s) in a given microstate, measured in discrete units of the time increment Dt. Although the original motivation for considering these age subspaces was to allow for the transition rate parameters to be functions of the ages of the components, we sought to demonstrate the soundness of such models by first considering the special case with all constant transition rate parameters. In terms of the nomenclature defined above, and putting q = 1/ Dt, we found the following steadystate solution 

_{} 

where 
_{} 

We then defined the variables b_{n} = b_{n}/a_{00} and g_{0} = c_{0}/a_{00}, and found that 

_{} 

and for brevity we have put 

_{} 

and 
_{} 

In these terms the value of a_{00} was found to be given by 

_{} 

where 
_{} 

We now wish to determine the solution in the continuous limit, i.e., as the size of the time increments goes to zero. Thus for any given age t = n Dt we let Dt goes to zero and n go to infinity. The characteristic roots go to 

_{} 

so equation (1) becomes 

_{} 

Recalling the limiting identity 

_{} 

we can express (2) for sufficiently small Dt as 

_{} 

In the limit as Dt goes to zero the quantities in the square brackets approach unity, so this can be written as 

_{} 

The other auxiliary parameters approach 

_{} 

and 
_{} 

Inserting these into the expressions for b_{0} and g_{0}, we get 

_{} 

Making these substitutions into the expression for a_{00} (for sufficiently small Dt), and simplifying, we get 

_{} 

Inserting these values of b_{0} and a_{00} into equation (3) and simplifying, we arrive at 

_{} 

The represents the incremental probability b for an incremental age range, so if we divide through by Dt and convert to differentials, this gives the continuous density distribution for the ages of the components in State 1 of the model (at steadystate) 

_{} 

To check this, we can integrate this density for component ages ranging from t = 0 to infinity, which gives the total probability 

_{} 

This is consistent with the steadystate solution presented in the earlier note. By the same analysis we find that the density function for the component ages in State 2 is 

_{} 

A plot of the age density for State 1 is shown below, based on the example specified in the earlier note, i.e., we have constant failure rate parameters of l_{1} = (50)10^{6} per hour and l_{2} = (20)10^{6} per hour, and repair rate parameters of m_{1} = 1/100 and m_{2} = 2/1000. 


The density is zero at t = 0, but increases rapidly to a maximum at about 400 hours, and on the larger scale the density decays approximately exponentially. Notice that, since States 1 and 2 have only a single operational component, they are represented by onedimensional spaces. In contrast, State 0 has two operational components, so it is represented by a twodimensional space. From the earlier note we have the relations 

_{} 

and we also know that consecutive values along any diagonal are reduced by the factor 

_{} 

Consequently we have 

_{} 

Replacing k+m with n, and choosing Dt to be arbitrarily small, this leads to 

_{} 

Hence for arbitrarily small increments we have 

_{} 

Inserting the previous expression for Db, we get for the result 

_{} 

where 
_{} 

The age density in State 0 with t_{2} greater than t_{1} is therefore 

_{} 

The symmetrical formula applies for the region where t_{1} is greater than t_{2}. Note that both of these density functions vanish identically on the diagonal t_{1} = t_{2}. 

We must now consider the diagonal points in the State 0 space, i.e., the conditions for which t_{1} equals t_{2}, which signifies that the components are of exactly the same age. It might seem as if this should not contribute any probability in the continuous limit, because the “area” of the diagonal in the age space is zero. Hence for any finite density it would contribute no probability. However, our expression for a_{00} above contains only a single factor of Dt, so the density is infinite at that point, and hence at every point on the diagonal. It’s true that the probability contributed by each of those diagonal points is zero, but when we integrate along the entire diagonal we get a finite contribution. The discrete sum of diagonal terms is 

_{} 

so, inserting the previous expression for a_{00}, we find that the total probability along the continuous diagonal is 

_{} 

This shouldn’t be surprising, because the doublefault transitions return the system to the fullyrenewed microstate (i.e., the microstate a_{00}), so there is always a significant portion of the population with both components of exactly the same age. The density along this singular diagonal is 

_{} 

To confirm these results we can integrate the expression for the age density of State 0 for the region with t_{1} less than t_{2} as follows 

_{} 

Similarly for the other region, i.e., with t_{2} less than t_{1}, we get the integrated probability 

_{} 

Summing these two quantities and adding the contribution of the diagonal, we get (focusing on just the numerators, since they all have the same denominator) 

_{} 

Hence the sum of the three contributions equals the steadystate value of P_{0} determined in the previous note. As an aside, note that the double integral of either one of the density distributions (for the region above or below the diagonal) evaluated over the entire space also equals the correct steadystate value of P_{0}, although apparently just by coincidence. This concordance applies not just to the integral over the entire age space of State 0, but even over each slice of constant t_{1} or t_{2}, a fact which is significant for evaluating the continuous form of the age distribution equations. 

Notice that the general case of equation (9) in the previous note can be expressed in the form 

_{} 

for any k greater than zero. (Boundary states are exceptions, because they lack one or the other of the aging transitions, but those will be treated appropriately when we sum these microstates to determine the probabilities of the macrostates.) To go over to the continuous case, we first define three density functions f, y_{1}, and y_{2} for microstates of incremental size by the following relations 

_{} 

Substituting these into the preceding equation (and noting that Dt = dt_{1} = dt_{2}) gives 

_{} 

Dividing through by dt_{2} and converting the summation to an integral, we get 

_{} 

By symmetry we also have the equation 

_{} 

Similarly, taking equations (8) from the previous note to the continuous limit gives 

_{} 

for t_{1} , t_{2} > 0. The restriction to positive arguments is necessary due to the fact that equation (4) doesn’t apply for the singular boundary case d_{0}(0,0). Note that the solution derived above satisfies these relations. The continuous limit of equations (9) of the previous note gives the governing equation for the age density function of State 0 

_{} 

and the preceding equations (6) give the boundary conditions for this equation in terms of the density functions of the singlefault states. By direct substitution of our previous solution for the age density of State 0 into this equation we can show that it satisfies the equation for every point other than the diagonal, where the function is discontinuous. The previous two equations, (4) and (5) relate the fullup and singlefault density functions, and together with the continuous gauge equation 

_{} 

they enable us to solve for the steadystate density functions for the three system states. Of course, the age densities are functions of time, so the overall probabilities for the three macrostates at any time t are 

_{} 

To show that the above equations reduce to the usual Markov model equations when the transition rate parameters l_{1} and l_{2} are constant, we can integrate equation (4) for t_{2} = 0 to infinity, giving the result 

_{} 

The first integral in this equation vanishes because 

_{} 

so we have the expected relation 

_{} 

which is the same as the first of the steadystate equations for the timeindependent model. Likewise integrating equation (5) leads to the second of the steadystate timeindependent equations. Of course, the gauge condition (8) reduces to P_{0} + P_{1} + P_{2} = 1, so we have recovered all three of the steadystate equations with constant transition rate parameters. 

It’s also interesting to see how our solution satisfies equation (4) directly, as opposed to the integral of equation (4). To show this, it’s convenient to define the constants 

_{} 

where 
_{} 

Then our solution for State 0 can be expressed as 

_{} 

and for States 1 and 2 we have 

_{} 

Now, by direct substitution and differentiation into (4) we find that the left hand side is 

_{} 

The right hand side of (4) is more difficult to evaluate, because the integration extends over all three of the regions of State 0. Thus the integral consists of the three terms 

_{} 

The third term on the right side might be surprising, since each point on the diagonal contributes zero probability. However, the terms of this equation are not probabilities, they are onedimensional densities, and the diagonal contributes a positive density at each point. Evaluating each of these three terms, we get 

_{} 

The first two terms of the first integral equal the left side of equation (4), so all the remaining terms must sum to zero. Hence we must have 

_{} 

The constants C_{1} and C_{2} each have a factor of l_{1}l_{2}/D, so we can factor that out of all the terms, and we get (after canceling terms) 

_{} 

This does indeed equal zero, confirming that the solution satisfies equation (4). We also find that equation (4) is satisfied even if we evaluate the entire right hand integral (from zero to infinity) using just the d_{0} function that strictly applies only for t_{1}<t_{2}. 
