We consider a discrete time finite-state Markov chain 
with stationary transition probabilities 
. Let 
denote the
matrix of transition probabilities. The transition probabilities
between 
and 
are noted 
and the transition
matrix 
.
We need to introduce a few properties of the states of a Markov chain.
A state i is said to be aperiodic if
. A recurrent state is a state to which the
chain returns with probability 1. A state which is not recurrent is
called a transient state. In fact, state i
is recurrent if 
, and transient if
. Recurrent, aperiodic states are
called ergodic (An extra condition for ergodicity is that the
expected recurrence time be finite. This is always satisfied for
recurrent states in a finite state chain).
A state j can be reached from a state i if and only if there
exists 
. If state j can be reached from state i and state i
can be reached from state j, then states i and j communicate. A Markov
chain is said to be irreducible if all its states communicate,
i.e. 
. An irreducible Markov
chain with ergodic states is called ergodic. We are now ready to state
the following theorem:
A special case of ergodic chain is a chain satisfying the
reversibility condition called detailed balance: There exists
such that for all i and 
, we have
To see that 
in this case, we just sum over
j:
since 
.
The validity of the MCMC estimates rests on another important result: the ergodic theorem.
