Talk:Markov chain

What is a Higher Order Markov Chain ?

External link relating to google is busted ... no longer works (if ever?).

Looks like they moved the page... I located it at Mathworks and pasted the new link into the "External links" section. Thanks for noticing, who knows how long it was broken. Happy editing, Wile E. Heresiarch 04:39, 3 Jun 2004 (UTC)

Might be a stupid question...but how do you calculate the "power" of the matrix in order to calculate the n-step transition matrix?

See matrix multiplication for the answer to the question above. (That article could use some polishing, BTW, but the answer is there.) Michael Hardy 01:49, 30 Sep 2004 (UTC)

What is meant by a _sequence_ of random variables? A r.v. is a fn. from the probability space to some measurable space, so we can think of assigning a 'number' to each elementary event in the probability space - is the sequence here a series of values from a single r.v. generated by a series of elementary events occurring, or is it actually a sequence of different functions (which would be the literal interpretation of "a sequence of random variables").--SgtThroat 10:37, 10 Dec 2004 (UTC)

Having read around I'm pretty sure that the sequence consists of a sequence of functions. Also these need to have the same domain and codomain (?). Then each point in the sample space would correspond to a sequence x_1,x_2...x_N of values of these functions. The set of all points then corresponds to the set of all sequences. Lets assume for simplicity that each point in the underlying probability space has equal probability - then the transition probability P(x_n|x_n-1) would be obtained by taking all the sequences in which X_n-1=x_n-1 and calculating the fraction of them that have X_n=x_n. Similarly P(x_n|x_n-1,x_n-2) would be obtained by calculating the same fraction among all those that have X_n-1=x_n-1 and X_n-2=x_n-2. The Markov property is the statement that these are equal for all x_n (i.e the conditional distributions are the same) and all n. Can anybody confirm/correct this understanding? --SgtThroat 12:04, 10 Dec 2004 (UTC)

In mathematics, a sequence has a first element, a second one, a third one, and so on, so a sequence of random variables has a first random variable, a second random variable, etc. And since a r.v. is a function whose domain is a probability space, a sequence of random variables is a sequence of such functions. Your remarks about "elementary" events assumes a discrete probability space, and that assumption is not warranted. Definitely all of the r.v.'s in this sequence have the same domain; that is true of any stochastic process.

each point in the sample space would correspond to a sequence x_1,x_2...x_N of values of these functions

Correct.

Lets assume for simplicity that each point in the underlying probability space has equal probability

Again, your assuming discrete without warrant. More later .... Michael Hardy 23:02, 10 Dec 2004 (UTC)

To continue:

It seems your difficulty stems mainly from the unwarranted assumption of discreteness. Consider the "uniform" probability distribution on the interval [0, 1]. The probability that a random variable with this distribution falls in, for example, the interval [0.15, 0.23] is just the length of that interval: 0.23 − 0.15 = 0.08, and similarly for any other subinterval of [0, 1]. This is a continuous, rather than discrete, probability distribution. Nota bene: the probability assigned to each one-point set is zero! Thus, knowing the probability assigned to each one-point set does not tell you what the probability distribution is. Similarly, the familiar bell-shaped curve introduced in even the most elementary statistics courses for non-mathematically inclined people is a continuous distribution, so we're not talking about anything the least bit esoteric here.

Now consider an infinite sequence of independent fair-coin tosses. What's the probability of getting one particular infinite sequence of heads and tails? It's zero, no matter which sequence you pick. So this is also not a discrete distribution. Michael Hardy 01:01, 11 Dec 2004 (UTC)

Hi Michael. Yes, I agree that the assumption of a discrete probability space is rather limited. I was trying to get the discrete case clear before moving on to trying to understand the continuous one, although I neglected to say so. I think I now have a much better understanding of some of this material thanks to your comments and to some additional reading (in particular the entry on random vector which I hadnàt read when I wrote the above. I think we ought to clarify that the sequence of random vbles is a sequence of functions from some probability space onto some domain, both of which are the same for all elements of the sequence. It could use a link to the entry on random vector - although a sequence is not quite the same as a vector, the components of a random vector form a sequence with the required properties and there is much useful content on that page to clarify. SgtThroat 12:21, 12 Dec 2004 (UTC)

Aperiodic

Did anyone notice that the definiton of aperiodic in the text is plain wrong, the condition mentioned implies that the chain is aperiodic but it's not neccesary.