Table of Contents

Preface 1. Probability Spaces and Random Variables 2. Expectations and Independence 3. Bernoulli Processes and Sums of Independent Random Variables 4. Poisson Processes 5. Markov Chains 6. Limiting Behavior and Applications of Markov Chains 7. Potentials, Excessive Functions, and Optimal Stopping of Markov Chains 8. Markov Processes 9. Renewal Theory 10. Markov Renewal Theory Afterword Appendix. Non-Negative Matrices References Answers to Selected Exercises Index of Notations Subject Index