• Good understanding of basic probability (e.g. ECE302)
  • Some familiarity with Markov chains (e.g. EECS454/EECS422/EECS495 Randomized Algorithms/IEMS460)


  • S. P. Meyn and R. L. Tweedie, Markov chains and stochastic stability, Second edition, Cambridge University Press, Cambridge, 2009.
  • J. R. Norris, Markov chains, Cambridge Series in Statistical and Probabilistic Mathematics, 2, Cambridge University Press, Cambridge, 1998.
  • N. Cesa-Bianchi and G. Lugosi, Prediction, learning, and games, Cambridge University Press, Cambridge, 2006.
  • P. R. Kumar and P. Varaiya, Stochastic systems: Estimation, identification and adaptive control, Prentice Hall, Englewood Cliffs, N. J., 1986.
  • M. Mitzenmacher and E. Upfal, Probability and computing: Randomized algorithms and probabilistic analysis, Cambridge University Press, Cambridge, 2005.
  • Relevant papers.

EXAMS: There will be no exam for this course. However, there will be a final project.

COURSE GRADE: Your final grade will be determined by a mixture of class participation, problems sets and final project. The weightings used will be the following.

  • Problem sets 20%
  • Class participation 20%
  • Final project 60%

more news