SYS 605 - Stochastic Systems
 
Prof. Randy Cogill
University of Virginia
Fall Semester 2007

Teaching Assistant: Himanshu Gupta (himanshu@virginia.edu)

Tuesdays and Thursdays, 2:00 PM - 3:15 PM
Mechanical Engineering Building, room 216

Instructor Office Hours: Tuesdays, 10AM-12PM in Olsson 102G
TA Office Hours: Mondays and Wednesdays, 4PM-6PM in Olsson 105B

Announcements • Lecture Notes • Homework

Announcements:

  • The final exam is HERE. The solutions are HERE.

  • The take-home final exam is scheduled for 12/8-12/12. The exam will be emailed to the class and posted on the class website at 12 noon on 12/8. It will be due by 12 noon on 12/12.
  • The midterm exam is HERE. The solutions are HERE.
  • There will not be any office hours on Monday 10/15 and Tuesday 10/16 during the midterm exam period.
  • The notes for the pre-midterm review on Thursday 10/11 are here
  • The take-home midterm exam is scheduled for 10/15-10/16. The exam will be emailed to the class and posted on the class website at 9AM on 10/15. It will be due by 3:15 PM on 10/16.
  • Tuesday 10/9 is a university-wide reading day. There will be no class on 10/9.
  • Details on course materials, grading, homework, and exams are covered in the "Course Overview" notes.
    • Lecture Notes:

    Lecture notes will be posted as each topic is covered in class.

    1. Course Overview
    2. Probability review
    3. Introduction to Markov chains
    4. Linear algebra review
    5. Finite-state Markov chains: steady-state behavior
    6. Finite-state Markov chains: transient behavior
    7. Countable-state Markov chains
    8. Applications of Markov chains
    9. Poisson processes
    10. Continuous-time Markov chains
    11. Martingales

    Some additional handwritten notes:

  • Wald's equation via the Martingale Stopping Theorem
  • Waiting for a pattern in a sequence of coin flips
  • Solving Poisson's equation with Matlab's symbolic math toolbox
    • Homework:

    1. Homework #1, due 9/11/2007 (Solutions)
    2. Homework #2, due 9/20/2007 (Solutions)
    3. Homework #3, due 10/2/2007 (Solutions)
    4. Homework #4, due 10/11/2007 (Solutions)
    5. Homework #5, due 10/25/2007 (Solutions)
    6. Homework #6, due 11/6/2007. Matlab file sequences.m (Solutions)
    7. Homework #7, due 11/13/2007. (Solutions)
    8. Homework #8, due 11/27/2007. (Solutions)
    9. Homework #9, due 12/4/2007. (Solutions)