TWiki> AILab Web>Research>Courses (revision 3)Edit | Attach | 
==Class Links==
<big>'''Spring 2007'''</big>
Check out the relevant papers here: MDPClassPapers

I am going to attempt to keep this page upto date... please make changes as you see fit:

Week 1

Overview of MDPs

*Definitions (state space, Markov property, closed sets, MDP's)

*Agent Action Based Observations

*Value Function for MDPs

*Observable, partially Observable, and Non-Observable

*Histories, Horizons, Rewards and Plan's

Week 2

*Bellman's Principal of Optimality

*Value Iteration

**Implementation of Value Iteration

*Howard's Optimal Stationary Policy

*Successive Approximation

*Policy Iteration

Week 3

*Heuristic Searches

**See Papers for papers regarding these topics..

*Goal Based MDPs



Week 4



*Labeled RTDP


Week 5

*Factored MDP's

*Definitions and Comparisons to Flat MDPs

** Robot Example with different arc's etc.


**Definition and Construction

Week 6


**Addition, Subtraction, Multiplication, MAX, MIN


**Using to model Flat MDPs


**Small Example and PsuedoCode

*Planit Suite Intro (Big Picture)

**DIET, HELL, LIMBO, BNB, PET, PlanIt and interactions

Week 7

*More PlanIt

**Using the preference elicitor, create a plan. Demo of coffee/robot domain.

**DIET and PlanIt code analysis. What is significant in each as far as programming.

**[ Using Eclipse and linking with SVN]

*More PlanIt

**Looking at the algorithms

**Discussion and dissection of the code within the planner

Week 8


Week 9

*Linear Programming

**MAX and MIN functions


**Conical Forms

*More Linear Programming

**Use for Flat MDPs


**Use with DBNs

Week 10


**Presentation by Renee

**Pseudo-Code, Examples and Discussion


**Presentation by Nick

**Pseudo-Code, Examples and Discussion

Week 11

*Crash on Paper

*More Linear Programming

**Factored MDP uses

**MIN and MAX etc.

Week 12

*Reinforcement Learning

**Supervised / Unsupervised Learning

**Model Free Learning


*More Reinforcement Learning

**Model-Based Learning

**Differences with Q-Learning

Week 13

*More Reinforcement Learning

**More Model-Free refinements and algorithms


**Temporal Abstraction


**Semi-MDPs and Options

Week 14

*Discussion of End of Semester Report

Edit | Attach | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r3 - 2010-09-24 - 16:17:17 - NickM
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback