Difference: Courses (1 vs. 2)

Revision 22010-09-24 - NickM

Line: 1 to 1
META TOPICPARENT name="Research"
info about the old MDP course.
 \ No newline at end of file
==Class Links==
<big>'''Spring 2007'''</big>
Check out the relevant papers here: MDPClassPapers

I am going to attempt to keep this page upto date... please make changes as you see fit:

Week 1

Overview of MDPs

*Definitions (state space, Markov property, closed sets, MDP's)

*Agent Action Based Observations

*Value Function for MDPs

*Observable, partially Observable, and Non-Observable

*Histories, Horizons, Rewards and Plan's

Week 2

*Bellman's Principal of Optimality

*Value Iteration

**Implementation of Value Iteration

*Howard's Optimal Stationary Policy

*Successive Approximation

*Policy Iteration

Week 3

*Heuristic Searches

**See Papers for papers regarding these topics..

*Goal Based MDPs



Week 4



*Labeled RTDP


Week 5

*Factored MDP's

*Definitions and Comparisons to Flat MDPs

** Robot Example with different arc's etc.


**Definition and Construction

Week 6


**Addition, Subtraction, Multiplication, MAX, MIN


**Using to model Flat MDPs


**Small Example and PsuedoCode

*Planit Suite Intro (Big Picture)

**DIET, HELL, LIMBO, BNB, PET, PlanIt and interactions

Week 7

*More PlanIt

**Using the preference elicitor, create a plan. Demo of coffee/robot domain.

**DIET and PlanIt code analysis. What is significant in each as far as programming.

**[https://soapbox.iraproject.org/Blog/tabid/71/EntryID/11/Default.aspx Using Eclipse and linking with SVN]

*More PlanIt

**Looking at the algorithms

**Discussion and dissection of the code within the planner

Week 8


Week 9

*Linear Programming

**MAX and MIN functions


**Conical Forms

*More Linear Programming

**Use for Flat MDPs


**Use with DBNs

Week 10


**Presentation by Renee

**Pseudo-Code, Examples and Discussion


**Presentation by Nick

**Pseudo-Code, Examples and Discussion

Week 11

*Crash on Paper

*More Linear Programming

**Factored MDP uses

**MIN and MAX etc.

Week 12

*Reinforcement Learning

**Supervised / Unsupervised Learning

**Model Free Learning


*More Reinforcement Learning

**Model-Based Learning

**Differences with Q-Learning

Week 13

*More Reinforcement Learning

**More Model-Free refinements and algorithms


**Temporal Abstraction


**Semi-MDPs and Options

Week 14

*Discussion of End of Semester Report

 \ No newline at end of file

Revision 12010-09-23 - NickM

Line: 1 to 1
META TOPICPARENT name="Research"
info about the old MDP course.
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback