Probabilistic Hierarchical Planning over MDPs

Yuqing Tang; Felipe Meneguzzi; Katia Sycara; Simon Parsons

Probabilistic Hierarchical Planning over MDPs

Yuqing Tang, Felipe Meneguzzi, Katia Sycara, Simon Parsons

Research output: Chapter in Book/Report/Conference proceeding › Published conference contribution

Abstract

In this paper, we propose a new approach to using probabilistic hierarchical task networks (HTNs) as an effective method for agents to plan in conditions in which their problem-solving knowledge is uncertain, and the environment is non-deterministic. In such situations it is natural to model the environment as a Markov decision process (MDP). We show that using Earley graphs, it is possible to
bridge the gap between HTNs and MDPs. We prove that the size of the Earley graph created for given HTNs is bounded by the total number of tasks in the HTNs and show that from the Earley graph we can then construct a plan for a given task that has the maximum expected value when it is executed in an MDP environment.

Original language	English
Title of host publication	Proceedings of the Tenth International Conference on Autonomous Agents and Multiagent Systems
Pages	1143-1144
Number of pages	2
Publication status	Published - May 2011
Externally published	Yes

Bibliographical note

Acknowledgement: This research was sponsored by the U.S. Army Research Laboratory and the U.K. Ministry of Defence and was accomplished under Agreement Number W911NF-09-2-0053. The views and conclusions contained in this document are those of the author(s) and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Army Research Laboratory, the U.S. Government, the U.K. Ministry of Defence or the U.K. Government. The U.S. and U.K. Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.

Access to Document

https://dl.acm.org/doi/pdf/10.5555/2034396.2034458Licence: Unspecified

Cite this

@inproceedings{a06d558263544b1bb72ce5ec5f7bad8a,

title = "Probabilistic Hierarchical Planning over MDPs",

abstract = "In this paper, we propose a new approach to using probabilistic hierarchical task networks (HTNs) as an effective method for agents to plan in conditions in which their problem-solving knowledge is uncertain, and the environment is non-deterministic. In such situations it is natural to model the environment as a Markov decision process (MDP). We show that using Earley graphs, it is possible tobridge the gap between HTNs and MDPs. We prove that the size of the Earley graph created for given HTNs is bounded by the total number of tasks in the HTNs and show that from the Earley graph we can then construct a plan for a given task that has the maximum expected value when it is executed in an MDP environment.",

author = "Yuqing Tang and Felipe Meneguzzi and Katia Sycara and Simon Parsons",

note = "Acknowledgement: This research was sponsored by the U.S. Army Research Laboratory and the U.K. Ministry of Defence and was accomplished under Agreement Number W911NF-09-2-0053. The views and conclusions contained in this document are those of the author(s) and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Army Research Laboratory, the U.S. Government, the U.K. Ministry of Defence or the U.K. Government. The U.S. and U.K. Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.",

year = "2011",

month = may,

language = "English",

pages = "1143--1144",

booktitle = "Proceedings of the Tenth International Conference on Autonomous Agents and Multiagent Systems",

}

TY - GEN

T1 - Probabilistic Hierarchical Planning over MDPs

AU - Tang, Yuqing

AU - Meneguzzi, Felipe

AU - Sycara, Katia

AU - Parsons, Simon

N1 - Acknowledgement: This research was sponsored by the U.S. Army Research Laboratory and the U.K. Ministry of Defence and was accomplished under Agreement Number W911NF-09-2-0053. The views and conclusions contained in this document are those of the author(s) and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Army Research Laboratory, the U.S. Government, the U.K. Ministry of Defence or the U.K. Government. The U.S. and U.K. Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.

PY - 2011/5

Y1 - 2011/5

N2 - In this paper, we propose a new approach to using probabilistic hierarchical task networks (HTNs) as an effective method for agents to plan in conditions in which their problem-solving knowledge is uncertain, and the environment is non-deterministic. In such situations it is natural to model the environment as a Markov decision process (MDP). We show that using Earley graphs, it is possible tobridge the gap between HTNs and MDPs. We prove that the size of the Earley graph created for given HTNs is bounded by the total number of tasks in the HTNs and show that from the Earley graph we can then construct a plan for a given task that has the maximum expected value when it is executed in an MDP environment.

AB - In this paper, we propose a new approach to using probabilistic hierarchical task networks (HTNs) as an effective method for agents to plan in conditions in which their problem-solving knowledge is uncertain, and the environment is non-deterministic. In such situations it is natural to model the environment as a Markov decision process (MDP). We show that using Earley graphs, it is possible tobridge the gap between HTNs and MDPs. We prove that the size of the Earley graph created for given HTNs is bounded by the total number of tasks in the HTNs and show that from the Earley graph we can then construct a plan for a given task that has the maximum expected value when it is executed in an MDP environment.

UR - http://www.meneguzzi.eu/felipe/pubs/aamas-earley-poster-2011.pdf

M3 - Published conference contribution

SP - 1143

EP - 1144

BT - Proceedings of the Tenth International Conference on Autonomous Agents and Multiagent Systems

ER -

Probabilistic Hierarchical Planning over MDPs

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this