Publication Details
Fuel in Markov Decision Processes (FiMDP): A Practical Approach to Consumption
Blahoudek František, RNDr., Ph.D.
CUBUKTEPE, M.
ORNIK, M.
THANGEDA, P.
TOPCU, U.
Behavioral research, Decision making, Model checking, Multi agent systems,
Polynomial approximation
Consumption Markov Decision Processes (CMDPs) are probabilistic decision-making
models of resource-constrained systems. We introduce FiMDP, a tool for controller
synthesis in CMDPs with LTL objectives expressible by deterministic Büchi
automata. The tool implements the recent algorithm for polynomial-time controller
synthesis in CMDPs, but extends it with many additional features. On the
conceptual level, the tool implements heuristics for improving the expected
reachability times of accepting states, and a support for multi-agent task
allocation. On the practical level, the tool offers (among other features) a new
strategy simulation framework, integration with the Storm model checker, and
FiMDPEnv - a new set of CMDPs that model real-world resource-constrained systems.
We also present an evaluation of FiMDP on these real-world scenarios.
@inproceedings{BUT196648,
author="NOVOTNÝ, P. and BLAHOUDEK, F. and CUBUKTEPE, M. and ORNIK, M. and THANGEDA, P. and TOPCU, U.",
title="Fuel in Markov Decision Processes (FiMDP): A Practical Approach to Consumption",
booktitle="Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
year="2021",
pages="640--656",
address="Pittsburgh",
doi="10.1007/978-3-030-90870-6\{_}34",
isbn="978-3-030-90869-0",
url="https://link.springer.com/chapter/10.1007/978-3-030-90870-6_34?getft_integrator=scopus"
}