REFINE-PLAN Tutorial

This demo shows the robot executing refined behaviours within the Pyrobosim simulation.

Starting the Docker Container for this Tutorial

Running the Docker Container

cd overview
docker compose run --rm -it base /bin/bash

The Problem

The robot must search for a loaf of bread as quickly as possible within the following house environment:

pyrobosim

The bread appears stochastically at:

  • The dining table with probability 0.5

  • The fridge with probability 0.1

  • The kitchen table with probability 0.1

  • The side table with probability 0.3

From this, it is clear that the bread is much more likely to appear in the dining room than the kitchen.

The Initial BT

We begin with an initial BT for this problem which does not consider uncertainty. In this instance, the BT does not know the bread’s location distribution and so searches for it by moving clockwise around the environment until it finds it. This is inefficient, as the robot searches the lower probability locations first. A portion of the initial BT for the fridge can be seen below. The sequence node is repeated for the kitchen table, side table, and dining table.

initial_bt

Running the Initial BT

After starting the Docker container, run:

cd /convince_ws/src/tutorials/refine_plan_demo/bringup
./run_demo.sh initial

From this, a tmux session will open with two panels: The first one is for the simulation, and the second one is for execution. The execution panel provides insights into the robot’s decision-making. Upon running the demo the robot will begin searching for the bread by moving clockwise through the house.

REFINE-PLAN

REFINE-PLAN is an automated tool for refining hand-designed BTs to improve robot performance under uncertainty. We give an overview of the REFINE-PLAN framework in the image below:

framework

REFINE-PLAN begins with an initial hand-designed BT and a simulator as input. From the initial BT we extract a state space for planning using the condition nodes and blackboard. Next, we learn two Bayesian networks per action node in simulation which capture the stochastic transition and cost dynamics of robot execution. We then construct a Markov decision process using the learned Bayesian networks and extracted state space, and solve it using standard techniques to synthesise a policy. This policy can then be converted back to a BT.

State Space Extraction

For this problem, we extract the following state factors:

  • The robot’s location in the house (hall, fridge, kitchen table, side table, dining table)

  • A state factor for each location declaring whether the bread is present, or whether this information is unknown.

Data Collection for Model Learning

To learn the stochastic transition and cost dynamics of the environment we require a data collection phase in simulation. Here the robot executes actions randomly in the simulator. The recorded transitions are then used to learn the Bayesian networks.

To run data collection, you can run the following after starting the Docker container, where <MONGO_CONNECTION_STRING> is a connection string to a Mongo database, e.g. localhost:27017:

cd src/refine_plan_demo/bringup
./run_data_collection.sh <MONGO_CONNECTION_STRING>

Synthesising the Refined Behaviour

Given the recorded transitions from the data collection phase, we can then:

  • Learn the Bayesian networks for each BT action node

  • Construct the MDP

  • Solve it using standard techniques

These three steps can be executed for this problem by running the following script. After running REFINE-PLAN, the robot learns the location distribution for the bread and prioritises searching higher probability locations first. Visually, the robot now moves anticlockwise through the house, finding the bread quicker on average.

Running the Refined Behaviour

After starting the Docker container, run:

cd src/refine_plan_demo/bringup
./run_demo.sh refined

From this, a tmux session will begin with two windows. The first window is for the simulation, and the second is for execution. The execution window provides insights into the robot’s decision-making. Upon running the demo the robot will begin searching for the bread by moving anticlockwise through the house.

Further Information

For more information on REFINE-PLAN, you can: