POMDPPlanners.planners.vectorized_planners package

GPU-vectorized online POMDP planners.

This package hosts planners whose whole search is expressed as batched tensor operations over the flat-tensor belief tree in POMDPPlanners.core.tree.vectorized_belief_tree. Unlike the Monte Carlo Tree Search planners in POMDPPlanners.planners.mcts_planners, these run tens of thousands of parallel simulations per planning step with no Python per-simulation loop and no host/device synchronization.

Currently provided:

  • VOPPPlanner – the Vectorized Online POMDP Planner (VOPP / PORPP).

class POMDPPlanners.planners.vectorized_planners.VOPPEpisodeRunner(planner, model, *, num_belief_particles=1000, max_steps=50, world_transition=None, world_observation=None)[source]

Bases: object

Runs closed-loop POMDP episodes with a VOPPPlanner.

The runner owns the interaction loop but no planning policy of its own: the planner decides actions, the vectorized model supplies the dynamics and the belief filter, and an optional pair of world hooks overrides the ground-truth transition / observation when a real simulator is the world.

Parameters:
num_belief_particles

Size of the particle belief carried between steps.

max_steps

Maximum number of actions per episode.

Example

>>> import torch
>>> from POMDPPlanners.environments.light_dark_pomdp.continuous_light_dark_pomdp import (
...     ContinuousLightDarkPOMDP,
... )
>>> from POMDPPlanners.environments.light_dark_pomdp.continuous_light_dark_vectorized_model import (
...     ContinuousLightDarkVectorizedModel,
... )
>>> from POMDPPlanners.planners.vectorized_planners import (
...     VOPPEpisodeRunner,
...     VOPPPlanner,
... )
>>> _ = torch.manual_seed(0)
>>> env = ContinuousLightDarkPOMDP(discount_factor=0.95, is_obstacle_hit_terminal=False)
>>> model = ContinuousLightDarkVectorizedModel(env, device=torch.device("cpu"))
>>> planner = VOPPPlanner(
...     model, num_actions=model.num_actions, num_particles=128,
...     max_depth=6, num_planning_iterations=8,
... )
>>> runner = VOPPEpisodeRunner(planner, model, num_belief_particles=256, max_steps=20)
>>> initial = torch.tensor([[0.0, 5.0]])
>>> result = runner.run_episode(initial)
>>> result.num_steps >= 1
True
run_episode(initial_state, initial_particles=None)[source]

Run one closed-loop episode and return its recorded trajectory.

Parameters:

  • initial_state (Tensor) – [1, ds] (or [ds]) ground-truth start state.

  • initial_particles (Optional[Tensor]) – Optional [num_particles, ds] initial belief; defaults to the start state replicated across the particle set.

Return type:

VOPPEpisodeResult

Returns:

A VOPPEpisodeResult holding the states, beliefs, actions, rewards, planning times, and root visit counts of the episode.

Raises:

ValueError – If initial_state does not describe a single state.

class POMDPPlanners.planners.vectorized_planners.VOPPPlanner(model, num_actions, *, temperature=2.0, num_particles=1000, max_depth=10, discount_factor=0.95, num_planning_iterations=100, value_heuristic=None, belief_capacity=1024, action_capacity=1024, value_dtype=torch.float32)[source]

Bases: object

Fully vectorized online POMDP planner (VOPP / PORPP).

The planner owns a VectorizedBeliefTree and repeatedly expands and backs it up entirely with batched tensor operations. A single call to plan() samples particles from the supplied root belief, runs the configured number of forward-search / preference-backup iterations, and returns the greedy root action.

Parameters:
device

Device every tensor lives on (taken from the model).

num_actions

Size of the fixed representative action set.

Example

See the module-level docstring for a runnable example.

plan(root_particles)[source]

Plan from a particle-set root belief and return the greedy action.

The tree is cleared, then num_planning_iterations forward-search / preference-backup passes refine the root action preferences. The action with the highest root preference is returned.

Parameters:

root_particles (Tensor) – [num_root_particles, ds] states representing the current belief; iterations resample num_particles of them with replacement.

Return type:

int

Returns:

The index of the greedy root action in [0, num_actions).

Raises:

ValueError – If root_particles is not a 2-D tensor on the planner’s device.

property tree: VectorizedBeliefTree

The belief tree built by the most recent plan() call.

tree_metrics()[source]

Root-tree analysis metrics for the most recent plan() call.

Returns the same TreeMetrics set the MCTS planners report (root action visit min / max / entropy, number of root actions, root visit count, max depth, leaf flag), making VOPP directly comparable to them.

Return type:

List[PolicyInfoVariable]

Returns:

A list of PolicyInfoVariable describing the current tree.

Subpackages