POMDPPlanners.environments.push_pomdp.push_pomdp_utils package

Submodules

POMDPPlanners.environments.push_pomdp.push_pomdp_utils.push_reward_models module

Reward models for the Push POMDP family.

Mirrors the abstract-base / concrete-subclass layout used by POMDPPlanners.environments.light_dark_pomdp.light_dark_pomdp_utils.light_dark_reward_models and POMDPPlanners.environments.laser_tag_pomdp.laser_tag_pomdp_utils.laser_tag_reward_models, so further Push reward variants can be added without growing the env class.

The reward model owns the parameters and pre-built buffers the reward computation needs (obstacle geometry, dangerous areas, penalty probabilities). The environment retains its own copies for transition / observation paths and delegates reward() / reward_batch() to the model.

Reward-model variants (selected via RewardModelType):

  • CONSTANT_HAZARD_PENALTY — dangerous-area penalty fires deterministically (or with dangerous_area_hit_probability Bernoulli) whenever the realised post-action robot position lies inside any zone.

  • ZERO_MEAN_HAZARD_SHOCK — dangerous-area penalty becomes ±dangerous_area_penalty with 50/50 split when in zone; obstacle penalty is unchanged. Expected dangerous-area contribution is zero, variance is dangerous_area_penalty**2. Useful for benchmarking risk-sensitive planners against expected-value MCTS on the same mean.

  • DISTANCE_DECAYED_HAZARD_PENALTY — dangerous-area penalty fires with probability exp(-min_dist / penalty_decay) based on the Euclidean distance to the nearest dangerous-area centre, with no radius cutoff. Obstacle penalty is unchanged.

class POMDPPlanners.environments.push_pomdp.push_pomdp_utils.push_reward_models.BasePushRewardModel[source]

Bases: ABC

Abstract reward model for Push POMDP variants.

abstractmethod compute_reward(state, action, next_state)[source]

Return the scalar reward for (state, action, next_state).

Return type:

float

Parameters:
abstractmethod compute_reward_batch(states, action, next_states)[source]

Return the per-row reward for a batch of states under a single action.

Return type:

ndarray

Parameters:
class POMDPPlanners.environments.push_pomdp.push_pomdp_utils.push_reward_models.ContinuousPushDistanceDecayedHazardPenaltyRewardModel(obstacles, robot_radius, obstacle_penalty, obstacle_hit_probability, dangerous_areas_arr, dangerous_area_radius, dangerous_area_penalty, dangerous_area_hit_probability, penalty_decay)[source]

Bases: ContinuousPushRewardModel

Distance-decaying dangerous-area contribution for ContinuousPushPOMDP.

Penalty fires with probability exp(-min_dist / penalty_decay) based on the Euclidean distance from the realised robot position to the nearest dangerous-area centre. No radius cutoff — every position feels a (vanishingly small at large distance) penalty risk. Obstacle penalty is unchanged.

Parameters:

  • obstacles (ndarray)

  • robot_radius (float)

  • obstacle_penalty (float)

  • obstacle_hit_probability (float)

  • dangerous_areas_arr (ndarray)

  • dangerous_area_radius (float)

  • dangerous_area_penalty (float)

  • dangerous_area_hit_probability (float)

  • penalty_decay (float)

class POMDPPlanners.environments.push_pomdp.push_pomdp_utils.push_reward_models.ContinuousPushRewardModel(obstacles, robot_radius, obstacle_penalty, obstacle_hit_probability, dangerous_areas_arr, dangerous_area_radius, dangerous_area_penalty, dangerous_area_hit_probability)[source]

Bases: BasePushRewardModel

Standard reward model for ContinuousPushPOMDP.

Reward structure mirrors DiscretePushRewardModel (distance penalty, goal bonus, obstacle penalty, dangerous-area penalty) but obstacles are axis-aligned bounding boxes ((cx, cy, half_x, half_y) rows) tested against a circular robot footprint of radius robot_radius. Dangerous areas remain circular point-vs-circle checks. Subclasses override _dangerous_area_penalty_batch() to substitute high-variance or distance-decaying contracts without touching the goal / obstacle terms.

Parameters:

  • obstacles (ndarray)

  • robot_radius (float)

  • obstacle_penalty (float)

  • obstacle_hit_probability (float)

  • dangerous_areas_arr (ndarray)

  • dangerous_area_radius (float)

  • dangerous_area_penalty (float)

  • dangerous_area_hit_probability (float)

compute_reward(state, action, next_state)[source]

Return the scalar reward for (state, action, next_state).

Return type:

float

Parameters:
compute_reward_batch(states, action, next_states)[source]

Return the per-row reward for a batch of states under a single action.

Return type:

ndarray

Parameters:
class POMDPPlanners.environments.push_pomdp.push_pomdp_utils.push_reward_models.ContinuousPushZeroMeanHazardShockRewardModel(obstacles, robot_radius, obstacle_penalty, obstacle_hit_probability, dangerous_areas_arr, dangerous_area_radius, dangerous_area_penalty, dangerous_area_hit_probability)[source]

Bases: ContinuousPushRewardModel

High-variance dangerous-area contribution for ContinuousPushPOMDP.

Parameters:

  • obstacles (ndarray)

  • robot_radius (float)

  • obstacle_penalty (float)

  • obstacle_hit_probability (float)

  • dangerous_areas_arr (ndarray)

  • dangerous_area_radius (float)

  • dangerous_area_penalty (float)

  • dangerous_area_hit_probability (float)

class POMDPPlanners.environments.push_pomdp.push_pomdp_utils.push_reward_models.DiscretePushDistanceDecayedHazardPenaltyRewardModel(obstacles, obstacle_radius, obstacle_penalty, obstacle_hit_probability, dangerous_areas, dangerous_areas_arr, dangerous_area_radius, dangerous_area_penalty, dangerous_area_hit_probability, penalty_decay)[source]

Bases: DiscretePushRewardModel

Distance-decaying dangerous-area contribution for PushPOMDP.

Penalty fires with probability exp(-min_dist / penalty_decay) based on the Euclidean distance from the realised robot position to the nearest dangerous-area centre. No radius cutoff — every position feels a (vanishingly small at large distance) penalty risk. Obstacle penalty is unchanged. Mirrors ContinuousLightDarkDistanceDecayedHazardPenaltyRewardModel and LaserTagDistanceDecayedHazardPenaltyRewardModel.

Parameters:
class POMDPPlanners.environments.push_pomdp.push_pomdp_utils.push_reward_models.DiscretePushRewardModel(obstacles, obstacle_radius, obstacle_penalty, obstacle_hit_probability, dangerous_areas, dangerous_areas_arr, dangerous_area_radius, dangerous_area_penalty, dangerous_area_hit_probability)[source]

Bases: BasePushRewardModel

Standard reward model for PushPOMDP (discrete actions).

Reward structure:

  • Base term: -distance(object, target) where positions are read from the realised next_state.

  • Goal bonus: +100.0 when the object lies within 0.5 of the target.

  • Obstacle penalty: obstacle_penalty is added when the realised post-action robot position (next_state[:2]) lies within obstacle_radius of any circular obstacle. When obstacle_hit_probability < 1.0 the penalty fires with that probability per call (one Bernoulli draw per state).

  • Dangerous-area penalty: see _dangerous_area_contribution_scalar() / _dangerous_area_penalty_batch(). The standard contract is dangerous_area_penalty with optional Bernoulli dangerous_area_hit_probability. Subclasses override the two helpers to substitute high-variance or distance-decaying contracts without touching the goal / obstacle terms.

Parameters:
compute_reward(state, action, next_state)[source]

Return the scalar reward for (state, action, next_state).

Return type:

float

Parameters:
compute_reward_batch(states, action, next_states)[source]

Return the per-row reward for a batch of states under a single action.

Return type:

ndarray

Parameters:
class POMDPPlanners.environments.push_pomdp.push_pomdp_utils.push_reward_models.DiscretePushZeroMeanHazardShockRewardModel(obstacles, obstacle_radius, obstacle_penalty, obstacle_hit_probability, dangerous_areas, dangerous_areas_arr, dangerous_area_radius, dangerous_area_penalty, dangerous_area_hit_probability)[source]

Bases: DiscretePushRewardModel

High-variance dangerous-area contribution for PushPOMDP.

Dangerous-area hits emit +dangerous_area_penalty or -dangerous_area_penalty with equal probability — expected contribution is zero, variance is dangerous_area_penalty**2. Obstacle penalty is unchanged. Mirrors ContinuousLDZeroMeanHazardShockRewardModel and LaserTagZeroMeanHazardShockRewardModel.

Parameters:
class POMDPPlanners.environments.push_pomdp.push_pomdp_utils.push_reward_models.RewardModelType(*values)[source]

Bases: Enum

Reward model variants supported by the Push POMDP family.

CONSTANT_HAZARD_PENALTY = 'constant_hazard_penalty'
DISTANCE_DECAYED_HAZARD_PENALTY = 'distance_decayed_hazard_penalty'
ZERO_MEAN_HAZARD_SHOCK = 'zero_mean_hazard_shock'