POMDPPlanners.environments.rock_sample_pomdp.rock_sample_pomdp_utils package

Submodules

POMDPPlanners.environments.rock_sample_pomdp.rock_sample_pomdp_utils.rock_sample_reward_models module

Reward models for the RockSample POMDP.

Mirrors the abstract-base / concrete-subclass layout used by POMDPPlanners.environments.light_dark_pomdp.light_dark_pomdp_utils.light_dark_reward_models and POMDPPlanners.environments.laser_tag_pomdp.laser_tag_pomdp_utils.laser_tag_reward_models, so that further RockSample reward variants can be added without growing the env class.

Three concrete variants are provided. They share all of the non-dangerous-area scoring (exit, sample, sense, step-penalty) via the RockSampleRewardModel base; each variant only customises the dangerous-area contribution:

• RockSampleRewardModel (CONSTANT_HAZARD_PENALTY): constant-probability penalty when the realised next position is in any dangerous area.

• RockSampleZeroMeanHazardShockRewardModel (ZERO_MEAN_HAZARD_SHOCK): ±dangerous_area_penalty 50/50 in-zone — zero expected contribution, high variance.

• RockSampleDistanceDecayedHazardPenaltyRewardModel (DISTANCE_DECAYED_HAZARD_PENALTY): penalty applied with probability exp(-min_dist / penalty_decay) based on the closest zone centre — no radius cutoff.

class POMDPPlanners.environments.rock_sample_pomdp.rock_sample_pomdp_utils.rock_sample_reward_models.BaseRockSampleRewardModel

Bases: ABC

Abstract reward model for RockSample POMDP variants.

abstractmethod compute_reward(state, action, next_state)

Return the scalar reward for (state, action, next_state).

Return type:

float

Parameters:

• state (ndarray)

• action (int)

• next_state (ndarray)

abstractmethod compute_reward_batch(states, action, next_states=None)

Return the per-row reward for a batch of states under a single action.

Return type:

ndarray

Parameters:

• states (ndarray)

• action (int)

• next_states (ndarray | None)

class POMDPPlanners.environments.rock_sample_pomdp.rock_sample_pomdp_utils.rock_sample_reward_models.RockSampleDistanceDecayedHazardPenaltyRewardModel(map_size, rock_positions, step_penalty, bad_rock_penalty, good_rock_reward, sensor_use_penalty, exit_reward, dangerous_areas, dangerous_area_radius, dangerous_area_penalty, dangerous_area_hit_probability, penalty_decay)

Bases: RockSampleRewardModel

DISTANCE_DECAYED_HAZARD_PENALTY variant.

Penalty is applied with probability exp(-min_dist / penalty_decay) where min_dist is the Euclidean distance from the realised next position to the closest dangerous-area centre. No radius cutoff — every step risks some (vanishingly small at large distance) penalty. Each call draws one uniform regardless of distance, matching the existing decaying-prob kernel and light-dark’s analogous reward model.

dangerous_area_radius and dangerous_area_hit_probability are ignored in this model.

Parameters:

• map_size (Tuple[int, int])

• rock_positions (List[Tuple[int, int]])

• step_penalty (float)

• bad_rock_penalty (float)

• good_rock_reward (float)

• sensor_use_penalty (float)

• exit_reward (float)

• dangerous_areas (List[Tuple[int, int]])

• dangerous_area_radius (float)

• dangerous_area_penalty (float)

• dangerous_area_hit_probability (float)

• penalty_decay (float)

class POMDPPlanners.environments.rock_sample_pomdp.rock_sample_pomdp_utils.rock_sample_reward_models.RockSampleRewardModel(map_size, rock_positions, step_penalty, bad_rock_penalty, good_rock_reward, sensor_use_penalty, exit_reward, dangerous_areas, dangerous_area_radius, dangerous_area_penalty, dangerous_area_hit_probability)

Bases: BaseRockSampleRewardModel

Standard RockSample reward model.

Reward structure:

• Exit (action 2 East from the rightmost column): +exit_reward.

• Sample (action 0) at a rock cell: +good_rock_reward if the rock is good, +bad_rock_penalty if it is bad.

• Check actions (>= 5): +sensor_use_penalty.

• Per-step: +step_penalty baseline applied to every action.

• Dangerous area: +dangerous_area_penalty is added whenever the realised next robot position lies inside a dangerous area, gated by a per-call Bernoulli with probability dangerous_area_hit_probability (deterministic when == 1.0).

Note that this model uses the additive convention: pass a negative dangerous_area_penalty to penalise danger entry.

Subclasses override _dangerous_area_contribution_scalar() and _apply_dangerous_area_contribution_batch() to express different stochastic penalty models; the rest of the scoring is identical.

Parameters:

• map_size (Tuple[int, int])

• rock_positions (List[Tuple[int, int]])

• step_penalty (float)

• bad_rock_penalty (float)

• good_rock_reward (float)

• sensor_use_penalty (float)

• exit_reward (float)

• dangerous_areas (List[Tuple[int, int]])

• dangerous_area_radius (float)

• dangerous_area_penalty (float)

• dangerous_area_hit_probability (float)

compute_reward(state, action, next_state)

Return the scalar reward for (state, action, next_state).

Return type:

float

Parameters:

• state (ndarray)

• action (int)

• next_state (ndarray)

compute_reward_batch(states, action, next_states=None)

Return the per-row reward for a batch of states under a single action.

Return type:

ndarray

Parameters:

• states (ndarray)

• action (int)

• next_states (ndarray | None)

class POMDPPlanners.environments.rock_sample_pomdp.rock_sample_pomdp_utils.rock_sample_reward_models.RockSampleZeroMeanHazardShockRewardModel(map_size, rock_positions, step_penalty, bad_rock_penalty, good_rock_reward, sensor_use_penalty, exit_reward, dangerous_areas, dangerous_area_radius, dangerous_area_penalty, dangerous_area_hit_probability)

Bases: RockSampleRewardModel

ZERO_MEAN_HAZARD_SHOCK variant.

Replaces the constant-probability penalty with a 50/50 split between +dangerous_area_penalty and -dangerous_area_penalty whenever the realised next position is in any dangerous area. Expected contribution is 0; variance is dangerous_area_penalty**2. Suitable for benchmarking risk-sensitive planners against expected-value planners on identical means.

dangerous_area_hit_probability is ignored in this model.

Parameters:

• map_size (Tuple[int, int])

• rock_positions (List[Tuple[int, int]])

• step_penalty (float)

• bad_rock_penalty (float)

• good_rock_reward (float)

• sensor_use_penalty (float)

• exit_reward (float)

• dangerous_areas (List[Tuple[int, int]])

• dangerous_area_radius (float)

• dangerous_area_penalty (float)

• dangerous_area_hit_probability (float)