POMDPPlanners.environments.pacman_pomdp.pacman_pomdp_utils package

Submodules

POMDPPlanners.environments.pacman_pomdp.pacman_pomdp_utils.numba_kernels module

PacMan-specific Numba-JIT kernels.

Single-pass batched reward kernel that mirrors the numpy implementation in PacManRewardModel.compute_reward_batch but fuses the terminal / collision / pellet / win / dangerous-area work into one loop over rows. The numpy path issues ~15-20 separate operator calls per invocation; folding them into one @njit kernel eliminates the per-call dispatch overhead and the temporary allocations.

Conventions

  • All array inputs are contiguous float64 / int32 / int64 / uint8 as declared on the kernel signature.

  • next_states is always passed as a real array. When the caller has no realised next-state batch, pass a zero-row sentinel (np.empty((0, state_dim), dtype=np.float64)) and set has_next_states to False; the kernel skips the collision contribution in that case.

  • No Python objects, no RNG: reward is deterministic given inputs.

POMDPPlanners.environments.pacman_pomdp.pacman_pomdp_utils.numba_kernels.compute_reward_batch_kernel(states, action, next_states, has_next_states, neighbor_table, pellet_positions, dangerous_areas, num_ghosts, step_penalty, ghost_collision_penalty, pellet_reward, win_reward, dangerous_area_penalty, dangerous_area_radius, idx_pac_row, idx_pac_col, idx_ghosts_start, idx_pellets_start, idx_terminal)[source]

Fused per-row reward kernel.

Replicates the numpy implementation exactly: terminal -> 0.0, otherwise step_penalty + (optional) collision + dangerous-area + pellet + win-bonus contributions. The pellet-mask iteration bound is derived from pellet_positions.shape[0]; no separate idx_pellets_end parameter is needed.

Return type:

ndarray

Parameters:
POMDPPlanners.environments.pacman_pomdp.pacman_pomdp_utils.numba_kernels.compute_reward_scalar_kernel(state, next_state, dangerous_areas, num_ghosts, n_pellets, step_penalty, ghost_collision_penalty, pellet_reward, win_reward, dangerous_area_penalty, dangerous_area_radius, idx_pac_row, idx_pac_col, idx_ghosts_start, idx_pellets_start, idx_score, idx_terminal)[source]

Single-row scalar reward kernel mirroring the Python implementation.

Reads the realised pacman / ghost positions directly from next_state (matching the Python scalar — pacman moves deterministically, so the neighbor-table lookup the batch kernel uses is redundant here). Pellet pickup is detected via the next_state.score > state.score invariant the transition kernel upholds; win-bonus fires only when next_state is terminal with no remaining pellets.

Return type:

float

Parameters:

POMDPPlanners.environments.pacman_pomdp.pacman_pomdp_utils.pacman_reward_models module

Reward models for the PacMan POMDP.

Mirrors the abstract-base / concrete-subclass layout used by POMDPPlanners.environments.light_dark_pomdp.light_dark_pomdp_utils.light_dark_reward_models and POMDPPlanners.environments.laser_tag_pomdp.laser_tag_pomdp_utils.laser_tag_reward_models, so further PacMan reward variants can be added without growing the env class.

Three concrete variants are provided. They share all of the non-dangerous- area scoring (step / collision / pellet / win) via the PacManRewardModel base; each variant only customises the dangerous- area contribution:

  • PacManRewardModel (CONSTANT_HAZARD_PENALTY): deterministic -dangerous_area_penalty whenever the realised next pacman position lies inside any configured circular hazard zone (squared-distance check against dangerous_area_radius).

  • PacManZeroMeanHazardShockRewardModel (ZERO_MEAN_HAZARD_SHOCK): ±dangerous_area_penalty 50/50 in-zone — zero expected contribution, high variance.

  • PacManDistanceDecayedHazardPenaltyRewardModel (DISTANCE_DECAYED_HAZARD_PENALTY): -dangerous_area_penalty is applied with probability exp(-min_dist / penalty_decay) based on the closest zone centre — no radius cutoff.

The reward model owns all the parameters and pre-built buffers that the reward computation needs (state-layout indices, pellet positions, dangerous areas, scalar penalties / bonuses, and a callable that returns the env’s lazily-built neighbor table). The environment retains its own copies of these values for the transition / observation paths and delegates reward() / reward_batch() to the model.

class POMDPPlanners.environments.pacman_pomdp.pacman_pomdp_utils.pacman_reward_models.BasePacManRewardModel[source]

Bases: ABC

Abstract reward model for PacMan POMDP variants.

abstractmethod compute_reward(state, action, next_state)[source]

Return the scalar reward for (state, action, next_state).

Return type:

float

Parameters:
abstractmethod compute_reward_batch(states, action, next_states=None)[source]

Return the per-row reward for a batch of states under a single action.

Return type:

ndarray

Parameters:
class POMDPPlanners.environments.pacman_pomdp.pacman_pomdp_utils.pacman_reward_models.PacManDistanceDecayedHazardPenaltyRewardModel(*, num_ghosts, pellet_positions_arr, dangerous_areas, dangerous_areas_arr, dangerous_area_radius, dangerous_area_penalty, step_penalty, ghost_collision_penalty, pellet_reward, win_reward, idx_pac_row, idx_pac_col, idx_ghosts_start, idx_pellets_start, idx_pellets_end, idx_score, idx_terminal, neighbor_table_getter, penalty_decay)[source]

Bases: PacManRewardModel

DISTANCE_DECAYED_HAZARD_PENALTY variant.

Penalty is applied with probability exp(-min_dist / penalty_decay) where min_dist is the Euclidean distance from the realised next pacman position to the closest dangerous-area centre. No radius cutoff — every step risks some (vanishingly small at large distance) penalty. Each call draws one uniform regardless of distance, matching the existing decaying-prob kernel and the light-dark / laser-tag analogous reward models. dangerous_area_radius is ignored in this model.

Parameters:
compute_reward(state, action, next_state)[source]

Return the scalar reward for (state, action, next_state).

Return type:

float

Parameters:
compute_reward_batch(states, action, next_states=None)[source]

Return the per-row reward for a batch of states under a single action.

Return type:

ndarray

Parameters:
class POMDPPlanners.environments.pacman_pomdp.pacman_pomdp_utils.pacman_reward_models.PacManRewardModel(*, num_ghosts, pellet_positions_arr, dangerous_areas, dangerous_areas_arr, dangerous_area_radius, dangerous_area_penalty, step_penalty, ghost_collision_penalty, pellet_reward, win_reward, idx_pac_row, idx_pac_col, idx_ghosts_start, idx_pellets_start, idx_pellets_end, idx_score, idx_terminal, neighbor_table_getter)[source]

Bases: BasePacManRewardModel

Standard PacMan reward model.

Reward structure:

  • Per-step: step_penalty baseline applied to every non-terminal transition.

  • Pellet collection: +pellet_reward when next_state records a score increase versus state (scalar path) or when the realised next position lands on an active pellet cell (batch path).

  • Ghost collision: +ghost_collision_penalty when any ghost in next_state occupies PacMan’s realised next cell.

  • Dangerous area: -dangerous_area_penalty when PacMan’s realised next position lies inside any configured circular hazard zone (squared-distance check against dangerous_area_radius).

  • Win bonus: +win_reward when the transition consumes the last remaining pellet (terminal with empty pellet mask in the scalar path; final-pellet pickup in the batch path).

Terminal states return 0.0 reward.

Subclasses override _dangerous_area_contribution_scalar() and _apply_dangerous_area_contribution_batch() to express different stochastic penalty models; the rest of the scoring is identical.

Parameters:
compute_reward(state, action, next_state)[source]

Return the scalar reward for (state, action, next_state).

Return type:

float

Parameters:
compute_reward_batch(states, action, next_states=None)[source]

Return the per-row reward for a batch of states under a single action.

Return type:

ndarray

Parameters:
class POMDPPlanners.environments.pacman_pomdp.pacman_pomdp_utils.pacman_reward_models.PacManZeroMeanHazardShockRewardModel(*, num_ghosts, pellet_positions_arr, dangerous_areas, dangerous_areas_arr, dangerous_area_radius, dangerous_area_penalty, step_penalty, ghost_collision_penalty, pellet_reward, win_reward, idx_pac_row, idx_pac_col, idx_ghosts_start, idx_pellets_start, idx_pellets_end, idx_score, idx_terminal, neighbor_table_getter)[source]

Bases: PacManRewardModel

ZERO_MEAN_HAZARD_SHOCK variant.

Replaces the deterministic in-zone penalty with a 50/50 split between +dangerous_area_penalty and -dangerous_area_penalty whenever the realised next pacman position is in any dangerous area. Expected contribution is 0; variance is dangerous_area_penalty**2. Mirrors ContinuousLDZeroMeanHazardShockRewardModel and LaserTagZeroMeanHazardShockRewardModel. All non-dangerous-area scoring (collision / pellet / win / step penalty) is identical to the standard model.

Parameters:
compute_reward(state, action, next_state)[source]

Return the scalar reward for (state, action, next_state).

Return type:

float

Parameters:
compute_reward_batch(states, action, next_states=None)[source]

Return the per-row reward for a batch of states under a single action.

Return type:

ndarray

Parameters: