Skip to content

Implement missing 4th-edition algorithms (game theory, EM, Kalman, DBN, DDN, SARSA) with tests and demo notebooks, plus bug fixes and CI modernization#1323

Open
dmeoli wants to merge 283 commits into
aimacode:masterfrom
dmeoli:master
Open

Implement missing 4th-edition algorithms (game theory, EM, Kalman, DBN, DDN, SARSA) with tests and demo notebooks, plus bug fixes and CI modernization#1323
dmeoli wants to merge 283 commits into
aimacode:masterfrom
dmeoli:master

Conversation

@dmeoli

@dmeoli dmeoli commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

This PR fills in several algorithms from the 4th edition that were not yet implemented, adds tests (including cases taken from the book and the 3rd-edition solutions manual) and runnable demo notebooks, fixes a few bugs, and modernizes the CI. The full test suite passes (446 tests).

New algorithms

game_theory.py (new module) - Chapter 18, Multiagent Decision Making

  • 18.2 Non-cooperative: dominates / dominant_strategy, iterated_dominance, pure_nash_equilibria, solve_zero_sum_game (von Neumann minimax via linear programming)
  • 18.3 Cooperative: shapley_value, is_in_core
  • 18.4 Collective decisions: plurality_winner, borda_winner, condorcet_winner, vickrey_auction, contract_net, alternating_offers_bargaining

probability.py

  • KalmanFilter / kalman_filter - Kalman filtering (Section 15.4)
  • DynamicBayesNet - dynamic Bayesian networks with unroll() and exact filtering (Section 15.5)
  • baum_welch - EM for learning HMM parameters (Section 20.3.3)

learning.py

  • gaussian_mixture_em - EM for a mixture of Gaussians (Section 20.3.1)
  • naive_bayes_em - EM for a Bayes net with a hidden variable, the candy-bags model (Section 20.3.2)

mdp4e.py

  • pomdp_lookahead and update_belief - online POMDP agent over a dynamic decision network via belief-state expectimax look-ahead (Section 17.4)

reinforcement_learning.py and reinforcement_learning4e.py

  • SARSALearningAgent - on-policy temporal-difference control (Section 21.3)

Bug fixes

  • return NotImplementedError changed to raise NotImplementedError in deep_learning4e.py and ipyviews.py (was returning the exception class instead of raising it)
  • Fixed SyntaxWarning from invalid escape sequences in docstrings (csp.py, logic.py) by using raw strings
  • Keras 3 compatibility: optimizers.SGD(lr=...) changed to learning_rate=... in deep_learning4e.py
  • Made test_value_iteration robust to floating-point rounding across numpy/BLAS versions (pytest.approx)

Tests

  • New tests/test_game_theory.py and new tests for the Kalman filter, DBN, Baum-Welch, both EM variants, the DDN look-ahead agent, and SARSA
  • Cases taken from the book / 3rd-edition solutions manual: the zero-sum game of exercise 17.17 (optimal mixed strategy [1/9, 1/9, 1/9, 1/3, 1/3], value 0), a saddle-point game, battle of the sexes, stag hunt, the umbrella DBN cross-checked against the HMM forward algorithm, and the Kalman
    steady-state variance

Demo notebooks

  • Added executed notebooks with output (and plots): game_theory.ipynb, expectation_maximization.ipynb, kalman_filter.ipynb, dynamic_decision_network.ipynb, sarsa.ipynb

Cleanup

  • Fixed spelling typos in comments, docstrings, and data labels (codespell)
  • Updated the README algorithm index for all the new entries and corrected the Python version section (now Python 3.9 and up)

CI

  • Added a GitHub Actions workflow running the full suite on Python 3.9, 3.10, 3.11, and 3.12
  • Bumped the Python versions in .travis.yml to match (3.9-3.12)
  • Updated the build-status badge

dmeoli added 26 commits June 23, 2020 01:16
…point, battle of the sexes, stag hunt, DBN sequence, Kalman steady state)
@dmeoli

dmeoli commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

@antmarakis @norvig

dmeoli added 3 commits June 23, 2026 14:54
Cover the previously untested non-book algorithms so they match the
baseline test coverage:

- test_dpll_branching_heuristics / test_cdcl_restart_strategies exercise
  every branching heuristic (moms, momsf, posit, dlis, dlcs, jw, jw2, zm)
  and restart strategy (no_restart, luby, glucose) on small SAT/UNSAT
  instances.
- test_nary_csp, test_ac_solver_classes, test_crossword, test_kakuro
  cover NaryCSP/Constraint, the ACSolver/ACSearchSolver classes directly,
  and the Crossword/Kakuro models.
- game_theory.py -> game_theory4e.py (module written against 4th-edition
  chapter 18 numbering, consistent with the other 4e-structured modules);
  test file, notebook and README links updated accordingly.
- making_simple_decision4e.py -> making_simple_decisions4e.py to match the
  book chapter title 'Making Simple Decisions'.
- planning_graphPlan.ipynb -> planning_graph_plan.ipynb and
  knowledge_FOIL.ipynb -> knowledge_foil.ipynb (snake_case).
- images/pluralityLearner_plot.png and images/knowledge_FOIL_grandparent.png
  renamed to snake_case; notebook references updated.
Make function and variable names consistent with the snake_case style
used across the rest of the codebase:

- planning.py: the *_graphPlan example helpers become *_graph_plan, plus
  graphPlan_solution, initialPlan, nConstraints, nPartial.
- nlp.py: loadPageHTML, initPages, stripRawHTML, determineInlinks,
  findOutlinks, onlyWikipediaURLS, getInLinks, getOutLinks and their local
  variables (pagesIndex, pagesContent, inLinks, outLinks, etc.).
- Updated all call sites in tests and notebooks, and snake_cased the
  affected test names and test-local variables.

Class names (PascalCase) and the canvas strokeWidth method, which mirrors
the JS canvas API, are intentionally left unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant