Implement missing 4th-edition algorithms (game theory, EM, Kalman, DBN, DDN, SARSA) with tests and demo notebooks, plus bug fixes and CI modernization

.github/workflows/tests.yml

@@ -0,0 +1,40 @@
+name: tests
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    branches: [master]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
+    env:
+      MPLBACKEND: Agg
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          submodules: recursive
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: pip
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+          pip install pytest pytest-cov flake8
+
+      - name: Lint with flake8
+        run: flake8 .
+        continue-on-error: true
+
+      - name: Run tests
+        run: pytest --cov=./

.gitignore

@@ -9,6 +9,8 @@ __pycache__/
 # Distribution / packaging
 .Python
 env/
+.venv/
+venv/
 build/
 develop-eggs/
 dist/

.travis.yml

@@ -1,10 +1,12 @@
 language: python
 
+dist: jammy
+
 python:
-  - 3.5
-  - 3.6
-  - 3.7
-  - 3.8
+  - 3.9
+  - 3.10
+  - 3.11
+  - 3.12
 
 before_install:
   - git submodule update --remote
@@ -14,10 +16,9 @@ install:
 
 script:
   - py.test --cov=./
-  - python -m doctest -v *.py
 
 after_success:
-  - flake8 --max-line-length 100 --ignore=E121,E123,E126,E221,E222,E225,E226,E242,E701,E702,E704,E731,W503 .
+  - flake8 --max-line-length 100 --ignore=E121,E123,E126,E221,E222,E225,E226,E242,E701,E702,E704,E731,W503,W504,F405,F841 .
 
 notifications:
   email: false

README.md

@@ -1,15 +1,15 @@
 
 
-# `aima-python` [![Build Status](https://travis-ci.org/aimacode/aima-python.svg?branch=master)](https://travis-ci.org/aimacode/aima-python) [![Binder](http://mybinder.org/badge.svg)](http://mybinder.org/repo/aimacode/aima-python)
+# `aima-python` [![tests](https://github.com/aimacode/aima-python/actions/workflows/tests.yml/badge.svg)](https://github.com/aimacode/aima-python/actions/workflows/tests.yml) [![Binder](http://mybinder.org/badge.svg)](http://mybinder.org/repo/aimacode/aima-python)
 
 
 Python code for the book *[Artificial Intelligence: A Modern Approach](http://aima.cs.berkeley.edu).* You can use this in conjunction with a course on AI, or for study on your own. We're looking for [solid contributors](https://github.com/aimacode/aima-python/blob/master/CONTRIBUTING.md) to help.
 
 # Updates for 4th Edition
 
-The 4th edition of the book as out now in 2020, and thus we are updating the code. All code here will reflect the 4th edition. Changes include:
+The 4th edition of the book is out now in 2020, and thus we are updating the code. All code here will reflect the 4th edition. Changes include:
 
-- Move from Python 3.5 to 3.7.
+- Move from Python 3.5 to 3.7, and on to modern Python (the code now runs on Python 3.9 and up).
 - More emphasis on Jupyter (Ipython) notebooks.
 - More projects using external packages (tensorflow, etc.).
 
@@ -23,9 +23,9 @@ When complete, this project will have Python implementations for all the pseudoc
 - `search_XX.ipynb`: Notebooks that show how to use the code, broken out into various topics (the `XX`).
 - `tests/test_search.py`: A lightweight test suite, using `assert` statements, designed for use with [`py.test`](http://pytest.org/latest/), but also usable on their own.
 
-# Python 3.7 and up
+# Python 3.9 and up
 
-The code for the 3rd edition was in Python 3.5; the current 4th edition code is in Python 3.7. It should also run in later versions, but does not run in Python 2. You can [install Python](https://www.python.org/downloads) or use a browser-based Python interpreter such as [repl.it](https://repl.it/languages/python3).
+The code for the 3rd edition was in Python 3.5; the 4th edition code targets Python 3.7 and runs on Python 3.9 and up, but does not run in Python 2. Continuous integration runs the full test suite (including the deep-learning modules) on Python 3.9, 3.10, 3.11 and 3.12; note that some optional dependencies (`tensorflow`, `keras`, `opencv-python`) do not yet ship wheels for the very latest releases (3.13+), so one of those versions is recommended for running everything. You can [install Python](https://www.python.org/downloads) or use a browser-based Python interpreter such as [repl.it](https://repl.it/languages/python3).
 You can run the code in an IDE, or from the command line with `python -i filename.py` where the `-i` option puts you in an interactive loop where you can run Python functions. All notebooks are available in a [binder environment](http://mybinder.org/repo/aimacode/aima-python). Alternatively, visit [jupyter.org](http://jupyter.org/) for instructions on setting up your own Jupyter notebook environment.
 
 Features from Python 3.6 and 3.7 that we will be using for this version of the code:
@@ -120,7 +120,7 @@ Here is a table of algorithms, the figure, name of the algorithm in the book and
 | 7.15   | PL-FC-Entails?                    | `pl_fc_entails`               | [`logic.py`][logic]             | Done | Included |
 | 7.17   | DPLL-Satisfiable?                 | `dpll_satisfiable`            | [`logic.py`][logic]             | Done | Included |
 | 7.18   | WalkSAT                           | `WalkSAT`                     | [`logic.py`][logic]             | Done | Included |
-| 7.20   | Hybrid-Wumpus-Agent               | `HybridWumpusAgent`           |                                 |      |          |
+| 7.20   | Hybrid-Wumpus-Agent               | `HybridWumpusAgent`           | [`logic.py`][logic]             | Done | Included |
 | 7.22   | SATPlan                           | `SAT_plan`                    | [`logic.py`][logic]             | Done | Included |
 | 9      | Subst                             | `subst`                       | [`logic.py`][logic]             | Done | Included |
 | 9.1    | Unify                             | `unify`                       | [`logic.py`][logic]             | Done | Included |
@@ -147,22 +147,38 @@ Here is a table of algorithms, the figure, name of the algorithm in the book and
 | 15.4   | Forward-Backward                  | `forward_backward`            | [`probability.py`][probability] | Done | Included |
 | 15.6   | Fixed-Lag-Smoothing               | `fixed_lag_smoothing`         | [`probability.py`][probability] | Done | Included |
 | 15.17  | Particle-Filtering                | `particle_filtering`          | [`probability.py`][probability] | Done | Included |
+| 15.4   | Kalman-Filter                     | `KalmanFilter`                | [`probability.py`][probability] | Done | Included |
+| 15.5   | Dynamic-Bayesian-Network          | `DynamicBayesNet`             | [`probability.py`][probability] | Done | Included |
 | 16.9   | Information-Gathering-Agent       | `InformationGatheringAgent`   | [`probability.py`][probability] | Done | Included |
 | 17.4   | Value-Iteration                   | `value_iteration`             | [`mdp.py`][mdp]                 | Done | Included |
 | 17.7   | Policy-Iteration                  | `policy_iteration`            | [`mdp.py`][mdp]                 | Done | Included |
 | 17.9   | POMDP-Value-Iteration             | `pomdp_value_iteration`       | [`mdp.py`][mdp]                 | Done | Included |
+| 17.4   | Dynamic-Decision-Network          | `pomdp_lookahead`             | [`mdp4e.py`](mdp4e.py)          | Done | Included |
+| 18.2   | Iterated-Dominance                | `iterated_dominance`          | [`game_theory4e.py`](game_theory4e.py) | Done | Included |
+| 18.2   | Pure-Nash-Equilibria              | `pure_nash_equilibria`        | [`game_theory4e.py`](game_theory4e.py) | Done | Included |
+| 18.2   | Zero-Sum-Game (LP)                | `solve_zero_sum_game`         | [`game_theory4e.py`](game_theory4e.py) | Done | Included |
+| 18.3   | Shapley-Value                     | `shapley_value`               | [`game_theory4e.py`](game_theory4e.py) | Done | Included |
+| 18.3   | Core (cooperative game)           | `is_in_core`                  | [`game_theory4e.py`](game_theory4e.py) | Done | Included |
+| 18.4   | Voting (plurality/Borda/Condorcet)| `plurality_winner` etc.       | [`game_theory4e.py`](game_theory4e.py) | Done | Included |
+| 18.4   | Vickrey-Auction                   | `vickrey_auction`             | [`game_theory4e.py`](game_theory4e.py) | Done | Included |
+| 18.4.1 | Contract-Net-Protocol             | `contract_net`                | [`game_theory4e.py`](game_theory4e.py) | Done | Included |
+| 18.4.4 | Alternating-Offers-Bargaining     | `alternating_offers_bargaining` | [`game_theory4e.py`](game_theory4e.py) | Done | Included |
 | 18.5   | Decision-Tree-Learning            | `DecisionTreeLearner`         | [`learning.py`][learning]       | Done | Included |
-| 18.8   | Cross-Validation                  | `cross_validation`            | [`learning.py`][learning]\*     |      |          |
-| 18.11  | Decision-List-Learning            | `DecisionListLearner`         | [`learning.py`][learning]\*     |      |          |
+| 18.8   | Cross-Validation                  | `cross_validation`            | [`learning.py`][learning]       | Done | Included |
+| 18.11  | Decision-List-Learning            | `DecisionListLearner`         | [`learning.py`][learning]       | Done | Included |
 | 18.24  | Back-Prop-Learning                | `BackPropagationLearner`      | [`learning.py`][learning]       | Done | Included |
 | 18.34  | AdaBoost                          | `AdaBoost`                    | [`learning.py`][learning]       | Done | Included |
+| 20.X   | EM (Mixture of Gaussians)         | `gaussian_mixture_em`         | [`learning.py`][learning]       | Done | Included |
+| 20.3.2 | EM (Bayes net hidden variable)    | `naive_bayes_em`              | [`learning.py`][learning]       | Done | Included |
+| 20.3   | Baum-Welch (HMM learning)         | `baum_welch`                  | [`probability.py`][probability] | Done | Included |
 | 19.2   | Current-Best-Learning             | `current_best_learning`       | [`knowledge.py`](knowledge.py)  | Done | Included |
 | 19.3   | Version-Space-Learning            | `version_space_learning`      | [`knowledge.py`](knowledge.py)  | Done | Included |
 | 19.8   | Minimal-Consistent-Det            | `minimal_consistent_det`      | [`knowledge.py`](knowledge.py)  | Done | Included |
 | 19.12  | FOIL                              | `FOIL_container`              | [`knowledge.py`](knowledge.py)  | Done | Included |
 | 21.2   | Passive-ADP-Agent                 | `PassiveADPAgent`             | [`rl.py`][rl]                   | Done | Included |
 | 21.4   | Passive-TD-Agent                  | `PassiveTDAgent`              | [`rl.py`][rl]                   | Done | Included |
 | 21.8   | Q-Learning-Agent                  | `QLearningAgent`              | [`rl.py`][rl]                   | Done | Included |
+| 21.8   | SARSA-Agent                       | `SARSALearningAgent`          | [`rl.py`][rl]                   | Done | Included |
 | 22.1   | HITS                              | `HITS`                        | [`nlp.py`][nlp]                 | Done | Included |
 | 23     | Chart-Parse                       | `Chart`                       | [`nlp.py`][nlp]                 | Done | Included |
 | 23.5   | CYK-Parse                         | `CYK_parse`                   | [`nlp.py`][nlp]                 | Done | Included |

csp.py

@@ -632,7 +632,7 @@ def queen_constraint(A, a, B, b):
 
 
 class NQueensCSP(CSP):
-    """
+    r"""
     Make a CSP for the nQueens problem for search with min_conflicts.
     Suitable for large n, it uses only data structures of size O(n).
     Think of placing queens one per column, from left to right.
@@ -818,7 +818,7 @@ def Zebra():
     Colors = 'Red Yellow Blue Green Ivory'.split()
     Pets = 'Dog Fox Snails Horse Zebra'.split()
     Drinks = 'OJ Tea Coffee Milk Water'.split()
-    Countries = 'Englishman Spaniard Norwegian Ukranian Japanese'.split()
+    Countries = 'Englishman Spaniard Norwegian Ukrainian Japanese'.split()
     Smokes = 'Kools Chesterfields Winston LuckyStrike Parliaments'.split()
     variables = Colors + Pets + Drinks + Countries + Smokes
     domains = {}
@@ -829,7 +829,7 @@ def Zebra():
     neighbors = parse_neighbors("""Englishman: Red;
                 Spaniard: Dog; Kools: Yellow; Chesterfields: Fox;
                 Norwegian: Blue; Winston: Snails; LuckyStrike: OJ;
-                Ukranian: Tea; Japanese: Parliaments; Kools: Horse;
+                Ukrainian: Tea; Japanese: Parliaments; Kools: Horse;
                 Coffee: Green; Green: Ivory""")
     for type in [Colors, Pets, Drinks, Countries, Smokes]:
         for A in type:
@@ -857,7 +857,7 @@ def zebra_constraint(A, a, B, b, recurse=0):
             return same
         if A == 'LuckyStrike' and B == 'OJ':
             return same
-        if A == 'Ukranian' and B == 'Tea':
+        if A == 'Ukrainian' and B == 'Tea':
             return same
         if A == 'Japanese' and B == 'Parliaments':
             return same

deep_learning4e.py

@@ -41,10 +41,10 @@ def forward(self, inputs):
 class Activation:
 
     def function(self, x):
-        return NotImplementedError
+        raise NotImplementedError
 
     def derivative(self, x):
-        return NotImplementedError
+        raise NotImplementedError
 
     def __call__(self, x):
         return self.function(x)
@@ -575,7 +575,7 @@ def AutoencoderLearner(inputs, encoding_size, epochs=200, verbose=False):
     model.add(Dense(input_size, activation='relu', kernel_initializer='random_uniform', bias_initializer='ones'))
 
     # update model with sgd
-    sgd = optimizers.SGD(lr=0.01)
+    sgd = optimizers.SGD(learning_rate=0.01)
     model.compile(loss='mean_squared_error', optimizer=sgd, metrics=['accuracy'])
 
     # train the model

dynamic_decision_network.ipynb

@@ -0,0 +1,152 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "07e68a51",
+   "metadata": {},
+   "source": [
+    "# Dynamic Decision Networks (Section 17.4)\n",
+    "\n",
+    "Online decision making for a POMDP modeled as a dynamic decision network, using the belief-state look-ahead agent in [`mdp4e.py`](mdp4e.py)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "bded6ebb",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-06-23T10:42:10.709493Z",
+     "iopub.status.busy": "2026-06-23T10:42:10.709226Z",
+     "iopub.status.idle": "2026-06-23T10:42:10.874538Z",
+     "shell.execute_reply": "2026-06-23T10:42:10.872073Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from mdp4e import POMDP, update_belief, pomdp_lookahead"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f0fb1298",
+   "metadata": {},
+   "source": [
+    "A two-state 'tiger-like' POMDP: action 0 pays off in state 0, action 1 in state 1, and action 2 is a sensing action (small cost, informative observation)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "cccf717e",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-06-23T10:42:10.877731Z",
+     "iopub.status.busy": "2026-06-23T10:42:10.877319Z",
+     "iopub.status.idle": "2026-06-23T10:42:10.883821Z",
+     "shell.execute_reply": "2026-06-23T10:42:10.882142Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "t_prob = [[[0.65, 0.35], [0.65, 0.35]], [[0.65, 0.35], [0.65, 0.35]], [[1.0, 0.0], [0.0, 1.0]]]\n",
+    "e_prob = [[[0.5, 0.5], [0.5, 0.5]], [[0.5, 0.5], [0.5, 0.5]], [[0.8, 0.2], [0.3, 0.7]]]\n",
+    "rewards = [[5, -10], [-20, 5], [-1, -1]]\n",
+    "pomdp = POMDP(('0', '1', '2'), t_prob, e_prob, rewards, ('0', '1'), gamma=0.95)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1b6315c8",
+   "metadata": {},
+   "source": [
+    "### Belief update (POMDP filtering, Equation 17.17)\n",
+    "Observation 0 (more likely in state 0) shifts a uniform belief towards state 0."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "e117ad73",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-06-23T10:42:10.888179Z",
+     "iopub.status.busy": "2026-06-23T10:42:10.887720Z",
+     "iopub.status.idle": "2026-06-23T10:42:10.910700Z",
+     "shell.execute_reply": "2026-06-23T10:42:10.909496Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "belief after sensing obs 0: [0.727, 0.273]\n"
+     ]
+    }
+   ],
+   "source": [
+    "print('belief after sensing obs 0:', [round(b, 3) for b in update_belief(pomdp, [0.5, 0.5], '2', 0)])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "578f7466",
+   "metadata": {},
+   "source": [
+    "### Look-ahead decisions\n",
+    "When the state is known the agent commits to the rewarding action; when it is unknown it prefers to gather information first."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "d904ab41",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-06-23T10:42:10.914146Z",
+     "iopub.status.busy": "2026-06-23T10:42:10.913844Z",
+     "iopub.status.idle": "2026-06-23T10:42:10.937111Z",
+     "shell.execute_reply": "2026-06-23T10:42:10.936126Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "belief [0.9, 0.1], depth 1 -> action 0\n",
+      "belief [0.1, 0.9], depth 1 -> action 1\n",
+      "belief [0.5, 0.5], depth 2 -> action 2 (sense)\n"
+     ]
+    }
+   ],
+   "source": [
+    "print('belief [0.9, 0.1], depth 1 -> action', pomdp_lookahead(pomdp, [0.9, 0.1], depth=1))\n",
+    "print('belief [0.1, 0.9], depth 1 -> action', pomdp_lookahead(pomdp, [0.1, 0.9], depth=1))\n",
+    "print('belief [0.5, 0.5], depth 2 -> action', pomdp_lookahead(pomdp, [0.5, 0.5], depth=2), '(sense)')"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}