[d-morrison]

Requested by Ezra · project thread

This is the second PR in a stack. Merge #191 first, then this one, then #194.

Why

Ezra asked for the worked solution to ITILA exercise 8.11 (MacKay 2003, p. 142) to live in ids. The book gives no solution. The exercise asks whether a text read backward carries more information per character than the same text read forward, given that people find guessing backward harder.

What

The section that #191 adds to the entropy page gains:

• rem-reversed-text-guessing, which states the guessing-game question, in my own words, with the MacKay citation
• exr-reversed-text-total-entropy and its solution: the conditional entropies of a whole text sum to the same total forward and backward, by the chain rule applied in both orders
• exr-forward-n-gram-conditional-entropy and its solution: under a stationary N-gram model, the forward conditional entropy of each character is H(X_1..X_N) − H(X_1..X_{N−1})

The backward half (#194) shows that a reversed N-gram model is again an N-gram model, with the same conditional entropy.

The first review asked for one exercise per part. The four-part exercise is now four exercises, split across this PR and #194 to stay under the size cap.

Checks

• check-term-links passes locally with the CI inputs.
• The source-overlap checker, run against the ITILA text, flags nothing.
• quarto render entropy.qmd --to html builds, and every cross-reference resolves.
• I checked the full result numerically on a random, non-reversible trigram model. The forward and backward conditional entropies matched to 15 digits.

🤖 Generated with Claude Code

https://claude.ai/code/session_019xfc9pXuEzzVRdTKMWnCh8

[d-morrison]

The review's findings are addressed in commits 65365f9 and ccca005:

• Multi-part exercise: It is now four exercises, each with its own solution, after a remark that states the question and cites MacKay. To stay under the size cap, the first two are in this PR and the last two are in Add the backward half of MacKay exercise 8.11 #194, stacked on this one.
• Line breaks: I broke the three lines the review flagged at their clauses.
• \eqdef for R (optional): The solution now writes "Write $R$ for the product" with a plain =.
• Locator (optional): I checked "exercise 8.11" against a copy of the book: it is Exercise 8.11 on p. 142, in the section "Entropies of Markov processes". The book gives no solution for it.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[github-actions]

Claude finished review — View run

---

Both new commits (65365f9 and ccca005) fix the blocking items from the prior round, and I found no new blocking issues. I checked the diff by reading it and grepping for cross-reference anchors. I did not render the site or run any suite.

What changed since the last round

• Rule 7 (split exercises). The four-part exr-reversed-text-entropy is gone. The new files are _rem-reversed-text-guessing.qmd, _exr-reversed-text-total-entropy.qmd and _exr-forward-n-gram-conditional-entropy.qmd. The backward half moved to Add the backward half of MacKay exercise 8.11 #194. Each remaining exercise has a single task.
• Rule 12 (line breaks). The old _sol-reversed-text-entropy.qmd is deleted, and with it the three double-clause lines I flagged. The two new solutions break at clauses ("because", "so", "and"). I found no line carrying two clauses.
• Cross-references. I grepped for every anchor the new files use, and each is defined in a _subfiles file:
  • thm-conditional-entropy-average
  • def-entropy-given-value
  • cor-entropy-chain-rule-reversed
  • thm-entropy-chain-rule and thm-entropy-chain-rule-n
  • def-joint-entropy
  • def-stationary-sequence
  • def-n-gram-model
  • def-conditional-entropy
• Includes. entropy.qmd includes all five new subfiles, and nothing refers to a removed file.
• Math. The proofs hold:
  • The forward solution reduces the conditioning set by the N-gram property, sums out the extra variables, then applies the chain rule and stationarity.
  • The total-entropy solution applies the chain rule in both orders.
  • \P, \entropyf and the other macros come from the pinned latex-macros/. I did not check \entropyf against macros.qmd.

Suggestions

• _subfiles/_rem-reversed-text-guessing.qmd:11: the citation [@mackay2003information, exercise 8.11] is still unverified. The PR description says only "ITILA exercise 8.11 (MacKay 2003, p. 142)" and quotes no passage. A short quote of the exercise in a PR comment from the owner or a member would settle it.
• _subfiles/_sol-forward-n-gram-conditional-entropy.qmd:3-5: the solution defines a local shorthand p(x_a, …) for \P(X_a = x_a, …). This is a bare p standing in for a probability mass function. It is fine as a stated local abbreviation. Using the library's mass-function macro would be more consistent if one exists.

Verdict

Ready for merge

The earlier blocking findings (rule 7 split and the three rule 12 line breaks) are resolved. The remaining items are non-blocking suggestions.

Structured Review Data (JSON)

{
  "schema_version": "1.1",
  "reviewer": "claude",
  "commit_sha": "ccca005e78d3b416aa8637925c4835259b235b5a",
  "verdict": "CLEAN",
  "findings": [],
  "detailed_assessment": "The split exercises in _exr-reversed-text-total-entropy.qmd and _exr-forward-n-gram-conditional-entropy.qmd are single-task, their solutions' cross-references all resolve, and no line holds two clauses.",
  "holistic_assessment": "The change is scoped to new subfiles plus includes in entropy.qmd, resolves the prior rule 7 and rule 12 findings, and leaves only an unverified MacKay locator as a non-blocking suggestion."
}

Reviewed commit: ccca005

[github-actions]

💰 Cost: $0.2227 (review) — run

[d-morrison]

Replies to the two optional suggestions. Neither changes the code:

• Locator: My earlier comment checked it against a copy of the book. It is Exercise 8.11 on p. 142, under "Entropies of Markov processes".
• The p(...) shorthand: latex-macros/macros.qmd has no macro for a probability mass function evaluated at subscripted values: \pmass and \mass are aliases of \P. def-entropy-given-value on the same page already uses a bare p(y \mid x) the same way, so the shorthand stays as it is.

Posted by Claude Code (AI agent) --- not written by a human.

---

Generated by Claude Code

[github-actions]

PR Preview Action v1.8.2
🚀 Preview available at: https://Morrison-Lab.github.io/ids/pr-preview/pr-193/

_{Built to branch gh-pages at 2026-11-12 09:16 UTC.}