Belgian Boxcars

.github/workflows/learning-checks.yml

@@ -27,5 +27,5 @@ jobs:
         uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
       - name: Generate .meta/known-words.md
         run: bin/extract-known-words
-      - name: Detect unknown words in concept exemplars
-        run: bin/detect-unknown-words concept
+      - name: Detect unknown words in exemplars and examples
+        run: bin/detect-unknown-words

bin/detect-unknown-words

@@ -163,12 +163,40 @@ def parse_solution(path):
     return defined, locals_, used
 
 
+# Syntactic features that don't tokenise as ordinary words but still
+# require a concept be in the prereq chain. The detector filters these
+# tokens (so they never appear in the per-word "unknown" list), so we
+# need an explicit presence check.
+LOCALS_SYNTAX = re.compile(r'(?:^|\s)(?:::|:>|\[\|)(?:\s|$)')
+
+
+def reachable_concepts(prereqs):
+    """Return the set of concept slugs reachable through the prereq chain."""
+    concepts = set()
+    for ex_slug in transitive_prereq_concept_exes(prereqs):
+        concepts.update(EX_META[ex_slug].get('concepts', []))
+    return concepts
+
+
+def syntax_violations(sol_path, reachable):
+    """Return concept slugs whose syntactic features appear in the source
+    but whose teaching exercise is not reachable through the prereq chain."""
+    text = sol_path.read_text()
+    violations = []
+    if LOCALS_SYNTAX.search(text) and 'locals' not in reachable:
+        violations.append('<missing locals prereq>')
+    return violations
+
+
 def check_concept(slug, sol_path):
     """Concept exercise: known = its own known-words.md (which already
     includes own intro plus transitive prereq intros)."""
     known = known_words_for_concept_ex(slug)
     defined, locals_, used = parse_solution(sol_path)
-    return sorted(used - known - defined - locals_)
+    unknowns = sorted(used - known - defined - locals_)
+    own_concepts = set(EX_META[slug].get('concepts', []))
+    reachable = own_concepts | reachable_concepts(EX_META[slug].get('prerequisites', []))
+    return unknowns + syntax_violations(sol_path, reachable)
 
 
 def check_practice(prereqs, sol_path):
@@ -178,7 +206,8 @@ def check_practice(prereqs, sol_path):
     for cx in transitive_prereq_concept_exes(prereqs):
         known |= known_words_for_concept_ex(cx)
     defined, locals_, used = parse_solution(sol_path)
-    return sorted(used - known - defined - locals_)
+    unknowns = sorted(used - known - defined - locals_)
+    return unknowns + syntax_violations(sol_path, reachable_concepts(prereqs))
 
 
 def main():

concepts/arrays/about.md

@@ -10,6 +10,7 @@ the stack:
 | `1array` | `( a     -- { a } )`                |
 | `2array` | `( a b   -- { a b } )`              |
 | `3array` | `( a b c -- { a b c } )`            |
+| `array?` | `( obj   -- ? )` — type predicate   |
 
 A few protocol words from `sequences` come up so often with
 arrays that they are worth knowing as a unit:

concepts/conditionals/about.md

@@ -10,6 +10,14 @@ when    ( ? quot -- )
 unless  ( ? quot -- )
 ```
 
+Two starred variants treat the boolean as a value worth using
+when it's truthy — the canonical "value-or-`f`" pair:
+
+```
+if*     ( ? true false -- )    ! truthy: true called WITH value
+unless* ( ? false      -- )    ! falsy: false runs and pushes default
+```
+
 ```factor
 : abs   ( x -- y ) dup 0 < [ neg ] [ ] if ;
 : shout ( s -- )   dup empty? [ drop ] [ >upper print ] if ;

concepts/hash-sets/.meta/config.json

@@ -0,0 +1,6 @@
+{
+  "authors": [
+    "keiravillekode"
+  ],
+  "blurb": "Track unique values with mutable hash-sets — the canonical visited set for graph traversals."
+}

concepts/hash-sets/about.md

@@ -0,0 +1,59 @@
+# About
+
+Hash-sets implement the [`sets`][sets] protocol with hashing
+under the hood, giving O(1) average insert, lookup, and delete.
+They're mutable in place, which makes them ideal for the
+*visited set* pattern in graph traversals.
+
+```factor
+USING: hash-sets kernel sets ;
+
+HS{ } clone                   ! fresh empty mutable set
+"NS-1024" over adjoin         ! insert (no-op if already present)
+"NS-1024" over in? .          ! => t  (membership test)
+"WB-203" over adjoin
+over cardinality .            ! => 2
+"NS-1024" over delete         ! remove
+over cardinality .            ! => 1
+```
+
+| word          | effect                                    |
+|---------------|-------------------------------------------|
+| `HS{ }`       | empty hash-set literal (shared — `clone` it!) |
+| `adjoin`      | `( elt set -- )` — destructive insert     |
+| `in?`         | `( elt set -- ? )` — membership           |
+| `delete`      | `( elt set -- )` — destructive remove     |
+| `cardinality` | `( set -- n )` — number of elements       |
+| `members`     | `( set -- seq )` — enumerate as sequence  |
+| `union`       | `( set1 set2 -- set )`                    |
+| `intersect`   | `( set1 set2 -- set )`                    |
+| `diff`        | `( set1 set2 -- set )`                    |
+
+A subtle point about `in?` versus `member?` (from
+`sequences`): both test membership, but `member?` does a
+linear scan over a sequence, while `in?` dispatches to whatever
+the set type's protocol method is — for `HS{ }`, that's a
+hash lookup. Use `in?` once your "visited" container has more
+than a handful of entries.
+
+## Pairs nicely with hashtables
+
+`HS{ }` for "is X visited?" pairs naturally with `H{ }` for
+"who are X's neighbours?". A textbook BFS is just:
+
+```factor
+visited adjoin
+queue push
+[ queue empty? not ] [
+    queue pop dup neighbours-quot call [
+        dup visited in? [ drop ] [
+            [ visited adjoin ] [ queue push ] bi
+        ] if
+    ] each
+] while
+```
+
+The visited set deduplicates work; the queue threads frontier
+nodes; the neighbours map (a hashtable) supplies the graph.
+
+[sets]: https://docs.factorcode.org/content/vocab-sets.html

concepts/hash-sets/introduction.md

@@ -0,0 +1,6 @@
+# Introduction
+
+Hash-sets are mutable, unordered collections that store each
+value at most once. Lookup, insert, and delete are all O(1)
+average. They're the natural choice for "have I seen this
+before?" — the canonical *visited set* of graph traversals.

concepts/hash-sets/links.json

@@ -0,0 +1,10 @@
+[
+  {
+    "url": "https://docs.factorcode.org/content/vocab-sets.html",
+    "description": "sets vocabulary reference"
+  },
+  {
+    "url": "https://docs.factorcode.org/content/vocab-hash-sets.html",
+    "description": "hash-sets vocabulary reference"
+  }
+]

concepts/higher-order-sequences/about.md

@@ -23,6 +23,7 @@ The full common cast:
 | `filter`     | `( seq quot -- newseq )`                   |
 | `reject`     | `( seq quot -- newseq )`                   |
 | `find`       | `( seq quot -- i/f elt/f )`                |
+| `find-last`  | `( seq quot -- i/f elt/f )`                |
 | `reduce`     | `( seq init quot -- result )`              |
 | `count`      | `( seq quot -- n )`                        |
 | `any?`       | `( seq quot -- ? )`                        |
@@ -33,6 +34,12 @@ The full common cast:
 | `infimum`    | `( seq -- elt )`                           |
 | `supremum`   | `( seq -- elt )`                           |
 
+When *your* word forwards a runtime quotation to one of these
+combinators, declare your word with `; inline` so the
+combinator's effect inference can see the quotation's shape at
+the call site. Words built only from literal quotations don't
+need it.
+
 Beyond the core sequence ops:
 
 | vocab              | provides                                      |

concepts/numbers/about.md

@@ -18,9 +18,11 @@ Most arithmetic lives in [`math`][math]: `+`, `-`, `*`, `/`, `mod`,
 `even?`, `odd?`, `positive?`, `negative?`. Comparison: `<`, `<=`,
 `>`, `>=`, `=`. From [`math.order`][math.order]: `min`, `max`,
 `between?`. From [`math.functions`][math.functions]: `floor`,
-`ceiling`, `round`. Constructors and conversions like `>integer`,
-`>float`, `>fraction` are nearby.
+`ceiling`, `round`, `divisor?`. From [`math.primes`][math.primes]:
+`prime?`. Constructors and conversions like `>integer`, `>float`,
+`>fraction` are nearby.
 
 [math]: https://docs.factorcode.org/content/vocab-math.html
 [math.order]: https://docs.factorcode.org/content/vocab-math.order.html
 [math.functions]: https://docs.factorcode.org/content/vocab-math.functions.html
+[math.primes]: https://docs.factorcode.org/content/vocab-math.primes.html

concepts/reductions/about.md

@@ -17,15 +17,32 @@ USING: math sequences ;
 ```
 
 A custom seed lets `reduce` express folds that `sum` or `product`
-cannot. For example, applying a per-step floor:
+cannot. For example, the largest value in a sequence with a
+default if every element loses:
 
 ```factor
-100 { 50 -200 30 } [ + 0 max ] reduce .   ! => 30
+USING: math.order ;
+
+{ 3 1 -4 5 -2 } 0 [ max ] reduce .   ! => 5
+{ -3 -1 -4 }    0 [ max ] reduce .   ! => 0
+```
+
+The seed `0` participates in the comparison, so an all-negative
+sequence still produces `0` rather than its smallest value.
+
+## `produce` — the unfold
+
+The dual of `reduce` is `produce` (in [`sequences`][sequences]):
+where `reduce` *consumes* a sequence, `produce` *generates*
+one. A predicate quotation tests the running state; while it
+returns truthy, a body quotation produces the next element.
+
+```
+produce ( pred quot -- seq )
 ```
 
-(`100 + 50 = 150`; `150 + (-200) = -50`, floored to `0`;
-`0 + 30 = 30`.) The floor at each step matters — `sum +` would
-produce `-20`.
+Used together, `reduce` and `produce` form Factor's fold/unfold
+pair.
 
 ## Cumulative reductions
 
@@ -50,7 +67,9 @@ USING: math.statistics ;
 ```
 
 Cumulative reductions are useful when you want to inspect *how*
-a quantity evolved across the sequence, not only its final value
-— running balances, peak watermarks, low watermarks, and so on.
+a quantity evolved across the sequence, not only its final
+value. Chaining them is also useful: the output of one
+cumulative is itself a sequence, ready to feed into another.
 
 [math.statistics]: https://docs.factorcode.org/content/vocab-math.statistics.html
+[sequences]: https://docs.factorcode.org/content/vocab-sequences.html

concepts/sequences/about.md

@@ -9,7 +9,12 @@ with the others.
 |-------------|-----------------------------------|
 | `length`    | `( seq -- n )`                    |
 | `first`     | `( seq -- elt )`                  |
+| `second`    | `( seq -- elt )`                  |
+| `third`     | `( seq -- elt )`                  |
+| `fourth`    | `( seq -- elt )`                  |
 | `last`      | `( seq -- elt )`                  |
+| `rest`      | `( seq -- tailseq )`              |
+| `but-last`  | `( seq -- headseq )`              |
 | `nth`       | `( n seq -- elt )` (0-based)      |
 | `head`      | `( seq n -- headseq )`            |
 | `tail`      | `( seq n -- tailseq )`            |
@@ -20,12 +25,16 @@ with the others.
 | `unclip`    | `( seq -- rest first )`           |
 | `unclip-last` | `( seq -- butlast last )`       |
 | `empty?`    | `( seq -- ? )`                    |
+| `if-empty`  | `( seq emptyquot nonemptyquot -- … )` |
 | `member?`   | `( elt seq -- ? )`                |
 | `reverse`   | `( seq -- newseq )`               |
 | `index`     | `( elt seq -- i/f )`              |
 | `concat`    | `( seqs -- seq )`                 |
 | `sum`       | `( seq -- n )`                    |
 | `product`   | `( seq -- n )`                    |
+| `find`      | `( seq quot -- i/f elt/f )`       |
+| `find-last` | `( seq quot -- i/f elt/f )`       |
+| `produce`   | `( pred quot -- seq )`            |
 
 Arrays are immutable; the `prefix`/`suffix`/`append` operations all
 return new sequences without modifying the original. Vectors are

concepts/windows/.meta/config.json

@@ -0,0 +1,6 @@
+{
+  "authors": [
+    "keiravillekode"
+  ],
+  "blurb": "View a sequence as windows: disjoint chunks, sliding windows, predicate breaks, and adjacent-pair runs."
+}

concepts/windows/about.md

@@ -0,0 +1,44 @@
+# About
+
+Four words from [`grouping`][grouping] and
+[`splitting`][splitting] cover almost every "cut a sequence into
+pieces" need:
+
+```factor
+USING: grouping kernel sequences splitting splitting.monotonic ;
+
+! Disjoint chunks of N elements
+{ 1 2 3 4 5 6 7 } 3 group .
+! => { { 1 2 3 } { 4 5 6 } { 7 } }
+
+! Overlapping sliding window of N elements
+{ 1 2 3 4 5 } 2 clump .
+! => { { 1 2 } { 2 3 } { 3 4 } { 4 5 } }
+
+! Split wherever a predicate fires on an element
+{ 1 2 0 3 4 0 5 } [ zero? ] split-when .
+! => { { 1 2 } { 3 4 } { 5 } }
+
+! Group runs where adjacent elements are "the same" by some test
+{ 1 1 2 2 2 3 } [ = ] monotonic-split .
+! => { { 1 1 } { 2 2 2 } { 3 } }
+```
+
+| word              | effect                                                                  |
+|-------------------|-------------------------------------------------------------------------|
+| `group`           | `( seq n -- groups )` — disjoint *n*-sized chunks                       |
+| `clump`           | `( seq n -- clumps )` — overlapping *n*-sized windows                   |
+| `split-when`      | `( seq quot: ( elt -- ? ) -- pieces )` — break when quot is truthy      |
+| `monotonic-split` | `( seq quot: ( a b -- ? ) -- pieces )` — break when *adjacent* quot is `f` |
+
+A useful mental model:
+
+- **`group` / `clump`** look at sizes; the *n* parameter says
+  how big each window is.
+- **`split-when`** looks at *single elements*; the predicate
+  decides whether each element is a "break."
+- **`monotonic-split`** looks at *pairs*; the predicate decides
+  whether two adjacent elements still belong in the same run.
+
+[grouping]: https://docs.factorcode.org/content/vocab-grouping.html
+[splitting]: https://docs.factorcode.org/content/vocab-splitting.html

concepts/windows/introduction.md

@@ -0,0 +1,7 @@
+# Introduction
+
+Sometimes you need to view a sequence as a series of *pieces*
+rather than element-by-element. Factor's grouping and splitting
+words give you four ways to do that: fixed-size disjoint chunks,
+overlapping sliding windows, predicate-driven breaks, and
+adjacent-pair "run" grouping.

concepts/windows/links.json

@@ -0,0 +1,10 @@
+[
+  {
+    "url": "https://docs.factorcode.org/content/vocab-grouping.html",
+    "description": "grouping vocabulary reference"
+  },
+  {
+    "url": "https://docs.factorcode.org/content/vocab-splitting.html",
+    "description": "splitting vocabulary reference"
+  }
+]