Localise every hotlinked image, and re-encode to stay under the artifact limit

AGENTS.md

@@ -775,6 +775,24 @@ should catch a mistake at PR time rather than letting it fail silently.
   catches the near-miss of copying `template/post.md` to `draft/<slug>.md`
   instead of `draft/<slug>/index.md`, and leftover template placeholder text.
 
+  **`//JAVA 21+`, and NOT the 17 it declared for months.** `TITLE_EMOJI` uses
+  `\p{IsExtended_Pictographic}`, and Java's Unicode emoji properties -- that one
+  and the `\p{IsEmoji}` its comment rejects -- only arrived in JDK 21. On 17 the
+  pattern does not fail at the line that uses it: it throws
+  `PatternSyntaxException` from the static initialiser, so the whole check dies
+  before a single rule runs. jbang honours the `//JAVA` line, so declaring 17
+  provisioned exactly the JDK that cannot compile the pattern.
+
+  **It stayed hidden because this script only runs on a PULL REQUEST**, and
+  pushes to `main` go through `build-deploy.yml`, which runs
+  `validate/BuiltSite.java` and not this. Frank pushes most fixes straight to
+  `main`, so the first PR in a long while (#59) was the first thing to execute
+  it in CI -- passing locally the whole time on JDK 26. Reproduced and fixed
+  deliberately rather than by guesswork: `jbang --java 17` crashes,
+  `--java 21` passes. Checked the other 19 `//JAVA 17+` scripts for the same
+  property; none uses it, and `transfer/Posts.java`, which shares the character
+  class by design, already declared 21+.
+
   This is why **`.github/PULL_REQUEST_TEMPLATE.md` is not a checklist.** It was
   one, unchanged since the scaffold commit, and every item had rotted: it asked
   for `tags` (no such taxonomy), for images under `static/images/` with absolute
@@ -1458,6 +1476,123 @@ should catch a mistake at PR time rather than letting it fail silently.
   **Keep the substitution list in step with `images.py`** — a new renaming pass
   there silently reverts on the next re-scrape otherwise, and nothing reports it.
 
+  **`rewrite_in_bundle` matches WHOLE FILENAMES, and a plain `str.replace`
+  corrupted two posts before it did.** Its docstring used to argue the
+  replacement was safe because "the name is distinctive". It is not: one bundle
+  holds both `dummies.png` and `image-764x1024-dummies.png`, so converting the
+  first rewrote the SUFFIX of the second and pointed its hero at
+  `image-764x1024-dummies.jpg`, a file nothing ever wrote.
+  `add-mnemonic-bookmark.png` went the same way. **This was latent long before it
+  fired** — any earlier run could have hit it — and nothing reported it, because
+  Hugo publishes a bundle resource that no longer matches a reference without a
+  word. `validate/BuiltSite.java` is what caught it, and it caught it *eight
+  times over*: a hero is the card thumbnail on every listing page the post
+  appears on. The guard is a lookbehind that refuses a match preceded by another
+  filename character, which still matches all four reference shapes. **Audit
+  every local image reference after any bulk image pass** — a reference to a file
+  that is not there is invisible in a build log.
+
+- **EVERY IMAGE A POST REFERENCES IS PULLED LOCAL, whatever host serves it, and
+  that is a change of policy rather than a tidy-up.** `HtmlToMarkdown.localizeImages`
+  used to localise only foojay-hosted images, on the reasoning that a third-party
+  URL "keeps working after cutover". The audit says otherwise: **393 posts
+  reference 1710 images on 115 hosts, and 129 of them across 46 posts are dead
+  now** -- a broken glyph mid-article on the live site today. A host nobody here
+  controls is exactly the one that stops serving without telling us; hotlinking
+  also spends a stranger's bandwidth on every page view, leaks each reader's IP
+  to whoever owns the host, and puts 455 MB of pictures outside anything this
+  repo can measure or shrink.
+
+  Two halves, and they agree on the filename by construction:
+  the scraper (so a re-scrape produces local files and nothing needs freezing)
+  and **`scripts/cleanup/external_images.py`**, the one-off sweep over what is
+  already in `content/`.
+
+  **AN EXTERNAL IMAGE IS STORED AS `<stem>-<8 hex of its URL>.<ext>`, and the
+  hash is not decoration.** Third-party basenames collide constantly: **58
+  bundles reference an external URL whose basename is ALREADY the name of a file
+  sitting in that bundle** (`lambda.gif`, `layers.png`, `banner-1.png`). Without
+  the hash, `Files.exists` short-circuits and the reference silently displays a
+  picture from a different source -- the one failure this store cannot have,
+  because nothing downstream would report it. With it, "the file exists" provably
+  means "this URL was already downloaded". A **foojay-hosted** image keeps its
+  bare basename exactly as before, so not one byte of what is already in
+  `content/` moves (verified: `twitter-362x510.png`, not a hashed name).
+
+  Four more behaviours are load-bearing:
+  - **The lookup is by STEM, ignoring the extension** (`findByStem`), which does
+    two jobs: it finds a file `images.py` later re-encoded (`.png` -> `.jpg`)
+    instead of re-downloading it, and it makes a re-run free for the **221
+    external URLs that carry no extension at all** -- Medium, Hashnode and Google
+    user-content serve images from extension-less paths, so the format is only
+    known from the response's `Content-Type`. Same rule as
+    `transfer/Authors.java`'s avatar matching, which is why 203 converted avatars
+    already survive a re-scrape.
+  - **What came back has to BE an image.** A dead third-party URL rarely 404s
+    cleanly -- it serves a login wall, a "post not found" page or a placeholder,
+    all of them HTML with a 200. Writing that as `foo.png` turns a visibly broken
+    image into a file that is broken for ever and *looks* localised.
+  - **WordPress's emoji images are restored to the CHARACTER**, not localised.
+    WP rewrites an emoji an author typed into `<img
+    src="s.w.org/images/core/emoji/.../1f680.svg" alt="🚀">`, and the alt is the
+    character itself -- so the original is recoverable exactly. 23 of these across
+    eight bundles become text instead of 23 downloaded SVGs, which is also what
+    this file already says a body should hold (`stripEmoji` takes emoji out of
+    titles only).
+  - **A GitHub Actions `badge.svg` stays hotlinked** (5 in `content/`): it reports
+    whether a build passes *now*, so a frozen copy would assert a stale CI result
+    for ever. That is the whole exclusion list. The 15 `mermaid.ink` diagrams ARE
+    localised -- 9 of them are already dead, which is the argument for it rather
+    than against, though the real fix there is a ```mermaid fence, which this site
+    renders itself.
+
+  **The sweep shrinks BEFORE the file enters the repo, and that ordering is the
+  point.** The originals average 330 KB and reach 15.5 MB; 455 MB of them would
+  put the built site past GitHub Pages' 1 GB artifact limit, whose warning arrives
+  on a run that is otherwise GREEN. Downloading into `content/` and shrinking
+  afterwards would also leave the full-size blob in git history for ever -- and
+  nothing here rewrites history. So the download lands in a temp directory, is
+  re-encoded there with `cleanup/images.py`'s own measured rules (imported, not
+  re-derived), and only the result is moved in. It also finishes each bundle
+  before starting the next, which is `images.py`'s first hard-won lesson, and it
+  leaves a reference alone and REPORTS it when a fetch fails -- a 403 is a bot
+  wall, not proof of deletion, so those stay hotlinked rather than being rewritten
+  to a file that is not there.
+
+  **DONE, all 393 posts: external image references went from 1710 to 212**, and
+  the 212 are the unrecoverable remainder rather than a backlog -- 44 dead
+  (404/400), 9 behind a bot wall (403, kept because a 403 is about the client),
+  4 mermaid.ink 503s, 3 connection failures, and the 5 live CI badges. Every one
+  is named with its post in the script's own report. 6257 local image references
+  resolve; the only 3 that do not are prose EXAMPLES that never had a file
+  (`diagram.png` in the submit guide, `ai-generated.jpg` in a code sample).
+
+  **THE IMPORT COST ALL THE HEADROOM, AND THAT IS THE PART TO REMEMBER.** 216 MB
+  of images on top of a 773 MB build is **0.99 GB** -- against a 1 GB limit whose
+  warning arrives on a run that is otherwise green. Three things bought it back,
+  in the order they were worth doing:
+
+  1. **48 images were being stored TWICE** (13.2 MB). The URL hash correctly
+     refused to reuse a same-named file, but where a bundle already held foojay's
+     copy of the same picture the sweep fetched the author's copy as well. They
+     are de-duplicated on *identical name plus identical pixel dimensions inside
+     one bundle* -- a strong enough signal, where the name alone was not.
+  2. **`--png-min` dropped from 300 KB to 100 KB, site-wide** (Frank's call, with
+     before/after crops at reading size). 983 of 1263 candidate PNGs converted;
+     content media 798 -> 685 MB. The other 280 kept their PNG for real
+     transparency or because JPEG did not win by 10%. **That floor is what makes
+     the policy self-selecting**: a flame graph -- 58 colours, small labels, i.e.
+     JPEG's worst case -- was measured and DECLINED, because flat line art
+     genuinely compresses better as PNG.
+  3. Two posts had an animated REMOTE hero. Localising made them visible to
+     `checkHeroImageStill`, which failed them immediately; `images.py` wrote still
+     posters. That check was always right and simply could not see a hotlink.
+
+  Result: **864 MB, 136 MB of headroom.** 40 of the imported GIFs legitimately
+  refuse to shrink -- WebP cannot beat them by 10%, exactly as `convert_gif`'s own
+  comment predicts for flat-colour UI recordings where GIF's palette+LZW is
+  efficient. That is 28.8 MB that stays.
+
 - **Idempotency everywhere**: any script touching `content/` must be safe
   to re-run without duplicating or destroying hand edits (the `frozen: true`
   flag pattern). This matters because these scripts get re-run repeatedly

content/ads/foojay-sustainability-ebook/index.md

@@ -3,7 +3,7 @@ title: "Free eBook: Sustainability for Java Developers"
 description: "Towards an understanding of sustainable software development, written by and for the friends of OpenJDK"
 link: "https://leanpub.com/sustainabilityforjavadevelopers"
 cta: "Get it on Leanpub"
-image: "sustainability-ebook.png"
+image: "sustainability-ebook.jpg"
 background: "#29184e"
 
 # A second button, because this banner has somewhere on foojay to send a reader

content/authors/aicha-laafia/_index.md

@@ -1,7 +1,7 @@
 ---
 title: "Aicha Laafia"
 avatar: "aicha-laafia.png"
-avatarFull: "aicha-laafia-full.png"
+avatarFull: "aicha-laafia-full.jpg"
 bio: "Java Developer ☕ | Green Tech Advocate 🌍 | WomenTechmakers Ambassador | Oracle Ace Associate | JetBrains Community Contributor | International Speaker & Writer"
 bluesky: "https://bsky.app/profile/aichalaafia.bsky.social"
 mastodon: ""

content/authors/geertjan-wielenga/_index.md

@@ -1,7 +1,7 @@
 ---
 title: "Geertjan Wielenga"
 avatar: "geertjan-wielenga.png"
-avatarFull: "geertjan-wielenga-full.png"
+avatarFull: "geertjan-wielenga-full.jpg"
 bio: "Geertjan is an open source and Java enthusiast, Java Champion, Apache Member, and leader of the Amsterdam Java User Group."
 bluesky: ""
 mastodon: ""

content/authors/gilbert-kwan/_index.md

@@ -1,7 +1,7 @@
 ---
 title: "Gilbert Kwan"
 avatar: "gilbert-kwan.png"
-avatarFull: "gilbert-kwan-full.png"
+avatarFull: "gilbert-kwan-full.jpg"
 bio: "Advisory Software Developer at IBM"
 bluesky: ""
 mastodon: ""

content/authors/hardiksinghbehl/_index.md

@@ -1,7 +1,7 @@
 ---
 title: "Hardik Singh Behl"
 avatar: "hardiksinghbehl.png"
-avatarFull: "hardiksinghbehl-full.png"
+avatarFull: "hardiksinghbehl-full.jpg"
 bio: "Backend developer | Technical writer"
 bluesky: ""
 mastodon: ""

content/authors/marit-van-dijk/_index.md

@@ -1,7 +1,7 @@
 ---
 title: "Marit van Dijk"
 avatar: "marit-van-dijk.png"
-avatarFull: "marit-van-dijk-full.png"
+avatarFull: "marit-van-dijk-full.jpg"
 bio: "Marit van Dijk is a Java Champion and Developer Advocate at JetBrains with over 20 years of software development experience. She's passionate about building great software with great people, and making developers' lives easier. Marit regularly presents at international conferences and shares her expertise through webinars, podcasts, blog posts, videos, and tutorials. She's also a contributor to the book \"97 Things Every Java Programmer Should Know\" (O’Reilly Media)."
 bluesky: "https://bsky.app/profile/maritvandijk.bsky.social"
 mastodon: "https://mastodon.social/@maritvandijk"

content/authors/mohibul-chowdhury/_index.md

@@ -1,7 +1,7 @@
 ---
 title: "Mohibul Hassan Chowdhury"
 avatar: "mohibul-chowdhury.png"
-avatarFull: "mohibul-chowdhury-full.png"
+avatarFull: "mohibul-chowdhury-full.jpg"
 bio: "Software Engineer. Aiming to become Software Craftsman. Java | Spring boot | Javascript | VueJS | NuxtJS | MySQL An AI enthusiast. Avid self learner."
 bluesky: ""
 mastodon: ""

content/authors/nataliia/_index.md

@@ -1,7 +1,7 @@
 ---
 title: "Nataliia Dziubenko"
 avatar: "nataliia.png"
-avatarFull: "nataliia-full.png"
+avatarFull: "nataliia-full.jpg"
 bio: "Senior Software Engineer at Azul"
 bluesky: ""
 mastodon: ""

content/authors/scott-sellers/_index.md

@@ -1,7 +1,7 @@
 ---
 title: "Scott Sellers"
 avatar: "scott-sellers.png"
-avatarFull: "scott-sellers-full.png"
+avatarFull: "scott-sellers-full.jpg"
 bio: "President & CEO at Azul Systems, with more than 30 years of leading high-technology companies. Before Azul, Scott co-founded 3dfx Interactive, helping pioneer 3D graphics for PCs and game consoles. After 3dfx’s IPO, the company was later acquired by NVIDIA."
 bluesky: ""
 mastodon: ""

content/authors/wei-lin/_index.md

@@ -1,7 +1,7 @@
 ---
 title: "Wei Lin"
 avatar: "wei-lin.png"
-avatarFull: "wei-lin-full.png"
+avatarFull: "wei-lin-full.jpg"
 bio: "Wei Lin is an ed-tech entrepreneur. He has been teaching Java to students of various ages and backgrounds, and decided that interactive autograded labs are the best way to learn programming."
 bluesky: ""
 mastodon: ""

content/posts/2020/04/25/foojay-a-place-for-friends-of-openjdk/index.md

@@ -24,7 +24,7 @@ Welcome to foojay, a place for **f** riends **o** f **O** pen**J** DK, sponsored
 #### Java Technologist Dashboards
 
 Together with Java enthusiasts around the world, the foojay team works to identify critical aspects of each new OpenJDK release and update -- with related information such as OpenJDK distributions, download locations, and JVM command line arguments -- and brings to the fore precisely the content and services that have value and relevance to those that use Java on a daily basis throughout the industry.
-![](Screenshot-2020-08-20-at-17.19.43-1024x647.png) OpenJDK Update Release Details
+![](Screenshot-2020-08-20-at-17.19.43-1024x647.jpg) OpenJDK Update Release Details
 
 [Go here to see all the fixes and CVEs of the recent OpenJDK update releases](https://javaalmanac.io/jdk/8/), while indicating their relevance to you by voting on them. Use the Highlights tab to see what the community as a whole thinks about them. More details on the motivations for the OpenJDK update release details dashboard [are described here on foojay](https://foojay.io/today/dashboard-for-openjdk-update-release-details/).  
 
@@ -33,7 +33,7 @@ Together with Java enthusiasts around the world, the foojay team works to identi
 A key contributor to foojay is [Marc Hoffmann](https://twitter.com/marcandsweep). On his [javaalmanac.io](http://javaalmanac.io), he presents extensive data on each Java release, with details on distributions, new features, and API differences between releases.
 
 That information he's also made available here on foojay, where it is known as the Java Version Almanac, consisting of a set of dashboards with detailed information on each version of Java:
-![](Screenshot-2020-08-21-at-09.18.59-1024x633.png)
+![](Screenshot-2020-08-21-at-09.18.59-1024x633.jpg)
 
 Take special note at the lower end of each page of the almanac, where you'll find details on the differences between the APIs of the currently selected Java version compared to all previous Java versions:
 ![](Screenshot-2020-08-21-at-09.19.51-1024x525.png)
@@ -45,7 +45,7 @@ Take special note at the lower end of each page of the almanac, where you'll fin
 Another key contributor to foojay is [Chris Newland](https://twitter.com/chriswhocodes), who's gathered extensive data sets on JVM command line arguments, which he hosts on his own [chriswhocodes.com](http://chriswhocodes.com).   
 
 At the same time, he's also been making his data available here as part of the integrated Java dashboard environment that is foojay:
-![](Screenshot-2020-08-20-at-17.23.09-1024x639.png) OpenJDK Command Line Arguments
+![](Screenshot-2020-08-20-at-17.23.09-1024x639.jpg) OpenJDK Command Line Arguments
 
 [Go here to explore the wealth of JVM command line arguments](https://chriswhocodes.com/hotspot_options_openjdk11.html), with thanks to Chris.
 

content/posts/2020/05/01/dashboard-for-openjdk-update-release-details/index.md

@@ -5,7 +5,7 @@ lastmod: "2021-08-23T12:15:26+00:00"
 description: "Foojay is a one stop solution for everything related to Java. The Dashboard offers analysis, highlights, and categorized lists of updates by Java users."
 authors:
   - "geertjan-wielenga"
-image: "Screenshot-2020-08-20-at-17.19.43-1-1024x647.png"
+image: "Screenshot-2020-08-20-at-17.19.43-1-1024x647.jpg"
 categories:
   - "Release Notes"
 related_posts:
@@ -41,7 +41,7 @@ A great example of this difference in perspective can be found in [JDK-8146115](
 Going forward, a clearer perspective of what changes a Java update contains is now provided at [foojay.io](https://foojay.io/). Foojay's user-focused Java and OpenJDK update descriptions offer a dashboard with updated analysis, selected highlights, and categorized lists of updates arranged for consumption by Java users.
 
 Working together with Java enthusiasts around the world, the foojay team works to identify critical aspects of each new OpenJDK update when it is released and bring to the fore precisely the issues that have value and relevance to those that use Java throughout the industry.
-![](Screenshot-2020-08-20-at-17.19.43-1-1024x647.png)
+![](Screenshot-2020-08-20-at-17.19.43-1-1024x647.jpg)
 
 For example, in the April 2020 update, foojay chose to highlight two P4 issues that might otherwise not be given the prominence they deserve and be lost in the noise:
 

content/posts/2020/07/31/from-azure-active-directory-via-openid-connect-to-open-liberty-and-java/index.md

@@ -5,7 +5,7 @@ lastmod: "2021-08-23T12:30:34+00:00"
 description: "Let's look at an example of how to configure the Liberty social login feature as OpenID Connect client to secure Java apps with Azure Active Directory."
 authors:
   - "m-reza-rahman"
-image: "https://dzone.com/storage/temp/13800230-1596136325278.png"
+image: "dzone-com-afac1be9.png"
 categories:
   - "Azure"
 related_posts:
@@ -100,7 +100,7 @@ The relevant configuration in `web.xml`:
 ```
 
 This is just standard Java EE security. The authentication and authorization workflow is shown in the following diagram.
-![](https://dzone.com/storage/temp/13800230-1596136325278.png)
+![](dzone-com-afac1be9.png)
 
 *Picture 1: OpenID Connect sign-in and token acquisition flow, from* [*Microsoft identity platform and OpenID Connect protocol*](https://docs.microsoft.com/azure/active-directory/develop/v2-protocols-oidc#protocol-diagram-access-token-acquisition)