Update dependency org.apache.tika:tika-core to v4 - #4853
Conversation
|
Seems early enough that we can move to tika 4? |
|
Commonmark for Markdown parsing is now part of tika-core. Our extraction handler currently does not rely on MD, but explicitly requests xml from tika-server. So we could attempt excluding these three new jar deps and see if it will fly. There may be some part of core depending on these classes even if we don't use them explicitly so maybe not worth the risk? |
|
@tballison somebody moved my cheese ;-). |
Edited/Blocked NotificationRenovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR. You can manually request rebase by checking the rebase/retry box above. |
…xml. This is mostly a "hey, look, I got us to tika 4", but it feels like its still trying to push Tika 4 into a tika 3 approach, when Tika wants to be different.
I'd say we stay on 3.x for Solr 10, and work on exploring how to best utilize Tika 4 on main only, i.e. no backport. to 10.x. Of course, if we get to a stable back-compat impl of Tika 4 on main, it can be back-ported without risk. Even if Tika gains features like AI stuff, we are not obliged to use those :-) |
|
On phone with @epugh now, lol. Let me know if I can help, or if I've done enough damage. 🤣 |
|
Umm... at least locally the bats test and unit tests passed!!! So maybe this isn't the lift I thought it was? |
That works for me... As @tballison pointed out, it's literally a |
There was a problem hiding this comment.
🟡 Changes recommended
License validation will fail on a stale checksum, while error and partial-extraction handling can produce incorrect request outcomes.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Upgrades Solr Cell to Tika 4 and adapts its server integration to Tika’s new endpoints, metadata keys, and per-request configuration.
Changes:
- Updates Tika and Docker images to 4.0.0, including dependencies and licenses.
- Adds multipart parser configuration and password handling.
- Updates extraction tests and documentation for Tika 4 behavior.
File summaries
| File | Description |
|---|---|
gradle/libs.versions.toml |
Updates Tika to 4.0.0. |
solr/modules/extraction/gradle.lockfile |
Locks Tika and CommonMark dependencies. |
TikaServerExtractionBackend.java |
Implements Tika 4 request and response handling. |
TikaServerParser.java |
Supports renamed recursive metadata content keys. |
ExtractionRequest.java |
Adds per-request Tika configuration. |
ExtractingParams.java |
Defines the new configuration parameter. |
ExtractingDocumentLoader.java |
Passes configuration from request parameters. |
TikaServerExtractionBackendTest.java |
Tests configuration, passwords, and Tika 4 behavior. |
TikaServerContainerRule.java |
Runs the Tika 4 container with optional configuration. |
ExtractingRequestHandlerTikaServerTest.java |
Enables per-request configuration in integration tests. |
ExtractingRequestHandlerTestAbstract.java |
Updates expected Tika metadata keys. |
tika-server-config.json |
Enables test-time per-request configuration. |
test_extraction.bats |
Updates the packaging test container. |
indexing-with-tika.adoc |
Documents Tika 4 and parser configuration. |
tutorial-diy.adoc |
Updates the tutorial container version. |
tika-core-4.0.0.jar.sha1 |
Adds the Tika checksum. |
commonmark-LICENSE-BSD.txt |
Adds the CommonMark license. |
commonmark-NOTICE.txt |
Adds the CommonMark notice file. |
commonmark-0.30.0.jar.sha1 |
Adds the CommonMark checksum. |
commonmark-ext-gfm-tables-0.30.0.jar.sha1 |
Adds the tables extension checksum. |
commonmark-ext-gfm-strikethrough-0.30.0.jar.sha1 |
Adds the strikethrough extension checksum. |
PR#4853-update-org-apache-tika-tika-core.yml |
Records the dependency update. |
Review details
- Files reviewed: 21/22 changed files
- Comments generated: 5
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| @@ -0,0 +1 @@ | |||
| f370d8619de9e19329b2cc117943c83468c2ff9f | |||
| if (code == 422 && !request.tikaServerRecursive) { | ||
| PushbackInputStream peekable = new PushbackInputStream(responseStream, 1); | ||
| int firstByte = peekable.read(); | ||
| if (firstByte == -1) { | ||
| throw new SolrException( |
| .tikaServerRecursive(tikaserverRecursive) | ||
| .tikaServerTimeoutSeconds(tikaTimeoutSecs) | ||
| .tikaServerRequestHeaders(Map.of()) | ||
| .tikaServerConfigJson(params.get(ExtractingParams.TIKASERVER_CONFIG_JSON)) |
| * have {@code allowPerRequestConfig=true} set, or the request is rejected with 403. Ignored for | ||
| * recursive (tikaserver.recursive) requests, since TikaServer has no XML-output variant of | ||
| * /rmeta/config. |
| // Tika 4.x renamed its metadata keys under a single lowercase tk: prefix (TIKA-4816) | ||
| "fmap.tk:parsed-by", | ||
| "ignored_parser", | ||
| "fmap.X-TIKA:Parsed-By", | ||
| "fmap.tk:parsed-by-full-set", | ||
| "ignored_parser", |
This PR contains the following updates:
3.3.1→4.0.0Release Notes
apache/tika (org.apache.tika:tika-core)
v4.0.0Compare Source
v3.3.2Compare Source
Configuration
📅 Schedule: (UTC)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Renovate Bot