fix(security): validate uploaded file types by content with Apache Tika in the AI plugin#42005
Open
wyattwalter wants to merge 8 commits into
Open
fix(security): validate uploaded file types by content with Apache Tika in the AI plugin#42005wyattwalter wants to merge 8 commits into
wyattwalter wants to merge 8 commits into
Conversation
… Tika The Appsmith AI plugin forwarded uploaded files to the AI server without any server-side content inspection, trusting the client-supplied filename and Content-Type. The allowed types (application/pdf, text/plain, text/markdown) existed only as a client-side "allowedFileTypes" hint in form.json, so a crafted request could upload a disallowed type such as a malicious SVG. Detect each uploaded file's true type from its content bytes with Apache Tika (tika-core; MimeTypes magic detection composed with TextDetector, instantiated directly so it stays robust once the plugin is shaded into an uber-jar) and reject any file whose real content type is not in a server-side allow-list. A spoofed extension or Content-Type no longer matters because magic detection wins. Files are buffered (BufferedFilePart) so they can still be forwarded upstream after inspection. No bespoke size/count caps or markup scanning are added; those belong in framework/proxy config. APP-15345 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…5-66516 Addresses the independent re-review of the AI-plugin upload validation. Finding 1 (bypass): Tika's text fallback reports SVG/HTML as text/plain when the markup does not begin at byte 0 (leading whitespace or a short text prefix), so a malicious SVG slipped through the allow-listed text/plain path. Add a narrow anti-smuggling check on the text/* branch only: after skipping a BOM and leading whitespace, scan the leading window for XML/HTML/SVG/script root markers (<svg, <?xml, <html, <!doctype html, <!doctype svg, <script) and reject if any is present. This is targeted, not the broad markup scan we deliberately avoided. Added regression tests for whitespace-prefixed SVG, SVG after a text prefix, markdown/text containing <script>/<html>, and whitespace-prefixed HTML - all rejected; genuine plain text and plain markdown still pass. Finding 2 (CVE): bump org.apache.tika:tika-core 3.2.1 -> 3.2.2, since 1.13-3.2.1 are affected by CVE-2025-66516 (XXE via crafted XFA in PDFs). Finding 3 (comment only, per captain): note that the 150MB multipart body limit is the size control for the buffering; no bespoke per-file/count cap. APP-15345 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…byte-scan Follow-up to the second re-review, which showed the byte-scan was the wrong control: UTF-16/UTF-32 SVG and markup padded past the 8KB scan window still slipped through as text/plain. Fix the root cause in detection instead. tika-core's bundled magic only recognises XML/SVG roots as ASCII bytes at offset 0, so an encoded or prolog-padded SVG fell through to the allow-listed text/plain. FileValidationUtils now does a two-pass, content-only detection: when the first pass is ambiguous (text/plain or octet-stream) it decodes the head from its real charset (BOM or null-pattern heuristic for UTF-16LE/BE and UTF-32), strips the BOM and XML-prolog whitespace, re-encodes as UTF-8 and re-detects. An encoded/padded SVG then surfaces as image/svg+xml and is rejected by the existing allow-list; a genuinely wide-encoded text file is accepted as text/plain; opaque binary stays rejected. This stays tika-core-only (its transitive commons-io is the only extra runtime jar - both are shaded in; verified), so no tika-parsers / ServiceLoader fragility. The containsSmuggledMarkup byte-scan and its markup markers/error message are removed entirely - the type allow-list now does the work. Empirically verified (probe + tests): UTF-8, UTF-16LE/BE (with and without BOM), UTF-32, namespace-prefixed and leading-whitespace SVG, and leading-whitespace HTML are all detected as image/svg+xml / text/html and REJECTED; genuine plain text, markdown, PDF and wide-encoded plain text are ACCEPTED. Keeps the tika-core 3.2.2 bump (CVE-2025-66516). APP-15345 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
WalkthroughUploaded files are buffered, size-limited, and validated using detected MIME content before reaching the AI server. Supported types, error messages, Tika detection, replayable file parts, encoding normalization, integration, and tests were added. ChangesUpload Validation
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant TriggerServiceCEImpl
participant FileValidationUtils
participant AiServerService
Client->>TriggerServiceCEImpl: upload FilePart list
TriggerServiceCEImpl->>FileValidationUtils: validateFileType(FilePart)
FileValidationUtils->>FileValidationUtils: buffer, size-check, and detect MIME type
FileValidationUtils-->>TriggerServiceCEImpl: validated BufferedFilePart
TriggerServiceCEImpl->>AiServerService: upload validated files
Suggested labels: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Follow-up to PR re-review: validateFileType joined the whole file into heap with no maxByteCount, bounded only by the global 150MB spring.webflux.multipart.max-in-memory-size (shared across every upload path, not scoped here, and not per-request). Pass a per-file byte cap to DataBufferUtils.join so the framework fails fast as the file is read instead of after it is fully buffered. The cap is 20 MiB, mirroring form.json's maxFileSizeInBytes (the limit the client already enforces) and well below the global ceiling - aligning server with client, the same way the type validation aligns with allowedFileTypes. join throws DataBufferLimitException on exceed; it is translated to an AppsmithPluginException with a clear message in the FILE_TYPE_NOT_SUPPORTED style, not a raw framework error. Added a test for the oversized-file rejection path. APP-15345 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to review: the previous oversized-file test was vacuous - a zero-filled MAX+1 buffer was rejected as octet-stream regardless of the cap (mutation-reverting join's maxByteCount still passed), and a single pre-allocated buffer never exercised join's mid-stream fail-fast. Replace it with three targeted tests (all in FileValidationUtilsTest): - oneByteOverSizeCap: 'a'-filled (valid text/plain) MAX+1 content, so only the size cap can reject it, and assert the exact FILE_TOO_LARGE message rather than any AppsmithPluginException. - withChunkedStreamExceedingCap: feed the content as many 1 MiB chunks from a Flux able to emit 3x the cap, assert size rejection AND that join cancelled the source near the cap (emitted <= capChunks + 2) - pinning real mid-stream fail-fast, not full buffering. - atExactlySizeCap: a file of exactly MAX is accepted, guarding the > vs >= boundary against an off-by-one. Verified by mutation: reverting join(content, MAX) to join(content) now fails 2 of these 3 tests. APP-15345 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
wyattwalter
marked this pull request as ready for review
July 17, 2026 21:42
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@app/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/utils/FileValidationUtils.java`:
- Around line 195-206: Update normalizeHeadToUtf8 to scan the bounded file
content for the first non-whitespace token rather than truncating the decoded
prefix and allowing an empty result. If the entire HEAD_DECODE_LIMIT window
contains only whitespace (including an optional BOM), fail closed by rejecting
or returning the existing invalid result; add a regression case covering more
than 64 KiB of leading whitespace before an SVG/XML root.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 4a0411d8-4196-4eb4-b30f-04e354e834e9
📒 Files selected for processing (7)
app/server/appsmith-plugins/appsmithAiPlugin/pom.xmlapp/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/constants/AppsmithAiConstants.javaapp/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/constants/AppsmithAiErrorMessages.javaapp/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/services/TriggerServiceCEImpl.javaapp/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/utils/BufferedFilePart.javaapp/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/utils/FileValidationUtils.javaapp/server/appsmith-plugins/appsmithAiPlugin/src/test/java/com/external/plugins/services/FileValidationUtilsTest.java
… head Addresses the CodeRabbit finding (CWE-20) on PR #42005: content-type detection only inspects the first 64 KiB, so an attacker can pad an SVG/HTML/XML file with >64 KiB of benign leading bytes to push the markup root past the detector; the head types as text/plain and the file is accepted, reopening the very spoofing bypass this PR closes. Add a fail-closed backstop: a file accepted as text/* is scanned in full (bounded by the existing per-file upload cap, so no new unbounded work) for markup roots (<svg, <?xml, <html, <!doctype, <script) and rejected if any is present. The scan is anchored on '<' (one comparison per benign byte) and, for wide encodings, collapses the bytes to single-byte first so encoded markup is still found. This does NOT regress legitimate large uploads: PDFs are detected by magic and never scanned (they may embed markup); a uniform large plain-text/markdown file contains no markup root and passes. A blanket "large file -> reject" was explicitly avoided. Tests (all in FileValidationUtilsTest): whitespace-padded SVG and non-whitespace- padded <script>, both past 64 KiB, are rejected with the FILE_CONTAINS_MARKUP error; >64 KiB uniform text, >64 KiB markdown, >64 KiB PDF, and a PDF embedding markup are all accepted. Mutation-verified: disabling the guard fails exactly the two padded-markup tests. APP-15345 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
app/server/appsmith-plugins/appsmithAiPlugin/src/test/java/com/external/plugins/services/FileValidationUtilsTest.java (1)
295-307: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCover padded markup in wide encodings.
The tests verify wide-encoded SVG and beyond-head markup separately, but not together. Add UTF-16/UTF-32 markup padded beyond the detection head to protect the full-buffer decoding path described by this PR.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/server/appsmith-plugins/appsmithAiPlugin/src/test/java/com/external/plugins/services/FileValidationUtilsTest.java` around lines 295 - 307, Extend the FileValidationUtilsTest padded-markup coverage by adding cases that combine UTF-16 and UTF-32 encoding with markup placed beyond OVER_HEAD_PAD. Use the existing padding and rejection helpers, and cover both SVG and script payloads as appropriate to verify the full-buffer decoding path rejects padded wide-encoded markup.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In
`@app/server/appsmith-plugins/appsmithAiPlugin/src/test/java/com/external/plugins/services/FileValidationUtilsTest.java`:
- Around line 295-307: Extend the FileValidationUtilsTest padded-markup coverage
by adding cases that combine UTF-16 and UTF-32 encoding with markup placed
beyond OVER_HEAD_PAD. Use the existing padding and rejection helpers, and cover
both SVG and script payloads as appropriate to verify the full-buffer decoding
path rejects padded wide-encoded markup.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 1edee018-1e4f-422f-a1b7-becf1e786297
📒 Files selected for processing (3)
app/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/constants/AppsmithAiErrorMessages.javaapp/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/utils/FileValidationUtils.javaapp/server/appsmith-plugins/appsmithAiPlugin/src/test/java/com/external/plugins/services/FileValidationUtilsTest.java
🚧 Files skipped from review as they are similar to previous changes (1)
- app/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/utils/FileValidationUtils.java
Follow-up review of the fail-closed guard (c3c954d): the substring scan rejected any accepted text/* containing <svg/<?xml/<html/<!doctype/<script anywhere, so legit markdown/plain-text that merely mentions markup (prose about HTML, code examples, "1 < 2") was false-rejected. Refine to document-root detection (reviewer Option 1) with tag-boundary matching: a file accepted as text/* is rejected only if, after stripping its leading BOM and whitespace/padding, the very first content is a markup root (<svg/<?xml/<html/ <!doctype/<script) followed by a tag boundary - i.e. it would actually parse/render as an SVG/HTML/XML document. Prose or a code example that mentions markup away from the root is not a renderable document and is accepted. The tag boundary stops "<scriptural" from matching "<script". Wide (UTF-16/32) content is collapsed first so an encoded document root is still caught; PDFs remain unchecked; work stays bounded by the per-file upload cap. Posture: - REJECT: whitespace-padded SVG/HTML/XML document whose root evades the 64 KiB head detector (the actual bypass), in any encoding. - ACCEPT: text/markdown that mentions/embeds markup in prose or code; PDF (incl. embedding markup); and - low-risk residual, now explicit - a file padded with non-whitespace bytes before its markup (types as text, not a renderable document; uploads are S3-stored, not app-origin served). Renamed FILE_CONTAINS_MARKUP -> FILE_IS_MARKUP_DOCUMENT. Tests updated: whitespace- padded SVG/HTML and wide-encoded padded SVG rejected; prose-mentioning-markup, markdown-embedding-markup-examples and non-whitespace-padded markup accepted; large text/markdown/PDF and PDF-embedding-markup still accepted. Mutation-verified. APP-15345 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@app/server/appsmith-plugins/appsmithAiPlugin/src/test/java/com/external/plugins/services/FileValidationUtilsTest.java`:
- Around line 313-317: Update
validateFileType_withWideEncodedWhitespacePaddedSvgDocument_isRejected to use
the exact-message assertion helper introduced earlier, verifying rejection
occurs through the markup-document guard rather than merely any rejection path.
Keep the UTF-16LE payload and padded SVG setup unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 392a07da-552c-462e-83a0-05e88f14fa82
📒 Files selected for processing (3)
app/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/constants/AppsmithAiErrorMessages.javaapp/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/utils/FileValidationUtils.javaapp/server/appsmith-plugins/appsmithAiPlugin/src/test/java/com/external/plugins/services/FileValidationUtilsTest.java
🚧 Files skipped from review as they are similar to previous changes (1)
- app/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/utils/FileValidationUtils.java
![hacktron-app[bot]](https://avatars.githubusercontent.com/in/1250513?s=60&v=4){kind=small}
| // any leading BOM/whitespace. Stored lowercase; matched case-insensitively and only at the document root | ||
| // with a following tag boundary, so a document survives whitespace padding but prose that merely mentions | ||
| // these strings does not trip the guard. | ||
| private static final List<String> MARKUP_ROOT_MARKERS = List.of("<svg", "<?xml", "<html", "<!doctype", "<script"); |
There was a problem hiding this comment.
The FileValidationUtils class implements a content-based validation mechanism to prevent the upload of malicious markup files (such as SVG, HTML, and XML) to mitigate Stored Cross-Site Scripting (XSS) and other file execution attacks. However, the custom validation implemented in isStructuredMarkupDocument is insufficient and can be trivially bypassed.
The validation relies on checking if the first non-whitespace characters of a file match a hardcoded list of markup root markers:
private static final List<String> MARKUP_ROOT_MARKERS = List.of("<svg", "<?xml", "<html", "<!doctype", "<script");
This approach is vulnerable to two primary bypass techniques:
- XML/SVG Comments Bypass: XML and SVG specifications allow comments (
<!-- ... -->) to appear before the root element. An attacker can pad an SVG file with a comment to push the SVG magic bytes past Apache Tika's head detection window (causing Tika to classify it astext/plain). SincefirstNonWhitespaceAfterBomstops at the first non-whitespace character (the<of<!--), and<!--does not match any of the hardcoded markers, the file is accepted. When rendered in a browser, the browser ignores the leading comment and executes any embedded scripts. - HTML Alternative Tags Bypass: A valid HTML document does not need to start with
<html>,<!doctype>, or<script>. It can start with any other HTML tag (e.g.,<div>,<body>,<iframe>,<img>). If the file is padded with text so Tika detects it astext/plain, the validator stops at the first non-whitespace character (e.g.,<of<div>), which does not match any marker, allowing the malicious HTML file to be uploaded and executed.
Steps to Reproduce
- Craft an SVG file where the root
<svg>tag is preceded by an XML comment (e.g.,<!-- comment --> <svg xmlns="http://www.w3.org/2000/svg" onload="alert(document.domain)"></svg>). - Alternatively, craft an HTML file starting with an alternative tag (e.g.,
<iframe src="javascript:alert(document.domain)"></iframe>). - Upload the crafted file via the Appsmith AI Plugin's file upload feature.
- Observe that the file bypasses the validation checks in
FileValidationUtilsand is successfully uploaded. - When the file is rendered or accessed in a browser, the embedded JavaScript executes.
# Create a malicious SVG file with comment padding to bypass both Tika and FileValidationUtils
cat << 'EOF' > exploit.svg
<!-- PADDING_TO_EXCEED_TIKA_HEAD_LIMIT_HERE... -->
<svg xmlns="http://www.w3.org/2000/svg" onload="alert(document.domain)"></svg>
EOF
# Create a malicious HTML file starting with an alternative tag to bypass FileValidationUtils
cat << 'EOF' > exploit.html
<iframe src="javascript:alert(document.domain)"></iframe>
EOFFix with AI
A security vulnerability was found by Hacktron.
File: app/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/utils/FileValidationUtils.java
Lines: 80
Severity: high
Vulnerability: Stored XSS and Malicious File Upload Bypass in FileValidationUtils
Description:
The `FileValidationUtils` class implements a content-based validation mechanism to prevent the upload of malicious markup files (such as SVG, HTML, and XML) to mitigate Stored Cross-Site Scripting (XSS) and other file execution attacks. However, the custom validation implemented in `isStructuredMarkupDocument` is insufficient and can be trivially bypassed.
The validation relies on checking if the first non-whitespace characters of a file match a hardcoded list of markup root markers:
`private static final List<String> MARKUP_ROOT_MARKERS = List.of("<svg", "<?xml", "<html", "<!doctype", "<script");`
This approach is vulnerable to two primary bypass techniques:
1. **XML/SVG Comments Bypass**: XML and SVG specifications allow comments (`<!-- ... -->`) to appear before the root element. An attacker can pad an SVG file with a comment to push the SVG magic bytes past Apache Tika's head detection window (causing Tika to classify it as `text/plain`). Since `firstNonWhitespaceAfterBom` stops at the first non-whitespace character (the `<` of `<!--`), and `<!--` does not match any of the hardcoded markers, the file is accepted. When rendered in a browser, the browser ignores the leading comment and executes any embedded scripts.
2. **HTML Alternative Tags Bypass**: A valid HTML document does not need to start with `<html>`, `<!doctype>`, or `<script>`. It can start with any other HTML tag (e.g., `<div>`, `<body>`, `<iframe>`, `<img>`). If the file is padded with text so Tika detects it as `text/plain`, the validator stops at the first non-whitespace character (e.g., `<` of `<div>`), which does not match any marker, allowing the malicious HTML file to be uploaded and executed.
Proof of Concept:
**Steps to Reproduce**
1. Craft an SVG file where the root `<svg>` tag is preceded by an XML comment (e.g., `<!-- comment --> <svg xmlns="http://www.w3.org/2000/svg" onload="alert(document.domain)"></svg>`).
2. Alternatively, craft an HTML file starting with an alternative tag (e.g., `<iframe src="javascript:alert(document.domain)"></iframe>`).
3. Upload the crafted file via the Appsmith AI Plugin's file upload feature.
4. Observe that the file bypasses the validation checks in `FileValidationUtils` and is successfully uploaded.
5. When the file is rendered or accessed in a browser, the embedded JavaScript executes.
```bash
# Create a malicious SVG file with comment padding to bypass both Tika and FileValidationUtils
cat << 'EOF' > exploit.svg
<!-- PADDING_TO_EXCEED_TIKA_HEAD_LIMIT_HERE... -->
<svg xmlns="http://www.w3.org/2000/svg" onload="alert(document.domain)"></svg>
EOF
# Create a malicious HTML file starting with an alternative tag to bypass FileValidationUtils
cat << 'EOF' > exploit.html
<iframe src="javascript:alert(document.domain)"></iframe>
EOF
```
Affected Code:
private static final List<String> MARKUP_ROOT_MARKERS = List.of("<svg", "<?xml", "<html", "<!doctype", "<script");
Acceptance criteria:
- Acceptance is defined by the **actual reported behavior**, not by tests passing.
- Reproduce the issue, or narrow the exact code path that produces it, *before* changing code. State what you confirmed.
- Fix the underlying cause. Mitigations that paper over the reported behavior do not count as a fix.
- Add a regression test that fails on the unpatched code and passes on the fix. If a regression test is genuinely impractical (e.g. race condition, infra-level issue), say so and explain why.
- Existing tests passing is **not** the bar. Do not declare done on tests-pass theatre.
Only change what is necessary to fix this vulnerability. Do not refactor adjacent code or modify unrelated files.
Triage: Reply !fp <reason> (false positive), !valid (confirmed), !accepted_risk <reason>, or !fixed (resolved). Any other reply is saved as a triage note.
Reason is optional but improves future scans — e.g. !fp internal endpoint, not user-facing.
Addresses CodeRabbit r3607077454: the UTF-16 padded-SVG test asserted only a generic rejection, but that payload never reaches the markup-document guard - the encoding-aware detection resolves an all-whitespace-padded wide document to application/octet-stream (and a near-start one to image/svg+xml), both rejected by the allow-list first. - Assert the exact path: FILE_TYPE_NOT_SUPPORTED with the detected type (application/octet-stream for the padded case, image/svg+xml for the near-start case), via a new expectTypeRejected helper, instead of a generic reject. - Simplify the guard accordingly: isStructuredMarkupDocument no longer takes/decodes a wide charset (that branch was dead for rejection, since a wide markup document can never reach the guard as text/*). It now inspects single-byte content only, with the wide case documented as handled by type detection. Empirically verified (tika-core 3.2.2): wide big-whitespace+<svg> -> octet-stream; wide small-whitespace+<svg> -> image/svg+xml. Non-wide behavior unchanged; module suite green (FileValidationUtilsTest 30 tests). APP-15345 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment on lines
+168
to
+218
| private static Charset detectWideCharset(byte[] b) { | ||
| int n = b.length; | ||
| if (n >= 4 && u(b[0]) == 0x00 && u(b[1]) == 0x00 && u(b[2]) == 0xFE && u(b[3]) == 0xFF) { | ||
| return Charset.forName("UTF-32BE"); | ||
| } | ||
| if (n >= 4 && u(b[0]) == 0xFF && u(b[1]) == 0xFE && u(b[2]) == 0x00 && u(b[3]) == 0x00) { | ||
| return Charset.forName("UTF-32LE"); | ||
| } | ||
| if (n >= 2 && u(b[0]) == 0xFE && u(b[1]) == 0xFF) { | ||
| return StandardCharsets.UTF_16BE; | ||
| } | ||
| if (n >= 2 && u(b[0]) == 0xFF && u(b[1]) == 0xFE) { | ||
| return StandardCharsets.UTF_16LE; | ||
| } | ||
| // BOM-less heuristic over the head: ASCII characters in wide encodings leave nulls at fixed offsets. | ||
| int window = Math.min(n, 64); | ||
| if (window < 4) { | ||
| return null; | ||
| } | ||
| int z0 = 0, z1 = 0, z2 = 0, z3 = 0, quads = 0; | ||
| for (int i = 0; i + 3 < window; i += 4) { | ||
| if (b[i] == 0) z0++; | ||
| if (b[i + 1] == 0) z1++; | ||
| if (b[i + 2] == 0) z2++; | ||
| if (b[i + 3] == 0) z3++; | ||
| quads++; | ||
| } | ||
| if (quads > 0) { | ||
| if (z0 == quads && z1 == quads && z2 == quads && z3 == 0) { | ||
| return Charset.forName("UTF-32BE"); | ||
| } | ||
| if (z0 == 0 && z1 == quads && z2 == quads && z3 == quads) { | ||
| return Charset.forName("UTF-32LE"); | ||
| } | ||
| } | ||
| int evenZero = 0, oddZero = 0, pairs = 0; | ||
| for (int i = 0; i + 1 < window; i += 2) { | ||
| if (b[i] == 0) evenZero++; | ||
| if (b[i + 1] == 0) oddZero++; | ||
| pairs++; | ||
| } | ||
| if (pairs > 0) { | ||
| if (evenZero == pairs && oddZero == 0) { | ||
| return StandardCharsets.UTF_16BE; | ||
| } | ||
| if (oddZero == pairs && evenZero == 0) { | ||
| return StandardCharsets.UTF_16LE; | ||
| } | ||
| } | ||
| return null; | ||
| } |
There was a problem hiding this comment.
The newly introduced FileValidationUtils.java validates uploaded files by their true content type rather than client-supplied headers. To support UTF-16/32 encoded text files, it implements a custom detectWideCharset heuristic that checks if even or odd bytes in the first 64 bytes of the file are all 0x00. If this condition is met, it assumes the file is encoded in UTF-16BE/LE or UTF-32BE/LE and decodes the entire head of the file using that charset, which is then re-detected. If the re-detected type is text/plain, it trusts this result and allows the file.
An attacker can exploit this heuristic to bypass file type validation for any arbitrary binary file (such as executables, DLLs, or ZIP archives). By structuring the first 64 KB of a malicious binary file so that all even bytes (or all odd bytes) are 0x00 and odd bytes are printable ASCII characters, the validator is tricked into detecting the file as text/plain. Since the validator returns the original, unmodified binary bytes upon successful validation, the malicious binary file is successfully uploaded. Any arbitrary binary payload can be appended after the first 64 KB.
Steps to Reproduce
- Run the provided Python script to generate a file named
malicious_bypass.bin. - Authenticate to Appsmith and navigate to the AI plugin file upload feature.
- Upload
malicious_bypass.binas a file. - The file validation logic will evaluate the file, identify it as UTF-16BE plain text, normalize the first 64 KB, and successfully validate it as
text/plain. - The original, unmodified binary file (including the arbitrary binary payload appended after the 64 KB head) is successfully accepted and stored.
# Python script to generate a spoofed binary file that bypasses the validation.
# The first 64 KB (HEAD_DECODE_LIMIT) is structured as UTF-16BE plain text (even bytes 0x00, odd bytes ASCII).
# This ensures that the normalized head decodes to clean ASCII text and is detected as text/plain.
# The remaining part of the file can contain any arbitrary binary payload (e.g., a zip or exe).
head_size = 64 * 1024 # 64 KB
spoofed_head = bytearray(head_size)
for i in range(head_size):
if i % 2 == 0:
spoofed_head[i] = 0x00
else:
spoofed_head[i] = ord('A') # Printable ASCII character 'A'
# Arbitrary binary payload (e.g., representing a malicious executable or zip archive)
binary_payload = b"\x7fELF" + b"\x00" * 1000
with open("malicious_bypass.bin", "wb") as f:
f.write(spoofed_head + binary_payload)Fix with AI
A security vulnerability was found by Hacktron.
File: app/server/appsmith-plugins/appsmithAiPlugin/src/main/java/com/external/plugins/utils/FileValidationUtils.java
Lines: 168-218
Severity: high
Vulnerability: File Type Validation Bypass via Spoofed Wide Charset Heuristics
Description:
The newly introduced `FileValidationUtils.java` validates uploaded files by their true content type rather than client-supplied headers. To support UTF-16/32 encoded text files, it implements a custom `detectWideCharset` heuristic that checks if even or odd bytes in the first 64 bytes of the file are all `0x00`. If this condition is met, it assumes the file is encoded in UTF-16BE/LE or UTF-32BE/LE and decodes the entire head of the file using that charset, which is then re-detected. If the re-detected type is `text/plain`, it trusts this result and allows the file.
An attacker can exploit this heuristic to bypass file type validation for any arbitrary binary file (such as executables, DLLs, or ZIP archives). By structuring the first 64 KB of a malicious binary file so that all even bytes (or all odd bytes) are `0x00` and odd bytes are printable ASCII characters, the validator is tricked into detecting the file as `text/plain`. Since the validator returns the original, unmodified binary bytes upon successful validation, the malicious binary file is successfully uploaded. Any arbitrary binary payload can be appended after the first 64 KB.
Proof of Concept:
**Steps to Reproduce**
1. Run the provided Python script to generate a file named `malicious_bypass.bin`.
2. Authenticate to Appsmith and navigate to the AI plugin file upload feature.
3. Upload `malicious_bypass.bin` as a file.
4. The file validation logic will evaluate the file, identify it as UTF-16BE plain text, normalize the first 64 KB, and successfully validate it as `text/plain`.
5. The original, unmodified binary file (including the arbitrary binary payload appended after the 64 KB head) is successfully accepted and stored.
```python
# Python script to generate a spoofed binary file that bypasses the validation.
# The first 64 KB (HEAD_DECODE_LIMIT) is structured as UTF-16BE plain text (even bytes 0x00, odd bytes ASCII).
# This ensures that the normalized head decodes to clean ASCII text and is detected as text/plain.
# The remaining part of the file can contain any arbitrary binary payload (e.g., a zip or exe).
head_size = 64 * 1024 # 64 KB
spoofed_head = bytearray(head_size)
for i in range(head_size):
if i % 2 == 0:
spoofed_head[i] = 0x00
else:
spoofed_head[i] = ord('A') # Printable ASCII character 'A'
# Arbitrary binary payload (e.g., representing a malicious executable or zip archive)
binary_payload = b"\x7fELF" + b"\x00" * 1000
with open("malicious_bypass.bin", "wb") as f:
f.write(spoofed_head + binary_payload)
```
Affected Code:
private static Charset detectWideCharset(byte[] b) {
int n = b.length;
if (n >= 4 && u(b[0]) == 0x00 && u(b[1]) == 0x00 && u(b[2]) == 0xFE && u(b[3]) == 0xFF) {
return Charset.forName("UTF-32BE");
}
if (n >= 4 && u(b[0]) == 0xFF && u(b[1]) == 0xFE && u(b[2]) == 0x00 && u(b[3]) == 0x00) {
return Charset.forName("UTF-32LE");
}
if (n >= 2 && u(b[0]) == 0xFE && u(b[1]) == 0xFF) {
return StandardCharsets.UTF_16BE;
}
if (n >= 2 && u(b[0]) == 0xFF && u(b[1]) == 0xFE) {
return StandardCharsets.UTF_16LE;
}
// BOM-less heuristic over the head: ASCII characters in wide encodings leave nulls at fixed offsets.
int window = Math.min(n, 64);
if (window < 4) {
return null;
}
int z0 = 0, z1 = 0, z2 = 0, z3 = 0, quads = 0;
for (int i = 0; i + 3 < window; i += 4) {
if (b[i] == 0) z0++;
if (b[i + 1] == 0) z1++;
if (b[i + 2] == 0) z2++;
if (b[i + 3] == 0) z3++;
quads++;
}
if (quads > 0) {
if (z0 == quads && z1 == quads && z2 == quads && z3 == 0) {
return Charset.forName("UTF-32BE");
}
if (z0 == 0 && z1 == quads && z2 == quads && z3 == quads) {
return Charset.forName("UTF-32LE");
}
}
int evenZero = 0, oddZero = 0, pairs = 0;
for (int i = 0; i + 1 < window; i += 2) {
if (b[i] == 0) evenZero++;
if (b[i + 1] == 0) oddZero++;
pairs++;
}
if (pairs > 0) {
if (evenZero == pairs && oddZero == 0) {
return StandardCharsets.UTF_16BE;
}
if (oddZero == pairs && evenZero == 0) {
return StandardCharsets.UTF_16LE;
}
}
return null;
}
Acceptance criteria:
- Acceptance is defined by the **actual reported behavior**, not by tests passing.
- Reproduce the issue, or narrow the exact code path that produces it, *before* changing code. State what you confirmed.
- Fix the underlying cause. Mitigations that paper over the reported behavior do not count as a fix.
- Add a regression test that fails on the unpatched code and passes on the fix. If a regression test is genuinely impractical (e.g. race condition, infra-level issue), say so and explain why.
- Existing tests passing is **not** the bar. Do not declare done on tests-pass theatre.
Only change what is necessary to fix this vulnerability. Do not refactor adjacent code or modify unrelated files.
Triage: Reply !fp <reason> (false positive), !valid (confirmed), !accepted_risk <reason>, or !fixed (resolved). Any other reply is saved as a triage note.
Reason is optional but improves future scans — e.g. !fp internal endpoint, not user-facing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
The AI plugin's upload path validated files by the client-supplied filename and Content-Type — both spoofable. The allowed types already existed as a client-side
allowedFileTypeshint inform.json(application/pdf,text/plain,text/markdown), but the server never enforced them. This brings server-side validation in line with that hint — closing a client/server mismatch, not adding a new restriction — by detecting the true type from the file's content with Apache Tika (MIME + magic bytes) and rejecting anything outside the allow-list (e.g. a malicious SVG).Scope / severity: defense-in-depth, not an XSS-serving fix. Uploaded files are never served back to the user from the app origin — they're forwarded to the AI service and stored in S3, reachable only via short-lived presigned URLs on a different origin. Independently re-reviewed (SHIP-WITH-RISKS).
How
FileValidationUtilsdetects the type via tika-core detectors constructed directly (noServiceLoader, so it survives the plugin uber-jar shading; notika-parsers).image/svg+xmland is rejected. Genuine wide-encoded text still passes.text/plain. A file accepted astext/*is rejected only if, after stripping its leading BOM and whitespace/padding, the first content is a markup root (<svg,<?xml,<html,<!doctype,<script) followed by a tag boundary — i.e. it would actually render as SVG/HTML/XML. This is narrower than a substring scan: text/markdown that merely mentions or embeds markup in prose or a code example is accepted; PDFs are never checked (they legitimately embed markup).DataBufferUtils.join's fail-fastmaxByteCount(20 MiB, mirroringform.json'smaxFileSizeInBytes), so an oversized upload is rejected as it is read rather than buffered into heap — scoped to this endpoint, unlike the shared 150MB global multipart ceiling.BufferedFilePartbuffers bytes so the file is inspected and still forwarded.tika-core3.2.2 (1.13–3.2.1 are affected by CVE-2025-66516).Guard posture (explicit)
1 < 2); PDF, including one embedding markup; and — low-risk residual, made explicit — a file padded with non-whitespace bytes before its markup (AAAA…<script>): it types as text and is not a renderable document at the root. Consistent with the threat model (S3-stored, not app-origin served).Tests cover the encoding matrix; the padded-document bypass (whitespace-padded SVG/HTML and wide-encoded, rejected) vs. legit markup-mentioning text/markdown (accepted); large plain-text/markdown/PDF (accepted); and oversized-file rejection. Module suite green.
Linear: https://linear.app/appsmith/issue/APP-15345
Summary by CodeRabbit
Warning
Tests have not run on the HEAD b509992 yet
Sat, 18 Jul 2026 00:23:48 UTC