Skip to content

BUG HTTPTarget stores responses as str(bytes) and the JSON callback can't address keys with digits or hyphens #2813

Description

Two problems in the HTTP targets.

First, HTTPTarget and HTTPXAPITarget save str(response.content) when there's no callback. content is bytes, so what ends up in memory is the Python repr: b'...' with every non-ASCII character escaped. The regex callback searches that same repr, so matches pick up escapes and a trailing quote:

server text:        'Sure — here’s the answer:\nStep 1: café'
stored:             "b'Sure \\xe2\\x80\\x94 here\\xe2\\x80\\x99s the answer:\\nStep 1: caf\\xc3\\xa9'"
regex 'Step 1: .*': "Step 1: caf\\xc3\\xa9'"

You can see it in the saved output of doc/code/targets/10_http_target.ipynb too (b'<!DOCTYPE html>...).

Second, _fetch_key tokenizes with ([a-zA-Z_]+), so keys with digits or hyphens get split. output2 looks up output, generated-text looks up generated, and both raise.

Fix: use response.text, and treat anything other than ., [ and ] as part of a key.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions