Skip to content

The cookbook example "Accessing HTTP Response Metadata While Streaming" doesn't stream #2232

Description

@dilr

The example in question is:

http get --allow-errors https://api.example.com/events.jsonl
| metadata access {|meta|
    print $"Status: ($meta.http_response.status)"
    print $"Content-Type: ($meta.http_response.headers | where name == content-type | get value.0)"

    if $meta.http_response.status != 200 {
        error make {msg: $"Failed with status ($meta.http_response.status)"}
    } else { }
  }
| lines
| each { from json }
| where event_type == "error"

This won't work because the input to the closure is fed to the first statement, which is print $"Status: ...", while the closure output is taken from else {}. If the http response status is 200, the closure will always output nothing because the input for the closure was consumed by print. We can't test this because api.example.com is a fake subdomain, so let's modify the example to use example.com and not assume the output is json:

http get --allow-errors https://example.com
| metadata access {|meta|
    print $"Status: ($meta.http_response.status)"
    print $"Content-Type: ($meta.http_response.headers | where name == content-type | get value.0)"

    if $meta.http_response.status != 200 {
        error make {msg: $"Failed with status ($meta.http_response.status)"}
    } else { }
  }
| lines

If you run this, you will see that the status and content type are printed, but the http body is not returned from the closure.

One solution is to use the $in variable to collect the input and return it from the else statement:

http get --allow-errors https://example.com
| metadata access {|meta|
    let input = $in
    print $"Status: ($meta.http_response.status)"
    print $"Content-Type: ($meta.http_response.headers | where name == content-type | get value.0)"

    if $meta.http_response.status != 200 {
        error make {msg: $"Failed with status ($meta.http_response.status)"}
    } else { $input }
  }
| lines

This does return the http body from the closure, but the example is called "Accessing HTTP Response Metadata While Streaming", and $in collects the input, so it's no longer a stream.

If we want to make this work while streaming, we can make the if statement into the only top level statement of the closure by moving the print statements into its conditional expression:

http get --allow-errors https://example.com
| metadata access {|meta|
    if (
        print $"Status: ($meta.http_response.status)";
        print $"Content-Type: ($meta.http_response.headers | where name == content-type | get value.0)";

        $meta.http_response.status != 200
    ) {
        error make {msg: $"Failed with status ($meta.http_response.status)"}
    } else {  }
  }
| lines

When run, this streams the http body after printing the status and content-type. However, it is pretty hacky. If we were to put this in the cookbook, it would need an accompanying explanation as to how sticking statements in the conditional expression prevents them from eating the closure input. Perhaps someone with more nushell experience than me can come up with a cleaner way to write this?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions