Skip to content

-o tsv emits CRLF on Windows, breaking for x in $(az ... -o tsv) shell loops with misleading 400 errors #33385

@dmytroslotvinskyy

Description

@dmytroslotvinskyy

Describe the bug

On Windows, az ... --output tsv writes each row terminated by CRLF (\r\n) instead of LF (\n). When that output is consumed by a POSIX shell (Git Bash / MSYS2 / WSL-on-Windows-files / bash-from-PowerShell) via for x in $(...), every token except the last carries a stray \r. That \r gets interpolated into the URL path of the next az call, and ARM responds with HTTP 400 Bad Request. The CLI's error message — Operation returned an invalid status 'Bad Request' — does not reveal the underlying cause.

Net effect: a common shell pattern shown in many Microsoft Learn tutorials silently fails on Windows for every iteration except the last one in the loop.

To Reproduce

Any namespace/topic combination works. Example using Service Bus:

RG=<your-rg>
NS=<your-namespace>

for topic in $(az servicebus topic list --resource-group "$RG" --namespace-name "$NS" --query "[].name" -o tsv); do
  count=$(az servicebus topic subscription list --resource-group "$RG" --namespace-name "$NS" --topic-name "$topic" --query "length(@)" -o tsv 2>&1)
  printf "%-40s %s\n" "$topic" "$count"
done

Actual output (14 topics, only the alphabetically last works):

topic1                        ERROR: Operation returned an invalid status 'Bad Request'
topic2                         ERROR: Operation returned an invalid status 'Bad Request'

Environment summary

azure-cli                         2.80.0

core                              2.80.0
telemetry                          1.1.0

Extensions:
account                            0.2.5
authV2                             1.0.1
azure-devops                       1.0.2
redisenterprise                    1.4.0
subscription                       0.1.3

Dependencies:
msal                            1.34.0b1
azure-mgmt-resource               23.3.0

Python location 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe'

OS:     Windows 11 Pro 10.0.26200
Shell:  Git Bash (MSYS2 / mintty); also reproducible from PowerShell-invoked bash subshells

Proof of root cause

az ... -o tsv writes CRLF row terminators on Windows:

The bash `$(...)` substitution then carries `\r` on every token except the last (bash strips a trailing `\n` and the `\r` just before it in that single edge case — the other tokens keep their `\r` because the `\r` precedes a non-terminal `\n`).

When the next `az` call receives `--topic-name "$topic"`, the inner CLI inserts the `\r` into the ARM REST URL path. ARM responds 400. The CLI surfaces only `Operation returned an invalid status 'Bad Request'` and does not echo the ARM error body, so the cause is invisible to the user.

### Workaround

Strip `\r` from the captured output:

```bash
for topic in $(az servicebus topic list --resource-group "$RG" --namespace-name "$NS" --query "[].name" -o tsv | tr -d '\r'); do
  count=$(az servicebus topic subscription list --resource-group "$RG" --namespace-name "$NS" --topic-name "$topic" --query "length(@)" -o tsv)
  printf "%-40s %s\n" "$topic" "$count"
done

Proof of root cause

az ... -o tsv writes CRLF row terminators on Windows:

$ az servicebus topic list --resource-group "$RG" --namespace-name "$NS" --query "[].name" -o tsv | head -3 | od -c
0000000   t   o   p   i   c   1  \r  \n   t   o   p   i   c   2  \r  \n
0000020   t   o   p   i   c   3  \r  \n
0000030

After applying | tr -d '\r' the loop returns the expected counts for all 14 topics.

Comparison of output formats on Windows

Output mode Line ending Safe in shell for loops?
-o tsv CRLF No — this bug
-o json LF n/a (not row-delimited)
-o jsonc LF n/a
-o yaml LF n/a
-o table CRLF (decorative) n/a

Only the machine-parseable, row-delimited format tsv is affected. csv likely has the same issue but I have not retested it for this report.

Suggested fix

The CLI's tsv formatter inherits Python's default text-mode line translation on Windows. Two options:

  1. Reconfigure stdout for tsv/csv output — at format-dispatch time, call sys.stdout.reconfigure(newline='\n') when the user selected tsv or csv. Smallest change, easiest to review.
  2. Write tsv as bytessys.stdout.buffer.write(line.encode() + b'\n'). More invasive but eliminates any platform-specific newline handling.

Option 1 should be a one-line change in the format dispatch (azure-cli-core/azure/cli/core/commands/__init__.py or wherever the formatter lookup happens).

Bonus suggestion (separate issue territory)

When the underlying ARM call returns 4xx, the CLI should print the ARM error body, not just Operation returned an invalid status 'Bad Request'. ARM almost always includes a code and message field that would have pointed at the bad URL character immediately. The current behavior of swallowing those fields turns a 30-second diagnosis into a 30-minute one.

Why this matters

The pattern for x in $(az ... -o tsv); do ...; done appears verbatim in Microsoft Learn tutorials, Azure samples, and many blog posts. Every Windows user copying those snippets hits this bug and assumes their script, their credentials, their permissions, or their resource state is wrong. Few think "the CLI's tsv output has the wrong line endings." A one-line formatter fix removes a recurring scripting trap.

Anything else

Confirmed not present in PowerShell — PowerShell splits tsv output on [\r\n]+, masking the bug. The issue is bash-on-Windows-specific in observed effect, but the root cause (CRLF emission by the CLI on Windows) is the CLI's responsibility, not bash's.



### Related command

```bash
RG=<your-rg>
NS=<your-namespace>

for topic in $(az servicebus topic list --resource-group "$RG" --namespace-name "$NS" --query "[].name" -o tsv); do
  count=$(az servicebus topic subscription list --resource-group "$RG" --namespace-name "$NS" --topic-name "$topic" --query "length(@)" -o tsv 2>&1)
  printf "%-40s %s\n" "$topic" "$count"
done

### Errors

Actual output (14 topics, only the alphabetically last works):

topic1 ERROR: Operation returned an invalid status 'Bad Request'
topic2 ERROR: Operation returned an invalid status 'Bad Request'

Issue script & Debug output

Actual output (14 topics, only the alphabetically last works):

topic1                        ERROR: Operation returned an invalid status 'Bad Request'
topic2                         ERROR: Operation returned an invalid status 'Bad Request'

### Expected behavior

### Comparison of output formats on Windows

| Output mode | Line ending | Safe in shell `for` loops? |
|---|---|---|
| `-o tsv` | **CRLF** | **No — this bug** |
| `-o json` | LF | n/a (not row-delimited) |
| `-o jsonc` | LF | n/a |
| `-o yaml` | LF | n/a |
| `-o table` | CRLF (decorative) | n/a |

Only the **machine-parseable, row-delimited** format `tsv` is affected. `csv` likely has the same issue but I have not retested it for this report.

### Environment Summary


azure-cli 2.80.0

core 2.80.0
telemetry 1.1.0

Extensions:
account 0.2.5
authV2 1.0.1
azure-devops 1.0.2
redisenterprise 1.4.0
subscription 0.1.3

Dependencies:
msal 1.34.0b1
azure-mgmt-resource 23.3.0

Python location 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe'

OS: Windows 11 Pro 10.0.26200
Shell: Git Bash (MSYS2 / mintty); also reproducible from PowerShell-invoked bash subshells


### Additional context

_No response_

Metadata

Metadata

Assignees

No one assigned

    Labels

    Auto-AssignAuto assign by botAuto-ResolveAuto resolve by botOutputPossible-SolutionService AttentionThis issue is responsible by Azure service team.Service Busaz servicebusSimilar-Issueact-observability-squadbugThis issue requires a change to an existing behavior in the product in order to be resolved.customer-reportedIssues that are reported by GitHub users external to the Azure organization.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions