Skip to content

Commit b33e389

Browse files
Richard Taylorclaude
authored andcommitted
DownloadManager: document redact_url kwarg for auth-bearing URLs
Companion to MicroPythonOS#136 (adds the kwarg to DownloadManager.download_url). Adds a new "Redacting Sensitive URLs" section explaining when callers should opt in, what gets redacted (URL path/query, response-headers, exception messages — host kept for failure triage), and why the default is False (preserves useful debug output for public-URL callers). Updates the parameters table with the new kwarg. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 6a478a8 commit b33e389

1 file changed

Lines changed: 30 additions & 1 deletion

File tree

docs/frameworks/download-manager.md

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@ This means you can use the same API in both async and sync code without any wrap
8585
```python
8686
def download_url(url, outfile=None, total_size=None,
8787
progress_callback=None, chunk_callback=None,
88-
headers=None, speed_callback=None)
88+
headers=None, speed_callback=None, redact_url=False)
8989
```
9090

9191
**Note:** This method works in both async and sync contexts. When called from an async function, it returns a coroutine. When called from a sync function, it runs synchronously.
@@ -101,6 +101,7 @@ def download_url(url, outfile=None, total_size=None,
101101
| `chunk_callback` | async function | Callback for streaming chunks (optional) |
102102
| `headers` | dict | Custom HTTP headers (optional) |
103103
| `speed_callback` | async function | Callback for download speed (optional) |
104+
| `redact_url` | bool | Opt-in: hide the URL in log output when it carries an auth secret in its path or query string. Default `False` keeps existing debug output for public URLs. See [Redacting Sensitive URLs](#redacting-sensitive-urls) below. |
104105

105106
**Returns:**
106107
- **Memory mode**: `bytes` on success
@@ -144,6 +145,34 @@ else:
144145
await DownloadManager.download_url(url, outfile=outfile)
145146
```
146147

148+
## Redacting Sensitive URLs
149+
150+
`DownloadManager.download_url` prints the request URL three times per download (start, finished, exception path) plus the full response-headers dict — useful debug output for typical callers fetching public URLs (app icons, OS updates, weather data), but it silently leaks any auth secret embedded in the URL.
151+
152+
Common cases where a URL carries a secret:
153+
154+
- **API-key-in-URL** auth: `https://api.example.com/v1/data?api_key=ABC123`
155+
- **OAuth-token-in-URL**: `https://service.example.com/resource?access_token=eyJhbGci…`
156+
- **Wallet xpubs** in indexer URLs: `https://btc1.trezor.io/api/v2/xpub/zpub6q…`
157+
- **LNBits readkey** in URL path on some endpoints
158+
159+
A leaked secret-bearing URL ends up in serial / REPL output, screenshots of debug logs, and anywhere those logs are shared. Pass `redact_url=True` to scrub it:
160+
161+
```python
162+
await DownloadManager.download_url(
163+
"https://btc1.trezor.io/api/v2/xpub/zpub6q…",
164+
redact_url=True,
165+
)
166+
```
167+
168+
When `redact_url=True`:
169+
170+
- The URL is logged as `scheme://host[:port]/...REDACTED...` (path + query stripped). The host is intentionally kept so failure triage (DNS, connectivity, wrong endpoint) is still possible.
171+
- The response-headers dump is suppressed entirely (`<redacted>`). Response headers often contain `set-cookie`, `cf-ray`, and other tokens that correlate to the secret-bearing request.
172+
- Exception messages have any embedded URL substring scrubbed (aiohttp's `ClientConnectorError` typically embeds the URL).
173+
174+
Default `False` is deliberate — most callers fetch public URLs and want the full log line for diagnostics. Opt-in is per call.
175+
147176
## Common Patterns
148177

149178
### Download with Timeout

0 commit comments

Comments
 (0)