Skip to content

Commit 2637c91

Browse files
Merge pull request #6 from bitcoin3us/docs/download-manager-redact-url
DownloadManager: document redact_url kwarg
2 parents aa551e6 + b33e389 commit 2637c91

1 file changed

Lines changed: 30 additions & 1 deletion

File tree

docs/frameworks/download-manager.md

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@ This means you can use the same API in both async and sync code without any wrap
8585
```python
8686
def download_url(url, outfile=None, total_size=None,
8787
progress_callback=None, chunk_callback=None,
88-
headers=None, speed_callback=None)
88+
headers=None, speed_callback=None, redact_url=False)
8989
```
9090

9191
**Note:** This method works in both async and sync contexts. When called from an async function, it returns a coroutine. When called from a sync function, it runs synchronously.
@@ -101,6 +101,7 @@ def download_url(url, outfile=None, total_size=None,
101101
| `chunk_callback` | async function | Callback for streaming chunks (optional) |
102102
| `headers` | dict | Custom HTTP headers (optional) |
103103
| `speed_callback` | async function | Callback for download speed (optional) |
104+
| `redact_url` | bool | Opt-in: hide the URL in log output when it carries an auth secret in its path or query string. Default `False` keeps existing debug output for public URLs. See [Redacting Sensitive URLs](#redacting-sensitive-urls) below. |
104105

105106
**Returns:**
106107
- **Memory mode**: `bytes` on success
@@ -163,6 +164,34 @@ else:
163164
await DownloadManager.download_url(url, outfile=outfile)
164165
```
165166

167+
## Redacting Sensitive URLs
168+
169+
`DownloadManager.download_url` prints the request URL three times per download (start, finished, exception path) plus the full response-headers dict — useful debug output for typical callers fetching public URLs (app icons, OS updates, weather data), but it silently leaks any auth secret embedded in the URL.
170+
171+
Common cases where a URL carries a secret:
172+
173+
- **API-key-in-URL** auth: `https://api.example.com/v1/data?api_key=ABC123`
174+
- **OAuth-token-in-URL**: `https://service.example.com/resource?access_token=eyJhbGci…`
175+
- **Wallet xpubs** in indexer URLs: `https://btc1.trezor.io/api/v2/xpub/zpub6q…`
176+
- **LNBits readkey** in URL path on some endpoints
177+
178+
A leaked secret-bearing URL ends up in serial / REPL output, screenshots of debug logs, and anywhere those logs are shared. Pass `redact_url=True` to scrub it:
179+
180+
```python
181+
await DownloadManager.download_url(
182+
"https://btc1.trezor.io/api/v2/xpub/zpub6q…",
183+
redact_url=True,
184+
)
185+
```
186+
187+
When `redact_url=True`:
188+
189+
- The URL is logged as `scheme://host[:port]/...REDACTED...` (path + query stripped). The host is intentionally kept so failure triage (DNS, connectivity, wrong endpoint) is still possible.
190+
- The response-headers dump is suppressed entirely (`<redacted>`). Response headers often contain `set-cookie`, `cf-ray`, and other tokens that correlate to the secret-bearing request.
191+
- Exception messages have any embedded URL substring scrubbed (aiohttp's `ClientConnectorError` typically embeds the URL).
192+
193+
Default `False` is deliberate — most callers fetch public URLs and want the full log line for diagnostics. Opt-in is per call.
194+
166195
## Common Patterns
167196

168197
### Download with Timeout

0 commit comments

Comments
 (0)