dev-licence: allow users to configure a dev license with diminished c…

[ness-david-dedu]

• Adds an embedded dev license activated by setting REDPANDA_CONNECT_DEV_LICENSE=1. When no production enterprise license is present and the env var is set, Connect grants full enterprise feature access
  under a 1 MB/s throughput cap per process.
• Implements a token-bucket Throttler (golang.org/x/time/rate, 1 MB/s sustained, 30 MB burst) shared across all pipelines in the process to prevent load-spreading bypass.
• Wraps all 9 enterprise BatchOutput implementations (iceberg, Redpanda/Kafka, OTLP gRPC/HTTP, Salesforce, Snowflake PUT/streaming, Splunk HEC, BigQuery Write API) with WrapBatchOutput, which injects the
  throttler only under the dev license — no-op under a production license.
• In agent mode, RegisterServiceFrom propagates the single shared throttler to all per-stream Resources objects, preserving the per-process cap invariant across multi-agent configs.
• Logs a WARN once when the cap is first hit; exposes redpanda_connect_test_license_throttle_active gauge (1 = throttling, 0 = below cap).
• When a corrupt/expired production license is present alongside the env var, logs a distinct Error-level message linking the license failure to the dev license fallback.

[prakhargarg105]

the dev license should be the default. No opt in required. The only two states should be: dev license running or prod/enterprise license

[prakhargarg105]

We also want to throttle in the enterprise inputs. Postgres_cdc -> redpanda should be throttled and not free. Option B

[prakhargarg105]

The WARN text says "Throttling enterprise output" even while hitting on input. Can we parametrize on the connector name?

[prakhargarg105]

internal/license/throttle.go — Wait clamps the charge to one burst, so oversized batches bypass the cap

The throttle clamps n to the burst window before reserving tokens:

if n > testLicenseBurstBytes { // 30 MB
n = testLicenseBurstBytes
}

I get why it's here — rate.Limiter.ReserveN can't reserve more than burst, so without the clamp a batch larger than 30 MB would produce a never-satisfiable reservation. But the side effect is that a single batch is only ever charged up to 30 MB, regardless of its real size.

Since batch size is user-controlled config, this lets a pipeline run well over the 1 MB/s cap just by batching larger — no tooling or tricks needed. Effective rate ends up ≈ batch_size / 30s:

┌────────┬────────────┬─────────┬────────────────┐
│ Batch │ True bytes │ Charged │ Effective rate │
├────────┼────────────┼─────────┼────────────────┤
│ 100 MB │ 100 MB │ 30 MB │ ~3.3 MB/s │
├────────┼────────────┼─────────┼────────────────┤
│ 300 MB │ 300 MB │ 30 MB │ ~10 MB/s │
├────────┼────────────┼─────────┼────────────────┤
│ 1 GB │ 1 GB │ 30 MB │ ~33 MB/s │
└────────┴────────────┴─────────┴────────────────┘
Suggested direction: instead of clamping, charge the full byte count by billing the batch in burst-sized chunks and waiting for each. A 100 MB batch then honestly costs ~100s and making batches bigger buys nothing.

[prakhargarg105]

Last comment: aws_dynamodb_cdc and redpanda_common input need to be gated as well. We should match info.csv enterprise flagged.