Skip to content

Support OpenAI-compatible endpoints via OPENAI_BASE_URL - #16

Open
abliteration-ai wants to merge 1 commit into
uber:mainfrom
abliteration-ai:openai-compatible-base-url
Open

Support OpenAI-compatible endpoints via OPENAI_BASE_URL#16
abliteration-ai wants to merge 1 commit into
uber:mainfrom
abliteration-ai:openai-compatible-base-url

Conversation

@abliteration-ai

Copy link
Copy Markdown

What type of PR is this? (check all applicable)

  • Feature

Related issue: N/A

What changed?

Detection/openai_config.py — the single place where OpenAI clients are constructed — now reads the optional OPENAI_BASE_URL environment variable:

  • get_openai_client() passes it to the SDK as base_url when set, so every consumer (create_chat_completion, create_reasoning_completion, the ADR baseline agent) can be pointed at any OpenAI-compatible API.
  • get_openai_config() includes a base_url key when the variable is set.
  • When OPENAI_BASE_URL is unset, behavior is byte-for-byte identical to before (default api.openai.com endpoint).

Also adds unit tests for both helpers (Detection/tests/test_openai_config.py) and a short README section documenting the option with a provider example (Abliteration.ai).

Why?

The triage model is configurable in config_detector.yaml, but the client was hardcoded to the hosted OpenAI API. Teams running OpenAI-compatible gateways or alternate providers (self-hosted vLLM, LiteLLM proxies, etc.) had no way to route detector traffic there. OPENAI_BASE_URL follows the OpenAI SDK's own convention, so it's the least surprising knob.

How did you test it?

  • uv run pytest tests/test_openai_config.py — 8 passed (4 existing + 4 new covering default endpoint, custom base URL, and the config dict with/without OPENAI_BASE_URL).
  • black --check clean on the touched test file; openai_config.py keeps the file's existing style.
  • Verified no new flake8 findings in the touched code (the file's pre-existing warnings are untouched).

Potential risks

None expected: the new code path only activates when OPENAI_BASE_URL is set; existing deployments without it are unaffected. The Anthropic/Claude path is untouched.

get_openai_client() and get_openai_config() now read the optional
OPENAI_BASE_URL environment variable and pass it to the OpenAI SDK as
base_url, so detectors can be pointed at any OpenAI-compatible API.
When unset, behavior is unchanged (default api.openai.com endpoint).

Adds unit tests for both helpers and documents the option in the
Detection README with a provider example.
@CLAassistant

CLAassistant commented Aug 2, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@pengyuzhang

Copy link
Copy Markdown
Collaborator

Thanks for the PR, and especially for the tests — the coverage added in test_openai_config.py is useful regardless of the outcome here.

We're going to close this one, for two reasons.

1. The behavior already works without a code change. The OpenAI Python SDK reads OPENAI_BASE_URL from the environment itself. Verified against the version pinned in this repo (openai 1.100.2):

>>> os.environ["OPENAI_BASE_URL"] = "https://example.test/v1"
>>> OpenAI(api_key="sk-test").base_url     # no base_url argument passed
https://example.test/v1/

So setting the env var already points the triage LLM at any OpenAI-compatible endpoint on main today, and the added branch in get_openai_client() duplicates SDK behavior. The get_openai_config() change has no effect either — that function currently has no callers in the repo, so the new base_url key isn't consumed anywhere.

2. The README section. We'd prefer to keep third-party endpoint documentation vendor-neutral rather than build the example around one provider's URL, key name, and model. Separately, we'd rather not recommend running triage on an uncensored model: triage is the high-recall gate in front of the reasoning agent, and the paper's Table 2 numbers assume gpt-4o.

We'll document the SDK's existing OPENAI_BASE_URL support with a short neutral note instead.

Thanks again for taking the time to look at the project.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants