logprobs

Open-source alternative to TypeSafe's Jev: a System One style model layer that gives typed, calibrated decisions from any open-weights LLM in one forward pass (HF + vLLM), with honest benchmarks

Typed choices and probability distributions from text and images with llama.cpp, Ollama, and OpenRouter

Super fast Jev like server, model agnostic, working with any OpenAI compatible endpoint

Jev-shaped (TypeSafe System One) classification wrapper over OpenAI-like clients: probabilities and confidence instead of prose

Typed decisions from local LLMs in one forward pass, via Ollama. No generation loop, no JSON parsing.

An experimental approach for AI search visibility.

Calibrated yes/no, pick-one and scale judgments from your own LLMs, read straight off the logprobs. Runs on vLLM.

Interactive Streamlit app that visualises token-level log probabilities from Azure OpenAI

Token-level entropy trajectories from LLM logprobs. Models can measure their own uncertainty — grounding it in truth requires external verification.

Jev-compatible /v1/systemone server reading typed decisions from LLM logits, benchmarked against TypeSafe's Jev on the same items via JevBench

Jev-style /v1/systemone API in front of any OpenAI-compatible LLM server (one-token logprob scoring, MIT)

Model-agnostic substrate for running small local LLMs well — code agents, chat, image/video, speech. Small capable main model + tiny fast worker.

Self-hosted System One decision engine (TypeSafe Jev's Choice/Score/Noul contract) running locally on LM Studio, with real probabilities from token logprobs.

A demo to show confidence probabilities in AI responses

Minimal demo of calibrated LLM-as-a-judge scores from the single-call token-logprob distribution (Wang et al. 2025; G-Eval).

An interactive client for exploring token-level log probabilities from OpenAI-compatible chat models.

A drop-in confidence gate for LLM agents: turn token logprobs into an accept / verify / ask-a-human routing decision, with honest per-model calibration. Local & open-model first, plus an MCP server.

Renders every token a model generated with the tokens it rejected, ghosted above it by probability

MSc thesis: mechanistic interpretability of Chain-of-Thought reasoning in LLMs via SAE features and logprob signals

Stream LLM tokens with logprobs, compute per-token Shannon entropy, render a color-coded HTML uncertainty map