[llm_server] Fix replay recovery and assistant header configuration

cd7b41f

Select commit

Loading

Failed to load commit list.

Uh oh!

There was an error while loading. Please reload this page.

PyTorch Bot

Dr.CI

Meta CLA

Meta CLA Check

Meta CodeSync

Meta Import Checks

Meta Internal-Only Changes Check

CodeQL on: dynamic

Analyze (actions)

Analyze (c-cpp)

Analyze (javascript-typescript)

Analyze (python)

Docathon Labels Sync on: pull_request_target 2

check-labels

Labeler on: pull_request_target 2

label

Check Labels on: pull_request_target 2

Check labels

Lint on: pull_request 5

Get changed files / get-changed-files

link-check / lint-urls

link-check / lint-xrefs

link-check / lint-file-size

lintrunner-mypy

lintrunner

Build documentation on: pull_request 3

Resolve CI docker image / resolve

build (buck2) / Build doc

upload-gh-pages

MLX on: pull_request 1

CI run decision / decide

test-mlx / test-mlx

test-mlx-supertonic / test-mlx-supertonic

test-mlx-qwen35-moe / test-mlx-qwen35-moe

test-mlx-muse-glimmer-build / test-mlx-muse-glimmer-build

backend-tester (models) / test-mlx-backend-models

backend-tester (operators) / test-mlx-backend-operators

test-mlx-parakeet / test-mlx-parakeet

test-mlx-voxtral / test-mlx-voxtral

test-mlx-voxtral-realtime / test-mlx-voxtral-realtime

test-mlx-whisper / test-mlx-whisper

test-mlx-stories110m / test-mlx-stories110m

test-mlx-llm (unsloth/Llama-3.2-1B-Instruct, llama-1b, false, 4w, macos-14-xlarge) / test-mlx-llm-llama-1b-4w

test-mlx-llm (unsloth/Llama-3.2-1B-Instruct, llama-1b, false, nvfp4, macos-14-xlarge) / test-mlx-llm-llama-1b-nvfp4

test-mlx-llm (unsloth/Llama-3.2-1B-Instruct, llama-1b, true, 4w, macos-14-xlarge) / test-mlx-llm-llama-1b-custom-4w

test-mlx-llm (unsloth/Llama-3.2-1B-Instruct, llama-1b, true, nvfp4, macos-14-xlarge) / test-mlx-llm-llama-1b-custom-nvfp4

test-mlx-llm (unsloth/Qwen3-0.6B, qwen3-0.6b, false, 4w, macos-14-xlarge) / test-mlx-llm-qwen3-0.6b-4w

test-mlx-llm (unsloth/Qwen3-0.6B, qwen3-0.6b, false, nvfp4, macos-14-xlarge) / test-mlx-llm-qwen3-0.6b-nvfp4

test-mlx-llm (unsloth/Qwen3-0.6B, qwen3-0.6b, true, 4w, macos-14-xlarge) / test-mlx-llm-qwen3-0.6b-custom-4w

test-mlx-llm (unsloth/Qwen3-0.6B, qwen3-0.6b, true, nvfp4, macos-14-xlarge) / test-mlx-llm-qwen3-0.6b-custom-nvfp4

test-mlx-llm (unsloth/gemma-3-1b-it, gemma3-1b, false, 4w, macos-14-xlarge) / test-mlx-llm-gemma3-1b-4w

test-mlx-llm (unsloth/gemma-3-1b-it, gemma3-1b, false, nvfp4, macos-14-xlarge) / test-mlx-llm-gemma3-1b-nvfp4

test-mlx-llm (unsloth/gemma-3-1b-it, gemma3-1b, true, 4w, macos-14-xlarge) / test-mlx-llm-gemma3-1b-custom-4w

test-mlx-llm (unsloth/gemma-3-1b-it, gemma3-1b, true, nvfp4, macos-14-xlarge) / test-mlx-llm-gemma3-1b-custom-nvfp4

test-mlx-llm (google/gemma-4-E2B-it, gemma4-e2b, true, 4w, macos-15-xlarge) / test-mlx-llm-gemma4-e2b-custom-4w

test-mlx-llm (google/gemma-4-E2B-it, gemma4-e2b, false, 4w, macos-15-xlarge) / test-mlx-llm-gemma4-e2b-4w

test-mlx-dflash / test-mlx-dflash

test-mlx-llm-offgraph (unsloth/Llama-3.2-1B-Instruct, llama-1b, llama3, macos-14-xlarge) / test-mlx-llm-offgraph-llama-1b

test-mlx-llm-offgraph (unsloth/gemma-3-1b-it, gemma3-1b, gemma, macos-14-xlarge) / test-mlx-llm-offgraph-gemma3-1b

test-mlx-llm-offgraph (google/gemma-4-E2B-it, gemma4-e2b, gemma4, macos-15-xlarge) / test-mlx-llm-offgraph-gemma4-e2b

Test QNN Windows MSVC build on: pull_request 3

build-qnn-windows-x64 / windows-job

build-qnn-windows-arm64

periodic on: pull_request 14

Resolve CI docker image / resolve

gather-models

test-models-linux (buck2, mv3, portable, mt-l-x86iavx512-8-64, 90) / linux-job

test-models-linux (buck2, mv3, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64, 90) / linux-job

test-models-linux (cmake, mv3, portable, mt-l-x86iavx512-8-64, 90) / linux-job

test-models-linux (cmake, mv3, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64, 90) / linux-job

test-models-linux (cmake, vit, portable, mt-l-x86iavx512-8-64, 90) / linux-job

test-models-linux (cmake, vit, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64, 90) / linux-job

Test ARM Backend on: pull_request 10

test-arm / Resolve CI docker image / resolve

test-arm / test-backend-macos

test-arm / test-backend-linux (arm_tosa_fp, models) / linux-job

test-arm / test-backend-linux (arm_tosa_fp, operators) / linux-job

test-arm / package-golden-artifacts

test-arm-vgf / Resolve CI docker image / resolve

test-arm-vgf / test-backend-macos

test-arm-vgf / test-backend-linux (arm_vgf_fp, models) / linux-job

test-arm-vgf / test-backend-linux (arm_vgf_fp, operators) / linux-job

test-arm-vgf / package-golden-artifacts

Build Windows Wheels on: pull_request 3

generate-matrix / generate

pytorch/executorch / build-wheel-py3_10-cpu

pytorch/executorch / upload / upload-wheel-py3_10-cpu

Build Presets on: pull_request 23

Resolve CI docker image / resolve

apple (macos) / build

apple (ios) / build

apple (ios-simulator) / build

apple (pybind) / build

apple (profiling) / build

apple (llm) / build

windows (pybind) / build

windows (windows) / build

zephyr (zephyr) / build

linux (linux, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clang12) / build

linux (linux, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc11-aarch64) / build

linux (pybind, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clang12) / build

linux (pybind, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc11-aarch64) / build

linux (llm, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clang12) / build

linux (llm, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc11-aarch64) / build

Test CUDA Builds on: pull_request 9

Get changed files / get-changed-files

CI run decision / decide

test-executorch-cuda-build-${{ matrix.cuda-version }}

test-models-cuda

export-cuda-multi-arch-a100

unittest-cuda

unittest-cuda-runtime

test-model-cuda-e2e-${{ matrix.model.name }}-${{ matrix.quant }}

test-muse-glimmer-cuda-e2e-kquant-17gb-solo-text / linux-job

test-muse-glimmer-cuda-e2e-kquant-17gb-dflash-image / linux-job

test-muse-glimmer-cuda-e2e-kquant-dynamic-solo-text / linux-job

test-muse-glimmer-cuda-e2e-kquant-dynamic-dflash-image / linux-job

check-all-cuda-builds

merge-and-test-cuda-multi-arch-a10g

test-cuda-merged-pte-a100

Test Cortex-M Backend on: pull_request

Get changed files / get-changed-files

test-cortex-m

Build Linux CUDA Wheels on: pull_request 4

generate-matrix / generate

filter-matrix

pytorch/executorch / build-manywheel-py3_10-cuda13_0

pytorch/executorch / upload / upload-manywheel-py3_10-cuda13_0

Test CoreML Backend on: pull_request

Get changed files / get-changed-files

test-coreml

Cadence Build & Test on: pull_request 14

Resolve CI docker image / resolve

hifi-build / Resolve CI docker image / resolve

hifi-build / hifi4

vision-build / Resolve CI docker image / resolve

vision-build / vision

cpu-build / build

hifi-op-test / Resolve CI docker image / resolve

hifi-op-test / hifi4

cpu-test / Resolve CI docker image / resolve

cpu-test / test-aot / test-aot

cpu-test / test-ops / test-ops

Test XNNPACK Backend on: pull_request 5

test-xnnpack / Resolve CI docker image / resolve

test-xnnpack / test-backend-macos

test-xnnpack / test-backend-linux (xnnpack, models) / linux-job

test-xnnpack / test-backend-linux (xnnpack, operators) / linux-job

test-xnnpack / package-golden-artifacts

Test QNN Backend on: pull_request 5

test-qnn / Resolve CI docker image / resolve

test-qnn / test-backend-macos

test-qnn / test-backend-linux (qnn, models) / linux-job

test-qnn / test-backend-linux (qnn, operators) / linux-job

test-qnn / package-golden-artifacts

Test NXP Backend on: pull_request

Get changed files / get-changed-files

test-nxp

Test WebGPU Backend on: pull_request 5

test-webgpu / Resolve CI docker image / resolve

test-webgpu / test-backend-macos

test-webgpu / test-backend-linux (webgpu, models) / linux-job

test-webgpu / test-backend-linux (webgpu, operators) / linux-job

test-webgpu / package-golden-artifacts

Build Aarch64 Linux Wheels on: pull_request 3

generate-matrix / generate

pytorch/executorch / build-wheel-py3_10-cpu-aarch64

pytorch/executorch / upload / upload-wheel-py3_10-cpu-aarch64

Test Vulkan Backend on: pull_request 5

test-vulkan / Resolve CI docker image / resolve

test-vulkan / test-backend-macos

test-vulkan / test-backend-linux (vulkan, models) / linux-job

test-vulkan / test-backend-linux (vulkan, operators) / linux-job

test-vulkan / package-golden-artifacts

pull on: pull_request 223

Resolve CI docker image / resolve

Get changed files / get-changed-files

CI run decision / decide

android / build-android

android / run-emulator

unittest / Resolve CI docker image / resolve

unittest / macos / macos-job

unittest / windows / windows-job

unittest / linux / linux-job

unittest-editable / Resolve CI docker image / resolve

unittest-editable / macos / macos-job

unittest-editable / windows / windows-job

unittest-editable / linux / linux-job

unittest-buck / Resolve CI docker image / resolve

unittest-buck / macos / macos-job

unittest-buck / windows

unittest-buck / linux / linux-job

test-qnn-testsuite-linux / Resolve CI docker image / resolve

test-qnn-testsuite-linux / test-backend-macos

test-qnn-testsuite-linux / test-backend-linux (qnn, models) / linux-job

test-qnn-testsuite-linux / test-backend-linux (qnn, operators) / linux-job

test-qnn-testsuite-linux / package-golden-artifacts

test-qnn-wheel-packages-linux (3.10) / linux-job

test-qnn-wheel-packages-linux (3.11) / linux-job

test-qnn-wheel-packages-linux (3.12) / linux-job

test-qnn-wheel-packages-linux (3.13) / linux-job

test-setup-linux-gcc / linux-job

test-models-linux-basic (mv3, portable, cmake, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job

test-models-linux-basic (mv3, portable, cmake, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc11-... / linux-job

test-models-linux-basic (mv3, portable, buck2, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job

test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, mt-l-x86iavx512-8-64, execu... / linux-job

test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, mt-l-arm64g4-16-62, executo... / linux-job

test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, mt-l-x86iavx512-8-64, execu... / linux-job

test-models-linux-basic (vit, portable, cmake, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job

test-models-linux-basic (vit, portable, cmake, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc11-... / linux-job

test-models-linux-basic (vit, portable, buck2, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clan... / linux-job

test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, mt-l-x86iavx512-8-64, execu... / linux-job

test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, mt-l-arm64g4-16-62, executo... / linux-job

test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, mt-l-x86iavx512-8-64, execu... / linux-job

test-models-linux (linear, portable, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (linear, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (add, portable, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (add, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (add_mul, portable, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (add_mul, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (ic3, portable, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (ic3, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (mv2, portable, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (mv2, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (resnet18, portable, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (resnet18, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (resnet50, portable, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (resnet50, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (mobilebert, portable, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (mobilebert, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (emformer_transcribe, portable, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, mt-l-x86iavx512-8-64) / linux-job

test-models-linux (ic4, portable, mt-l-x86iavx512-16-128) / linux-job

test-models-linux (ic4, xnnpack-quantization-delegation, mt-l-x86iavx512-16-128) / linux-job

test-models-linux (emformer_join, portable, mt-l-x86iavx512-16-128) / linux-job

test-models-linux (emformer_join, xnnpack-quantization-delegation, mt-l-x86iavx512-16-128) / linux-job

test-models-linux (phi_4_mini, portable, mt-l-x86iavx512-16-128) / linux-job

test-models-linux (llama3_2_vision_encoder, portable, mt-l-x86iavx512-16-128) / linux-job

test-models-linux (w2l, portable, mt-l-x86iavx512-16-128) / linux-job

test-parakeet-xnnpack-linux / linux-job

test-voxtral-realtime-xnnpack-linux / linux-job

test-llama-runner-linux (fp32, xnnpack+custom+qe, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-c... / linux-job

test-llama-runner-linux (fp32, xnnpack+custom+qe, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-gcc... / linux-job

test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, mt-l-x86iavx512-8-64, executorch-ubunt... / linux-job

test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, mt-l-arm64g4-16-62, executorch-ubuntu-... / linux-job

test-llama-runner-linux (fp32, xnnpack+quantize_kv, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04... / linux-job

test-llama-runner-linux (fp32, xnnpack+quantize_kv, mt-l-arm64g4-16-62, executorch-ubuntu-22.04-g... / linux-job

test-llama-runner-linux (bf16, custom, mt-l-x86iavx512-8-64, executorch-ubuntu-22.04-clang12) / linux-job

test-llama-runner-linux-android / linux-job

test-custom-ops-linux / linux-job

test-selective-build-linux / linux-job

test-multimodal-linux (gemma3-4b) / linux-job

test-moshi-linux / linux-job

test-quantized-aot-lib-linux / linux-job

test-binary-size-linux-gcc / linux-job

test-binary-size-linux / linux-job

test-arm-cortex-m-size-test (bare_metal) / linux-job

test-arm-cortex-m-size-test (zephyr-preset) / linux-job

test-mcu-cortex-m-backend / linux-job

test-qnn-buck-build-linux / linux-job

test-arm-backend-no-driver (test_pytest_ops_no_target) / linux-job

test-arm-backend-no-driver (test_pytest_ops_tosa) / linux-job

test-arm-backend-no-driver (test_pytest_models_tosa) / linux-job

test-arm-backend-no-driver (test_run_tosa) / linux-job

test-arm-backend-public-api-backward-compatibility / linux-job

test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job

test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job

test-static-llama-qnn-linux (stories_110m) / linux-job

test-static-llama-qnn-linux (stories_260k_bc) / linux-job

test-sqnr-static-llm-qnn-linux (smollm2_135m) / linux-job

test-qnn-models-linux (mv2) / linux-job

test-qnn-models-linux (mv3) / linux-job

test-qnn-models-linux (dl3) / linux-job

test-qnn-direct-build-linux / linux-job

test-qnn-passes-linux / linux-job

test-qnn-delegate-linux / linux-job

test-phi-3-mini-runner-linux / linux-job

test-qnn-python-imports-linux / linux-job

test-eval_llama-wikitext-linux / linux-job

test-llama_runner_eager-linux / linux-job

test-lora-linux / linux-job

test-lora-multimethod-linux / linux-job

test-mediatek-models-linux / linux-job

test-openvino-linux / linux-job

test-build-wasm-linux / linux-job

unittest-wasm-bindings / linux-job

unittest-wasm-bindings (--enable-etdump) / linux-job

unittest-nxp-neutron / linux-job

test-vulkan-models-linux / linux-job

test-vulkan-operators-linux / linux-job

nxp-build-test / linux-job

nxp-mcuxpresso-test / linux-job

test-arduino-library / Resolve CI docker image / resolve

test-arduino-library / run / arduino-library

test-minimal-wheel-linux / linux-job

llm-server / Resolve CI docker image / resolve

llm-server / linux / linux-job

test-coreml-bc-macos (macos-m1-stable) / macos-job

test-coreml-bc-macos (macos-m2-stable) / macos-job

Build Aarch64 Linux CUDA Wheels on: pull_request 4

generate-matrix / generate

filter-matrix

pytorch/executorch / build-wheel-py3_10-cuda-aarch6413_0-aarch64

pytorch/executorch / upload / upload-wheel-py3_10-cuda-aarch6413_0-aarch64

Build macOS Wheels on: pull_request 6

generate-matrix / generate

pytorch/executorch / build-wheel-py3_10-cpu

pytorch/executorch / upload / upload-wheel-py3_10-cpu

Build Linux Wheels on: pull_request 3

generate-matrix / generate

pytorch/executorch / build-manywheel-py3_10-cpu

pytorch/executorch / upload / upload-manywheel-py3_10-cpu

Code scanning results

CodeQL

Select a check to view from the sidebar