Specscheduler#90

Open

chenzhuofu wants to merge 759 commits intoinferencefrom

This pull request is big! We're only showing the most recent 250 commits

Commits on Sep 7, 2024

Commits on Sep 8, 2024

fix: request expected latency
chenzhuofu
committed

Commits on Sep 11, 2024

Commits on Sep 12, 2024

Merge pull request #1494 from flexflow/specscheduler-request-emission
chenzhuofu
authored

Commits on Sep 16, 2024

Commits on Sep 17, 2024

Commits on Sep 20, 2024

feat: add emission output
chenzhuofu
committed

Commits on Sep 21, 2024

Dynamically control tree width to not exceed max_tokens_per_ssm_batch.
chenzhuofu
committed

Commits on Sep 22, 2024

Simplified the method to add tokens to the token trees.
chenzhuofu
committed

Commits on Sep 26, 2024

feat: add TraceEmissionMachine
chenzhuofu
committed

Commits on Sep 28, 2024

Add back old scheduler
chenzhuofu
committed

Commits on Oct 2, 2024

Commits on Oct 3, 2024

feat: add old_scheduler option
chenzhuofu
committed
feat: cherry-pick https://github.com/flexflow/FlexFlow/commit/9784b5c6516bafe272fc6555daaa9b867a5eacfa

jiazhihao
authored and
chenzhuofu
committed
update legion version

goliaro
authored and
chenzhuofu
committed
Fix nccl-induced segfault (#1481)

goliaro
authored and
chenzhuofu
committed
add page_manager and request_manager functions
Qinghan Chen
committed
add batch_config
Qinghan Chen
committed

Commits on Oct 6, 2024

Commits on Oct 8, 2024

Commits on Oct 9, 2024

chore: add more profiling
chenzhuofu
committed

Commits on Oct 10, 2024

ckpt before build
chenzhuofu
committed
some fix
Bob-Chen222
committed
ready for sanity check
Bob-Chen222
committed
Merge remote-tracking branch 'origin/specscheduler' into paged_attention_new
Bob-Chen222
committed
fix last commit index
Bob-Chen222
committed

Commits on Oct 11, 2024

chore: minor
Bob-Chen222
committed
fix request id error
Bob-Chen222
committed
fix: allreduce should handle `elts==0`
Bob-Chen222
committed
fix spec token num
chenzhuofu
committed
fix small error in free_multiple_blocks
chenzhuofu
committed
ckpt single request
Bob-Chen222
committed
add cleanup
Bob-Chen222
committed
ckpt before index error in prepare_parameters
Bob-Chen222
committed
fix: embedding use real batch_size
chenzhuofu
committed
fix: residualRMSNorm uses real batch size
chenzhuofu
committed
fix token error in prepare_batch_config
Bob-Chen222
committed
ckpt, something wrong in the prefilling
Bob-Chen222
committed

Commits on Oct 12, 2024

fix: SigmoidSiluMulti uses real batch size
chenzhuofu
committed
style: format
chenzhuofu
committed
ckpt
Bob-Chen222
committed
Merge remote-tracking branch 'origin/specscheduler' into paged_attention_new
Bob-Chen222
committed
update
Bob-Chen222
committed
chore: minor
chenzhuofu
committed

Commits on Oct 13, 2024

Commits on Oct 14, 2024

Commits on Oct 15, 2024

Commits on Oct 16, 2024

feat: add two naive scheduling policies
chenzhuofu
committed

Commits on Oct 20, 2024

Merge remote-tracking branch 'origin/specscheduler' into paged_attention_new
Bob-Chen222
committed

Commits on Oct 21, 2024

add some docuementation and delete print
Bob-Chen222
committed
add additional flag max-kv-cache-size
Bob-Chen222
committed
chore: typo
chenzhuofu
committed

Commits on Oct 22, 2024

fix tokenizer conversion

zhihao
authored and
sfc-gh-goliaro
committed
update

zhihao
authored and
sfc-gh-goliaro
committed
update

zhihao
authored and
sfc-gh-goliaro
committed
update
sfc-gh-goliaro
committed
add special tokens
sfc-gh-goliaro
committed
Update LLAMA tokenizer (#1524)
sfc-gh-goliaro
committed
rope
sfc-gh-goliaro
committed
fix
sfc-gh-goliaro
committed
fix
sfc-gh-goliaro
committed
linting
sfc-gh-goliaro
committed
fix
sfc-gh-goliaro
committed
fix
sfc-gh-goliaro
committed
Merge pull request #1530 from flexflow/sd
chenzhuofu
authored

Commits on Oct 24, 2024

Commits on Oct 26, 2024

fix
jinhongyii
committed

Commits on Oct 30, 2024

feat: update weight file naming style

goliaro
authored and
chenzhuofu
committed
fix: file_loader

goliaro
authored and
chenzhuofu
committed
fix

goliaro
authored and
chenzhuofu
committed
fix

goliaro
authored and
chenzhuofu
committed
feat: update file_loader to latest ver. on `peft`
chenzhuofu
committed
Merge branch 'specscheduler' of github.com:flexflow/FlexFlow into specscheduler
chenzhuofu
committed

Commits on Oct 31, 2024

fix: alignment issue
chenzhuofu
committed

Commits on Nov 2, 2024

fix: use double for latencies
chenzhuofu
committed

Commits on Nov 4, 2024

feat: modified the logic of distributing the budget across requests
chenzhuofu
committed
Merge remote-tracking branch 'origin/specscheduler' into paged_attention_new
Bob-Chen222
committed
fix for merge
Bob-Chen222
committed
init page manager at request manager init and clean the format
Bob-Chen222
committed

Commits on Nov 5, 2024

ckpt
Bob-Chen222
committed
refactor and add kv cache flag via page manager
Bob-Chen222
committed
ckpt for performance issue
Bob-Chen222
committed
first attempt in incr decoding with page attention
Bob-Chen222
committed

Commits on Nov 6, 2024

ckpt for nothing
Bob-Chen222
committed

Commits on Nov 7, 2024

feat: modify the logic of the scheduler
chenzhuofu
committed
fix compilation error
Bob-Chen222
committed
all good for spec, now test incr
Bob-Chen222
committed
typo
Bob-Chen222
committed
workable incrdecoding!
Bob-Chen222
committed
Merge remote-tracking branch 'origin/specscheduler' into paged_attention_new
Bob-Chen222
committed

Commits on Nov 8, 2024

Commits on Nov 14, 2024

feat: load weights in parallel

goliaro
authored and
chenzhuofu
committed
fix: compile bug
chenzhuofu
committed
feat: upgrade to llama3 rope
chenzhuofu
committed

Commits on Nov 15, 2024

Commits on Nov 17, 2024

fix: argtopk memory
chenzhuofu
committed

Commits on Nov 21, 2024

feat: add flashinfer ResidualRMSNorm
chenzhuofu
committed

Commits on Nov 28, 2024

Commits on Nov 29, 2024

Commits on Dec 4, 2024

Commits on Dec 5, 2024

fix: update tree depth
chenzhuofu
committed

Commits on Dec 6, 2024

Commits on Dec 8, 2024

Commits on Dec 10, 2024

Commits on Jan 24, 2025

:Merge branch 'specscheduler' of https://github.com/flexflow/flexflow-serve into paged_attention_new
chenzhuofu
committed

Commits on Jan 31, 2025