Feature/isaac 0.1

[oscardev256]

Purpose

Support https://huggingface.co/PerceptronAI/Isaac-0.1 on vLLM: [New Model]: Issac 0.1 #25448

Test Plan

Compare output to Hugging Face example repo:
https://github.com/perceptron-ai-inc/perceptron/tree/main/huggingface

Test Result

Output from Hugging Face example repo:

<|endoftext|><think>

</think>

No, it is not safe to cross the street at this time. The <point_box mention="traffic light"> (808,248) (863,386) </point_box> in the background is showing a red signal, which means it's not safe or legal to cross the street. This red light indicates that vehicles have the right of way, and pedestrians should wait until the light changes before proceeding to cross. It's important to always follow traffic signals for your safety and the safety of others on the road. When in doubt, it's best to wait until the signal changes to green before crossing the street.<|im_end|>

Output from vLLM implementation:

<think>

</think>

No, it is not safe to cross the street at this time. The <point_box mention="traffic light"> (808,248) (863,386) </point_box> is red, indicating that pedestrians should not cross. Additionally, there are <point_box mention="cars"> (712,461) (966,608) </point_box> visible on the road, which means it's important to wait until the signal changes before proceeding to cross. It's crucial to always follow traffic signals and be aware of your surroundings when navigating busy urban areas to ensure your safety and the safety of others.<|im_end|>

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

[KKSK-DON]

Hi Oscar. I'm very interested in this feature. Considering you meet a obstacle, if you do not mind, could I take over this feature ? I'm working on it and checking if we can use exist SiglipEncoder now. @oscardev256

[oscardev256]

Hi @KKSK-DON, I'm open to collaborating on this if that works with you. Let me know how you'd like to proceed.

I just so happened to update the code to use the existing SiglipEncoder from siglip2navit.py, and I pushed the changes after testing.

I also rebased previous commits to include the Signed-off-by line in the commit message.

[KKSK-DON]

Hi @oscardev256 , I’m very happy to collaborate on this. May I push commits directly to this PR? This is my first time co-authoring an open-source PR, and as far as I understand I probably need to be added as a collaborator on your fork in order to be able to push commit.

[oscardev256]

@KKSK-DON Invitation sent. Note that I rebased yesterday, so you may need to pull the latest changes.

[KKSK-DON]

vllm serve PerceptronAI/Isaac-0.1 --trust-remote-code --max-model-len 16384 --max-num-batched-tokens 16384 --tensor-parallel-size 1

is showing load correctly

[mergify]

Documentation preview: https://vllm--28367.org.readthedocs.build/en/28367/