Draft: Add new pre-configured model #138045
Conversation
jonathan-buttner
left a comment
jonathan-buttner
left a comment
There was a problem hiding this comment.
Looking good, left a few suggestions.
| } | ||
| """; | ||
|
|
||
| // Should we add gp-llm-v2 to the response? |
There was a problem hiding this comment.
Naa, you can leave this as is.
| private void assertChatCompletionEndpointExists() { | ||
| var eisEndpoints = getEisEndpoints(); | ||
| assertThat(eisEndpoints.size(), is(1)); | ||
| assertThat(eisEndpoints.size(), is(2)); |
There was a problem hiding this comment.
Let's leave this file unchanged, the logic here is just to make sure an endpoint gets created. I think it's fine if we stick with rainbow-sprinkles.
| .filter(endpoint -> endpoint.getService().equals(ElasticInferenceService.NAME)) | ||
| .toList(); | ||
| assertThat(eisEndpoints.size(), is(1)); | ||
| assertThat(eisEndpoints.size(), is(2)); |
There was a problem hiding this comment.
Same as above, let's remove these changes.
|
|
||
| // gp-llm-v2 | ||
| public static final String DEFAULT_CHAT_COMPLETION_MODEL_ID_V2 = "gp-llm-v2"; | ||
| public static final String DEFAULT_CHAT_COMPLETION_ENDPOINT_ID_V2 = ".gp-llm-v2"; |
There was a problem hiding this comment.
Based on the naming doc I've seen, I believe we want .gp-llm-v2-chat_completion
| } | ||
| } | ||
|
|
||
| // Should we add another test for gp-llm-v2? |
There was a problem hiding this comment.
Naa let's leave this as is.
| public static final String DEFAULT_CHAT_COMPLETION_ENDPOINT_ID_V1 = ".rainbow-sprinkles-elastic"; | ||
|
|
||
| // gp-llm-v2 | ||
| public static final String DEFAULT_CHAT_COMPLETION_MODEL_ID_V2 = "gp-llm-v2"; |
There was a problem hiding this comment.
I realize the convention here is DEFAULT_..., I think that's going to get confusing now that we'll have multiple of them.
How about we change this to be:
| public static final String DEFAULT_CHAT_COMPLETION_MODEL_ID_V2 = "gp-llm-v2"; | |
| public static final String GP_LLM_V2 = "gp-llm-v2"; | |
| public static final String GP_LLM_V2_ENDPOINT_ID = ".gp-llm-v2"; |
3 participants
gradle check?