Ignore flakey daily AI Logic test#8288
Conversation
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. |
Generated by 🚫 Danger |
📝 PRs merging into main branchOur main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released. |
| } | ||
|
|
||
| @Test | ||
| @Ignore("Model is prone to return specifically London instead of New York") |
There was a problem hiding this comment.
Also update the coordinates below.
| toolConfig = | ||
| TemplateToolConfig( | ||
| RetrievalConfig( | ||
| latLng = LatLng(latitude = 30.2672, longitude = -97.7431), |
There was a problem hiding this comment.
What is this test testing? The city based on the latitude = 30.2672, longitude = -97.7431 is near Austin, Texas. The test is expecting New York, and AI is returning London.
Cannot tell what do we want to test with this. The request is not rejected?
4 participants
As the comment says, the model is prone to return "London" instead of "New York". This is a reasonable test to keep around to base other tests on in the future, but it's introducing noise into our daily testing so we need to disable it.