Skip to content

improved query extraction#1652

Merged
Artuomka merged 1 commit intomainfrom
backend_unsafe_query_logging
Mar 6, 2026
Merged

improved query extraction#1652
Artuomka merged 1 commit intomainfrom
backend_unsafe_query_logging

Conversation

@Artuomka
Copy link
Collaborator

@Artuomka Artuomka commented Mar 6, 2026

No description provided.

Copilot AI review requested due to automatic review settings March 6, 2026 12:20
@Artuomka Artuomka enabled auto-merge March 6, 2026 12:20
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the robustness of SQL query extraction from AI responses in two use cases that generate dashboard panels/widgets via AI. It strengthens the system prompt to more explicitly request raw SQL only, and adds logic to cleanQueryResponse to extract SQL statements when the AI includes surrounding explanatory text.

Changes:

  • Updated the query correction prompt in both use cases to use a stronger "CRITICAL" instruction telling the AI to respond with raw SQL only.
  • Added SQL extraction logic to cleanQueryResponse in both files to handle cases where the AI wraps the SQL statement in explanatory text, by finding SQL keywords at the start of a line and stripping surrounding non-SQL content.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
backend/src/entities/visualizations/panel-position/use-cases/generate-panel-position-with-ai.use.case.ts Strengthened AI prompt wording and added SQL extraction fallback logic to cleanQueryResponse
backend/src/entities/visualizations/panel-position/use-cases/generate-table-dashboard-with-ai.use.case.ts Identical changes: strengthened AI prompt wording and added the same SQL extraction fallback logic to cleanQueryResponse

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

);
const match = cleaned.match(sqlPattern);
if (match) {
const sqlStart = cleaned.indexOf(match[0]);
Copy link

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: cleaned.indexOf(match[0]) may find the wrong occurrence of the SQL keyword. When the regex matches a keyword at the start of a line (due to the m flag), it might not be the first occurrence of that keyword in the text. For example, if the AI response is "The SELECT query has been fixed:\nSELECT * FROM table;", the regex correctly matches SELECT at the start of the second line, but indexOf('SELECT') returns the position of SELECT in the first line's explanatory text.

Use match.index instead of cleaned.indexOf(match[0]) — the match object from a non-global regex already includes the position of the match.

Suggested change
const sqlStart = cleaned.indexOf(match[0]);
const sqlStart = match.index!;

Copilot uses AI. Check for mistakes.
);
const match = cleaned.match(sqlPattern);
if (match) {
const sqlStart = cleaned.indexOf(match[0]);
Copy link

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Same issue as in generate-panel-position-with-ai.use.case.tscleaned.indexOf(match[0]) may find an earlier occurrence of the SQL keyword in explanatory text rather than the one the regex matched at the start of a line. Use match.index instead, which directly gives the position of the regex match.

Suggested change
const sqlStart = cleaned.indexOf(match[0]);
const sqlStart = match.index ?? cleaned.indexOf(match[0]);

Copilot uses AI. Check for mistakes.
Comment on lines +506 to 533
// If AI included explanatory text around the SQL, extract the SQL statement.
const sqlPrefixes = ['SELECT', 'WITH', 'SHOW', 'DESCRIBE', 'DESC', 'EXPLAIN'];
const upperCleaned = cleaned.toUpperCase();
const startsWithSql = sqlPrefixes.some((prefix) => upperCleaned.startsWith(prefix));

if (!startsWithSql) {
const sqlPattern = new RegExp(
`^(${sqlPrefixes.join('|')})\\b`,
'im',
);
const match = cleaned.match(sqlPattern);
if (match) {
const sqlStart = cleaned.indexOf(match[0]);
let sqlBody = cleaned.slice(sqlStart);

sqlBody = sqlBody.replace(/```\s*$/, '').trim();

const lastSemicolon = sqlBody.lastIndexOf(';');
if (lastSemicolon !== -1) {
const afterSemicolon = sqlBody.slice(lastSemicolon + 1).trim();
if (afterSemicolon.length > 0 && /[a-zA-Z]{3,}/.test(afterSemicolon)) {
sqlBody = sqlBody.slice(0, lastSemicolon + 1);
}
}

return sqlBody.trim();
}
}
Copy link

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The entire cleanQueryResponse method (and buildQueryCorrectionPrompt, normalizeWhitespace, mapPanelType) are duplicated verbatim across this file and generate-panel-position-with-ai.use.case.ts. This duplication means any bug fix or improvement must be applied in both places, which is error-prone. Consider extracting the shared SQL-cleaning logic into a shared utility function (e.g., in a utils file alongside check-query-is-safe.util.ts) that both use cases can import.

Copilot uses AI. Check for mistakes.
@Artuomka Artuomka merged commit b476df9 into main Mar 6, 2026
21 of 23 checks passed
@Artuomka Artuomka deleted the backend_unsafe_query_logging branch March 6, 2026 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants