🧠 DeepQuery

AI-Powered SQL Intelligence & Optimization Console

A deep learning framework that profiles SQL execution plans, predicting performance bottlenecks and ranking alternative query formulations without requiring expensive test executions.

---

⚠️ Important Notice Regarding Model Weights & Data
Due to file size constraints, the pre-trained neural network artifacts (.keras, .pkl) and the benchmark databases (.db) have not been included in this public repository.

If you need access to the pre-trained weights or the generated datasets to run the project locally, please reach out directly:
📧 Contact: m.tahanasir.cs@gmail.com

---

⚡ Overview

Modern relational databases can execute a single logical data retrieval task using dozens of syntactically valid SQL formulations. However, their execution efficiency varies wildly based on CPU consumption, memory footprint, and execution plan cost.

DeepQuery is an intelligence console that bridges the gap between machine learning and database engineering. By parsing SQL Abstract Syntax Trees (AST) and dynamic DBMS execution statistics, it extracts a 14-Dimensional Feature Vector. This vector is evaluated by a trained 4-Layer Feedforward Multi-Layer Perceptron (MLP) to predict the query's efficiency, saving database administrators hours of manual inspection.

📸 Visual Tour

Side-by-side Multi-Candidate SQL Evaluation Console

Neural Network Performance Rankings with XAI Diagnostic Badges

Comparative Execution Time & Estimated Cost Analytics

3-Tier Progressive ER Benchmark Diagram

✨ Key Features

• 🔬 Multi-Candidate Evaluation: Input up to 8 candidate SQL formulations side-by-side to predict performance ranks before execution.
• 🧠 Explainable AI (XAI) Diagnostics: DeepQuery doesn't act as a black box. It automatically flags warnings (e.g., scalar functions bypassing indexes, SELECT * penalties) and highlights green flags (e.g., optimal index scan utilization).
• 📈 3-Tier Progressive Benchmark Suite:
  • Tier 1: Synthetic baseline (10,000 customers / 25,000 orders).
  • Tier 2: DVD Rental Schema (15 normalized tables).
  • Tier 3: TPC-H Decision Support (Industry standard analytical benchmark).
• 🖥️ Interactive Developer Dashboard: A modern UI featuring SVG circular score ring gauges, comparative performance bar charts, and floating toast notifications.

🏗️ Neural Network Architecture

DeepQuery utilizes a 4-Layer Sequential Artificial Neural Network engineered with Keras. It maps the extracted structural and heuristic query features to a normalized performance score ranging from 5.0 to 99.0.

Input: 14-Dimensional Feature Vector
  │
  ├─► Layer 1: Dense (128 units) ─► BatchNorm ─► ReLU ─► Dropout (0.3)
  ├─► Layer 2: Dense (64 units)  ─► BatchNorm ─► ReLU ─► Dropout (0.2)
  ├─► Layer 3: Dense (32 units)  ─► LeakyReLU (α=0.1)
  └─► Output Layer: Dense (1 unit) ─► Sigmoid Activation

• Performance: R² = 0.4256 | MAE = 0.1127 | RMSE = 0.1659
• Optimization: Adam Optimizer (lr = 0.001) with Mean Squared Error (MSE) Loss.

🧮 14-Dimensional Feature Matrix

Every candidate SQL query is evaluated against these normalized metrics:

Feature Dimension	Origin Source	Preprocessing & Transformation
query_length	Textual AST	$\log(1+x)$ + Z-Score Standardized
num_tables	SQL AST Parser	Z-Score Standardized
num_joins	SQL AST Parser	Z-Score Standardized
num_where_conditions	SQL AST Parser	Z-Score Standardized
has_select_star	AST Payload Analyzer	Direct Penalty Flag
func_in_where	AST Scalar Inspector	Direct Index Bypass Flag
estimated_cost	EXPLAIN QUERY PLAN	$\log(1+x)$ + Z-Score Standardized
index_scan_ratio	Plan Graph Analyzer	Direct Pass-Through
execution_time_ms	DBMS Profiler	$\log(1+x)$ + Z-Score Standardized

(Remaining dimensions are derived structural features calculated during the AST token walk).

💻 Installation & Quickstart

(Note: Ensure you have obtained the model weights and datasets from the author before running).

# 1. Clone the repository
git clone https://github.com/MuhammadTahaNasir/DeepQuery.git
cd DeepQuery

# 2. Install dependencies
pip install -r requirements.txt

# 3. Launch the master environment
python run_demo.py

run_demo.py automatically initializes the 3-tier benchmark databases, verifies dataset features, loads the pre-trained model artifacts, and serves the Flask web dashboard at http://127.0.0.1:5000.

📂 System Architecture

DeepQuery/
├── app.py                      Flask REST API server (/api/analyze)
├── run_demo.py                 Master launcher orchestrator
├── requirements.txt            Python dependencies
│
├── src/
│   ├── benchmark_db.py         3-tier database initialization engine
│   ├── dataset_generator.py    AST feature extraction & profiling
│   ├── explainer.py            XAI diagnostics & heuristic rules engine
│   ├── model.py                Keras Neural Network architecture definitions
│   └── train.py                Model training and evaluation loop
│
├── templates/
│   └── index.html              AI SQL Intelligence Console frontend
│
├── deepquery_model.keras       [NOT UPLOADED] 4-Layer ANN Keras model
├── scaler.pkl                  [NOT UPLOADED] Fitted StandardScaler artifact
├── benchmark.db                [NOT UPLOADED] Tier 1 database
├── pagila.db                   [NOT UPLOADED] Tier 2 database
└── tpch.db                     [NOT UPLOADED] Tier 3 database

---

Developed by Muhammad Taha Nasir with ❣️