Youtube_Tool with gemini support

[OmkarBansod02]

No description provided.

[greptile-apps]

Greptile Overview

Greptile Summary

This PR adds YouTube video analysis capabilities using Gemini's API, allowing the agent to summarize videos, answer questions about video content, and extract transcripts.

Key Changes:

• New YouTubeTool that integrates with Gemini API for video analysis
• Comprehensive prompt instructions to guide the agent when to use YouTube analysis vs browser automation
• Support for three actions: summarize, ask (with question), and transcript
• Auto-detection of YouTube video URL from current tab

Major Issues Found:

• Critical: The Gemini API call uses incorrect format - file_data.file_uri with a YouTube URL directly, which likely won't work with Gemini's API
• Style Violation: Hardcoded default value 'gemini-2.5-flash' instead of using a constant as required by style guide

Positive Aspects:

• Good separation of concerns with helper functions
• Proper error handling and validation
• Debug logs were cleaned up per custom instructions
• Tool registration follows existing patterns

Confidence Score: 2/5

• This PR has a critical implementation issue that will likely cause runtime failures
• The core functionality (Gemini API call for YouTube video analysis) uses an incorrect API format that will likely fail at runtime. The file_data.file_uri field expects uploaded file references, not direct YouTube URLs. This needs to be fixed before merging.
• Pay close attention to src/lib/tools/YouTubeTool.ts lines 159-170 - the Gemini API request format needs correction

Important Files Changed

File Analysis

Filename	Score	Overview
src/lib/tools/YouTubeTool.ts	2/5	New YouTube video analysis tool with critical API usage issue - uses incorrect format for Gemini file_data parameter
src/lib/agent/BrowserAgent.prompt.ts	4/5	Added comprehensive YouTube tool instructions to agent prompts with clear examples and warnings

Sequence Diagram

sequenceDiagram
    participant User
    participant BrowserAgent
    participant YouTubeTool
    participant LLMSettingsReader
    participant GeminiAPI
    participant BrowserContext

    User->>BrowserAgent: Task: "Summarize this YouTube video"
    BrowserAgent->>BrowserAgent: ClassificationTool analyzes task
    BrowserAgent->>BrowserAgent: PlannerTool creates execution plan
    BrowserAgent->>YouTubeTool: Call youtube_tool(action='summarize')
    
    YouTubeTool->>LLMSettingsReader: Check if Gemini is configured
    LLMSettingsReader-->>YouTubeTool: Return Gemini provider config
    
    alt No Gemini API key
        YouTubeTool-->>BrowserAgent: Error: Gemini API required
    else Gemini available
        YouTubeTool->>BrowserContext: Get current page URL
        BrowserContext-->>YouTubeTool: Return YouTube video URL
        
        YouTubeTool->>YouTubeTool: Extract video ID from URL
        YouTubeTool->>YouTubeTool: Prepare prompt based on action
        
        YouTubeTool->>GeminiAPI: POST /generateContent<br/>(with video URL + prompt)
        GeminiAPI-->>YouTubeTool: Return video analysis
        
        YouTubeTool-->>BrowserAgent: Success: { summary: "..." }
    end
    
    BrowserAgent-->>User: Display video summary

Loading

_{4 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

logic: incorrect file_data format - Gemini API expects YouTube URLs to be passed differently

The current code passes the YouTube URL directly as file_uri, but Gemini's API doesn't accept YouTube URLs in the file_data.file_uri field. That field is for files uploaded via the File API. For YouTube videos, you should pass the URL directly in a text part or use a different approach.

Check Gemini documentation for correct YouTube video analysis format.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/lib/tools/YouTubeTool.ts
Line: 163:163

Comment:
**logic:** incorrect `file_data` format - Gemini API expects YouTube URLs to be passed differently

The current code passes the YouTube URL directly as `file_uri`, but Gemini's API doesn't accept YouTube URLs in the `file_data.file_uri` field. That field is for files uploaded via the File API. For YouTube videos, you should pass the URL directly in a text part or use a different approach.

Check Gemini documentation for correct YouTube video analysis format.

How can I resolve this? If you propose a fix, please make it concise.

[felarof99]

@OmkarBansod02 Even after I set the gemini API key, I still get this error.

Could you test it out again? Also, could you pull changes from HEAD?

And when gemini API key is not present, in the prompt mention that model should alert that to the user.

[felarof99]

Doesn't work still! I tested the gemini API key is correct because test works. Implementation very brittle.

[felarof99]

@OmkarBansod02

[felarof99]