Medical Image Segmentation Task

Segmenting Pancreatic & Liver Tumors in Ultrasound Images using SAM2

This project implements an automatic segmentation pipeline for identifying pancreatic and liver tumors in 2D ultrasound (US) images. It utilizes the Segment Anything Model 2 (SAM2) from Meta AI to perform promptable, zero-shot segmentation.

The core of this project is to evaluate SAM2's effectiveness on this specialized, limited medical dataset and analyze how different prompt types (bounding boxes vs. points) impact segmentation accuracy.

This repository contains the complete pipeline in the Medical_Image_Segmentation_Task.ipynb notebook, covering data preprocessing, model setup, inference, evaluation, and visualization.

📊 Example Results

The model demonstrates a strong ability to segment the region of interest when provided with a bounding box prompt. Point-based prompts were significantly less effective for this specific task.

Caption: Example segmentation from the test set. Left: Original US Image. Middle: Ground Truth Mask. Right: SAM2 BBox-prompted Prediction.

🚀 Key Findings & Performance

The evaluation was performed on a 20% hold-out test set (18 images). To simulate a "perfect" user input for a fair zero-shot evaluation, prompts (bounding boxes and points) were generated directly from the ground truth masks.

Key Finding: Bounding box prompts significantly outperform point-based prompts for this dataset, achieving a mean Dice score of 0.8770. Single and multi-point prompts struggled to identify the correct object boundary, often segmenting only a small, ambiguous portion of the tumor.

Performance Metrics (on Test Set)

Prompt Type	Metric	Mean	Std. Dev.	Min	Max
Bounding Box	Dice	0.8770	0.0596	0.7498	0.9386
	IoU	0.7854	0.0898	0.5997	0.8842
Single Point (Centroid)	Dice	0.2095	0.3264	0.0147	0.9068
	IoU	0.1664	0.2879	0.0074	0.8294
Multi-Point (5 points)	Dice	0.3343	0.4065	0.0141	0.9230
	IoU	0.2878	0.3785	0.0071	0.8571

🛠️ Methodology

The pipeline follows these steps, as implemented in the notebook:

1. Data Preparation: The 105 US images and their corresponding masks were loaded. The notebook found 86 valid image-mask pairs. All images and masks were resized to 1024x1024 for compatibility with the SAM2 model.
2. Model Setup: The sam2_hiera_large model was loaded from the official SAM2 GitHub repository using the SAM2ImagePredictor.
3. Prompt Generation: To evaluate zero-shot performance, prompts were automatically generated from the ground truth masks:
   • Bounding Box: The smallest box enclosing the mask (with 10px padding).
   • Single Point: The centroid of the mask.
   • Multi-Point: 5 equidistant points sampled from the mask's contour.
4. Evaluation: The dataset was split 80/20. The model's predictions on the test set were compared against the ground truth masks using Dice Score and Intersection over Union (IoU).
5. Analysis: Results were aggregated, saved to CSV in the sam2_results/ folder, and visualized.

🔧 How to Run

🚀 Usage

1. Clone the repository:

   git clone https://github.com/saoud30/Medical-Image-Segmentation-Task.git
   cd Medical-Image-Segmentation-Task

2. The Dataset Already Present :

   • Download the dataset from the source: 100+ 2D US Images and Tumor Segmentation Masks
   • Zip the contents into a file named ultrasound_dataset.zip and place it in the root directory.

3. Install Dependencies:

   • The notebook (Cell 1) handles all installations, including sam2 from GitHub.

pip install -q torch torchvision opencv-python pillow matplotlib pandas tqdm scikit-learn
pip install -q 'git+[https://github.com/facebookresearch/sam2.git\](https://github.com/facebookresearch/sam2.git)'

4. Download Checkpoint:
   • Cell 1 also automatically downloads the sam2_hiera_large.pt checkpoint into a ./checkpoints/ directory.
5. Run the Notebook:
   • Open and run all cells in Medical_Image_Segmentation_Task.ipynb.
   • The notebook will automatically unzip the data, run the evaluation, and save results (CSV files and visualizations) to the sam2_results/ directory.

Acknowledgements

• Model: This project uses the SAM2 model by Meta AI.
• Dataset: The dataset of 105 2D US Images and Tumor Segmentation Masks was provided by P. G. C. N. M. B. Ekanayake et al.
• Reference Paper: [https://arxiv.org/abs/1710.02984\]