Serverless Async Image Processor ☁️

This project implements an Event-Driven architecture on AWS using Infrastructure as Code (IaC). It automates the ingestion and metadata processing of images uploaded to the cloud, decoupling the upload from processing via asynchronous events.

🏗 Architecture

The data flow is as follows:

1. The user uploads an image to the Source S3 Bucket.
2. S3 emits an ObjectCreated event that automatically triggers a Lambda function.
3. The function (written in Python) extracts technical metadata (size, MIME type) without downloading the full file to optimize costs and latency.
4. The results are persisted in a DynamoDB table with a NoSQL design.

🚀 Tech Stack

• Cloud Provider: AWS
• IaC: AWS CDK (Python Edition)
• Compute: AWS Lambda (Python 3.9)
• Storage: Amazon S3 & Amazon DynamoDB
• Best Practices: Principle of Least Privilege (IAM), Event-Driven Architecture.

💡 Technical Decisions & Architecture

Infrastructure as Code (CDK) Usage

AWS CDK in Python was chosen to define the infrastructure. This allows versioning the architecture alongside the application code, facilitates environment replication (dev/prod), and avoids manual configuration ("ClickOps") in the AWS console, following DevOps best practices.

Decoupled Event-Driven Architecture

The system does not have servers running waiting for requests. The entire flow is triggered by S3 events. This guarantees near-zero cost when the system is idle and near-infinite scalability during sudden load spikes.

Data Separation (Raw vs. Processed)

A "Source Bucket" and "Destination Bucket" pattern was implemented. The original image is never overwritten. This is crucial for maintaining data integrity (raw "data lake") and allowing future reprocessing if business logic changes.

Lambda Layers for Complex Dependencies (Pillow)

The Pillow library (required for image processing) contains C-compiled dependencies (libjpeg, etc.) that must be compatible with the Lambda Amazon Linux environment. Instead of packaging these heavy libraries in every function deployment, Lambda Layers were used. This offers several advantages:

1. Faster Deployments: The Lambda function code package remains very lightweight (containing only business logic).
2. Reusability: The Pillow layer can be shared by other Lambda functions in the same account.
3. Clean Dependency Management: Clearly separates proprietary code from third-party libraries.

DynamoDB for Metadata

The nature of image metadata (key-value, flexible schemas) fits perfectly with NoSQL. DynamoDB in PAY_PER_REQUEST mode offers millisecond latency without fixed provisioning costs.

🛠 Installation and Deployment

This project uses AWS CDK. To deploy it to your own AWS account:

# 1. Clone repository
git clone [YOUR_GITHUB_URL]
cd image-processor

# 2. Install dependencies
pip install -r requirements.txt

# 3. Generate the Pillow layer (Linux Binaries)
mkdir -p layers/pillow/python
pip install Pillow --target layers/pillow/python --platform manylinux2014_x86_64 --implementation cp --python-version 3.9 --only-binary=:all: --upgrade

# 3. Synthesize CloudFormation template
cdk synth

# 4. Deploy to AWS
cdk deploy