Skip to content
View AMC-hawk's full-sized avatar
  • Kanpur, India

Block or report AMC-hawk

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
AMC-hawk/README.md
Anmol Mishra

Anmol Mishra

Software Engineer, Distributed Data Platform · Bangalore, India

Portfolio LinkedIn Email


What I actually do

I care about what happens to data before anyone models it.

Most of the interesting failures in a data platform never happen in the model. They happen three systems upstream, in a schema nobody owned, in a CDC stream that quietly dropped a column, in a join that was fine until the partition skewed. That is the layer I work at.

At Hewlett Packard Enterprise I design and operate the batch and streaming pipelines behind the Distributed Data Platform:

Postgres to Debezium CDC to Kafka to Spark to Iceberg to Trino to 15+ BI models

Scale What it means
1+ TB / day processed across batch and streaming pipelines, with reliable CDC and low-latency sync
15+ BI data models delivered for Customer Success, FinOps and Engineering
95%+ production incident SLA compliance

Most of my week is spent on the unglamorous half of that picture: partitioning and join strategy when a table outgrows its plan, resource tuning on Kubernetes, and the alerting that tells us something broke before a dashboard does.


Building

RaftDB, a mini distributed database written from scratch. I did not want consensus, recovery and query optimization to stay black boxes, so I built them: Raft consensus with leader election and log replication, a storage layer with WAL, snapshots, SSTables, Bloom filters and background compaction, and a cost-based SQL layer on Apache Calcite over the top. Java · Raft · LSM Tree · RocksDB · Apache Calcite

Data quality research, working toward a cloud-native data quality framework for open-source datasets. Right now that means reading widely across validation systems, error detection, label noise and LLM-driven cleaning, and writing up what actually holds.

Public experiments: AIOps Platform · JanusGraph · Airflow + Spark · MCP Server · MLflow · Scala REST API


Stack

Data Apache Spark Kafka Airflow Trino Iceberg

Languages Python Scala Java SQL

Platform AWS Kubernetes Docker Terraform Argo CD

Observability Prometheus Grafana OpenTelemetry


B.Tech Computer Science & Engineering, Vellore Institute of Technology · 8.81 / 10

Interested in distributed systems, database internals, performance engineering and AI-powered data platforms.

amc-hawk.github.io

Pinned Loading

  1. Crustapp Crustapp Public

    Forked from hasnain40247/Crustapp

    JavaScript

  2. Political-NLP Political-NLP Public

    Forked from NikhilC2209/Political-NLP

    A NLP based approach to use Twitter data to summarize the recent elections

    Jupyter Notebook 1

  3. Instagram-Backend-API Instagram-Backend-API Public

    This is the task done for Appointy

    Go

  4. Android-QR-Code-App-SCAMS Android-QR-Code-App-SCAMS Public

    App to scan the BAR code at Airport. Built in React Native & still work is going on!

    JavaScript 2

  5. VITSIC_Educational-Platform VITSIC_Educational-Platform Public

    IWP Project regarding VITSIC

    PHP

  6. Edit-Function_Integration-In-Real-Time Edit-Function_Integration-In-Real-Time Public

    Here, I have written the code to integrate a edit button into a basic database table which would retrieve the data of the row we want to edit and store the data into the edit module which would eas…

    PHP