Architected large-scale batch and streaming data pipelines using Python, Kafka, Airflow, Snowflake, and Delta Lake to power various workloads.

Developed platform APIs (REST/gRPC) enabling seamless integration between internal systems, orchestration layers, and data platform services.

Led warehouse workload optimization and performance tuning, delivering infrastructure cost savings.

Designed governance and compliance frameworks (deletion, lineage, classification, retention) ensuring secure and regulated data usage across enterprise datasets.

Implemented automated data quality and observability systems with custom monitoring, validation, and alerting for mission-critical datasets.

Collaborated cross-functionally with ML, analytics, and engineering teams to build reliable, AI-ready data foundations and developer tooling.

Built and operated batch and event-driven ETL pipelines using Spark, Python, and AWS services, processing 250M+ records/day at scale.

Led migration from monolithic architecture to microservices with concurrent and asynchronous processing patterns, improving runtime performance by ~50% and enhancing system observability.

Designed end-to-end ML pipelines covering data ingestion, preprocessing, training, deployment, and continuous inference for production ML workloads.

Developed scalable inference APIs using FastAPI and async frameworks to support real-time classification and translation services.

Implemented GPU-based model serving infrastructure on Kubernetes, reducing inference latency and improving throughput for AI workloads.

Part of NGINX Controller analytics module development team, enabling better monitoring & management of metrics/events data and providing API driven development support.

Worked on containerized and microservices based architecture, enabling different REST based API's and gRPC based communication between services, using golang .

Managed authentication using JWT tokens , metrics/events processing using NATS / Kafka message brokers

Developed a monitoring system which analyzes traffic based on moving averages, over different nginx agents using python.

Skills Used : Python , GoLang, Analytics and Containerization powered by kubernetes, NATS and Kafka (streaming and Publish/Subscribe)

Engineered and Developed cloud automation solutions using python programming , AWS cloud and machine learning techniques like clustering, regression, neural networks and dimensionality reduction

Worked on Various AWS services like EC2, Lambda, AWS Batch, IAM, CloudWatch, API Gateway, S3, Elastic Search, Elastic Cache (Redis) and Dynamo DB.

This Research Internship experience introduced various concepts of data mining and worked on a research problem "Image Compression using Pattern Mining"

https://www.researchgate.net/publication/328372307_Lossy_Image_Compression_using_Frequent_Pattern_Mining_based_Huffman_Encoding