AI for Large Scale Systems

Take AlloyDB 0 to 1 and continue improving it.

Tech lead in the Database High Availability and Service Infrastructure team:

* Led support for near-infinite scaling using dynamic SSD allocation and management

* Led support for multi-major versions in AlloyDB

* Led design and development of the AlloyDB database Stop and Start functionality

* Led development and scaling of robust logging infrastructure

* Designed and deployed an auto-tracking mechanism for temporary disk state in per-instance configs, eliminating 4–6 downtimes weekly for large workloads

* Implemented a smarter startup sequence that improved instance creation times by 20–22%

* Built deployment and parallel rollout infrastructure to keep customers happy and AlloyDB secure

* Designed and led the development of atomic cluster creation with clean rollback mechanisms, ensuring zero partial-failure states in production

* Contributing to several new initiatives in high availability and auto-attribution using LLMs, RAG and Gemini

DynamoDB Transactions service, Distributed Ledger Sweeper, NoSQL Workbench, DynamoDB Local, NoSQL Data Modelling

• Implemented several features in the core Transactions service and led implementation of distributed sweeper

• Performance/load testing for DynamoDB Transactions service

• Internet-scale infrastructure management using scripting for Hailstone host cleaning/release

• Test cluster bootstrap automation for DynamoDB Transactions

• Designed and built out multiple parts of NoSQL data modelling tool (DynamoDB Workbench) from scratch and led to GA

• Led feature development for DynamoDB Local and NoSQL Workbench

Helped to build the R13n Platform from scratch. Led development of Datapath service and ROLSS which are the core components of that system. Owned integration into upstream/downstream services.

Led design, development and operations for software platform that quantified $1.2B worth of cost opportunity for Amazon.

# Data Processing Framework based on Apache Spark, Kafka, Zookeeper, YARN and AWS EMR, S3.

• Developed and tested highly configurable Apache Spark based data processing ETL framework

• Automated complex data processing workflows & operationalized pipelines using Kafka, Zookeeper, Jenkins, EMR

• Re-architected and optimized release process for the offline data processing pipelines to reduce deployment time of software and configuration by 75%

• Prototyped Apache Spark based streaming pipeline for real-time incremental updates

# Web service for EMR Cluster management and YARN Job Submission

• Developed RESTful web service for centralized cluster management of all deployed AWS EMR clusters

• Implemented frontend and activity dashboard for this web service using Flintjs & Javascript

• Implemented single-click YARN job submission & tracking using predefined job templates and job submit history

• Deployed the distributed software stack using Docker, Jetty, AWS RDS, AWS EC2 and AWS EMR.

# Content Discovery and Publishing tool

• Architected & developed scraper for AWS S3/Aliyun OSS that discovers content using topological folder access

• Prototyped content injection tool that leverages AWS SNS and SQS for reliable and seamless content discovery

• Operationalized software using Jenkins, Zookeeper and Kafka

Technologies used: Apache Spark, Apache Kafka, Apache Zookeeper, AWS EMR, AWS S3, AWS RDS, flintjs, Bash, Jenkins, Scrum, Git/gitflow