Storage Infrastructure - Cassandra, Kafka, CRDB

Building Google scale infrastructure for stream-based data processing. The framework provides infra-level features necessary for running a data pipeline in production such as event batching, error retrial, and scheduling of work units.

Stack: C++

Built a Cassandra orchestration service in Java which facilitated the rollout of a major DB version upgrade; the service automates cluster operations needed to carry out tests for data loss and performance regressions in large scale clusters.

Stack: Java, Kubernetes, Cassandra

• Improved performance of the storage engine by eliminating wasteful disk writes and implementing lazy file creation during the Raft snapshotting process in Go

• Increased compatibility of Cockroach DB with PostgreSQL by adding support for missing system catalogs

• Created a web app to demo Cockroach without requiring a local download

• Redesigned the Routing config architecture to make it possible to infer the city of a routing request from its GPS coordinates and thus apply a per-city configured Routing stack to each request

• Enabled switchback experiments to run in parallel across different cities which unblocked several teams’ iteration cycles and increased the overall Routing experimentation throughput by 20x

• Created a city-id lookup service in Java to serve over 200k requests per second