Feature Platform/Training Data

• Re-architected a Dataflow system for point-in-time data generation (See prior work: https://eng.snap.com/speed-up-feature-engineering), cutting costs by 30% and accelerating feature experimentation. Identified and resolved critical issues, including fixing a memory consumption bug in parquet-java and added support for complex types in Apache Dataflow's managed I/O iceberg reader.

• Drove various cost and latency reductions across core data pipelines used for feature logging (20% cost and latency reduction through round-robin partitioning) and offline dataset generation (10x cost reductions on feature skew)

Languages: Java, Python

Technologies: Spark, Flink, Kafka, Iceberg

• Worked on Amazon Music Podcasts focusing on Personalized Recommendations

• Designed and implemented a nearest neighbor vector search service. Stress-tested service at scale (> 500 TPS).

• Optimized EMR Spark batch workloads by up to 10x by investigating query plans, metrics, and cluster configurations. Experimented with Flink as a replacement for real-time feature ingestion and storage

• Profiled existing lambdas to identify and fix cold starts resulting in timeouts on both Alexa and Personalization. Saw a 90% reduction in Lambda Init latency for Cold Starts in Alexa by experimenting with Snapstart for Lambda

Languages: Java, Typescript, Python, NodeJS

Technologies: AWS CDK, DynamoDB, Express, EMR, OpenSearch, Step Functions, Lambda, Spark

• Worked on Data Infrastructure

• Built and maintained low latency, high throughput streaming pipelines at scale to deliver data to data warehouses

• Built data products for end-users to easily and reliably query information

Languages: Scala, Java, Python, JavaScript, Go, SQL

Technologies: Change Data Capture (Maxwell, Debezium, TiCdC), Kafka, Spark, Hudi (DeltaStreamer), Druid, Presto/Trino