• Led the migration of model inference from hosted on the external cloud service Modal to self-hosted Ray cluster running on Kubernetes to bring inference in-house for lower costs and optimized pipelines

• Set up tools and infrastructure to allow spinning up new models in hours

• Designed and generated Python and Typescript SDK to get models in the hands of customers within the day of the request

• Optimized various model inference with batching, vLLM and PyTorch, reducing latency of pipelines by up to 80%

• Led the setup of tracing by integrating OpenTelemetry with Jaeger and augmented Ray logs and Prometheus metrics to allow SDK developers to get features out in hours rather than days and help pinpoint bottlenecks in customers’ AI pipelines

• As first backend hire, laid down the foundation of the company’s microservices through implementing the original services, setting up Kubernetes cluster with Helm and Terraform on EKS, and the CI/CD pipeline with Github Actions

• Architected and led the implementation of the Control Plane-Data Plane split utilizing Kubernetes Operator pattern to make the system declarative, resulting in the creation of the egress only Data Plane agents to be installed in customers’ VPC that are orchestrated by the hosted Control Plane, leading to significant reduction of deployment time from weeks to just hours

• maintain CLI for Confluent Cloud and Confluent Platform (on-prem)

• add observability to the CLI by integrating Segment Analytics to track user actions

• using the observability metrics and customer's feedback to improve user experience

• add automatic refreshing of authentication token, and support for JSON and YAML output to allow scripting of the CLI

• Implemented automated CLI binaries build, testing and release pipeline involving Goreleaser tool, bash script, GNU Make script, AWS S3 and Azure DevOps Pipelines

• Automated end-to-end cloud system test for Cloud CLI

• Taught two different lectures of five students weekly and hosted office hours

• Taught data structures using Java