• Developed and maintained scalable backend data pipelines using Go to handle business logic and data ingestion for user accounts, class schedules, and rostering management, ensuring high performance and reliability.

• Implemented server-less architectures with AWS Lambda to automate and streamline data processing and data pipelining tasks, including event-driven updates and integration with external EdTech systems like Clever, ClassLink, Edlink and OneRoster.

• Designed and optimized relational databases in MySQL, managing and querying data related to user profiles, class assignments, and scheduling, resulting in improved data retrieval times and system efficiency.

• Worked with Devops teams to set up HoneyComb tracing and setup AWS Cloudwatch dashboards for monitoring improving observability into Data platform.

• Refactored and cleaned up legacy code within the rostering data pipeline to improve maintainability and pipeline ingestion performance, ensuring compatibility with modern technologies and reducing technical debt.

• Led the design and implementation of event-driven micro-services using Scala and RabbitMQ, automating the delivery of 1099-K forms to over 350,000 users, resulting in $250k of additional revenue.

• Used DBT to transform data from various sources such as transactional databases, API’s into clean, analytics ready datasets.

• Used Python to clean, wrangle unstructured data in varied formats such as ACH, API logs.

• Helped implement and maintain ETL jobs for our Snowflake data warehouse with dimensional modeling.

• Collaborated with cross-functional teams to design, scale, and implement ETL data pipelines to derive analytics from Ad-campaign data from multiple sources using Spark (AWS Glue), AWS S3 and Snowflake data warehouse.

• Developed interactive dashboards using Looker to visualize key metrics and enhance data driven decision-making.

• Automated critical data processing workflows by orchestrating pipelines with Airflow, improving scheduling efficiency and reducing manual intervention.

• Designed and implemented scripts for legacy data ingestion from external sources like AWS S3 and FTP servers, using Pandas for data cleaning and transformation.