Created various frameworks from scratch using Java (TestNG, Junit, Serenity, Cucumber) as well as Python using Pytest. Contributed and lead efforts to test various projects around big data and ETL validations in multiple projects. Utilized cutting edge technologies like Snowflake, AWS, ADO, Kafka and Spark to drive tests and deliver products.
Created test suites to validate critical files generated by the emergency response system using Python, Pytest, Pandas and Pyspark utilizing fixtures and markers to data drive smoke and regression tests.
Created the ETL test suite that validates the transformation of critical emergency response system outputs for Northern California customers.
Created pytest tests to validate parquet file transformations through multiple data layers in a Cloud environment.
Created a utility to automatically pull files under test from S3 and run the tests. This helped reduce time it takes to prepare automation.
Validated Airflow data ingestion to Snowflake data warehouses.
Created a python and pytest test suite that lived on server to perform sanity validation of critical api
and data flow.
Oversaw data quality testing during our data flow changes and achieved 100 percent automation.
This was achieved by automating Kafka message production to validate data flows to critical Data lakes as well as creating tests to validate important information is properly ingested into newer data infrastructure.
Bootstrapped API and end-to-end testing for existing flows using JavaScript/Python and configured them to run on Saucelabs on AWS.