Developing Snowpark pandas

Acquired by Snowflake

I developed an end-to-end data pipeline to write broker axe data to Parquet files from a Java app server with Spark and Scala and load into a Snowflake database using Python and SQL. I then utilized query optimization using PrestoDB and Snowflake analytics and improved quantitative model query times from 134 to 0.084 seconds.

I automated front-end web browser testing for NERSC JupyterLab extensions using Selenium framework, Python pytest, and Travis CI continuous integration service and implemented data mocking to test REST API back-end functionality for Slurm workload manager extension. This increased JupyterLab testing capabilities and efficiency by fivefold and I presented the project at JupyterCon 2020.