> Designed and built end-to-end ETL machine learning pipeline from ingestion output

> Setup Kubernetes and Argo Workflow on local server to handle container orchestrating and job dispatching

> Optimized process build times to maximize CPU efficiency using all 20 threads

> Minimized training time while maintaining model performance by tuning hyperparameters

> Designed custom model architectures in Tensorflow and custom loss functions in Numpy to increase model performance

> Built backend for efficient asyncronous job processing using a DAG

> Used Data Science principles to design features leading to increased model performance

Machine Learning Pipelines:

> Data Modeling | Data Architecture | Database Architecture | Timeseries Forecasting

> Optimize Performance | Parallel Processing

> Hyperparameter Tuning | Preprocessing | Custom Loss Function | Feature Engineering

> Scikit-learn | SciPy | TensorFlow | Keras | PyTorch | CUDA | GPU vs CPU

Design and Build ETL pipeline:

> Data Modeling | Data Architecture | ML | AI

Development Tools:

>Python | PySpark | Spark | Jupyter Notebooks | Containerized Development

Data Ingestion:

> API | REST / RESTful | asyncronous client-server data streaming

Scalable Data Processing Solutions:

> Docker products | Kubernetes | Argo Workflow | Apache Airflow | Luigi | Container Orchestrator

Database Technologies:

> CSV | JSON | Relational Database

Self-motivated personal project I have been consistently working on, 10-30 hours/week.

> Build and maintain highly reliable computed tables, incorporating data from various sources, including unstructured and highly sensitive data

> Access, manipulate, and integrate external datasets with internal data

> Building analytical and statistical models to identify patterns, anomalies, and root causes

> Leverage SQL and Python to shape and aggregate data

• Support and maintain the primary production pipeline, ensuring accurate and on-time delivery of data for data consumers

• Business owner for a dev team (6) to ensure Continuous Improvement Continuous Deployment (CI/CD) - - Reporting process of ongoing enhancement, deployment, and monitoring of pipeline health through key metrics

• Oversaw transition into production for our cloud based data service "Skywise"

• Assessing the impact of upstream changes allowing prioritization and proper communication to customers

• Headed data governance initiative to lead new program and to provide algorithms to internal clients

• Developed a solution to enhance visibility into pipeline throughput, resulting in faster identification of various issues

• Investigated two algorithms designed to forecast system failures in advance of scheduled maintenance intervals

• Monitoring and improving data pipeline (ingestion to delivery) to meet KPI’s

Programming:

> C++ | OOP | Debugging

> Used system knowledge to solve issues in production

> Worked with system engineers to support with domain specific knowledge

> Supported real-time programing in time critical applications

> Memory Management