Developed the Azure data factory pipelines for a load of sales files into Azure Data lake Storage (ADLS) with the time taken to load under 2 minutes for files in GB sizes.

Designed the notebooks that control the entire data load process and insert the data to DW.

Developed notebooks with effective transformations and optimized timings using Pyspark with azure data bricks as IDE.

Created and enabled email notifications using Azure Logic Apps for delivery, success, failures, and discrepancy of file load.