Worked on the core platform & RESTful APIs to provide self-serve data orchestration
• Streamlined deployment of data extraction resources using state-management
• Aggregated logs for deployment & runtime operations to simplify troubleshooting for DevOps teams
• Automated product provisioning to provide the ELT pipeline in a trial setting
• Utilized caching to speed up API responses by a factor of 8
Analyzed technical stack in Amazon Web Services to identify cost-reduction improvements, leading to a cost reduction of 50% in development accounts
• Developed command-line tools to clean up excess resources in AWS S3 & CloudWatch
• Modified provisioning of AWS DynamoDB tables & Kinesis data-streams to optimize cost performance
Maintained ELT pipelines used by customers on a routine basis to ensure the flow of data
• Utilized AWS Console to investigate microservices to determine the cause of data gaps
• Coordinated efforts to address gaps in data & restore the pipeline