Unified Data Warehouse (UDW):
Migrated Petabyte-Wise Data and its Associated Services from PostgreSQL, HBase, S3, etc. to Snowflake.
• Developed and maintained 20+ (50%) App Annie customer APIs using Flask in Python; conducted unit test, functional test, robustness test, load test, and smoke test to ensure API qualities
• Initiated a testing architecture to automatically collect user-queried URLs from production and compared response results between refactored APIs and original ones to avoid data inconsistency, reducing estimated 1-week QA effort into 3 hours
• Migrated data sources of 15 Web APIs and Ajax from PostgreSQL and HBase to Snowflake; increased the overall API/Ajax efficiency by 50%, e.g., reducing the response time of App Summary API (60% of overall API traffic) from 4s to 2s
• Collected missed 1-year, 1T log from AWS Cloud Watch, analyzed the lost log, and optimized the original cron job from 10mins to 45s with 25.8% less memory usage to generate daily customer API usage report for PMs to reduce churn rate
• Designed 3 data models in App Annie Query Language (AQL), an internal OLAP-based query system, allowing users to query desired metrics from different dimensions and reducing the processing time of data aggregation
