o Designed and built a Data Lake using Azure’s HDInsight, which stores marketplace data, fulfillment center, SKU information, catalogs, pricing for analysis.

o Built a streaming analytics layer using Apache Kafka to support real-time analytical reporting in addition to a batch processing layer using the Lambda implementation pattern.

o The current architecture provides an event-driven real-time processing engine using Spark and Oozie, alongside a batch processing layer transforming structured, semi & unstructured data stored in Hive & MySQL tables.

o Operationalized data and ML pipelines using Automic for orchestration and monitoring, using the TICK stack (Telegraf, InfluxDB, Chronograf, and Kapacitor).

o Currently integrating data pipeline between Walmart and Jet.com environments in-order to share inventory & customer data.

• Currently developing an Integrated Web & Mobile solution to bring real time customer interaction data from Web & Mobile platforms and build a data pipeline for backend enterprise systems using Apache Spark.

• Collate events logs from different systems supporting real-time streaming mechanism using MQ’s like Solace and Apache Kafka.

• Creating an abstract data layer using Apache Spark for data quality and parsing and neutralization and loading the data real-time into M7 HBase and Elastic Search.

• Built a visualization matrix for the advanced analytics team on the real-time data using Kibana

• Developed analytical capabilities to build insights on data usage patterns, security triggers and auditing as per the audit requirements.

• Built an automated Prediction Model helping business assess the top 3 reasons a card member could potentially close the account. Wrote multiple map-reduce jobs and hive scripts that would evaluate the card members based on their spending habit and come up with a strategic pattern for retention.

• Developed a capability that allowed business users to categorize merchants into "interests", create customer spending patterns, and segment customers based on different types of spend behavior within an interest.

• Developing tools and reporting capabilities using HBase as the enterprise Big Data - data warehouse to help access and analyze data resulting in higher revenues and margins alongside better end user experience

• Developing a Map-reduce job that provides matching capabilities to reduce redundant prospect information coming from different sources. The existing algorithm in Perl is being migration to Map-reduce to improve performance and scalability

• Built automated Map-reduce jobs and processes to consolidate multiple tasks and create an efficient workflow when managing huge amounts of structured and semi-structured data

• Creating an abstract data layer in Hive for data mining and reporting purposes

• Built a Mobile App that provided personalized recommendations to the users based on their spending habits

• Built an application UI that provided ETL capabilities which included defining metadata for files coming into the Hadoop cluster and provided scheduling and monitoring of jobs that refined and staged the data

• Built an approval workflow using JSP that provided secure administration of PII files being ingested for further processing and data mining

• Developed Map-reduce jobs for address parsing of customers that helped create a distinct and accurate set for marketing teams