* Spoke at Analytics Vidhya Datahack summit on "Semantic Search, QnA and Chat using LLMs and Vector databases."

https://github.com/deepanshululla/SemanticSearch/tree/main

https://www.analyticsvidhya.com/datahack-summit-2023/session/building-your-own-search-engine-using-vector-databases/

* Panelist on "Advancements in recommendation systems". Discussed state of the art LLM based recommendation systems, pros, cons and challenges while building them.

https://www.globalbigdataconference.com/virtual/global-artificial-intelligence-conference/schedule-139.html

Tech Stack: Java, Python, PHP, Spring Boot, Spring Data, React, PySpark, Graphql, Protobuffs, Twirp, MySql, Elasticsearch, GCP, AWS, Splunk, Wavefront, Anomaly Detection, Peer Benchmark analysis, Best Send Daytime predictions.

• Architected, Built, and Led a high-performing team of five engineers to deliver external. Customer-facing complex product called Custom Reports on time, working with multiple partners and stakeholders while ensuring a safe and successful release within four months. The product is still the highest revenue-generating product for the team and has generated more than 5 Million USD so far.

• Collaborated on developing a Customer Data Platform to help create a single source of customers using change data capture, Kafka, and Debezium.

• Data Engineered clean and complete collection for email analytics data derived from journeys in marketing campaigns to ensure Change Data capture pipelines downstream work well.

• Improved the aggregation for clicks and opens a data pipeline that processes 10 billion events and 30 million jobs a day, improving job performance from 600 seconds to 2.4 seconds (99.6% reduction).

• Drove and led the team's AI/ML (and Generative AI) efforts by educating different stakeholders about the technology and possibilities of enhancements within our products and creating APIs for other engineers to build products upon. Developed Prompt manager service using Python and Flask to enable prompt engineering as a platform across the organization.

• Prompt engineered AI solutions for analytics products, and A/B tested different prompts to improve the product iteratively

Tools: C++, Python, Rabbitmq, Multithreading, Splunk, Kafka, Zookeeper, SQL, Bash, Cmake, and other Bloomberg in-house tools like Comdb2, BPKG, DPKG

• Developed, enhanced, and maintained a highly scalable, efficient and robust platform that processes over 50 million transactions a day with guaranteed SLAs.

• Led a team of 3 people in timely and efficiently delivering product features spanning multiple products and teams.

• Bootstrapped a BAS API microservice to deliver critical client configuration data for internal customers.

• Used test-driven development along with integration and unit testing to build software with no bugs.

• Integrated CI/CD pipeline to automate the end-to-end testing and deployment of software.

• Created tools using python for faster triaging of customer and alarm tickets like late trades analyzer, and fix message parser.

Tech Stack: Python3, Celery, RabbitMQ, Postgres, Elasticsearch, Kubernetes, Prometheus, Grafana, helm, Jenkins

• Implemented pipeline retry logic to automatically retry hanging or failed jobs hence decreasing downtime by 300%.

• Developed status endpoints for viewing the current status of ETL jobs and exposing them over rest API.

• Wrote custom Prometheus exporter to expose application pipeline metrics to monitor exceptions, latency of API and overall runtime of jobs.

• Added monitoring and alerting to the application by exporting metrics using Prometheus, Elasticsearch and using Grafana to create dashboards. Created alerts by using pagerduty and logic for automatic healing of the application.

• Scaled the application to accommodate 200% increase in load while increasing speed by over 75%.

• Use Test-Driven Development to ensure code quality always remains high and always pertain to business requirements.

• Implement a PostgreSQL syncing service that processes ~1B rows every 4-6 hours

• Lead the migration of a critical Python2 service to Python3 by establishing unit and functional testing best practices and policies, refactoring faulty code, leveraging code coverage reporting, and implementing CI workflows

• Created end-to-end black-box testing framework in python for testing the backend rest API and integrated it with Jenkins and Kubernetes pipeline for CI/CD. Handed it off to QA for test cases development.

• Automated Database change management using sqitch and kubernetes making database changes more reliable and agile.

• Working on Redesigning Database Schema for Data Export pipeline to make pipeline more reliable and resilient to failures.

• Created helm charts to move an application from bare metal to the Kubernetes cluster.

• Attended 4-day training sanctioned by oracle on each of the following topics: Kubernetes, Big data and data science/machine learning.