Experienced backend software engineer with a focus on distributed data systems, high-scale job orchestration, and infrastructure reliability. Proven track record of delivering resilient systems handling petabytes of data and millions of operations daily.
Experience
2025 — Now
2019 — 2025
2019 — 2025
New York City Metropolitan Area
Software architect and engineering lead for large scale data platform supporting over 50+ PB of enterprise-scale data.
Designed and led development of a highly reliable data replication and backup system for petabyte-scale Hadoop clusters using Apache Flink and Java, significantly reducing lag between primary and secondary clusters.
Extended backup system to the cloud, enabling scalable, cost-efficient storage and recovery via AWS S3 and Glacier, with a modular architecture built using Spring Boot.
Modernized job scheduling infrastructure, transitioning to an elastic and fault-tolerant system handling 1M+ daily jobs, improving SLA adherence and developer productivity.
Built and maintained a data validation and quality assurance tool processing 50+ TB/day, used to block critical data issues prior to report generation and onboarding.
Integrated a GenAI-powered assistant using LLMs and prompt engineering to auto-suggest schema validation rules and generate SQL constraints, enhancing data governance and user experience.
Developed and optimized near real-time data pipelines using Apache Kafka and Flink, reliably moving terabytes of daily data between heterogeneous systems.
Contributed core components to internal ETL libraries built on Apache Spark, enabling scalable ingestion and transformation of structured and semi-structured data.
Built the initial MVP and adoption framework for GCP BigQuery, leading to wider adoption of cloud-native analytics across the team.
Delivered robust and secure RESTful APIs in Java Spring Boot, used by trading desks to manage data operations with strict access controls and audit requirements.
Tuned Flink applications by introducing multithreading and resource management strategies, reducing runtime latency and improving resource utilization.
Early contributor to a firmwide data processing framework.
Open-source contributor: Finos Legend Engine (PR #3000) — enhancements to core data lineage features.
2017 — 2019
2017 — 2019
Greater New York City Area
Worked on improving search relevancy on moodys.com, using Lucidworks Fusion and Apache Solr. Building data pipelines for the search infrastructure using Apache Spark and Kafka
Lead the design and development of infrastructure and application automation along with disaster recovery automation using Terraform and Puppet for the whole product comprising 9 different applications stacks like Python, Node, React, Clover, C++, R.
Working as a full-stack developer, part of the team building a complex tool for screening and filtering data and displaying as per user selection. The application is built on top of variety on technology like React, Java, C#, NodeJS and MongoDB.
Building new features and functionalities, enhancing the existing features and doing bug fixes are some of my daily responsibilities. Creating UI components and developing RESTful web services.
Developed a test suit for sanity testing of microservices and working on developing a suite for load testing for the microservices. Developed a middleware component from scratch as a reusable library to intercept the request.
Integrated code on front-end as well as back-end to collect and log data how the user interacts with the website which help understanding user behavior and help analyze the areas of improvement for the tool.
Responsible for AWS cloud cost management. Developed a scheduler for the servers to reduce the cloud usage bill using AWS Lambda, AWS DynamoDB and CloudWatch.
Developed a custom monitoring system using Python and AWS Lambda to monitor all the application nodes, clover jobs and sql tables.
2016 — 2017
2016 — 2017
Piscataway, New Jersey
Software Development for a Cloud based product providing Iaas.
Worked on a Heat Orchestration Template (HOT) feature in ECM implemented in Java Spring MVC framework, where our team upgraded the existing HOT functionalities to include features like stack preview, support both yaml and json formats, multiple files to specify architecture following Agile development methodologies.
Part of a research team to investigate feasibility to add a Custom Workflow Design feature to Ericsson Cloud Manager (ECM). Implemented a business use case using Camunda BPMN. Yaml file and swagger API were used to generate classes for ECM northbound API web services request and response. Ansible was used for monitoring and configuration.
Developed a generalized multi yaml file North Bound API code generator using swagger and Node.js that contains all ECM RESTful request and response for developing web application and can be used as 3rd party APIs.
Modifying code in Java Spring framework in the rest layer as per new industry and customer requirements and write JUnit tests for that new upgraded feature. Working on proprietary RDBMS software for ORM and ETL operations.
Take care of JIRA issues raised for the team and testing customer files for compatibility with the upgraded feature. Work with Jenkins to check the stability from regression every night, find and resolve issues if present.
2014 — 2015
2014 — 2015
Ahmedabad Area, India | www.softvan.in
Designed and developed Amazon S3 based personalized cloud data storage web application using S3 API along with front end in JavaScript, jQuery and CSS and back end of the application in J2EE. The application had functionalities like creating buckets, uploading objects in buckets, versioning in uploads, managing users and access to different buckets and logging user activity.
Developed Amazon EC2 based load balancer application by fetching live data from the servers in the group to redirect the incoming request using AWS API, Java and shell script.
Designed and developed a web-based editor for online PHP coding majorly working on front end, with JavaScript, Ajax,AngularJS and BootStrap. The user can create directories and PHP files. Write code and edit those files and finally run them live on the server (Apache Tomcat or JBoss).
Part of the development team in the real-time location finder web application, which included using JQuery and Google Maps API and REST Web Services using JAX-RS API, Jersey Servlet. Used MySQL and NoSQL database like MongoDB and Java frameworks like Spring MVC and Hibernate / JPA for server side.
Education
New Jersey Institute of Technology
Master of Science (M.S.)
2015 — 2017
Pandit Deendayal Energy University
Bachelor of Technology (B.Tech.)
2010 — 2014