About
Generalist software engineer. Data, Distributed Systems, Python, Scala, Erlang.
Writing
Work Experience
• I was part of an effort to develop a new data platform for Goibibo, a data platform based on Kafka, a S3 based data-lake, Redshift and Spark. Goibibo-MakeMyTrip is India’s largest online travel portal.
• I developed a Spark-based batch ingestion tool, taking care of data consistency, exactly-once semantics across sinks(Delta, Redshift, Athena). As part of the efforts, I also developed a Delta-Redshift Spectrum connector. • I architected and developed a data-streaming platform using Kafka and SQLite, as part of an user-personalisation project.
• I was part of a 2-person team that built a high-performance, open-source, distributed API Gateway called Proxy42.
• The API gateway provides services such as routing, load-balancing, rate-limiting, authentication, logging and monitoring to APIs in a microservice environment.
• Proxy42 is written in Erlang, which provided an ideal platform for building distributed, low-latency, soft-realtime systems.