The growth and wide adoption of edge computing have introduced several issues such as load balancing, resource provisioning, and workload placement which are all developing as optimization problems. An algorithm, in particular, was successful in solving some of these optimization problems, namely PSO. Particle Swarm Optimization (PSO) is a nature-inspired stochastic optimization algorithm that emulates the behavior of bird flocks, whose objective is to iteratively improve the solution of a problem over a given objective. The execution of PSO at the edge would result in the offload of resource-intensive computational tasks from the cloud to the edge, leading to more efficient use of existing resources. However, it would also introduce several performance and fault tolerance challenges due to the resource-constrained environment with a high probability of faults. Given the benefits and the challenges above, the need for a new PSO algorithm to be efficiently executed in an edge network arises. This thesis introduces multiple distributed synchronous and asynchronous variants of the PSO algorithm implemented with the Kotlin programming language, built on the Apache Spark distributed computing framework and Kubernetes container orchestration platform. These algorithm variants aim to address the performance and fault tolerance problems introduced by the execution in an edge network. Moreover, they want to provide a greater level of scalability. By designing a PSO algorithm that distributes the load across multiple executor nodes, effectively realizing both coarse- and fine-grained parallelism, we can significantly increase the algorithms’ performance, fault tolerance, and scalability. The experimental results show the benefits of the proposed distributed variants of the PSO algorithm versus a baseline, the traditional PSO, testing different values of specific configuration parameters of the algorithms.

Thesis Abstract

Software Engineer

WebValley

WebValley is the FBK summer school for disseminating interdisciplinary research: almost 400 students (17-18y old) have attended the WebValley camps since its first edition in 2001. WebValley started as a regional initiative for high school students from Trentino Alto-Adige SüdTirol, and it is now an opportunity for national and international students to experiment with scientific research.

2nd place local

Won 2nd place locally at the biggest hackathon ever organized in Italy. I have developed a Java SDK for the SPID service which is the digital authentication system for Italian citizens.

Member

Nova is the merit-based access network where the top 3% of talent connect with each other, develop and accelerate their careers.

Mentee

Among the few Italian students selected to be mentees for LeadTheFuture, a leading mentorship non-profit organization for students in STEM, with acceptance rate below 20%. LeadTheFuture empowers top-performing students to achieve their goals and contribute to their communities by giving them one-on-one guidance from high-impact mentors coming from the world's leading STEM innovation hubs such as Silicon Valley and CERN.

English C1

German B2

- Chose the software engineering specialization.
- Prepared lectures to help students grasp complex topics.
- Among the top-performing students with a 4.0 GPA.
- Graduated with 110L/110 with thesis on distributed systems.

Edge computing poses a range of optimization problems such as load balancing, resource provisioning, and workload placement. Particle swarm optimization (PSO) is a bio-inspired stochastic optimization algorithm, with the objective to iteratively improve the solution of a problem for a given objective. The distribution of PSO workloads to the edge would transfer resource-intensive computational tasks from central large cloud data centers to the edge, resulting in more efficient use of existing resources there. However, this edge architecture introduces performance and fault tolerance challenges, due to the resource-constrained edge environment with a high probability of faults. We present here an asynchronous variant of an edge-distributed PSO algorithm built on top of the Apache Spark distributed computing framework. This PSO variant aims at solving performance problems introduced by the execution in an edge setting. Our asynchronous PSO algorithm that distributes the load across multiple executor nodes can effectively realize both coarse- and fine-grained parallelism, allowing us to obtain a substantial performance increase.

Asynchronous Distributed Particle Swarm Optimization for Edge Cloud Architectures

The wide adoption of edge computing has introduced several issues such as load balancing, resource provisioning, and workload placement as optimization problems. Particle swarm optimization (PSO) is a nature-inspired stochastic optimization algorithm, whose objective is to iteratively improve the solution of a problem over a given objective. The distribution of PSO to the edge would result in the transfer of resource-intensive computational tasks from the cloud to the edge, leading to more efficient use of existing resources. However, it introduces challenges related to performance and fault tolerance, due to the resource-constrained edge environment with a high probability of faults. We introduce multiple distributed synchronous variants of the PSO algorithm built on top of the Apache Spark distributed computing framework and Kubernetes container orchestration platform. These variants of the algorithm aim at addressing the performance and fault tolerance problems introduced by the execution in an edge network. A PSO algorithm that distributes the load across multiple executor nodes can effectively realize coarse-grained parallelism, thus can obtain a significant increase in performance, but also more fault tolerance and scalability.

Distributed Synchronous Particle Swarm Optimization for Edge Computing

- Implemented a Reliable Causal Broadcast (RCB) protocol with Rust and Actix.
- Implemented on top of the protocol several operation-based CRDTs (CmRDTs) such as ORSet, LSEQ and RGA.
- Designed the system following Rust idiomatic programming principles around an actor-based architecture.

Causal

- Designed and implemented multiple distributed variants of the Particle Swarm Optimization (PSO) stochastic optimization algorithm with Kotlin, Apache Spark, and Kubernetes.
- Obtained 5x performance increase, added fault tolerance, and improved scalability with respect to the traditional PSO.

Spark Distributed PSO

- Designed the Android application with clean architecture and Kotlin coroutines.
- Developed the backend service in Go with a custom rate-limiting mechanism implemented with goroutines.

eléncho

- Designed a custom use-case-oriented architecture in TypeScript with composable functional constructs and MongoDB integration.
- Developed the web app in React with custom hooks for API state handling.

Books

I have helped other students to understand complex Computer Science topics, via additional lectures, 1 to 1 problem solving classes and more.

Tutor

- Achieved ~50% reduction in the number of errors of asynchronous jobs.
- Identified eventual consistency issues and evaluated solutions with distributed locking and transactions.
- Explored caching for multimedia assets and implemented a solution with AWS CloudFront.
- Migrated code base and test suite of a Ruby micro-service to a new architecture.

Backend  Engineer Intern

- Enhanced the notification inbox with an algorithm written in Kotlin and Swift.
- Streamlined error management in the Android apps with a reactive error dispatcher.
- Improved the notification experience on Android with notification channels and the supporting micro-service in Ruby.

Software Engineer Intern

- Designed and implemented key features of the Fatture in Cloud mobile app with React Native and Redux.
- Worked in an agile environment with scrum software development.

Mobile Developer

- Improved researchers' workflow by implementing an SDK in Python to manage datasets containing MRI scans.
- Experimented with neural networks to perform MRI scans coregistration.

Software Developer

- Optimized performance of internal gateway service.
- Designed backend service in Ruby for deep links management.
- Extended and improved core backend micro-services along with unit/integration testing.

Backend Engineer Intern

* Designed and implemented an observable, durable, and idempotent mass migration system for transitioning our alerting infrastructure from stored transactions to metrics.
* Identified and participated in resolving production incidents and led post-mortems.
* Worked on a new metrics querying language called MQL, which is used to power a generic metrics platform for querying metrics across the entire product.
* Designed and implemented the metrics API responsible for parsing, transforming, executing, and post-processing all metrics queries.

Software Engineer II

* Developed a new format for managing JS bundles and co-designed a novel indexing mechanism for scalable stack trace symbolication.
* Designed and implemented version 2 of our dynamic sampling system, which optimized the sampling of billions of daily events through a set of interdependent smart heuristics.
* Reengineered an internal system for implementing sampling heuristics, aiming to facilitate the addition of new heuristics by other developers.
* Initiated an effort to enhance documentation across various areas within my team.
* Mentored new senior engineers on several aspects of infrastructure, backend, and internal tooling.

Riccardo Busetti

2022