As AI (specifically Large Language Model), blown up thanks to the popularity of ChatGPT, I started to take a look deeper about it. From the first sight, it's seems like many product launched lately is more about "stuff + openai", which not really a bad thing, but makes me wonder, is Open AI will monopoly the AI fields?

Then, a project called llama.cpp comes out, that claims able to runs Facebook LLM model just utilizing CPU. A quick look into the repository documentations, and it blows my mind. As a believer in ability to self hosting application, keeping privacy in check in an unknown world of Artificial Intelegence seems beneficial.

I dig through the repository, reads some issues, and play around with it, and it runs really well, both on my M1 Pro MacBook, and my Orange Pi 5. The accuracy is not highest, and the speed is usable for a simple local “ChatGPT” like application. From this experience, I think we could start considering build stuff on top of it more seriously.

Luckily, that llama.cpp project not only proof it possible, but also encourage many other open source project to build their own LLM models, one of them is GPT4All, which claims has decent performance and accuracy compares to ChatGPT with GPT 3.5 model. And this project has an official NodeJS binding. This is great since I’m quite confident develop tools / apps using NodeJS.

## Ideas: Personal Email Summarizer

A simple ideas for LLM is summarizer. I‘m using Spark (a really great mail client btw, you should definitely check that out), and it already did great job to filter out low priority email. Low priority email includes newsletter, which sometimes, I’m curious and wants to learn more.

With that said, I start to tinkering the GPT4All binding, and mix and match it with other self hosted tools that make sense for this use case. Then I remember a great workflow tools called n8n, that easily hosted and extended, using NodeJS. N8n basically a workflow tools that enable users to define a workflow of many 3rd party integrations like SMTP, GMail, Telegram, etc. Considering the LLM capability and the main missing piece is access to data, n8n basically provide the bridge of it.

## Designing Workflow

on n8n I start with creating a workflow that listen SMTP of my gmail, hook it with my custom gpt4all node to do AI Summary, and then push it to Telegram, all runs on Orange PI 5 beind Tailscale, so I could access it from all my device, and keep it secure.

At first it works as expected, all emails got summarized and pushed to my Telegram bot. But then some of the jobs got stuck. Turns out the event from SMTP might contains multiple email, and n8n by nature, runs all of them in parallel. Considering my Orange Pi 5 compute resources (4 performance cores, 4 efficiency cores), the jobs easily piled up and takes so long to complete.

```cv
gallery
{"layout":"hscroll","images":[{"src":"https://firebasestorage.googleapis.com/v0/b/maitake-project.appspot.com/o/pages%2FN2CDXKStFwTljP6TlGJZ7Kn3O4J3%2FStz5DUWZT8SWW5HqBwQz%2Fbc757444-0b28-41c7-8817-a833245c9de3.jpg?alt=media&token=ae51b460-3b1b-49d4-aab2-418e74b9751a","id":"bc757444-0b28-41c7-8817-a833245c9de3","width":1932,"height":1190,"filename":"IMG_0160.jpeg","type":"image/jpeg","caption":"","border":false}]}
```

Based on my needs, I don’t need a quick summary when each of the email arrives. So I start updating my workflow to break all the events into a serial job in queue, with a simple built in n8n node called Split In Batches. Configure it to only forward 1 message each batch, process it, then send it to telegram, now I had my own AI Email Summarizer.

## Conclusion

I personally believe that AI could help humanity going forward, but I also believe that we need to learn from our past about data privacy. We know giants like Google, Microsoft, Meta, and now OpenAI had all it takes on compute resources to built this awesome AI stuff, but we can’t easily give up all our data to them, and let them do monopoly in this area. This AI model will be an additional great tools alongside other existing tools, and democratizing it (by let anyone host their own AI with their own data), is the correct way to move AI forward.

GPT4All integration to n8n

DevOps Engineer

Kubernetes Resource Quota Problem

A journey finding potential bugs around Kubernetes Resource Quota that prevent a burst action on pod creation.

Integrate SOPS into ArgoCD

A descriptive articles about how to integrate SOPS into ArgoCD by utilizing ArgoCD Config Management Plugin (CMP).

A simple kubectl extensions to find unused volumes across the cluster. Distributed via KREW

kubectl extensions for finding unused volume

A simple n8n extension to integrate self hosted GPT4All

A kubernetes operator for running Pipelinewise. Pipelinewise is a simple ETL platform to transform data from various location like MySQL, Postgres to AWS Redshift etc.

Pipelinewise Kubernetes Operator

I had 2 main roles in this period, as QA Engineer from 2016 to 2018, and Platform Engineer starting 2019 until 2020.

- Improves CI/CD Pipeline processing time by distributing queue for each application. This improvement also opens the possibility to use the same application for PR checkers.
- Lead initiative to implement improved observability company-wide. Conduct presentation about tracing, implement library and tools for onboard software engineers to observability. 
- Reduce infrastructure costs by about 30% by cleanup unused Kubernetes PVCs, and also converting disks type to cheaper ones by sacrificing little-to-no performance and reliability.
- Simplify setup for running automation test. Wrapping all the bash commands to a one-liner CLI command without sacrificing any customization.
- Automate testing grid setup on Kubernetes to have a scalable and reliable test runner. This setup is able to run about 120 UI Automation tests in 40 minutes.
- Develop an in-house test management system in ReactJS.

Platform Engineer

Work as Individual Contributor under DevOps subteam of Infrastructure Team. Focusing on CI/CD and Developer Experience

- Establish company-wide CI/CD system using the GitOps approach with comprehensive documentation and automation, including a migration plan and guidance to onboard developers into the new flow.
- Deep dive into Kubernetes long overdue issues around Pod creation delay for burst pod creation.

Junior Software Engineer

Software Test Engineer

Act as Infrastructure Technical lead of 6 team members with 4 subteams: DevOps, SRE, Database, and InfoSec, with a Team Lead. Focusing on technical deep dive, technical direction, and technical decision-maker.

- Lead service mesh (Linkerd) research and rollout throughout the entire Infrastructure.
- Lead of project to migrate ElasticSearch cluster from 3rd party into our Google Kubernetes Engine, and serve it to production users in 2 weeks.
- Learn to groom team member to improve in technicality by provide supports, technical deep dive, and regular 1-on-1.
- Resolving long overdue Kubernetes issue around bottleneck to execute burst pod creation. Working closely with the Google Cloud team to find the resolution.
- Exposes false-postive metrics from core system, suggest the fixes and assist the implementation to the production.

Infrastructure Technical Lead

Act as DevOps team lead, driving infrastructure improvement and platform initiatives

- Establish company-wide Continuous Delivery flow using GitOps methodology so the deployment process could be easy to use,  transparent, and auditable.
- Being an evangelist for monitoring and observability by establishing a standard monitoring dashboard and alerting.
- Establish a Data Pipeline process to enable Data Scientists to do their own ELT with simple customization and requires minimum supervision from DevOps.

Aldira Putra Raharja

2023