Call recording with AI analysis

2026-02-19

I have started a handful of fun side projects. Mainly to see if I could replace existing tools I use with local open source alternatives that I build.

I am on calls all day long sometimes. I use Otter to join my calls, transcribe, and then provide AI analysis. It works great. But, I realize I am mostly paying for expensive hosting and that kills me. Yes, they had to develop it, keep is functional, etc., but I figured I could build a tool that did similar, never changed, and was hosted locally. Anything I can do to contribute to the downfall of SaaS is a worthy endeavor. I have no plans of pushing it out to the world. That wasn’t the point initially. The point was to gain experience making a tool for myself and if it worked out and was of a quality that others might find helpful - then I would make it available to the world.

The tool works completely independent of the internet. I wanted a local only and always approach.

I made a first run at this a while back and abandoned it to work on other things. When I came back to work on it I was then using SSOTs/ADRs and wanted to start over. The restart was extremely helpful. The end product is turning out great. It’s not perfect yet and I had to pause to focus on real work, but it’s going to end up a finished product.

How it was built

I went with a CLI first approach. It ended up being far more efficient than bum rushing a UI that I would also have to keep straight while working on functionality. I also had no idea what I wanted it to look like so CLI first made the function the main show and the UI could come later.

This allowed me to run quick commands to begin recording, process recording, see transcripts, etc.

This is the highlights of how it was built:

• Written in Go (compiled binary, no framework)
• CLI-first architecture — no daemon, no web server
• Filesystem-as-database (UUID directories + Markdown metadata)
• Used Go’s standard library almost exclusively (minimal dependencies)
• Manual frontmatter parsing (no heavy YAML library early on)
• Designed to integrate with local LLMs instead of cloud APIs
• Targeted running against Ollama for local model execution
• Whisper for transcription
• Diarization with pyannote.audio
• Focused on offline-first, self-hosted AI workflows
• Built and tested entirely in Linux (Mint environment)

The current state has diarization turned off. I am trying to iron out some details with the transcription accuracy and then I will turn back on and fix (hopefully) the diarization accuracy.

I also built in the ability to upload audio files so that I could easily move out of Otter and migrate older recordings into my new system. I do need to work out file format issues as Otter exports in one file type and my processing assumes another file type. I could just convert the files but should make it flexible to accept all common file types. I shelved this for the time being as I am currently working on real work and this is a side project.

Another neat idea is that this would be perfect for people concerned about HIPPA requirements. If nothing is stored in the cloud or can even hit the internet, it might be a good tool for people required to follow HIPPA requirements.

I have no idea when it will be finished. If I can’t get the transcription and diariazation accuracy improved, I may never finish. I just don’t know if the local models are capable of better and the ability to adjust it might be out of my reach. We’ll see.