Three Years of Building a Custom Music Server: Part One — The Journey Begins

Have you ever spent three years crafting something from scratch, only to realize the real challenge was just beginning? That's the story of a developer who set out to build their own music server — a project that started as a weekend hack and turned into a multi-year obsession. In this first part of a series, we dive into the motivations, technical decisions, and early struggles that shaped this ambitious endeavor.

Why Build a Music Server from Scratch?

The idea of a personal music server isn't new. Services like Plex, Jellyfin, and Navidrome have been around for years, offering off-the-shelf solutions for streaming your own music collection. But for some, the limitations of these platforms become unbearable. The developer behind this project wanted total control: over metadata, over the playback engine, over the user interface, and over the integration with other smart home devices. They wanted something that could scale from a Raspberry Pi to a full-fledged home server cluster without compromises.

The article on Habr (a popular Russian tech blog) describes how the author began with a simple question: "Why does every music server I try fail to handle my library of 50,000 tracks without lag or metadata errors?" That frustration led to a decision that would consume the next three years of their life.

The Technical Stack: A Deliberate Choice

From the start, the developer chose a stack that emphasized performance and flexibility. The backend was built in Go, a language known for its concurrency and speed — ideal for handling simultaneous streams, metadata parsing, and database queries. Go's simplicity also meant fewer dependencies, making the server easier to maintain over time.

For the frontend, they opted for a React-based interface, but with a twist: no heavy frameworks like Redux. Instead, they used a lightweight state management approach to keep the UI snappy, even on older devices. The database was PostgreSQL, chosen for its robustness with complex queries (like searching across artists, albums, genres, and custom tags).

One of the most interesting technical decisions was the audio player. Instead of relying on HTML5 audio elements (which have notorious limitations with large playlists and gapless playback), they built a custom Web Audio API pipeline. This allowed precise control over buffering, crossfading, and even real-time DSP effects like equalization.

Relevant Insight: If you're building a custom music server, consider the Web Audio API for advanced playback. It's supported in all modern browsers and gives you low-level control over audio graphs.

The Metadata Nightmare

Anyone who has managed a large music library knows the pain: inconsistent tags, missing album art, duplicate artists, and genre lists that make no sense. The developer spent the first six months writing a metadata scraper that could handle this chaos. They integrated with multiple sources: MusicBrainz for album IDs, Discogs for release dates, and Last.fm for artist bios. But the real innovation was a custom fuzzy-matching algorithm that could identify tracks even when filenames were gibberish (e.g., "01-track.mp3" with no ID3 tags).

The algorithm used a combination of acoustic fingerprinting (via Chromaprint) and pattern matching against the file path. For example, if a file was in a folder named "Beatles/Abbey Road", it would assume the artist and album, then verify against MusicBrainz. This approach achieved a 94% accuracy rate on a test set of 10,000 files — a significant improvement over most commercial solutions.

Early Challenges: The First Prototype

The first working version — built in just two weeks — was a disaster by the developer's own admission. It could play music, but the UI was ugly, the search was slow, and it crashed every time someone tried to add a playlist with more than 200 songs. The article describes a moment of despair: "I looked at the code, and I knew it was a dead end. I had to rewrite everything from scratch."

And that's exactly what they did. The second rewrite took three months, but it introduced a modular architecture that would prove crucial later. The core was split into three services:
- Metadata Service: Handles all database operations, caching, and external API calls.
- Audio Service: Manages file streaming, transcoding (for formats like FLAC to MP3), and audio output.
- Web Service: Serves the frontend and handles user authentication.

Each service could be scaled independently. If the developer wanted to add a mobile app later, they could reuse the Audio Service without touching the rest.

What's Next in Part Two?

This first part of the series lays the groundwork for the real challenges that came later: building a recommendation engine, integrating with voice assistants (like Home Assistant), and optimizing for low-latency streaming over the local network. The developer also hints at a major design flaw in the second rewrite that forced yet another rewrite — but that story is for the next installment.

For now, the key takeaway is this: building a custom music server is not just about code. It's about understanding your own listening habits, tolerating imperfection, and being willing to throw away months of work when the architecture is wrong. The developer's journey is a reminder that even in a world of off-the-shelf solutions, there is value in building something truly your own.

Read the original article on Habr

Conclusion

Three years is a long time to spend on a single project, especially one that (so far) only serves one person's music library. But for this developer, the process was as valuable as the product. They learned Go deeply, mastered the Web Audio API, and built a metadata system that could rival commercial tools. If you're considering a similar project, start small, expect to rewrite, and don't underestimate metadata. The second part of this series promises to reveal how the server evolved into a smart home hub — and whether the developer ever truly finished it.

Stay tuned for Part Two, coming soon to asibiont.com/blog.

← All posts

Comments