Experimental composer Holly Herndon is pioneering a new era in music, one where artificial intelligence isn’t just a tool but a collaborative partner in the creative process. Herndon, whose background spans from church choirs to doctoral studies in computer music, has developed AI models capable of replicating her voice and style, opening up possibilities for collective performance and artistic experimentation.
The Rise of “Protocol Art”
Herndon describes her work as “protocol art,” where the creative act shifts from composing with instruments to designing the very systems that create. This means meticulously crafting training datasets – not just scraping existing content, but composing original material specifically for AI learning.
“The making of the dataset is part of the artwork. I often write music not necessarily for human ears but for a computer to learn something.”
This approach is a direct response to the limitations of commercial AI models, which often rely on broad, unfocused datasets resulting in generic outputs. Herndon believes in curating unique sonic palettes, much like an electronic musician building custom samples.
The result? Models that generate truly distinctive sounds, unbound by the constraints of typical song structures.
The Power of Voice Cloning and Real-Time Identity
One breakthrough came with “Holly+,” a real-time voice clone enabling anyone to sing in Herndon’s vocal style. The implications are striking: performers can inhabit another artist’s identity on stage, blending human and machine expression. This technology is still in its early stages, but Herndon envisions a future where vocal morphing allows for hybrid performances – blending operatic highs with whale-like lows, or even merging voices in unpredictable ways.
Beyond Automation: Collaboration and IRL Connection
Herndon’s work isn’t about replacing human artists; it’s about amplifying collective creativity.
One recent project in Berlin involved training an AI model on the compositions of Hildegard von Bingen, then inviting performers and the public to collaborate in real-time. The goal is to bring people together, not isolate them behind screens. Herndon and partner Mat Dryhurst advocate for a shift away from passive scrolling towards active “strolling” – real-world interaction facilitated by AI tools.
A Future Where AI Liberates Creativity
Herndon acknowledges fears surrounding AI, particularly in an attention-driven economy. But she remains optimistic, suggesting that AI could filter out noise and connect people in meaningful ways. She envisions developers “vibe coding” with AI agents while living their lives, building a future where technology enhances, rather than replaces, human experience.
Ultimately, Herndon’s work challenges the very definition of creativity, forcing us to reconsider our role as the sole authors of art. The emergence of AI as a creative partner may be unsettling, but it also holds the potential for liberation and unexpected beauty.
