Music – and How Computers Hear It

This post was written for my client Spectralmind and appeared initially on their blog:

Spectralmind works with music. But what is “music”? A look into Wikipedia  gives some helpful clues about music, and unwittingly, even about Spectralmind:

”Music is an art form whose medium is sound and silence. Its common elements are pitch (which governs melody and harmony), rhythm (and its associated concepts tempo, meter, and articulation), dynamics, and the sonic qualities of timbre and texture.”

In fact, these described elements of music are the ingredients Spectralmind uses for the creation of music tech products. Music is the base material from which we explore, analyze and extract information:

Algorithms, packaged into software, “listen” to music. What the algorithm “hears”, are music properties, including rhythm, timbre and many more.

Of course, a computer does not perceive music like humans do. Computers just calculate, they cannot take into consideration the cultural heritage, emotions and interpretations human listeners feel or are aware of.

”The border between music and noise is always culturally defined—which implies that, even within a single society, this border does not always pass through the same place; in short, there is rarely a consensus … By all accounts there is no single and intercultural universal concept defining what music might be.” (musicologist Jean-Jacques Nattiez, quoted in Wikipedia).
Applying a uniform algorithmic evaluation across a large number of music titles creates an objective mathematical description of each piece of analyzed music and, derived from here, an approach of comparability. We call it “music intelligence”. Such intelligence can be exploited in various ways like identifying music, determining similarities between music titles or organizing music. Still, there will always remain a gap between ”human understanding” and ”machine understanding” of music, as there will always be a gap in the understanding of music between human listeners.

“The creation, performance, significance, and even the definition of music vary according to culture and social context.”

Ever increasing sophistication of algorithms and availability of computational power lets us apply the music intelligence approach on large catalogs of music, thus eliminating great portions of cost and manual labor for large inventory music classification.