Popularity in music is a science. But it doesn’t seems so from conversations like this:

“I feel like I’ve heard this song before.”

“Is this a remake?”

“This sounds so much like a different song – it’s on the tip of my tongue!”

We’ve all been there. We’ve had the phantom-limb sensation of knowing a new song that everyone seems to love, yet it’s maybe the first time we’ve ever heard it played. What gives?

There’s a reason some songs are hits. There’s a science behind popularity in music, and it’s affecting what you listen to and why.

Let’s get down to the basics. According to Derek Thompson, author of the new book The Science of Popularity in an Age of Distraction, content creation has skyrocketed in the US since 1980 – by a multiplier of seven, to be exact. That means there’s more creative, culture content than there’s ever been, which leads to the idea that popularity is harder than ever to achieve.

He goes on to explain that there’s no formula for hits because if there was, everything would be a hit by now. And that would subvert what makes a hit an actual hit – it’s more popular than the rest of the material it competes with.

While there is no exact formula for a hit, popularity occurs in the sweet spot between neophilia and neophobia. Neophilia is the love of things that are new, and neophobia is a fear of anything new (that might force someone to change their habits).

In simpler terms: popularity occurs when you’re familiar enough with something whether it be a song, artist, concept, or ideology, but it is foreign enough it still causes you excitement by discovering something new.

Which brings us to the music industry, and the main critique of top 40 hits – they all ‘sound the same’, or don’t feel new. Why do the little guys (independent artists) feel neglected by major labels, or feel the need to rebel in a world where there is so much free access to culture content?

Who is ‘the man’? Why do we insist on sticking it to him?

Derek Thompson delves deeper into the concept of marketing as a ploy in popularity in his book. Take what we know about popularity; it’s the desire for things that are familiar, yet not overtly so. Through that lens, we can understand how and why top 40 artists seem to be so successful.

Raymond Loewy, known as the father of industrial design, sums up this idea as the “MAYA” principle, the “most advanced, yet acceptable”. They’re not focused on revolutionizing the industry, but more so differentiating themselves slightly from the norm without straying too far into the new and weird.

Thompson goes on to emphasize the ideology that “familiarity beats originality, and distribution beats content”. This means even if a song is good, no – great, if it doesn’t have a large distribution, it simply will not succeed on a large scale. The local rapper on your block pushing his mixtape might have more raw talent than a well-established artist signed to a major label, but if his only means of sending out his music is by handing out flyers on a street corner or asking girls on dating apps to listen to his Soundcloud, he simply won’t succeed in the way national artists do.

So in the music world, the ‘man’ to stick it to, so to speak, is the large scale labels that have the distribution and marketing networks to get their artists’ music out in front of audiences. After all, familiarity invokes a sense of endearment. If the same song is playing on all the radio stations, you’ll be bound to recognize it eventually. And once you recognize it, you’ll be more pre-disposed to like it. Annoying, isn’t it? There’s a network of marketers out there choosing what song you’ll be listening to next Friday night at the bar. It’s not really your choice… Or is it?

Discovering new artists through curated exploration

Let’s say you’re someone who prides themselves on being interested in broadening your taste in music further than the hits. Yeah, there’s the big top 40 dance songs you hear at the club or the classic throwbacks in your favorite dive-bar, but you love to find new stuff, too. How do you find new music? It’s probably through one of the major streaming platforms riddled with algorithms to recommend you new music “based on” your tastes, right? Tidal, Soundcloud, Pandora, Spotify, Apple – they all build off your previous listens to curate music you’re going to enjoy.

Popularity affecting what artists you discover can best be summed up by the Spotify case. Spotify established a playlist called Discover Weekly. This function, as we currently know it, creates a playlist which is purportedly to your taste, including songs you’ve previously listened to and introducing new songs that you haven’t listened to on Spotify.

However, in the first iteration of Discover Weekly, the inclusion of songs that you had previously heard was considered a bug in the algorithm. Since it was a bug and was unanimously reported as a bug, Spotify “fixed” it and made the playlist play exclusively new music. Usage tanked – it was too foreign, and lacked familiarity. It alienated the users. The “bug” was reinstated, and use skyrocketed. The science of popularity strikes again.

This phenomenon, the sweet spot of popularity, is referred to by psychologists as ‘aesthetic aha’. When discovering new things, or conceptualizing a new idea, there’s a range of thinking between ease and difficulty. There’s conflict between someone telling you what to do versus understanding a foreign concept, like hearing a song where you can’t identify what the chorus is. The human mind can only leap so far. It’s the same reason we have trouble conceptualizing the far future. So a playlist composed of brand new songs related back to songs you’ve previously played on repeat (like that go-to shower song) is an aesthetic aha, right in the sweet spot.

Gianluca Consoli of the University of Rome explains it simply, “During the interpretation of a great artwork, when the viewer has already solved a certain quantity of difficulties, [they begin] to feel good”. It’s the “transition from an initial state of uncertainty, associated with unpleasant and negative affect, to a subsequent state of increased predictability and fluency [that] is highly rewarding”.

Understanding out of the chaos is that ‘ooooh, yeah! This does remind me of my favorite band’. Spotify has hit the sweet spot of new music without alienation, including repetition without boredom that other services have sought to varying levels of success (I’m looking at you, Pandora-John-Mayer-radio-composed-of-only-John-Mayer-covers).

This controlled exploration takes a little bit of the adventure out of quest of new music, but aids those who don’t feel the need to actively seek out new music. This algorithm enables the discovery of new music, but this music provided to you by Spotify is still signed with a label. You haven’t fully stuck it to the man. How do we fully escape the influence of popularity?

The rise of the bedroom DJ

Optimal newness as a concept was introduced on a song by song basis in the infamous “4 Chords” online video. Essentially, Axis of Awesome proves that tons of songs are based on the same chord progression: I–V–vi–IV. The list includes the Beatles’ “Let It Be”, Journey’s “Don’t Stop Believin’ ”, Elton John‘s “Can You Feel the Love Tonight?”, and Bob Marley’s “No Woman, No Cry”.

These songs could not be considered knock-offs of one another,by sense of any imagination. These “songwriters aren’t [even] retracing one another’s steps. They’re more like clever cartographers given an enormous map, each plotting new routes to the same location”. Songs are like one another, even if they don’t sound like it. So what about a song that actually sounds like songs you’ve heard before? A song like… dare we say it, a remix?

Spotify does a great job of introducing you to songs and musicians. But what about your favorite dance remix of Blink-182‘s “I Miss You”? Where’s that space on Spotify? Let’s introduce you to the bedroom DJ, and their (and Chance the Rapper‘s) preferred platform: Soundcloud.

Soundcloud provides an ideal platform for showcasing talent because of its low barrier to entry. All you need is an account to sign up and start uploading music. But how does this relate to popularity?

Bedroom DJs, or unsigned DJs/producers, do a phenomenal job of embodying the aesthetic aha. They use remixes of classics and songs you used to love as a gateway to their original content. Remixes combine familiarity (your old favorite song) with a perceived newness. Maybe they’ve been sped up, slowed down, chopped to pieces. Maybe they added a jazz riff. It’s still the song you loved, but now… it’s new, it’s different.

The one catch is songs are usually protected by copyright, and remixes without consent are legally considered derivative content and thus are technically copyright infringement. This has no place on Spotify, but it has found a home on Soundcloud.

These bedroom DJs can bridge listeners to their new original content by creating remixes of hits that draw the crowd in and work as an integration of familiarity, allowing listeners an introduction without alienation. Then, the producer can introduce original content, and, voila – convert a listener without scaring them away. It’s almost magic. Or maybe it’s the science of popularity rearing its head again – even the unsigned producer on Soundcloud is relying on science to get you to listen to their newest song. And you, dear listener, will never really beat your human instincts. But this is about as close as you’ll get to ‘sticking it to the man’.

Don’t get me wrong – the science of popularity is not a bad thing in the world of music. It lets you discover new music without the risk of investing too much time without payoff. It often makes good business sense too. Nobody in the history of humanity has even successfully run too far out in front of the current times. No matter how innovative you want to be, you can’t beat human biology. And thus you see a proliferation of these algorithms in music companies, and large investments in their future.

The best ones slowly push us forward into discovering new music, coaxing us out from the familiar and steadily expand our horizons. At Jukely [Disclosure: Four Over Four is published by Jukely], there’s an algorithm driving the curation of GoLists for concerts in your area, driving the discovery of music through the affinity you develop after seeing a band live.

Recommending shows that match your taste, but pushing your boundaries into making you find a new band is the end-goal. It makes you enjoy the music when it seems to cater to your taste, but excites you because it feels like discovery. It’s biology. Don’t question it.