Is Facebook listening in to my conversations?

Have you ever been talking to a friend about a specific product or service and then said product or service appears in your newsfeed shortly after?

You are not alone.

A quick Google search of “Is Facebook listening to conversations?” returns 15.5 million results—including numerous forum posts and threads about this exact topic.

So is this possible?

Well, yes it is. To do it, you’d need to capture an audio file and convert it into text:

This could be as simple as accessing your smartphone’s microphone—which you have given Facebook permission to access.

There are alternatives, though.

Researchers at Stamford University have proven that the accelerometer in your smartphone (the gyroscopic sensor inside which detects movement and direction) can be sensitive enough to record and capture sound waves via the vibrations they produce through the air.

Machine learning could then be used to interpret the data as speech. This approach can even be used to analyse the sound of keystrokes on a keyboard.

There are several approaches to transfer audio files into text.

Both acoustic modelling and language modelling are important parts of modern speech recognition algorithms. These first came to prominence in the 1980s with the rise of neural networks.

More recently, new approaches have come to the fore. These include ‘hidden Markov models’ (HMMs), which can efficiently translate speech into text and form the basis of voice assistants like Siri, Cortana, Bixby, etc.

And speech processing technology is continually evolving.

With the increasing availability of computing power, including graphics cards, technology has begun to sway back in the direction of neural networks for many processes—including audio to text translation.

There is no doubt that technology and approaches will continue to be refined and improved over the next few years.

So it’s possible that Facebook is listening, but is it probable?

Almost certainly not.

Being technically possible is a long way from actually implementing something, as any nuclear fusion scientist or Aston Villa manager will vouch for.

The most efficient and cost-effective way to do this ‘listening’ would be to crunch the data on the phone itself then send it to Facebook—after all a single iPhone is 32,600x more powerful than the Apollo spacecraft that went to the moon.

iPhones don’t have a low-powered audio processing chip, though. This means that to “always listen” would burn through your battery in a very short space of time and get very, very hot. And it would be pretty obvious what was going on.

Some Android devices do have the hardware to “always listen” without being too detrimental to battery life. But you still have to trigger the processing of the audio.

At the moment, your phone is probably only listening for one or two audio signatures to trigger full audio processing e.g. “OK Google”. For Facebook to capture conversations relevant to its advertisers, your phone would need to be listening out for virtually all known advertisers’ brands and products, then sending this data back to Facebook for processing…

Are they doing this at Facebook’s end instead?

Let’s crunch some numbers to show how infeasible this is. The lowest possible quality mp3 file recording generates roughly 3.6mb of data per hour.

The Facebook app has a reach of around 500 million.

This crude calculation would mean that Facebook would need to run 1.8 petabytes of raw data through its servers every hour. That’s 21.6 petabytes of data a day.

Facebook as a whole platform consumes around 600 terabytes total per day—including everyone’s likes, dislikes, pictures and videos. That’s 34 times less than the above estimate.

It’s debatable whether storing and processing this much data would even be possible, let alone commercially viable.

According to Facebook, it spends ~$50million a year on data centre costs. Increasing this by 34 times would push this cost over $1.5 billion: close to doubling its current total operating expenses.

So the answer is no?

Yes, the answer is no.

Ignoring all the technical challenges, on the crudest level it would not be commercially viable for Facebook to be listening to you in any meaningful capacity.

What’s really happening?

It’s a combination of marketing and psychology.

The average consumer is exposed to around 4,000 marketing messages every day. It’s almost inevitable that some of those messages will occur coincidentally alongside your conversations and stand out because of their perceived relevance. You don’t notice the 1,000s of times relevant ads didn’t follow your conversations—only the times they did.

But why do we only notice this on Facebook?

The reason we’re not paranoid about our TVs, the radio or billboards is in part down to OG.

No, not “original gangsta”. Open Graph.

OG allows Facebook to gather information about where a Facebook user has been on the wider web, wherever OG is present. It’s like having a little bit of Facebook in another website.

OG everywhere you go

According to Quantcast and BuiltWith…

43% of the top 1 million websites have OG protocols (including Facebook Root aka Like button) .

That’s ~6-7% of the entire web.

For comparison, Google Analytics has ~8% of total web coverage.

This is used most obviously for direct remarketing and competitor buying.

You can see this in action for yourself. Visit a large double-glazing website and you will almost immediately get Facebook ads for that firm and most likely competitors in your newsfeed.

This allows brands to advertise to prospects who have not converted—however, they are also giving this data away to Facebook and allowing other competitors to pay to get in front of potentially qualified leads. It’s a fine balancing act that Jaywing can help you with if you so wish.

We’ve all had those ads that seem to follow you around, but where Facebook is particularly impressive is the extrapolation of less obvious, more nuanced data points.

For instance, if you’re looking to re-decorate your living room you may browse on Pinterest and click on a few articles:

‘Dark Blue’s and Brown’s for 2018 ’

‘10 modern sofas which don’t suck’

‘Stripping Wallpaper The Easy Way’

These aren't actually on Pinterest—Pinterest has a browser that sends info to Facebook via OG.

Facebook can now deduce that you may be thinking of re-doing your living room, without you ever visiting a brand directly.

Ten different brands who might be paint suppliers, sofa manufacturers or hardware stores are now going to pay to get in front of you several times in the next four weeks, because there is a higher probability that you are interested in their product/service than someone who isn’t looking up re-decorating tips.

The chances of you now having a discussion about a sofa manufacturer you don’t like and then seeing their advert has just got hundreds of times more probable.

This is a fairly obvious scenario, but Facebook completes this profiling at even higher levels than simply device-level—it also works on a network or router IP level.

As a personal example, over Christmas I was visiting family in the UK. I was browsing on my mobile phone, connected to a family member’s WiFi, and happened to view several property listings in a rural area of southern Spain called Competa, which is where my partner is from.

Twenty 20 minutes later, a family member burst in to the room, thrust their phone into my hand exclaiming:

“What are the chances of that!”

On the screen was a newsfeed advert for new build villas in Competa. My random browsing behaviour on a different device directly affected another family member’s newsfeed advertising on Facebook.

Crucially, the only reason this family member had even heard of Competa was because of my partner. And this is why it had grabbed their attention. Otherwise they would have scrolled right past.

If we take this back to our decorating example, your partner may have looked at an article, but you are still going to get an advert for the sofa in your newsfeed because as a ‘household’ you are interested in re-decorating. This increases the perceived disconnect in behaviour and the advert itself.

So to answer the original question, yes Facebook is listening to you. But not in the way you may think.

For more GDPR news and views, visit gdpr.jaywing.com.

Mitch Vidler is Head of Marketing Technology and Digital Analysis at Jaywing.