Did you know Social Media Users Unknowingly Participate in Marketing Experiments, Research Reveals

 

You think you're just scrolling. Checking in. Catching up. Maybe liking a post or skipping an ad.

But behind the scenes, something else is happening.

A quiet experiment.

Not the kind in a lab with white coats and clipboards, but one that unfolds in real time, on your screen, in your feed, shaping what you see and what you don’t.

Researchers at the UBC Sauder School of Business took a closer look. And what they found? Even the companies running these tests don’t fully understand the results.

The question is, Are you really in control, or just another data point in an endless experiment?

The study (published in the International Journal of Research in Marketing) analyzed all available peer-reviewed research on A/B testing conducted by Facebook and Google. This form of testing involves showing varied ads to different users to determine which ones drive engagement and purchases. Despite its widespread use, the research identified critical flaws in how such experiments are conducted and interpreted.

At any given moment, millions of social media users are unknowingly subjected to these tests. Marketers hope to gauge which ads effectively influence consumer behavior, assuming clear conclusions can be drawn. However, the reality is far more complex.

Facebook’s A/B testing tool provides access to an immense audience, allowing researchers to observe real-time behavior. Because users remain unaware of their participation, their interactions are considered more authentic. But the process is fundamentally skewed by intricate algorithms dictating which individuals see which ads. This results in unpredictable targeting decisions, even the engineers who built these systems cannot fully explain why specific ads appear to certain users.

The core issue stems from the absence of random assignment, a principle in controlled experiments where different versions of an ad are presented randomly to distinct groups. Instead, algorithms selectively deliver ads to users predicted to engage with them, leading to misleading results.

For instance, if one version of an ad receives higher engagement, it may not necessarily be due to superior design. The algorithm itself might have refined the audience, favoring those already predisposed to click. This undermines the assumption that ad performance directly correlates with creative effectiveness.

Another complication arises from personalized ad targeting. Social media platforms often display ads based on users’ browsing history. If someone has already decided to purchase an item and later sees an ad for it, researchers might mistakenly attribute their decision to the ad rather than their pre-existing intent.

These systems rely on more than just visible demographics such as age or location. They also factor in behavioral patterns, past interactions, and even variables not directly measurable by the platform itself. Machine learning models process vast amounts of data, identifying behavioral trends that remain opaque even to the companies deploying them.

The consequences extend beyond marketing inefficiencies. Advertisers rely on these tests to refine campaigns, yet the results can exclude certain demographics from receiving vital information. Some academic studies have highlighted cases where women are disproportionately excluded from STEM-related advertisements, not due to deliberate bias, but because social media algorithms optimize for cost-effectiveness. Since targeting women can be more expensive in specific ad auctions, algorithms might deprioritize them, reinforcing existing disparities.

These systems also adapt dynamically. If women interact with certain ads less frequently, platforms further reduce their exposure to similar content, deepening the divide.

Although this study primarily focused on Facebook and Google, similar practices are prevalent across all major social media platforms, including Instagram and TikTok. Large-scale A/B testing has become an industry standard. A social media insider once disclosed that every Facebook user simultaneously participates in approximately ten different marketing experiments — an estimate likely increasing with AI-driven ad strategies.

The research team, which includes experts from UBC, Erasmus University, and Hong Kong Polytechnic University, warns marketers against placing excessive trust in A/B testing results from platforms like Facebook. A surge in ad engagement might simply indicate that the system identified a niche audience with strong interest rather than demonstrating broad appeal.

Blindly restructuring entire campaigns around such findings could alienate larger consumer groups. Furthermore, ad delivery algorithms are so sophisticated that they can target individuals rather than traditional audience segments. Advertisers remain unaware of the precise mechanisms governing ad placements, but AI systems continuously refine them in the background.

Ultimately, while social media ad testing provides valuable insights, the study cautions that businesses must interpret results with skepticism. The data fueling these systems operates within a complex, opaque framework, one where even those overseeing it cannot fully predict its outcomes.