It’s a challenge to coax insights from today’s crop of social media listening tools. In expert hands they’re wonderful at providing information in the aggregate for things like audience sentiment and share of conversation. However, unique insights that lead to new opportunities rarely come from the vox populi.

To eke insights from online conversations you need a human with an authentic interest in the subject matter to study them in their context. This is rarely practical due to the sheer volume of mentions and the dearth of talent in this new discipline. We took this on as a challenge at my company  and have been developing tools and techniques that make this process fruitful and scalable.

I’d like to share an example. We were engaged by a global manufacturer of intrauterine contraceptives—more commonly referred to as IUD’s, to conduct a social media listening exercise. They wanted us to uncover new market opportunities and get a sense of competitor vulnerabilities.

Right from the outset interesting avenues to explore came to light. For example, people who mentioned IUD’s also talked about physical activities such as running and yoga significantly more often than individuals who mentioned other types of contraception. Contrast this to the company’s advertising at the time that depicted women relaxing and in mostly sedentary situations—the exact opposite of what their consumers were actually doing day-to-day.

Another unexpected tidbit appeared. In a few instances anonymous individuals were recommending the IUD as a viable form of contraception for women who were having extramarital affairs. As we dug deeper we learnt why: for a woman who is on oral contraception—the Pill, to continue taking it after her husband has a vasectomy would increase her risk of getting caught—after all, why would she need the Pill if he’s effectively sterile? The IUD on the other hand is invisible and won’t inadvertently fall out of her purse.

Perhaps this is too niche a target? Perhaps not; consider that up to 60% of all married individuals will engage in infidelity at some point and that the infamous Ashley Madison website has over 10 million members. Also consider that market share battles in this mature category are won or lost by fractions of a point.

Real insights are rare. In a hypercompetitive market there is no such a thing as too small an insight.

Technologies like addressable TV, the increasing sophistication of intent and location-based marketing, and the eventual ubiquity of digital signage—all these point to a real-time future. It’s easy to envision small digital displays on every grocery store shelf—under every SKU, all networked into head office. From there brands will control, monitor and test offers. Decisions made at head-office will be reflected at the point of sale in real-time. If you think the CPG space is commoditized now, just wait.

Smartphones, location-aware devices and gamified experiences will play a key role, as will the next evolution of listening platforms. Creativity will matter more than ever—as it does when a natural system gets hypercompetitive.

People like Shiv Singh, Head of Digital for PepsiCo Beverages America will gamble their futures on it. In a recent article Singh stated that he is staking his whole career on real-time marketing. I had to raise an eyebrow when reading his post. A couple of years ago when meeting with some senior marketing folks at Pepsi Canada I used an image of Barack Obama drinking a Pepsi to illustrate the point that they, like most of their competitors, were organizationally incapable of moving at speed. Here was an opportunity that fell into their lap: an image of the most beloved presidential hopeful since JFK consuming their product—and they did nothing to amplify or own the moment.

The technology and tools of real-time marketing will be here well before most organizations can fight the inertia created by corporate affairs, legal, PR and inter-agency fiefdoms.

Like Singh, I too believe the future of marketing will be real-time and that all organizations will need to play there in order to thrive. The aggressive and agile ones will grab the spoils that come with first-mover advantage.

Back in high school Business Class 101 most of us learned that for a company to thrive it has to invest at least 10% of what it makes back into research and development.

The same should apply to marketing. To maintain relevance and effectiveness marketers need to be pushing boundaries and changing up tactics and strategies often. What works today will likely not work tomorrow.

As a starting point I suggest dedicating 10% of your next campaign’s budget to something new and untried. If your company has never played with mobile—try it. Never did anything with social media—try that. You’ve been there, done that, and won the Lion? Then push into the more esoteric fringes of our craft and experiment with things like neuromarketing and decision-science.

You’ll need to reframe expectations. This cannot be about squeezing maximum ROI from that 10% of your budget. The other 90% should be doing the heavy lifting. This is about exploration and discovery. It’s as much about uncovering new paths to scale as it is about learning which to avoid.

Do whatever it takes to get approval. Save some trees and drop the direct mail component. Run less mass. Call it a pilot program. Then apply that other great lesson we learned in high school: the scientific method. Have a hypothesis, conduct the experiment, record the results then apply what you’ve learned to your next campaign.

By institutionalizing and rigorously applying this 10% rule you just might be the first to uncover the next great insight that redefines your industry.

Stay curious my friends.