The Service-Side Selling of Self Interest-based Salamander-Like Statistics
No doubt MadAve Journal readers are well up to speed on the recent Harvard University Biological Sciences Research which illuminates the behavior of Eastern Redback Salamanders (Plethodon cinereus) and Juvenile Eastern Red-Spotted Newts (Notophthalmus viridescens) found in abundance in Hemlock and Hardwoods.
Like virtually everyone else on MadAve, we believe that the study's learnings has great relevance to our business, specifically as it relates to the use of statistics in all aspects of advertising.
Here's why: we all know that Salamanders and Newts change the nature of their color to adapt to the environment they are in. We also know that service-side marketing vendors change the nature of their research to adapt it to the audience and environment they are in.
Salamanders and Newts do so for self-preservation purposes. So do researchers. However, while this is a natural deceptive mechanism to protect amphibians from predators in the jungle, those Madison Avenue agencies and publishers that change the nature and color of their marketing research act as predators on their own clients!
Colored, massaged and infested statistics provides a deceptive mechanism to justify service-side self-interests at the expense of the clients it is supposed to support. In essence, while Salamanders and Newts prevent themselves from extinction by changing colors, marketers who allow/accept statistics that’s been colored or changed face extinction.
Today, there is a sea of data available to MadAve as never before. Years ago, with fewer sources, statistics was less apt to be manipulated due to two reasons:
1. The sources were more widely known and universally used by major players in each category.
2. Researchers were nerds. They had pocket protectors, wore thick glasses and were committed to excellence vs. pressured to draw outside the lines to help their company’s P&L.
There’s little doubt today that research company CEO’s do not regret the extinction of data-geeks, just like industrialists do not regret the extinction of the rare white-spotted owl.
The days of Geekiness at data companies is now gone with the wind. Today, we face an environment where objectivity is virtually extinct, replaced with hyping database strengths with 80-point fonts while downsizing weaknesses with 8 font small type.
Account Planning – When Jay Chiat introduced the use of Account Planners in the US from Great Britain, this was universally received since the industry saw the benefits statistics would help smart clients become smarter, better, faster. This was also due to the fact that data-geeks were still back-room staff. Management would often flinch when their presence was required in new business meetings since they dressed and spoke very different vis-à-vis other departmental heads also in the room.
Account Planning changed all this. While we joke about “planners” today are often British, always wear black, wear funny glasses and smoke incessantly, they were easy on the eyes, ears and over drinks later after the meeting. They were (and are today) cool!
They also provided a fundamentally new role in the agency; to be the one source that all departments could go, to find the “truth” on literally every aspect of the client’s business. Seemed like a great idea. Unfortunately, instead of sticking them back in research cubicles near the bathroom, air conditioning units or file cabinets, lucky for them, they shared offices near or in creative departments.
They mingled with art directors, copywriters, and producers. Good for them, bad for clients. What began as a purist's discipline and love of the truth, over the years became a department to justify the creative solution. Agencies loved “planning” since it made selling creative (mostly TV) easier to convince research-loving clients, who were much more comfortable approving creative which had "statistics" to back it up, than simply the creative director’s passionate presentation. Changing colors of data became synonymous with a planners’ ability to tap dance. You can probably figure out the rest of the story.
2. Media Planning & Buying – The MadAve Journal recently published two articles on manipulating media planning and buying research statistics. The first is titled, "Should You Get An MRI Before Using MRI?" and the other is titled, "The Mommy Reach Theory of Effective Reach" which details how infested media research statistics are colored to to justify media "planner" recommendations. Please review at your convenience. Needless to say, if coloring research by account planners has run is a problem, it is a tiny white spot on an extinct owl as compared to the tainted statistics being used to justify $$ million plans and buys.
3. Results ROI Analysis – A relatively new type of data on Madison Avenue. Other than DR, much research on sales effectiveness has been has yet to be fully known. More on this at another time.
Solution: Statistical Planning:
Marketers, who audit their performance and come up with soft sales, may find that the data their service-side “partners” used to justify their strategies may be softer than the bedrock rationale they claimed. What to do?
1. Hire a media auditor in the planning stage, not only on the backend audit stage.
2. Require partners to justify the statistical databases before planning begins. Virtually any strategy can be backed up with research. The abundance of data available today could justify murder. Unfortunately, brands risk being murdered if they allow their agency’s to get away with research murder.
3. Make sure research databases are agreed to long before planning begins. The longer the better. That’s why agencies make the big bucks today, for their expertise. If they change their mind on a database’s validity at the time the campaign needs to be sold in, question why did their confidence change? This includes agreeing up front what sources should be excluded and why.
4. Have the auditor or agency begin with their statistical planning cycle with a methodology audit. Take a close look at it. Use common sense in your analysis. Do some role playing to see if you would answer honestly or even take the time to allow yourself to be queried. If it seems artificial or superficial to you, it most likely does to the “respondents” as well. If it feels stupid to you, you need to question why you would accept research that only idiots would sit for.
The dawn of natural research will soon be upon us and in some ways is already here. The irony about natural feedback is that it is primarily delivered without human tainting or tweaking. The only real confidence marketers should have in statistics today is based on unweighted rough data that is readable without being massaged. Here are a few examples:
1. Blog feedback
2. Straight sales data
3. Non-sample-based sources
4. Research two or more of your competitors use
5. Feedback from your objective and presumably honest distributors
Here are a few types of statistics to take with a grain of salt:
1. Syndicated data which measures 500 brand categories and 6000 brands
2. Focus-group research
3. Telephone coincidental research
4. Mall testing
5. Creative recall testing
6. Intent to buy measurements
7. Statistical modeling bases tools
8. Any research you have not heard of
9. Any research more than 1 year old
Besides being unhelpful, the largest problem marketers should be aware of is research that has no relevance or use to the brand. As we enter a new phase of data intelligence, service-side vendors should be required to provide a work-flow application to all the output campaigns are generating. If there’s no relevant application, it raises the question of why it is being collected.
statistical Research should strengthen - not weaken - the sophistication of the marketing message and campaign. The analysis should be a 360 degree measurement. When it is not, it gives vendors the opportunity to allow their accountability to fall into the abyss of the unknown.
Harvard’s research helps us understand how nature finds a way to give each of its creatures (however large or small) a fighting chance to survive and avoid extinction. Unfortunately Mother Nature does not afford marketers the same protection.