Lyft Director of Data Science & Marketing; Getting Measurement of Marketing Spend Right!

The following is a guest blog post from Ernan Roman, President of ERDM Corp., DMA Hall of Famer and author of “Voice of the Customer Marketing”

Alok Gupta is currently the Director of Data Science/Head of Marketing Science at Lyft. Previously he was the Director of Data Science at Airbnb and an Affiliated Researcher at Stanford University. Ahead of the 2018 Marketing Analytics Conference (May 17-18 in Atlanta), he shared how he has helped right-size the marketing spend through a strategic use of testing and analytics:

  1. What is one marketing topic that is most important to you as an innovator?

Accurate measurement of marketing spend is the single most critical factor in determining the success of a marketing program. It seems simple but measurement is actually very tricky in marketing. Digital channels such as Search and Display are difficult and offline channels such as TV and print are even harder still.

The goal of measurement is to understand the return of the next dollar spent on any particular marketing campaign. But this is not easy to do since we often do not know the counterfactual of what would have happened if we did not spend that dollar. Instead we, as Data Scientists, run experiments that try to estimate this counterfactual.

Sometimes we run incrementality tests, sometimes we run lift tests, sometimes we run market-level experiments – in all cases, the goal is try to create a control group that is not exposed to the advertising, and a treatment group that is exposed and measure the difference in outcomes. Outcomes are typically business goals like purchases or brand awareness.

  1. Why is this so important?

For most companies, the goal of marketing is to maximize the return of a fixed budget. But to do this one has first to be able to measure the return of each marketing channel or campaign, and then based on this correctly allocate the spend. Optimality is reached when one cannot transfer a dollar from any campaign to any other campaign without reducing the overall return.

It is becoming ever more important and harder to maximize return from marketing spend because marketing has become more democratized – every business can bid on Search or Display platforms – and there is a greater abundance of advertising channels. As well as the traditional offline platforms, each person has multiple electronic devices that advertisers can reach them on.

For example, recently at Airbnb, we have been measuring the effectiveness of campaigns goaled with increasing app installs. This has meant exposing some devices in mobile ad networks to seeing a campaign and making sure others are not exposed. Although very preliminary, early data suggests the campaign is causing some incremental app installs, and at potentially acceptable dollar return.

  1. How will this improve the customer experience?

The added benefit of using experimentation and measurement to value channels is that it enables a marketer to glean where the advertising is having greater impact. Moreover, the advertiser can identify exactly where the incremental impact is – that is, where a consumer would otherwise not have converted if they had not been exposed to the advertising.

This effectively enables the customer to ‘vote with their feet’: if they see a campaign they dislike or has no effect on their consumption choice then the advertiser will measure this feedback and stop spending. Measurement deliberately provides a feedback loop from the customer to the marketer, which has the effect of empowering the customer and thus improving their experience.

Experimentation can help discern the effects of all aspects of advertising: the creative content (text and images), the placement, the frequency, the medium, the sequence, the duration, etc. Each aspect of the campaign can be tested and rapidly iterated on based on what the customer likes and dislikes.

  1. How will this improve the effectiveness of marketing?

As more companies build the technology and framework needed for rapid experimentation in marketing, it enables marketers to more quickly ‘test and learn’. Indeed, less time needs to be spent debating which creative or channel should be selected, a marketing team can just try all combinations (within reason) and select the best.

This becomes very liberating for a marketing team as all ideas – ‘hypotheses’ – are welcome and can be tested without bias or judgment. By creating this feedback loop with the customer, decision-making becomes less subjective and more scientific.

That is not to say that everything can be tested and measured. One-off campaigns like a Super Bowl halftime advert are difficult to run an experiment on – which is ok. But for the majority of recurring spend, especially on digital, marketers can make their dollars work harder through proper measurement.

The best way to start testing is to work with channel partners. The major platforms like Google, Facebook, and others offer very sophisticated tools for running experiments to measure return. Even within Google AdWords, marketers can run experiments on different copy and bid strategies. Within Facebook, marketers can launch lift studies on their users.

How-To Tips

Q. How do I start testing?

A: Have a play around in Google AdWords with one of your campaigns – change the title/copy/link text and see what happens!

Q. How do I build a team to improve advertising measurement?

A: Begin by hiring a senior Data Scientist with experience in experimentation, preferably in marketing, and they can be expected to bring in more people.

Q. How do I handle change management at my company?

A: Start with quick small wins for measurement that feel collaborative and inclusive; this will build desire and momentum for more.

Q. How do I optimize my marketing spend?

A: Try to start at channel level and work out roughly what your return is from each channel (like Facebook, Google, etc.) at different levels of spend – then pick the combination which maximizes this return.