Online Advertising Incrementality Testing: Practical Lessons And Emerging Challenges

Tutorial Website to come soon.

Tutorial schedule to come soon.

Tutorial Organisers

  • Joel Barajas
    Yahoo Research, Verizon Media
  • Narayan Bhamidipati
    Yahoo Research, Verizon Media
  • James G. Shanahan
    Church and Duncan Group Inc and UC Berkeley

Tutorial Abstract

Online advertising has historically been approached as an ad-to-user matching problem within sophisticated optimization algorithms. As the research and ad-tech industries have progressed, advertisers have increasingly emphasized the causal effect estimation of their ads (incrementality) using controlled experiments (A/B testing). With low lift effects and sparse conversion, the development of incrementality testing platforms at scale suggests tremendous engineering challenges in measurement precision. Similarly, the correct interpretation of results addressing a business goal requires significant data science and experimentation research expertise. We propose a practical tutorial in the incrementality testing landscape, including: item The business need Literature solutions and industry practices Designs in the development of testing platforms The testing cycle, case studies, and recommendations. We provide first-hand lessons based on the development of such a platform in a major combined DSP and ad network, and after running several tests for up to two months each over recent years.