Using AI to measure ad creativity at scale: how ad distinctiveness and consistency affect ad effectiveness

Blog by Xiongkai Tan – PhD student at the University of Groningen

This blog is based on the presentation that was given during the yearly EMAC conference in Bucharest last May. DVJ hosted a special session where Lisette Kruizinga- de Vries shared insights from internal studies and the PhD students we cooperate with shared the latest insights from their studies.

CREATIVITY CRISIS

Research indicates that the impact of advertising is declining over time (Sethuraman et al. 2011). This trend, thought to be due to diminishing creativity in advertising, is concerning for advertisers. In recent years, the creativity of ads has been overshadowed by technology-driven trends. This shift has led advertisers to prioritise metrics such as click-through rates and immediate sales results, pushing brands to focus on “what works” and over-optimize the same ad formats (Roach 2019). Ads cannot stand out due to high ad clutter and low distinctiveness (Ha 2017). At the same time, brands also face a trade-off between distinctiveness and consistency, as straying too far can confuse consumers or dilute brand identity.

USING AI TO MEASURE AD CREATIVITY AT SCALE

Although creativity is widely recognised as a crucial determinant of advertising success, the challenges in objectively measuring it have limited field research on its impact on actual consumer purchasing behaviour (brand sales). The limited existing studies generally adopt the human coding approach, which makes evaluating large-scale ads costly. Additionally, human coders may be influenced by current trends or their subjective interpretations, and they cannot maintain the consistency of data interpretation across time. Put differently, an ad that was unique fifteen years ago may no longer be perceived as unique by today’s consumers.

We employ a state-of-the-art AI algorithm, Vision Transformer (ViT) (Dosovitskiy et al. 2020), to measure ad distinctiveness, the core component of ad creativity that reflects the extent to which an ad diverges from those of competing brands (Rosengren et al. 2020). We use ViT to transform the rich information in an ad image into a representation vector and measure the similarity between each pair of ads using the cosine similarity of their representation vectors. For each ad, we measure its distinctiveness using the weighted average similarity with ads released by competing brands before its own release. The intensity of brand competition, time decay, and share of voice are considered for weighting. Similarly, we measure its consistency using the weighted average similarity with ads released by its brand before its own release.

A large consumer survey dataset is used to validate the AI-based measurement. A total of 4,537 participants evaluated all 245 print ads released in the Dutch automotive industry in 2023, with each ad being reviewed by an average of 74.07 participants. We used Smith et al. (2007)’s scale on ad creativity in the survey. Preliminary results indicate that AI-based algorithms evaluate ad uniqueness and consistency similarly to consumers. The correlation between the two types of measurement for distinctiveness (consistency) is 0.57 (0.55), p<0.001 in both cases.

OUR ANALYSIS

We conducted research on advertising in the automotive industry in the Netherlands. The automotive industry was chosen due to its considerable economic significance, large volume of ads, and intense competition. Our dataset includes all advertising campaigns across six major media channels—TV, radio, out-of-home, online, magazines, and newspapers—and monthly sales data for 73 different automotive brands in the Netherlands from 2007 to 2023 (204 months). We use AI to measure the distinctiveness and consistency of 8,548 unique print ad creatives and aggregate these measurements to the brand-month level. Our analyses focus on ad distinctiveness and consistency in print advertising while controlling for all other advertising channels.

Our preliminary analysis shows that ad distinctiveness has significantly declined, while ad consistency has significantly increased. We standardised ad distinctiveness and consistency in our sample to a scale with a maximum of 100 and a minimum of 0. Over the 17-year period, the results show that ad distinctiveness in the Dutch automotive market decreased by 11.63 points, while ad consistency increased by 5.30 points.

Moreover, our analyses reveal that the impact of ad distinctiveness on ad effectiveness follows an inverted U-shape, indicating that being too distinctive may harm ad effectiveness. In contrast, the impact of ad consistency on ad effectiveness is a positive linear relationship.

KEY TAKEAWAYS

AI Measurement Tool: Utilising AI tools, such as Vision Transformers, enables effective and fast measurement of ad distinctiveness and consistency, providing valuable insights into the effectiveness of advertising strategies.

Decreased Ad Creativity: There has been a notable decline in ad creativity over the past 17 years in the Dutch automotive market, with distinctiveness decreasing and consistency increasing, highlighting a shift in creative approaches.

Distinctiveness and Consistency are Relevant: Both distinctiveness and consistency are crucial for ad effectiveness. The best ad creatives strike a balance between standing out from competitors and maintaining brand continuity, optimising the impact on ad effectiveness.

REFERENCES

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2020). An image is worth 16×16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.

Ha, L. (2017). Digital advertising clutter in an age of mobile media. Digital advertising: Theory and research, 69-85.

Roach, J. (2019). Why digital advertising killed creativity and how to make your brand remarkable again. https://wearescs.com/digital-marketing/why-digital-advertising-killed-creativity-and-how-to-make-your-brand-remarkable-again/.

Rosengren, S., Eisend, M., Koslow, S., & Dahlen, M. (2020). A meta-analysis of when and how advertising creativity works. Journal of Marketing, 84(6), 39-56.

Sethuraman, R., Tellis, G. J., & Briesch, R. A. (2011). How well does advertising work? Generalizations from meta-analysis of brand advertising elasticities. Journal of Marketing Research, 48(3), 457-471.

Smith, R. E., MacKenzie, S. B., Yang, X., Buchholz, L. M., & Darley, W. K. (2007). Modeling the determinants and effects of creativity in advertising. Marketing science, 26(6), 819-833.