A/B Testing at Scale - Risks and Opportunities

WOE 14:00 - 14:30

Zaal 1


Booking.com is known for running A/B tests at scale to optimise every small and big change to our products. Guiding product development by monitoring well-defined metrics allows dozens of teams to work on the same product simultaneously and iterate quickly. However, A/B testing methods can easily be misunderstood, unintentionally abused, or become a limiting factor. 

In this talk, we present common pitfalls and shortcomings of classical A/B testing we've experienced at Booking, together with some concrete remedies and alternative solutions. After listening to this talk, you can go back to your teams and constructively challenge them. 


About Booking.com

Founded in 1996 in Amsterdam, Booking.com has grown from a small Dutch start-up to one of the world’s leading digital travel companies. Part of Booking Holdings Inc. (NASDAQ: BKNG), Booking.com’s mission is to make it easier for everyone to experience the world.

By investing in technology that takes the friction out of travel, Booking.com seamlessly connects millions of travelers to memorable experiences, a variety of transportation options, and incredible places to stay – from homes to hotels, and much more. As one of the world’s largest travel marketplaces for both established brands and entrepreneurs of all sizes, Booking.com enables properties around the world to reach a global audience and grow their businesses.

Booking.com is available in 43 languages and offers more than 28 million reported accommodation listings, including over 6.2 million homes, apartments, and other unique places to stay. Wherever you want to go and whatever you want to do, Booking.com makes it easy and supports you with 24/7 customer support."