Control Group Selection for A/B Testing Through Optimization
  IJCTT-book-cover
 		          
 
© 2024 by IJCTT Journal
Volume-72 Issue-5
Year of Publication : 2024
Authors : Manasa Gudimella
DOI :  10.14445/22312803/IJCTT-V72I5P107

How to Cite?

Manasa Gudimella, "Control Group Selection for A/B Testing Through Optimization," International Journal of Computer Trends and Technology, vol. 72, no. 5, pp. 56-64, 2024. Crossref, https://doi.org/10.14445/22312803/IJCTT-V72I5P107

Abstract
A/B testing is critical for big retail giants to adapt to rapidly changing consumer preferences and maintain market leadership. In a situation where the test group is predetermined, effective A/B testing requires selecting control stores that are comparable to test stores, a challenge addressed in this paper. The focus of this study is on the methodology for selecting homogeneous experimental units in physical retail settings evaluating them through a time series of key performance indicators. The methodology demonstrates adaptability to multiple KPIs, enhancing its applicability. Two methods for control store selection are introduced and compared: a statistical sampling technique and an optimization approach, with findings indicating the superiority of the optimization method for achieving more accurate and reliable A/B testing results. This research offers significant insights for retailers aiming to optimize their in-store strategies and improve overall business strategies and performance, making it essential for retail decision-makers seeking to optimize operational efficiency.

Keywords
A/B testing, Controlled experiments, Experimental Design, Optimization, Population Sampling, Optimization.

Reference

[1] Howard Anton, Elementary Linear Algebra, 9th Edition, John Wiley & Sons, 1973.
[Google Scholar] [Publisher Link]
[2] Amitav Banerjee, and Suprakash Chaudhury, “Statistics without Tears: Populations and Samples,” Industrial Psychiatry Journal, vol. 19, no. 1, pp. 60-65, 2010.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Ilker Etikan, and Kabiru Bala, “Sampling and Sampling Methods,” Biometrics & Biostatistics International Journal, vol. 5, no. 6, pp. 215-217, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Julius Hokka, “Controlled Experiments for Data-driven Retail Optimization,” Master’s Thesis, pp. 1-82, 2023.
[Google Scholar] [Publisher Link]
[5] Kirthi Kalyanam et al., “Cross Channel Effects of Search Engine Advertising on Brick & Mortar Retail Sales: Meta Analysis of Large Scale Field Experiments on Google.com,” Quantitative Marketing and Economics, vol. 16, pp. 1-42, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Ron Kohavi et al., “Online Controlled Experiments at Large Scale,” Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1168-1176, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Rembrand Koning, Sharique Hasan, and Aaron Chatterji, “Experimentation and Start-up Performance: Evidence from A/B Testing,” Management Science, vol. 68, no. 9, pp. 6355-7064, 2022.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Sharon L. Lohr, Sampling: Design and Analysis, Duxbury Press, pp. 1-494, 1999.
[Google Scholar] [Publisher Link]
[9] Douglas C. Montgomery, Design and Analysis of Experiments, John Wiley & Sons, pp. 1-734, 2017.
[Google Scholar] [Publisher Link]
[10] Douglas C. Montgomery, Cheryl L. Jennings, and Murat Kulahci, Introduction to Time Series Analysis and Forecasting, John Wiley & Sons, 2015.
[Google Scholar] [Publisher Link]
[11] Katta G. Murty, Linear Programming, Springer, pp. 1-482, 1983.
[Publisher Link]
[12] Federico Quin et al., “A/B Testing: A Systematic Literature Review,” Journal of Systems and Software, vol. 211, pp. 1-28, 2024.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Hamdy A. Taha, Operations Research: An Introduction, Prentice Hall Upper Saddle River, NJ, USA, 2013.
[Google Scholar] [Publisher Link]
[14] Hamed Taherdoost, “Sampling Methods in Research Methodology; How to Choose a Sampling Technique for Research,” SSRN, vol. 5, no. 2, pp. 18-27, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Ya Xu et al., “From Infrastructure to Culture: A/B Testing Challenges in Large Scale Social Networks,” Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2227-2236, 2015.
[CrossRef] [Google Scholar] [Publisher Link]