Evaluating Performance Consistency in Competitive Programming: Educational Implications and Contest Design Insights
Article Sidebar
Main Article Content
Abstract
Competitive programming (CP) contests are often treated as interchangeable proxies for algorithmic skill, yet the extent to which results at lower contest tiers anticipate performance at higher tiers, and how closely any tier resembles the ubiquitous online-contest circuit, remains unclear.
We analyze ten years (2015–2024) of International Collegiate Programming Contest (ICPC) standings, comprising five long-running superregional championships (Africa & Arab, Asia East, Asia West, North America, and Northern Eurasia), associated local regionals of North America and Northern Eurasia, and the World Finals. For 366 World Finalist teams (2021–2024), we augment the dataset with pre-contest Codeforces ratings. Pairwise rank alignment is measured with Kendall’s τ.
Overall, superregional ranks predict World Final ranks only moderately (weighted τ = 0.407), but regional-to-superregional consistency varies widely: Northern Eurasia exhibits the strongest alignment (τ = 0.521), while Asia West exhibits the weakest (τ = 0.188). Internal consistency within a region can exceed its predictive value for Worlds—for example, Northern Eurasia and North America regionals versus superregionals (τ = 0.666 and τ = 0.577, respectively). Codeforces ratings correlate more strongly with World Final results (τ = 0.596) than any single ICPC tier, suggesting that high-frequency online contests capture decisive skill factors that many superregional sets miss.
We argue that contest organizers can improve both fairness and pedagogical value by aligning problem style and selection rules with the formats that demonstrably differentiate teams, in particular the Northern Eurasian model and well-curated online rounds. All data, scripts, and additional analyses are publicly released to facilitate replication and further study.
Article Details
Data Availability Statement
Data, scripts, and additional analyses are available in the public replication repository. Please cite this article and the authors’ original work when reusing the repository.
Issue
Section
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
JCL provides immediate open access to published articles. Unless an article states otherwise, the work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (https://creativecommons.org/licenses/by-nc-sa/4.0/). Users may share and adapt the work for non-commercial purposes provided they give appropriate credit, link to the license, indicate changes, and distribute adaptations under the same license. Copyright and license details shown on the article page and in the article PDF govern that specific version of record.
How to Cite
References
icpc.foundation. The ICPC Regional Results. https://icpc.global/regionals/results.
Federico Nahuel Quijada. Teams going to ICPC WF 2024 (Kazakhstan 2024) — WIP List. https://codeforces.com/blog/entry/129887.
Federico Nahuel Quijada. Teams going to ICPC WF 2023 (Egypt 2023, 2nd final) — WIP List. https://codeforces.com/blog/entry/117183.
Federico Nahuel Quijada. Teams going to ICPC WF 2021 (Dhaka 2022) — WIP List. https://codeforces.com/blog/entry/102593.
Luo, Zhongtang. (2024). The Reason You are Bad at Codeforces — You are Not Russian Enough. https://zhtluo.com/cp/the-reason-you-are-bad-at-codeforces-you-are-not-russian-enough.html.
Di Mascio, Tania; Laura, Luigi; Temperini, Marco. (2018). A Framework for Personalized Competitive Programming Training. 2018 17th International Conference on Information Technology Based Higher Education and Training (ITHET), 1-8. https://doi.org/10.1109/ITHET.2018.8424620.
Lertbanjongngam, Sila; Chinthanet, Bodin; Ishio, Takashi; Kula, Raula Gaikovina; Leelaprute, Pattara; Manaskasemsak, Bundit; Rungsawang, Arnon; Matsumoto, Kenichi. (2022). An Empirical Evaluation of Competitive Programming AI: A Case Study of AlphaCode. 2022 IEEE 16th International Workshop on Software Clones (IWSC), 10-15. https://doi.org/10.1109/IWSC55060.2022.00010.
Bandeira, Ian Nery; Machado, Thiago Veras; Dullens, Vitor F.; Canedo, Edna Dias. (2019). Competitive programming: A teaching methodology analysis applied to first-year programming classes. 2019 IEEE Frontiers in Education Conference (FIE), 1-8. https://doi.org/10.1109/FIE43999.2019.9028518.
Yuen, Kevin K. F.; Liu, Dennis Y. W.; Leong, Hong Va. (2023). Competitive programming in computational thinking and problem solving education. Computer Applications in Engineering Education, 31(4), 850-866. https://doi.org/10.1002/cae.22610.
Luo, Yongjun; Zheng, Hong. (2021). An Innovative Teaching Mode Based on Programming Contest. Application of Intelligent Systems in Multi-modal Information Analytics, 469–477.
Raman, Raghu; Vachharajani, Hardik; Achuthan, Krishnashree. (2018). Students motivation for adopting programming contests: Innovation-diffusion perspective. Education and Information Technologies, 23(5), 1919–1932.
Bloomfield, Aaron; Sotomayor, Borja. (2016). A programming contest strategy guide. Proceedings of the 47th ACM technical symposium on computing science education, 609–614.
Blum, Jeremy J. (2023). Competitive programming participation rates: an examination of trends in US ICPC regional contests. Discover Education, 2(1), 11.
Madeira, R Neves; Silva, Bruno; Grueau, C. (2009). A methodology and a framework proposal for competitive learning in programming courses. Research, Reflections and Innovations in Integrating ICT in Education, 1, 315–319.
Ribeiro, Pedro; Guerreiro, Pedro. (2008). Early introduction of competitive programming. Olympiads in Informatics, 2, 149–162.
Guerra, Luis H González; Juárez, Gilberto Huesca; Pérez-Lezama, Claudia V; González, Elda G Quiroga; Campos-García, Gabriela A; Cacho, Mónica Larre Bolaños. (2024). Enhancing Problem-Solving Skills: The Synergy of Competitive Programming and Gamification Strategy. 2024 IEEE Global Engineering Education Conference (EDUCON), 1–7.
Gonzalez-Escribano, Arturo; Lara-Mongil, Victor; Rodriguez-Gutiez, Eduardo; Torres, Yuri. (2019). Toward improving collaborative behaviour during competitive programming assignments. 2019 IEEE/ACM Workshop on Education for High-Performance Computing (EduHPC), 68–74.
Moreno, Julian; Pineda, Andres F. (2018). Competitive programming and gamification as strategy to engage students in computer science courses. Revista Espacios, 39(35).
Garcia, Jose R; Aguirre, Vanessa E. (2014). The learning curves of competitive programming. AIP Conference Proceedings, 1618(1), 934–937.
Kurtz, Chas. (2013). The World’s Smartest Programmers Compete: ACM ICPC. Communications of the ACM. https://cacm.acm.org/blogcacm/the-worlds-smartest-programmers-compete-acm-icpc/.
Pelech, James. (2010). The comprehensive handbook of constructivist teaching: From theory to practice. IAP.
de Picanço, Wollace Souza; de Lucena, Juliana M.V. Martinez; de Lira, Antônio Fonseca; de Lucena, Vicente Ferreira. (2018). Didactic Framework for Teaching C Programming Language: A Proposal Based on Cooperative and Competitive Learning Techniques. 2018 IEEE Frontiers in Education Conference (FIE), 1-9. https://doi.org/10.1109/FIE.2018.8658825.
Jara, Carlos A; Candelas, Francisco A; Torres, Fernando; Dormido, Sebastián; Esquembre, Francisco. (2012). Synchronous collaboration of virtual and remote laboratories. Computer Applications in Engineering Education, 20(1), 124–136.
Moreno-Ger, Pablo; Martínez-Ortiz, Iván; Gilmartin, Virginia Francisco; Ballesteros, Raquel Hervás. (2013). TrivialCV: Competitive activities for the classroom integrated in a moodle virtual campus. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, 8(1), 31–38.
Reeve, Johnmarshall. (2012). A Self-determination Theory Perspective on Student Engagement. Handbook of Research on Student Engagement, 149–172. https://doi.org/10.1007/978-1-4614-2018-7_7.
Wulf, Tom. (2005). Constructivist approaches for teaching computer programming. Proceedings of the 6th Conference on Information Technology Education, 245–248. https://doi.org/10.1145/1095714.1095771.
Robins, Anthony; Rountree, Janet; Rountree, Nathan. (2003). Learning and teaching programming: A review and discussion. Computer science education, 13(2), 137–172.
Roger Newson. (2002). Parameters behind “Nonparametric” Statistics: Kendall's tau, Somers’ D and Median Differences. The Stata Journal, 2(1), 45-64. https://doi.org/10.1177/1536867X0200200103.
Di Luigi, William; Farina, Gabriele; Laura, Luigi; Nanni, Umberto; Temperini, Marco; Versari, Luca; others. (2016). oii-web: An interactive online programming contest training system. Olympiads in Informatics, 10(1), 207–222.
Luo, Zhongtang. (2025). Curriculum Design of Competitive Programming: a Contest-based Approach. arXiv preprint arXiv:2504.00533.
Manzoor, Shahriar. (2008). Common mistakes in online and real-time contests. XRDS: Crossroads, The ACM Magazine for Students, 14(4), 10–16.
Dickey, Ethan. (2025). Removing the Competition from Competitive Programming: A Novel Curriculum Design. arXiv preprint arXiv:TBA.
Trotman, Andrew; Handley, Chris. (2008). Programming contest strategy. Computers & Education, 50(3), 821–837. https://doi.org/10.1016/j.compedu.2006.08.008.
Wicklin, Rick. (2023). Weak or strong? How to interpret a Spearman or Kendall correlation. SAS Blogs. https://blogs.sas.com/content/iml/2023/04/05/interpret-spearman-kendall-corr.html.
ICPC Beijing Headquarter. ICPC Beijing Headquarter. https://icpc.pku.edu.cn.
SUA Competitive Programming Judges. SUA Competitive Programming Judges. https://sua.ac/about.html.
Meza, Federico. (2025). ICPC Regional Structure – Research Inquiry. Private communication.