Use your statistical intuition to save Christmas!

A Christmas-themed statistical game and experiment

(The experiment is over, but you can see a version of it on github)

The Jinglies and the Sparklies face off in a toy-making competition. Who is faster?

It’s nearly Christmas, but Santa has a problem: his elves have stopped working! The elves have have divided into two gangs, the Jinglies and the Sparklies, each believing that their gang can make toys faster. Santa has promised them that the fastest team can take next Christmas season as a holiday; but now, the elves have demanded an authoritative answer to who is faster. Santa has called on your expertise to help settle the question. The elves won’t work until you give Santa your expert opinion on the matter!

It won’t be easy though; Santa is very secret. You aren’t allowed to know how many elves you’re testing. To prevent you from accessing the secret raw data, you can only use his custom statistical software, Santa’s Pretty Simple Statistics™, which doesn’t show any numbers, only colors! On the other hand, you can order as many experiments as you like. How will you solve the problem? Can you decide which team is faster, if either? What will you tell Santa?

Rink Hoekstra and I have devised this game as part of an experiment to see how people use the results of simple experiments. We’d be very grateful if you’d give it a go. It will take a few minutes, depending on how you approach the problem — it is an open-ended experiment. We’ve tried to make it fun enough that you will want to try multiple times. If you like, you can even download your own data at the end and import it into your favorite statistical software (just don’t tell Santa you smuggled the secret data away from the North Pole!).

If you’re an instructor of a science course at the University level, we’d also love if you’d send a link to our study to your students. We’re curious how budding scientists will tackle the problem as well.

Thanks for participating!

(The experiment is over, but you can see a version of it on github. When we release the data, I will link it on my Medium blog and share the pre-print.)