I like the card game bridge and play in local duplicate club tournaments about twice a week. Depending on the game or the preference of the director, the hands that are played are either shuffled and dealt by the players at the beginning of the session or are hands that have been previously generated by a computer and arranged by a dedicated card sorter.
In bridge, each of the four players starts with 13 cards and if the deck of cards has been completely randomized before being dealt, the distribution of the four suits (in any order) can vary from somewhat even distributions such as 4-4-3-2 (21.6% probability) to the next most likely 5-3-3-2 (15.5%), 5-4-3-1- (12.9%). 5-4-2-2- (10.6%), 4-3-3-3 (10.5%) and then starts dropping sharply until it gets to 9-3-1-0 (0.01%). More skewed distributions are even rarer. (The Official Encyclopaedia of Bridge (1984))
A common refrain that I hear from players at the table is that they feel that the hands that are generated by the computer tend to have more skewed distributions than the ones shuffled and dealt at the table. They think that whoever is in charge of the computer that generates the hands tend to program it that way in order to provide greater challenge. I heard this so often that I became curious if this was the case and looked it up to see if there was anything to this bit of bridge folklore.
It turns out that the American Contract Bridge League expressly prohibits tinkering with the settings to create more skewed distributions. The hands that are generated by the computer are thus purely random.
Rumors circulate about how computer dealt hands are more distributional than seen at a Club game. First, neither the ACBL Director In Charge, nor anyone else involved with making boards are allowed to pick and choose hands based on certain characteristics. The master ACBL computer which “deals” hands for all tourneys in North America is based on random numbers to ensure the hands are truly random. So the ACBL files sent to the Director receives hands that closely represent hand patterns you would find in bridge tables. This may seem surprising since some Club players seem to encounter flatter hands, as opposed to wilder distribution with computer-generated hands. Why is this true?
So are the players mistaken in their gut sense that the computer generates hands that are more skewed? No, it turns out that they are right, but for a very different and interesting reason than what they think, dealing with the counter-intuitive nature of randomness.
It turns out that if one starts with a completely randomized deck of cards, the hands that are generated have the distributions listed with the above probabilities. But the hands that are created with decks shuffled at the table tend to be ‘flatter’ (i.e., have more even distributions) because it is hard to create a completely randomized deck by shuffling.
This turns out to be an interesting mathematical problem with many applications where the results have been published in journals and books. Most people attempt to create a randomized deck by the so-called ‘riffle-shuffle’ or ‘faro’ method, where you split a deck into two and then holding each half in a hand you interleave them, as shown in the video below. However this only begins to produce randomization after five such shuffles and gets close to complete randomization only if you repeat it at least 11 or 12 times. The authors of the study say that seven shuffles is sufficient to produce approximate randomization for most (but not all) purposes. But most people only do it once and the more conscientious may do it three times but I have never seen anyone do it more than that. Magicians take advantage of the fact that a riffle shuffle does not randomize a deck for some of their tricks.
Interestingly, the seemingly primitive method known as ‘smooshing’ (where you just spread the cards on the table and then randomly move them around with your hands before bringing them back together, takes between 30-60 seconds but, although looking childish, produces more randomization than a few riffle-shuffles. You can also improve the randomness by not dealing the deck of cards in the same pattern (i.e,, say entirely clockwise or counter-clockwise) but by alternating (one round clockwise, the next round counter-clockwise, and so on)
Randomness often does not look quite like what we expect. If you ask people to write down a long sequence of heads H and tails T that they think should look as if it was randomly generated by (say) tossing a coin (like HHTHTT…), professional statisticians can almost always tell if the sequence was generated by a human who simply wrote it down or by a truly randomized method such as actually tossing coins. This is because humans think that randomness means somewhat even distributions and will avoid long consecutive strings of H’s or T’s while a randomly generated sequence will contain them. In other words, a more even distribution is a sign of non-randomness, not the opposite.
Hence the bridge players are right to think that computer-generated hands have more skewed distribution but are wrong in attributing it to the computer program being adjusted to produce hands that are not random. It is the decks shuffled by people that are not properly randomized and hence produce flatter distributions than the randomly generated computer hands.
And of course, a perfect faro shuffle is absolutely predictable, and can be used to deterministically reorder a deck -- a fact which is used in some card tricks.
During WWII one time pads* would be generated by having an operator take a ball from a bingo machine, the instructions being that the operator remove a ball from the machine without looking at the ball selected.
But some operators thought the resulting sequence was less than random so they would try again to obtain a more “random” looking sequence.
Turns out that these one time pads were less than random and the Germans were able to break some of them.
* In cryptography, the one-time pad is an encryption technique that cannot be cracked, but requires the use of a single-use pre-shared key that is larger than or equal to the size of the message being sent. In this technique, a plaintext is paired with a random secret key.
Fun fact: I taught a course on simulation for a few years a while back, and one of the assignments was that students make their own random number generator, sample it and compare the result with the RNG of software of their choice (usually Excel, R, or Python)
Many of them built a RNG that was very repetitive (repeated every 32 draws or so), and concluded that their samples, which had extremely flat distributions, were more random, and thus better than the built in RNG of their computer.
When I taught statistics, I would always take part of a day when we were talking about probability theory to give half the class a ’70s-era dollar coin and telling them to flip it 50 times and record the results, while the other half of the class was to write down what they thought 50 coin flip outcomes would be, then I’d leave the room while they split up (so that I wouldn’t know a priori which half of the class did what) and generated their data sets. I was always able to guess correctly which of the data sets produced was the one produced by actually flipping the coin, since that one would contain longer strings of Hs or Ts than the other, for the reasons you describe above.
I used to work in a manufacturing environment where we tested every fiftieth object to destruction. My supervisor said that if you get the same result more than twice in a row, you should modify it a little so it doesn’t look like you are just making up numbers. I tried to explain, but my probability explanation didn’t overpower his experience.
Electronic and mechanical sources of entropy are tricky to get right. The card shuffler manufacturers are probably aware of the state of the art today, but it’s an evolving field that has taken unexpected turns in the past.
It’s slow, tedious, but fair to start with a deck, three six-sided dice (you’re counting permutations, don’t get them mixed up), and a lookup table, then proceed to do a Fisher-Yates shuffle.
I’ve suspected this for awhile.
The frequency with which perfect hands are dealt in some clubs strongly suggests that some players are not shuffling the cards well enough. (Or perhaps they are performing a technique so well, they are effectively reordering the cards in a deterministic fashion.)
If some players are not so keen to find out that they have been doing it wrong for years, there might well be interesting times ahead …..
@mathman85, yes, indeed.
When I played Magic: the Gathering early on I found that groups both in college and back home (a 5 hour straight drive on interstates the whole way) were aware of the shuffling problem. Home was in the middle of nowhere.
The talk around the game was to shuffle at least 7 times for randomness. Each player in Magic has a 60 card deck (or more). Players care about randomness because one type of card is free to play and then used to play other types of cards. If you don’t get enough of one or the other type of card, you’re kinda just sitting there while your opponent(s) do whatever they want.
This is a pretty good demonstration of the predictability of the randomness people come up with.
It’s been relatively common knowledge for a while that people absolutely suck at randomness, and often end up getting it the wrong way around, as described above.
One of the best examples of this is the classic ‘shuffle’ option on music players once CDs and then later MP3s became a thing. When it was first introduced it usually was completely random… then people started complaining about it not being random enough because it played three songs in the same order as they were on the CD. (Which is, of course, exactly as likely as any other combination of three songs to be played.) The end result is that a lot of media players deliberately de-randomize the ‘shuffle’ play to avoid playing songs in order and to avoid too many of the same band showing up consecutively. Because people just don’t actually understand probabilities or randomness.
With regards to card shuffling in particular, this one might be fun; I can’t remember whether I got this one from Martin Gardner or from John Scarne of “Scarne on Cards” (though Gardner reported on Scarne more than once, so it could have been both). Basically there was a stage magician who was trying to figure out ways to bury the card on top of the pile a certain number of cards deep using perfect riffle shuffles, so he started playing around with attempts. He used ‘O’ to represent an ‘outside’ shuffle (where the half of the pack that was on the top would end up still on top) and ‘I’ to represent an ‘inside’ shuffle (where the other half would end up with its card on top, so the previous top card would be the next one down). And he started writing down which sequences led to where the top card wound up.
After getting down to levels where an inside-outside-inside (or IOI) resulted in the card being five cards down from the top, and noticing a resemblance to the binary numbers… he determined that this was actually a general thing. As long as the card of interest is still in the top half of the deck, a perfect outside riffle shuffle doubles the number of cards on top, while a perfect inside riffle shuffle doubles the number of cards but adds one. So if you can do a perfect riffle shuffle, you can actually guarantee how many cards down the previous top card will end up.
lanir @8
That must have been some time ago. There are still 60 card decks in still popular formats (Standard or Modern), but the most popular format right now has 100 card decks (well, 99 because one card starts outside of your library and usually stays outside of it).
Not only does this mean a larger deck to shuffle, but because so many Magic cards can be worth a fair amount of money, they’re usually kept and played in plastic sleeves, often even double sleeved (a thinner inner sleeve and a larger, more sturdy outer sleeve).
The most common shuffle you’ll see is the overhand shuffle, and sometimes you’ll see pile shuffles (dealing the cards into different piles, sometimes changing the order to try to randomize even more). You’ll still see riffle shuffles in 60 or 40 card formats, but usually only from the professionals. More casual players with decks that can be worth hundreds or thousands of dollars don’t want to keep bending them like that.
another fun thing about a randomly shuffled deck of cards is what are the odds of two random shuffles producing the same order of cards.
the number is 52!=52*51*50…3*2*1 which is about 8x10^67, a HUGE number.
if everyone on earth, say 10 billion people all shuffled a deck of cards every second since the big bang (4.32x10^17 s) they would have produced 4.32x10^27 different shuffled decks.
the result is that all 10 billion people only generated a fraction of about 5x10^-41 of the 52! possible decks.
this is a very very small number, which means, the odds of having two decks shuffled into the same card order is vanishingly small.
every randomly shuffled deck of every card game in the history of the world has been a unique shuffled deck.
mistake:
52! is the number of unique ways of shuffling a deck of 52 cards. that is NOT the odds.
Tabby Lavalamp @11
Whoops. I see that was less clear than I meant it to be. By “early on” I meant early on in Magic, but it could be read as early on in my experience of it. After looking some very old sets up it looks like I got into and out of Magic several times in the 90’s.
The experience that stuck in my mind was when I gave a ride to two brothers who were begging everyone at the local hobby shop for a ride home -- they stole my cards while I was doing them a favor so they wouldn’t have to walk home in soon to be rainy weather. They were adults and even chatted about the cards right before they stole them. Only $20 or less in worth at the time because I was not playing then and had only saved some cards for artistic rather than monetary value. They probably got less than $5 for it all though, as they weren’t welcome at that hobby shop again and the only other local shop dealing in Magic cards at the time was run by a scumbag. The owner would tell you your card was only worth a couple bucks, buy it from you, but when you go back you find it’s front and center in the card display with a $50 price tag on it.
I never really got into cheap little pieces of cardboard with inflated kinda-sorta-maybe monetary value again. I know the vast majority of players are much better behaved and most of them probably even have reasonably okay moral values. But a game that seems to prompt that sort of absurd behavior is going to be hard for me to enjoy. So I got out long before the game publisher was sending Pinkertons after players.