Corey and Lori's Quest Log


Corey and Lori’s Quest Log

Random Acts – The Drunkard’s Walk

I (Corey) just read a fascinating book called, “The Drunkard’s Walk: How Randomness Affects Our Lives,” by Leonard Mlodinow. Since I found myself quoting all sorts of interesting tidbits from the book to Lori, I guess it’s time for another book review – what “The Drunkard’s Walk” is all about and how it relates to games and life.

Meep PrintsA “drunkard’s walk,” also known as a “random walk,” is a mathematical term for randomness. Suppose you take one step in a random direction, turn in a random direction, take another step, and so on? Will you end up at your starting point? It’s possible, but it’s far more likely you’ll end up somewhere else. If you flip a balanced coin and it comes up Heads, the next flip is equally likely to come up Heads or Tails. Over the long run, you’ll probably get about half of each, but you can expect to see a lot of “clusters” of 3, 4, 5, or more Tails in a row. That’s why you can’t just pick the best team in a sport and expect them to win every time. There are so many random factors to any significant event that you can never be sure of the outcome until it happens.

There are a lot of counter-intuitive results in probability. Perhaps you’re familiar with the “Let’s Make a Deal” puzzle from the old television game show. Your host, Monty Hall, shows you three doors. Behind one is a brand-new Mercedes, while the other two have live donkeys. After you choose one of these doors, Monty opens one of the remaining two and shows you that there’s a donkey behind it. He then offers you the choice to stick with your original door or switch to the last one. So, what IS behind Door number One? Should you switch? Does it make a difference?

What do you think?

What Are the Odds?

“The Drunkard’s Walk” has some crossover with the previously-reviewed book, “The Black Swan.” One of the important points is that there’s a huge difference between “unlikely” and “impossible.” Over enough trials, every unlikely result is likely to occur. And in a single trial, anything can happen. If you roll two dice together, the most likely result is that they will total 7, but that only happens 1/6 of the time. It is twice as likely that you will roll one of the “unlikely” results of 2, 3, 4, 10, 11, or 12, because their combined chance is 1/3.

That’s really what the Black Swan theory is about – When you look at enough highly unlikely possibilities, the combined chance that at least one of them will happen is actually very high. Of course it’s impossible to predict which unlikely chance will come up – except as a random guess – but unlikely things occur all the time. A friend is fond of saying, “All that probability shows is how unlikely it was for the thing that just happened to occur.”

You’re probably all familiar with the “bell curve” – also known as a “normal distribution”. Basically, “average” and near-average results are the most common, while very low and very high results are rarer. The problem is that our minds are not wired very well for understanding randomness. As a result, we tend to overemphasize the high probabilities and underestimate the lower ones. If we roll two dice, if we expect them to total 6, 7, or 8, we’ll be right a lot, but still wrong more than half the time. If we note that the stock market has historically risen 8-10% per year, we may find ourselves expecting our stock holdings to go up 9% next year… but as we’ve seen, the fluctuations that make up that total trend can be huge, and there’s no guarantee the trend will continue. The outliers – strings of low-probability random results – actually happen quite a lot and can make life very interesting.

Patterns in Chaos

Our brains are programmed to look for and recognize patterns. This is a valuable survival trait, but has the unfortunate side-effect that we tend to see patterns where there are none. We tend to think that a heavily-downloaded song must be good. It might be, but it’s just as likely that it got a few extra downloads early from fans or random chance, then after that benefited from the snowball effect of others assuming that its early success was meaningful.

A related phenomenon is the “confirmation fallacy,” which basically says that we see what we expected to see. Sneaky researchers did a blind taste test of cola brands. First they asked the 30 participants which they preferred – Pepsi or Coke. They then tasted both colas, and 21 out of 30 found they liked the brand they had said they preferred. However, the researchers had switched the bottles, putting Coke in the Pepsi bottle and vice versa. In another test, researchers put the same wine into five bottles with price tags ranging from $10 to $90. The $90 bottle got much higher ratings than the $10 bottle. We tend to believe “authority”, in this case, that the $90 wine must be better for them to be able to charge that much.

Are you concerned that you got a “B” on an important essay test when you thought you should get an “A”? That’s just another example of randomness. In one study, a group of eight faculty members independently graded 120 term papers on the A-F scale. In some cases, their grades differed by two full marks. The average range was one full grade. I remember a friend in High School getting marked down for misspelling “Trinity” in the title of his short story, “A Threnody for Reason.” The teacher didn’t bother to look up “threnody” – a funeral dirge – which was in fact a perfect title for the story. Or the college writing instructor who thought that Lori made up the word, “Ragnarok” on a poem… Teachers, just can’t trust ’em… er… except for the ones at Our School!

Randomness happens… but how we react to it affects how we live our lives. We think that Bill Gates must be much smarter than other software entrepreneurs because Microsoft has been so successful. And yet, the story of Microsoft points to a huge series of lucky incidents that resulted in that success. Any of a number of less-successful entrepreneurs could be just as smart, and run their businesses just as well, but got fewer “heads” in a row on the coin flips of fate. Sports team managers and executives are judged on the success of their team/company, but pretty much all the winning streaks and team records follow normal distribution patterns. They match random results much better than anything predictive based on management. “The Drunkard’s Walk” has dozens of similar examples of events which are best explained by randomness, but which we tend to think of as having a deeper pattern and meaning.

Make Your Best Deal

Meep MartiniDid you answer, “It doesn’t matter,” to the Let’s Make a Deal puzzle? Most people do. In fact, when Marilyn vos Savant said in her syndicated newspaper column that you should switch, she received a lot of angry letters from pretty intelligent people. However, she was correct. Look at the problem this way – You started with a 1/3 chance of picking the correct door. There was a 2/3 chance that the car was behind one of the other doors. Now Monty – who knows which door hides the car – eliminates a door. Your door still has a 1/3 chance of being correct, and the other two doors still have a 2/3 chance. But now there’s only one door to switch to, so the 2/3 chance applies to it alone. Switching gives you twice the chance of driving home in a new car as staying with your original pick. Results on the show confirm this – People who chose a door and stayed with it won the big prize about 1 time in 3. People who switched won 2 times in 3 – double the odds of staying with your original pick.

You can see this more clearly by saying there were 100 doors at the beginning. After you pick one, the host opens 98 of the remaining 99 doors to show they’re empty (or filled with donkeys). There’s a 99/100 chance that the prize is behind the last door, vs. the 1/100 chance that you picked correctly originally. Even if you think there’s a good chance your host is trying to cheat you, you should switch. After all, isn’t it almost as likely that he’s trying to use reverse-psychology on you by trying to keep you from switching?

We all tend to be stubborn about choices. Once we make one, we hate to switch. But when the original decision is purely random, and there is any evidence at all in favor of switching, it pays to be flexible.

Getting Superior Results in a Random World

Are you depressed at the idea that so much of what happens in our lives is random? You shouldn’t be. The important message I took away from “The Drunkard’s Walk” was that we can make randomness work to our advantage. Failures happen, but people who refuse to give up after a setback are the ones most likely to find eventual success. If you “win on a 6”, keep rolling the dice. Sooner or later, you’re likely to hit. If you give up after the first roll, you’re out of the game.

Most of us have had the unpleasant experience of being turned down for a job. Writers and salespeople face constant rejection; one writer papered his cabin with rejection slips. The successful ones are those who keep going and try again. My father has been a successful real estate investor. He once told me that the important thing was never to fall in love with a property. He would make an offer far below the asking price, and if it was rejected, move on to the next property. The first Harry Potter book was rejected nine times before J. K. Rowling found a publisher and became the wealthiest writer in the world. The winners in the game of life are those who keep going and keep trying.

“Never give up; never surrender. Full speed ahead!” Most success comes from trying and failing and trying and failing and yet, trying again. Keep trying and the random factors will eventually align (probably!). Make the odds work in your favor! If you never give up, you’ll never fail.

Happy Holly Days!

Similar Posts:

Tags: ,

Tweet Me!Share on Twitter

Comments

  1. Nagath Says:

    Here in the Netherlands there’s woman ‘Lucia de Berk‘ who was convicted, by aid of statistical proof, for murdering her patients and was released a few years later because the stastical proof was no longer deemed valid. There is an english article on her and her case on wikipedia.

    As for the Dutch article on the down syndrome test, it states that the probability that the test is positive while the fetus doesn’t have the down syndrome is 1%.

    Since im not a Wizard, I might not get this proof communicated with magical precision. I spent some hours getting it all together, so I hope you don’t mind me opting to post it here.

    So, first some orientation: In the proof I use ’conditional probability’, ‘law of total probability’, and multiplication of probabilities.

    P(D) = 0.01, is the probability that the fetus has down syndrome
    P(D’) = 0.99, is the probability that the fetus doesn’t have down syndrome
    P(TP|D) = 0.9, is the probability that the test is positive given that we know the fetus has down syndrome
    P(TP|D‘) = 0.01, is the probability that the test is positive give that we know the fetus doesn‘t have down syndrome

    Our goal, P(D|TP), is the probability that the fetus has down syndrome given that we know the test is positive. We can’t find this probability directly so we use the general multiplication rule for conditional probabilities. P(D /\ TP), the probability that the baby has down syndrome and the test is positive is the same situation as P(D|TP) x P(TP), the probability the fetus has down syndrome given that we know the test is positive multiplied by the probability that the test is positive.

    P(D|TP) x P(TP) = P(D /\ TP),
    P(D|TP) x P(TP) / P(TP) = P(D /\ TP) / P(TP),
    P(D|TP) = P(D /\ TP) / P(TP)

    Now we have a new goal, P(D /\ TP) / P(TP).
    P(D /\ TP), the probability that the baby has down syndrome and the test is positive is the same situation as P(TP|D) x P(D), the probability that the test is positive given that we know the fetus has down syndrome multiplied by the probability that the baby has down syndrome.

    P(D /\ TP) = P(TP|D) x P(D) = 0.9 x 0.01 = 0.009

    P(TP), the probability that the test is positive, can be found using the total probability rule:
    the probability that the test is positive given that we know the fetus has down syndrome multiplied by the probability that the fetus has down syndrome
    added to
    the probability that the test is positive give that we know the fetus doesn‘t have down syndrome multiplied by the probability that the fetus doesn’t have down syndrome.

    P(TP) = P(TP|D) x P(D) + P(TP|D‘) x P(D’) = 0.9 x 0.01 + 0.01 x 0.99 = 0.009 + 0.0099 = 0.0189

    With both P(D /\ TP) and P(TP) we come back to P(D|TP)
    P(D|TP) = P(D /\ TP) / P(TP) = 0.009 / 0.0189 = 0.476

  2. Marquillin Says:

    It’s a known fact that 45.7% of all statistics are made up to make people sound informed. However, only 3% of the world is aware of this, so your chances of sounding like an ass are very low, and within that 3% your chances are something like 55.3% to the square root of your charisma stats and- what the fruitcake am I saying?

    But that was very interesting about the chances regarding the doors, something that wasn’t obvious until you had raised the stakes to 1/100 vs 1/2. I think I’m hardwired to think if someone tries to make me second guess my decisions, they must be up to something, but logic ought to win out over obstinance.

  3. Corey Says:

    “The Drunkard’s Walk” mentions a similar case in England where a woman (Sally Clark) lost two children to suffocation in a way that might have been Sudden Infant Death Syndrome (SIDS). A pediatrician testified that the odds of both children dying from SIDS was 73 million to 1, and she was convicted. Besides his calculation being wrong (the British Medical Journal said the actual odds were 2.75 million to 1), the argument was fallacious. Given that the children did die, the question is why. One mathematician calculated that two infants are 9 times as likely to die from SIDS as from murder, which is the relevant statistic. Ms. Clark was released after 3-1/2 years in prison.

    On your Down’s Syndrome study, what is the percentage of false positives on the test? If the fetus does not have Down’s Syndrome, what percentage of the time does the test say he does? I’d be interested in seeing the full calculation if you want to either email it to me (corey at the school URL) or post it here.

  4. Nagath Says:

    I spend some time on probability and statistics course a year ago, and this article was a good incentive to flex that muscle a bit! I would like to add what I thought was the most interesting bit of the course:

    I remember discussing a test for the probability of down syndrome for unborn babies. Down syndrome happens to 1 out of 100 people, and while this test is 90% accurate (which means that if the unborn child has the syndrome, 9 out of 10 tests will be positive), the fact alone that the test is positive doesn’t mean the chance has changed from 1% to 90%! The first probability is still true, and the latter is also. Both are used in a conditional probability to draw the conclusion that, in this case, only 47,6 percent of the babies tested positive might actually have the syndrome. Which is close to a 50% and even then it’s slightly more likely that the baby doesn’t have the syndrome. (I can post the math for this if anyone is interested.)

    So with every test or following event you gain information, which needs to be considered with the previous knowledge, instead of viewing the event or test result as a new situation. Just like in the ‘lets make a deal’ example.

Leave a Reply

Follow these comments with the RSS 2.0 feed.