Fun With Statistics – Predictably Private Edition

By Kevin Meyer

In this edition of Fun With Statistics we'll tackle the brouhaha over the past couple days regarding the discovery that the iPhone as well as phones running Google's Android software are tracking their users' every move.  This has been entertaining to some, disturbing to others. 

I'm personally not all that worked up as location-based services create a lot of benefits – like being able to find the best Italian restaurant close to me in San Francisco last night.  Besides, I do encrypt the contents of my iPhone just as a standard precaution, and unless you live under a rock you realize that your identity and location are logged indirectly a hundred other ways already – when you use your ATM or credit card, log onto any computer network or wi-fi, fly, etc.

Running the iPhoneTracker app that taps into the no-longer-hidden file with the history of where my iPhone has been over the last 10 or so months was interesting.  Here's the large global graphic:

Several countries and of course my quarterly sojourns to the Big Island.  The app lets you zoom in to a square mile resolution, scroll by date, and the works.  Pretty interesting to watch – sort of a diary of where I've been.  Whoopee.

Today's Wall Street Journal describes the real potential of this data even further.

For almost two years, Alex Pentland at the Massachusetts Institute of Technology has tracked 60 families living in campus quarters via sensors and software on their smartphones—recording their movements, relationships, moods, health, calling habits and spending. In this wealth of intimate detail, he is finding patterns of human behavior that could reveal how millions of people interact at home, work and play.

Through these and other cellphone research projects, scientists are able to pinpoint "influencers," the people most likely to make others change their minds. The data can predict with uncanny accuracy where people are likely to be at any given time in the future.

Uncanny accuracy is right.

Advances in statistics, psychology and the science of social networks are giving researchers the tools to find patterns of human dynamics too subtle to detect by other means. At Northeastern University in Boston, network physicists discovered just how predictable people could be by studying the travel routines of 100,000 European mobile-phone users.

After analyzing more than 16 million records of call date, time and position, the researchers determined that, taken together, people's movements appeared to follow a mathematical pattern. The scientists said that, with enough information about past movements, they could forecast someone's future whereabouts with 93.6% accuracy.  The pattern held true whether people stayed close to home or traveled widely, and wasn't affected by the phone user's age or gender.

93% – incredible.  Imagine how further refined that could be with the combined (presumably anonymous) records of the 75% of the world's population that now carries cell phones.  But where you will be is just the tip of the iceberg.

The data can reveal subtle symptoms of mental illness, foretell movements in the Dow Jones Industrial Average, and chart the spread of political ideas as they move through a community much like a contagious virus, research shows.

Dr. Bollen and his colleagues, for example, found that the millions of Twitter messages sent via mobile phones and computers every day captured swings in national mood that presaged changes in the Dow Jones index up to six days in advance with 87.6% accuracy.

And back at MIT, scientists who tracked student cellphones during the latest presidential election were able to deduce that two people were talking about politics, even though the researchers didn't know the content of the conversation. By analyzing changes in movement and communication patterns, researchers could also detect flu symptoms before the students themselves realized they were getting sick.

So now are you worried, or intrigued with the potential of crowd statistics?  As a glass half full kind of guy who still remembers the folks scared about how the new networked world was going to destroy privacy… back in the early 1990s, I'm intrigued.  And as that tune oft attributed to Ozzy but really by the Swollen Members goes, "… paranoia will destroy ya…"