Idea: The Correlation Project
I want to see a website that lets me keep track of something simple for a set period of time, and then compares it to a database to search for correlations. For example, over the course of a week or month or year, or even on an ongoing basis, it could ask, “What did you have for breakfast today?” or “What color shirt did you wear today?” or “How many calls did you get on your cell phone today?” Maybe a daily e-mail would remind you to log in with the answers.
Then it would compare the answers to other things that are already tracked such as the stock market, phases of the moon, sports scores, etc., and spit out some correlations.
It could tell you “On 93% of the days on which you ate eggs for breakfast, the stock market went up.” Or, “When the moon is waxing, you are 88% more likely to wear a green shirt than when it is waning.” Or, “On days when you get more than 7 phone calls, the Yankees win their games.”
Or, if you opt in to share your information with others, it could tell you, “For the past year, you’ve been on the same Green Shirt Schedule as Joe Shmoe of Hackensack, NJ, who has worn a green shirt every day that you did.”
There would need to be a notice reminding people that there is no causation implied by these findings, just correlations. People have a tendency to apply too much meaning to these sorts of things, and think they are evidence of paranormal phenomenon or conspiracy. But I think it’s interesting just to look at these things for the sake of seeing how easy it is to find coincidental correlations retrospectively. If such a project existed, maybe it could show people just how common coincidence really is with an experiment they can participate in themselves.
Comments
I like this idea. I’d certainly participate in something like this.
Posted by: jayne d'Arcy | May 24, 2006 6:58 PM
I may actually look into what it would take to (at least) collect the info…
Posted by: Tim | May 24, 2006 7:53 PM
I’m a programmer who also thinks this is a fascinating idea. I’m up for taking a shot at making this happen. Perhaps some collaboration is called for. If somebody wanted to be the “project manager” I’d happily join the team…if nobody else steps up to the plate, I can organize. One way or another I’m going to take a crack at it - this would just be too cool. (Unless of course it turns out to be prohibitively complicated after I actually start thinking about the underlying requirements…) :-)
Posted by: Matt Winckler | May 24, 2006 11:23 PM
It is not hard to do at all, you just need a basic form to accept the data and then set up a query form that will then query the data which you can hold in an sql database. To set it up would not take long but getting enough people to use it, now thats a different matter.
Posted by: barcrest | May 25, 2006 7:49 AM
I just struggled through a grad level stats class, and found it interesting in a masochistic way. How do you start something like this? I figure you need to collect the info, find the right formulas, plug it all into excel and away you go …worth looking into
Posted by: John | May 25, 2006 8:30 AM
Where’s the money? Ads? Membership? It seems like the people who would find this useful (and would actually make the most out of it) could probably do it themselves at varying levels of sophistication.
I think it would be neat, but that’s a big hurdle, I think.
Posted by: Bill Brown | May 25, 2006 1:20 PM
Yes. Please do it. Do it.
There is nothing more fascinating than comparing random mundane aspects of your life to those that others may have. I constantly do it.
And statistics just rule. Hard.
Posted by: Luci | June 4, 2006 1:00 PM
So, did anybody get around to doing anything about this?
Posted by: jon | August 25, 2006 5:21 AM
Hmm, a day after reading this entry, I came across this:
http://swivel.com/
Posted by: h3 | December 6, 2006 6:01 PM
How weird, I was thinking about the exact same thing…
Posted by: Alski Malawski | January 22, 2007 9:12 AM
ooooh i like this. i’m a statistics phD student, and i’m going to try to think of a way to use it with my classes.
i think Luci (4 comments above) makes and excellent point about statistics. having the similar names must cause people to have the same feelings about blogs and statistics.
Posted by: Lucy | February 28, 2007 4:57 PM