Replies to This Discussion

Permalink Reply by timothy vogel on February 28, 2010 at 4:33am

Dear Patricia,

While you're there, can you ask the LinkedIn brain-trust co-sponsoring this deep-dive into "data mining" to stop connecting us all with Barack Obama already? As in "people who looked at your profile recently looked at Barack Obama's profile, as well."

DUH.

This is Social.Graph 101. And to think I was told to my face by an analytics honcho at LinkedIn that they didn't need any help with their "social.graph algorithms." I agree. They don't need help with their algorithmic approach. Nor do they need to be sponsoring classes in it. What they really need is to take a real statistics class or two. Data Mining, whatever it has come to mean, starts and ends with inference.

I have great respect for the ACM and have participated in their meetings and panels for years. But perhaps the ASA ought to help the ACM refocus that lesson toward "the new data miners", who seem to think computing is more important than understanding.

My rates are not exactly cheap, DJ, but I daresay I can help you fix the "what to do when an actor's degree in an n-node-network = n" bugaboo in the first five minutes of my lecture!

Sheesh.

TV

Permalink Reply by Paul Wilson on March 2, 2010 at 2:44pm

"Data Mining, whatever it has come to mean, starts and ends with inference."

Not quite clear what you're trying to say here.

Care to elaborate?

Permalink Reply by timothy vogel on March 2, 2010 at 3:43pm

Why does anyone analyze anything? Attaching meaning to low-level patterns in nonsensical relationships like "people interested in Barack Obama are also interested in me" is not analysis, and is actually little more than a superficial scrape of the most obvious data-collection-traps online.

nonsense. useless. it will never generate even spurious revenue or move our social connectivity any closer.

Amazon, LinkedIn, eBay, Google, and all the ad-networks are among the largest purveyors of online data mining. do they ever manage ot offer you something you want in a timely fashion?

No. Never. They wallow in brokered-irrelevance since that is what pays the bills. We should rename what these ostensible experts, all sponsors of "teaching US how to mine data", "data dreaming".

That is far more fitting moniker for what they are doing.

Data Mining 80% pre-processing/data hygiene/massage and 20% inference seeking. Fail to do the first 80, for whatever reason, and the latter 20 becomes an exercise in futility

TV
http://www.heur-e-ka.com
http://commonsensical.wordpress.com/

Permalink Reply by Paul Wilson on March 3, 2010 at 8:19am

"data dreaming"... ha, ha , ha...never heard of it before, but it sure made me laugh to hear the term :)

I agree with this statement "Data Mining 80% pre-processing/data hygiene/massage and 20% inference seeking. Fail to do the first 80, for whatever reason, and the latter 20 becomes an exercise in futility"

Not sure about LinkedIn, eBay and Google, but that "XYZ % of customers who bought this also buy that" feature on Amazon doesn't seem all that bad. From what I can tell it's not some fancy-schmancy cutting edge algorithm, but probably a simple market basket analysis....not always relevant of course, but it made me look a few times.

Permalink Reply by Steffen Springer on March 4, 2010 at 2:51am

I agree with Paul.

Data Mining is not all about finding a cure for cancer automatically. Sometimes it is just to help finding people what they want. I cannot count the times I have looked for books in special fields, when Amazon first helped me to find one book I am interested in (search engine, ok computing :)) and then showed me similar books which I otherwise had to find manually piece by piece (col. filtering).

Anyway, timothy, in one line you say "it will never generate spurious revenue", in the other line "that is what pays the bills". Sounds to me like the ranting about the brainless tv-program, which is after all watched by masses. Sad, but true.

my cents

steffen

Permalink Reply by Patricia Hoffman, Ph.D. on March 5, 2010 at 11:25am

Prof Robert Tibshirani has a humurous but true comparison:

Machine learning ......................... vs .................. Statistics
--------------------------------------------------
network, graphs ............................. vs .................. model
weights ........................................... vs .................. parameters
learning ........................................... vs .................. fitting
generalization .................................. vs .................. test set performance
supervised learning .......................... vs .................. regression/classication
unsupervised learning ....................... vs .................. density estimation, clustering
large grant = $1,000,000....................... vs .................. large grant= $50,000
nice place to have a meeting:.................. vs ..................nice place for meeting:
Snowbird, Utah, French Alps ................... vs .................. Las Vegas in August

Attachments:

• glossary.pdf, 9 KB