This contest is focused on forecasting Standard & Poor's 500 (S&P500) closing prices from 1st January, 2012 to 31st March, 2012. Its the first of many aimed at building a quality market prediction engine.Prizes include one for community participation and collaboration.

Timelines: The contest will begin on November 28th, 2011 and the final date of submission of entries will be December 30th,…

By Brian Bloch | Investopedia – Fri, Nov 25. 

It is no secret that various people or bodies within the financial services industry will say or write whatever will sell the products, rather than what is actually true and correct. For this reason, buyers may be more inclined to rely on what they think is hard evidence – in the form of statistics and charts. Regrettably, one…

We've experienced a number of small Internet problems that were easily fixed, just before Thanksgiving. But there was one attack that looked very weird, and the motivation behind this failed plot is obscure. Most likely, our publication of new material to better and automatically detect scammers on social networks probably triggered the retaliation. We've experienced this type of retaliation in the past, when…

By  Derrick Harris Nov. 24, 2011, 6:02am PT.

Hadoop gets plenty of attention from investors and the IT press, but it’s very possible we haven’t seen…

(Reuters) - Exchange operator CME Group Inc (NasdaqGS:CME) fined Infinium Capital Management, a U.S. high-frequency trading firm, a total of $850,000 for three separate computer malfunctions that rattled futures markets in 2009 and 2010.

Infinium trading programs malfunctioned on October 7 and again on October 28, 2009, causing uncontrolled selling of e-mini contracts on the Chicago Mercantile Exchange. Then on February 3, 2010, the firm…

In this recent editorial about the Occupy Wall Street movement, Richard Kim profiles a protestor that despite having a master’s degree can’t find a job. This particular protestor quit his job as a school teacher three years ago and took out a $35K student loan to obtain a master’s degree in puppetry from the University of Connecticut. I…

Most analytic people would agree that lottery is a tax on the innumerate. However, for this statement to be true, we need to assume that winning numbers (and any combination of digits from these numbers) are truly random. If not, of course smart people are going to find and devise algorithms to increase their odds of winning, and/or will try to sell their magic "winning numbers" to idiots. …

...

read more here: …

And to feed billions more humans! There has been lots of research and development to improve farming over the last 6,000 years. This includes irrigation, crop rotation, fertilizers, insecticides, genetic engineering, building skycrapers where vegatables are grown on each floor, etc.…

The recent attack on Facebook (with almost a billion users exposed to porn and violence spread by spammers), as well as the unreported / undetected attack on Yahoo Finance - with thousands of fake members posting comments against "Occupy Wall Street" (where 5 days ago thousands of possibly fake posters were posting comments in favor of this movement) raise questions about the business value of hiring data scientists, based on the…

We tested how Google and Bing are vulnerable to sophisticated algorithmic attacks that could potentially erase all paid ads and organic search results on search engines. 

An algorithmic attack is defined by the following…

Cloudera founder Christophe Bisciglia unveiled his new startup Odiago this morning, giving to the business side scoop to TechCrunch and the technical details to Curt Monash. The company is launching a product called Wibidata (“we be data”) specializing in…

Source: Joshua Kitlas.

I have been working on this (mostly) annotated collection of tools and articles that I believe would be of help to both the data dabbler and professional. If you are a data…

I have become interested in the new micro-economic foundations of Macro - Rational Expectations Economics (in the context of irrational macro-policy design, particularly visible in the EuroZone); here are two interesting papers from one seminar at Warwick, link below abstracts ..

FIRST PAPER

A MODEL OF REFERENCE-DEPENDENT PREFERENCES

*BOTOND KO˝SZEGI AND MATTHEW RABIN

ABSTRACT: We develop a model of…

I was working on the post related to the relation between IFS fractals and Analytical Probability density, when an IT manager contacted me asking an informal opinion about a tool for Neural Networks.

There's apparently a shortage of analytic talent to fill these new "yield optimization scientist" and similar positions created by ad agencies. Since I get lots of resume each week from stellar, perfectly qualified candidates who want their resume posted on DataShaping, and few if any request from hiring managers looking to fill these positions, I think that

1. Either this New York Times article is just hyping a non-existent shortage, or maybe, only one hiring manager is…

by Ravi Kalakota

A fantastic article from one of our company's very talented engineers:  http://blog.synerscope.com/2011/11/fraud-detection-using-interactive.html

FRAUD DETECTION USING INTERACTIVE CONSTRUCTION AND ANALYSIS OF DECISION TREES…

By Erika Morphy

E-Commerce Times 

11/09/11 5:00 AM PT

Innovations in the management of big data are getting encouragement to the tune of $100 million in potential investments from a new venture fund established by Accel. "Databases have gotten so big that we need to develop the next generation of technology that can capture, store, analyze and report on datasets that are measured in quadrillions of bytes," noted Richard McCarthy, head…