Subscribe to DSC Newsletter

Great statistical analysis: forecasting meteorite hits

This example can be used for training purposes, as part of the curriculum to become a data scientist. Here I describe the different steps involved in a typical, real analysis: from understanding the problem (how to forecast meteorite hits) to providing an answer. The Excel spreadsheet with all the details can be downloaded at the bottom of this article.

Conclusions are based on manual estimates - which make sense in this context. I also describe how much a consultant would charge for this analysis, disproving the fact that data scientists are expensive. 

The conclusions of my statistical analysis are different from what you've read in the press recently: the Russian meteor is an event said to occur once every 40 years, according to journalists. Read more to find out what my forecast is.

A statistical analysis in 8 steps

1. Define scope of analysis.

This is a small project to be completed in 10 hours of work or less, billed at $100/hour. Provide risk of meteorite hit, per year per meteorite size.

2. Identify data and caveats

Data set used: http://osm2.cartodb.com/tables/2320/public#/map. You can find and download the dataset here

Data about identified meteors - those that have not hit Earth yet but occasionally fly by - is not included in this analysis. Identifying and forecasting the number of such meteors would be the scope of another, very similar project.

3. Data cleaning

The data seems comprehensive, but it is messy. The database seems to take years to get updated (much fewer meteorites in 2000-2010 than one decade earlier). Some years have very little data. The number of meteorites started to explode around 1974, probably the year when collecting data started. Five fields are of special interest: meteorite type, size, year, and whether the date corresponds to" when it fell" or "when it was found". Another field not considered here is location. We discarded data prior to 1900.

4. Exploratory analysis

Some strong patterns emerge, despite

  • Messy data,
  • Recent years with missing observations or data glitches,
  • Massive meteorites that are both extremely rare and extremely big - creating significant volatility on number of tons per year
  • The fact that most meteorites are found (rather then being seen falling) with unknown time lags between "discovery" and "fall". And "found" sometimes means a fragment rather than the whole rock.

These patterns are:

  • More smaller meteorites are now detected thanks to (1) growing surface of inhabited land, and (2) better instruments
  • The frequency of hits decreases exponentially with size (measured in grams in the dataset)
  • Grouping by size using a logarithmic scale, and grouping by decade, makes sense

5. The actual analysis

See attached spreadsheet below. It helps to include the "found" category in the analysis (despite its lower accuracy) otherwise the data set is too small. The data is summarized (grouped by decade and size) in the Analysis tab in the spreadsheet below, using basic Excel function such as Concatenate, Countif, Averageif, log, int.

We have 9 group sizes with at least one observation: from size 0 for the smallest meteorites, to size 8 representing a 9 to 66 tons meteorite - like the one that exploded over Russia. According to the data set, no size 8 was found in the last 40 years, and maybe this is why people claims that the 2013 Russian bang occurs every 40 years. But we had 5 occurrences between 1900 and 1950. Not sure how reliable these old numbers are. Here, the size is computed as INT(0.5 * log m) where m is the mass of the meteorite, in grams.

We are going to look at size ratios, defined as the number of meteorites of size k divided by the number of meteorites of size k+1 (for a given decade), to build a predictive model. We are going to analyze how variations occur in these ratios, as well as ratio variations across decades for a fixed size k.

A quick regression where Y = ratio (1950-2010 aggregated data) and X = size, shows a perfect linear fit (R^2 > 0.98). I have excluded size 0 (it has its own problems), as well as sizes 8 and 7 which are very rare (only one occurrence for size 8 during the time period in question). Total number of meteorites across all sizes is 31,690.


Meteorites: ratios (Y axis) vs. size (Y axis)

I tried separate regressions (size vs. ratios) for each decade, to see if a pattern emerge. A pattern does emerge, thought it is more easy to detect with the brain than with a computer: the slope of the regression decreases over time (very very roughly, and on average), suggesting that more small meteorites are found recently, while the number of large meteorites does not increase much over time (for size 5+).

6. Model Selection

Two decades show relatively good pattern stability, and recency: 2000-2010, 1990-2000. Likewise, sizes 5 and 4 show pattern stability (growth over time for size 4, rather flat for size 5). I used these two time periods and sizes to build initial forecasts and estimating slopes, eventually providing two predictions for the chance of a hit (lower and upper bound) for any size. The span (or confidence interval!) grows exponentially with size, as one would expect.

The ratios in white font / red background in the spreadsheet represent manual estimates (sometimes overwriting actual data) based on interpolating the most stable patterns. No cross-validation was performed here (more on this in my next paper). Much of my validation consisted in identifying islands of stability in the ratio summary table in cells Y1:AG8.

7. Forecasts

A formula to compute yearly occurrences, as a function of actual weight, can easily be derived:

Yearly_Occurrences(weight) = 1/(A + B* log(weight))

Exercise: estimate A and B based on the above table. 

Based on my table, it makes the "every 40 year" claim for the 2013 Russian bang plausible. The Russian meteorite might have exploded louder than expected because of the angle, composition or velocity. Or maybe the weight of the meteorite is based on the strength of its explosion, rather than the other way around.

There were no meteorites of sizes 9, 10, 11, 12 in the data set, and very few of size 8. So the associated risks (for these sizes) are based on pure interpolation. Yet they constitute the most interesting and valuable part of this study.

8. Follow up

A more detailed analysis would involved predictions broken down by meteor type (iron, water), angle, velocity. Also, the impact of population growth could be assessed in this risk analysis. In this small analysis, it is indeed implicitly factored in. Also, we could do some more conventional cross-validation, maybe making prediction for 2013 and see how close we are, for meteorites of size 0 to 4.

Discussing this with the client can provide additional work for the consultant, and more value for the client.

General Comments

This is small analysis and there is not real metric to measure model performance or ROI on this project. Many projects do, just not this one. It is also unusual for a statistician or data scientist to make intelligent guesses and hand-made predictions. It is common place in engineering though.

I will provide examples of more complicated analyses later, with multivariate logistic regression done with Excel, interactive Excel spreadsheets used for semi-visual model selection and parameter fine-tuning. And of course, stuff that does not use Excel because of its limited capabilities.

Click here to download the spreadsheet meteors4.xlsx, with data, analysis and forecasts.

Related articles:

Views: 7526

Comment

You need to be a member of AnalyticBridge to add comments!

Join AnalyticBridge

Comment by Vincent Granville on May 24, 2013 at 12:12pm

Comment from one of our readers:

It would be very interesting to perform an equivalent analysis for this other meteorites database, although of Mars: 

http://craters.sjrdesign.net/

Comment by Vincent Granville on March 2, 2013 at 12:36am

Interesting comment by J.D., one of our (anonymous) readers:

The following link presents a decent overview of meteor/meteorite knowledge.

http://www.tulane.edu/~sanelson/geol204/impacts.htm

The above link also presents a linear [log-log] relationship between size and rate which is congruent with your analysis.

I would remind your readership that these numbers that you are using only include those objects that:

  1. Have fallen over or on land;
  2. Were found; and/or
  3. Were observed presumably over or on land by humans [not automated systems].

The first and third items above may be substantially incorrect as there now are systems that collect such observations as you noted in your text.

The actual flux of meteoric objects must be greater - and significantly so - due to the size of the oceans and the relatively sparse population density over the land surface of the globe. Additionally, the number of observers decreases during the night - the best time to see these objects - and that occurs between midnight and say 5AM, although this may now be a secondary effect conditioned by item 3 above. Prior to the 1970's that may have been significant.

Large impactors are obviously of greatest concern. The K-T event [now K-Pg] is presented here:

http://en.wikipedia.org/wiki/Cretaceous%E2%80%93Paleogene_boundary

I will be editing this wiki page as the Alvarez' chronological references are categorically wrong. I read the original Nature article by Luis and Walter Alvarez and Fred Hoyle no later than 1978.

Again, thanks for your data analytics expose' on meteor flux. The scientific community has realized that the geologic record is the true "book of life" albeit heavily edited by the processes of continental drift - an explosively controversial idea finally confirmed in the 1970's decade - and erosion. The human race is slowly realizing how contingent our existence actually is. Without histrionics, we're in a race with time, and we all know how that ends.

Follow Us

On Data Science Central

On DataViz

On Hadoop

© 2016   AnalyticBridge.com is a subsidiary and dedicated channel of Data Science Central LLC   Powered by

Badges  |  Report an Issue  |  Terms of Service