New statistical method discovers hidden correlations in complex data

Nature : Simultaneous dependencies within large data sets can be effectively invisible to statistical analysis. David Reshef of the Broad Institute of MIT and Harvard in Massachusetts, Yakir Reshef at the Weizmann Institute of Science in Rehovot, Israel, and their colleagues have devised a method called the maximal information coefficient (MIC) to find superimposed correlations between variables and measure how tight each relationship is. The MIC is calculated by plotting data on a graph and looking for all the ways of dividing up the graph into blocks or grids that capture the largest possible number of data points. The team applied their method to data concerning global health, gene expression, major-league baseball, and human gut microbiota to identify both known and novel dependencies.