One of the hottest topics in business analytics today is “big data,” defined by Wikipedia as “a term applied to data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time.”

How big is “big data?”

Last year, consumers and businesses around the world are estimated to have stored more than 13 exabytes of information on PCs, laptops and other devices — the equivalent of more than 52,000 times the information housed in the Library of Congress. An exabyte is 1 followed by 18 zeros, or a billion gigabytes.  And the amount of data stored in such “technological memories” is growing 25 percent a year, said Martin Hilbert, a researcher at the University of Southern California.  These were some of the estimates shared at the The Economist Big Data Conference last June in Santa Clara, CA. (for complete story see http://pittsburghlive.com/x/pittsburghtrib/business/s_745039.html).

The experts at the conference had strong opinions on the value of learning to exploit big data.  “Businesses that analyze data about customers increasingly have an edge over their competitors.  Retail companies that are savvy about sifting through their data can improve operating margins by 60 percent,” said James Manyika, director of the McKinsey Global Institute.

Sounds great, let’s all build our big data systems and get back to making money.

Or maybe it’s not that simple.  Jeff Jonas, is the intelligent systems engineer who developed the casino fraud detection system that caught the infamous “MIT Team” portrayed in the movie “Bringing Down the House.” He observes that as the information overload is increasing, the ability of organizations to keep up with it has actually flattened.  Corporations are becoming “dumber” as the percentage of data that is used for decision-making gets smaller (for an awesome video on this subject see http://wp.me/p10LZV-Gng#ooid=lnajByMTocIKwCbdUFmzL2LRbU-P9GmV).

Still, increasing operating margins by 60% is a pretty attractive incentive.  It’s a safe bet that eventually we will find a way to develop computer systems that will guide us to make fast decisions with reliable outcomes.  What can chain store real estate planners be doing to prepare for this new paradigm?

The first step is to get used to working with computer-based information systems.  One of the hottest companies in the Big Data space is Opera Solutions, who just received an $84 million round of financing to fund their astounding growth.  The company is grounded in science, with 160 of their 600 employees holding advanced degrees in quantitative analysis (half are PhD).  However, their philosophy is that it’s the combination of Big Data, intelligent computer systems, and people that produces the best decisions.  As Opera’s CEO Arnab Gupta explains, “We often create false dichotomies for ourselves, but the truth is it’s never been man vs. machine; it has always been man plus machine. Increasingly, I think it’s an article of faith that the machine beats the human in most large-scale problems, even chess. But though the predictive power of machines may be better on a large-scale basis, if the human mind is trained to use it powerfully, the possibilities are limitless”(see http://radar.oreilly.com/2011/08/data-human-machine-analysis.html).

Another important step is to make sure that all of our important data sources are accurate and in one place.  I am continually amazed at the way real estate and customer data are managed in chain store companies, even the ones with large IT departments.  The excel spreadsheet is still the leading “database.” In fact, it’s not ONE spreadsheet, but MANY spreadsheets with data of different formats and vintages.  Sometimes there’s a great enterprise database but the real estate department can’t get access to the data on a timely basis in a format that can be used for mapping or analytics.  Third party data such as competitor locations or important retail traffic generators are often unreliable and there may not be a process in place for people in the field to correct the errors.

Finally, we need to begin thinking about how to use the data we have in CONTEXT.  For now, we need to develop associations between the data using categories and “tags” such as “urban” or “freestanding.”  In the future we will have smart databases that can create context “on the fly” based on the data, such as “urban store where competitor dominates and brand loyalty is growing.”  Until the “Big Data Machine” arrives we can at least make sure that we have all the data we need, stored and maintained in a single database, and accessible for all decision support tools such as maps, reports, and analytics.

( Cartoon courtesy of Thierry Gregorius)