Behavioral economist Dan Ariely once compared Big Data to teenage sex: “everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.” The common reasons businesses give for using Big Data, whether they’re actually using it or not, include improving customer experiences, increasing process efficiency, and helping to launch new products or business models. But can you really do all this with Big Data?
The biggest obstacle any business faces when trying to realize the full potential of Big Data is how difficult it can be to draw meaningful conclusions from such large stores of information, especially information that’s poorly organized—if it’s organized at all. The new technology could possibly provide businesses with any number of actionable insights, but for the most part you have to be a data scientist to discover any of them.
As Serendipity’s Mare Lucas points out, “The tools are designed for those who speak the language of algorithms and statistical analysis. It’s simply too hard for the everyday user to ‘ask’ the data any questions—from the routine to the insightful.”
What Lucas is highlighting here is the gulf separating people who have a strong grasp of the data from people who have a strong grasp of the business context which would give that data meaning. Self-service Business Intelligence tools like Tableau and Power BI go some way toward bridging this divide, but Big Data will probably never fully take the place of traditional BI. And that’s because of yet another gap—the one between the sorts of questions Big Data is best suited to answering and the sorts of questions traditional BI still works really well for.
As Aptera’s Business Intelligence Practice Leader Aaron Crouch explains, “Big Data is one tool in your tool belt. That flies in the face of a lot of marketing hype. While it’s true that the technologies are really exciting, they exist to solve certain types of problem. The key, as always, is to use the right tool for the job.”
What is Big Data?
To get a sense of what situations call for which approach, we have to understand what separates one from the other. Traditional BI is about putting in place technology and creating processes that help you measure your business’s performance overall, as well as your progress toward established goals. This often means taking data originally generated in your CRM, ERP, or financial systems, storing it in a Data Warehouse, and then using it to create reports that focus on key performance indicators. Many businesses govern this entire process with a Master Data Management strategy to ensure that the reports are both accurate and meaningful.
Traditional BI is about getting answers you already know are important, and because you know they’re important you put mechanisms in place to produce the key metrics. Big Data, on the other hand, is often about finding answers to questions you didn’t even know you had, based on data that may have been originally recorded for some completely different purpose. It usually involves a much greater amount of information which can be culled from multiple sources, and thus requires the extra computing capacity you get through distributed processing and cloud technology.
You could, for instance, pull in and analyze a huge number of tweets or Facebook comments that mention your brand. The goal may be to glean some sort of insight into the aspects of customer experiences it would be most productive for you to focus on enhancing, as well as some indication of what direction to take with future marketing efforts.
The 3 Vs
In general, there are three areas where Big Data excels. The first two are volume and variety. This is when you’re dealing with massive amounts of data, from all kinds of sources, in all kinds of forms. Traditional BI relies on what’s called structured data—it’s usually arranged in columns and rows, with each field corresponding to some known source. Big Data meanwhile is often completely unstructured, so you usually need to have a developer create some kind of mechanism for interpreting the information.
The third V is for velocity, since the extra capacity you get with distributed processing can also dramatically accelerate computing. A factory could, for instance, use this kind of high-capacity accelerated processing to calibrate a programmable logic controller for an automated tooling function. This would be immensely beneficial in instances where the machine relies on real-time feedback on the condition of raw materials throughout the production process. This scenario, while it may not fall under the BI label, illustrates how the new technologies are transforming information-gathering.
The Drawbacks of Big Data
While in some instances, you know exactly what you’re looking for in all that copious data, it’s often the case that your analysis will be somewhat open-ended. In other words, you may be mining the data for potentially actionable insights, with little idea at the outset what those insights will even look like. If you don’t even know the question you may—or may not—find the answer to when you begin a project, it’s pretty much impossible to determine what your ROI will be.
The biggest challenges businesses face as they try to incorporate Big Data into their BI strategies, however, are the complexity of the underlying processes and the divide between data experts and business experts. In traditional BI scenarios, it’s relatively straightforward to set up reports and dashboards that allow business users to get the information they need to make important decisions. But Big Data analysis is more developer-centric, since you usually have to create the tools required to make sense of the massive amounts of unstructured information. And you may end up having to create another of these tools each time you pose a new question.
The key takeaway here is that, as exciting as the new technologies are, Big Data isn’t the be-all-end-all of Business Intelligence. For most decision-making processes, traditional BI is still the most reliable and cost-effective approach. But it can also be immensely beneficial to have processes in place to explore high volumes of data from a variety of sources at lightning velocity. And once the data scientists have tamed the Big Data beast, giving it structure and helping you derive meaning from it, there’s nothing stopping you from moving it into a traditional data warehouse. What this means is that Big Data and traditional BI are by no means mutually exclusive—a strong BI vision will incorporate both approaches.
Other popular posts like this: