Analytics has come a long way in last two decades. It started as back of the envelope numbers scribbled by top level cigar smoking business executives from experience and data they watched, analyzed and inferred from. Then came the ubiquitous RDBMS based web applications that made data entry job of end users rather than backend employees. This generated enormous amounts of data over the years. Hence the birth of analytics technologies and frameworks.
The Analytics Technology Landscape
Since the bulk of data resided in RDBMSes, it became imperative that the products and tools that helped organizations leverage this data to predict upcoming opportunities and customers needs, were very compatible and coherent with these technologies. This spawned a whole analytics software industry with products like Vertica, Netezza and Teradata for large scale storage. This also gave birth to a few other product families.
One such conspicuous family is the ETL tools like Ascential DataStage, Informatica, Ab Initio and Talend. Add to this some other products that help leverage these by going above and beyond the basic analytics features and come standard with bulk storage. This category includes Cognos, Microstrategy, SAP’s Analytics products, Oracle OBIEE, SAS.
As these products provide features on top of other products and add value to existing IT investments, they tend to command a high price tag. Professional services for these also get priced at premium rates. Therefore, the inherent cost-prohibitive nature of these products made them only available to the top 1% of Fortune 500 companies. The lower rung management was left with the same old archaic ways of predicting marketing, planning and forecasting, figuring out operational efficiencies and all such services that analytics tools help with.
The Big Data Impact On The Analytics Ecosystem
Then came a new generation of tools and technologies with Google and the likes leading the pack. Google’s Big Table started off with the need to store massive amounts of data i.e. crawling whole of the web’s content, indexing and tagging it. However the real value was realized when Google performed analytics on this data to figure out what a user was looking for and displayed relevant advertisements.
Today Google uses Big Table, Facebook uses Cassandra and LinkedIn uses Kafka with the one common requirement of crunching massive amounts of data and using it to suit their business model and needs. These frameworks are open-sourced and available freely. Tens and hundreds of developer communities and organizations are constantly contributing to these frameworks, their tooling and missing functionalities. One commonality that all these share is that they are device independent and run well on any commodity hardware. They are also inherently cluster-able therefore scaling up implies buying more commodity hardware and stacking up the data centers as market, revenue and user base grow bigger and bigger.
The Other Game Changing Advancements
Coupled with some other advancements, all these come together as a game changing proposition. Analytics is not the domain of the top 1% of management anymore. It is as commoditized as the hardware that these frameworks run on. Add to this, the internet bandwidth that is becoming faster and cheaper with every passing day and Gigibit speeds becoming a standard.
Now mid-sized companies and middle management of big companies can afford to leverage the latest analytics technologies that were, till recently, thought of as elitist tools. Managers on shop floors or in the field can get this information on the go on their tablets and smartphones while driving on the streets. The affordability goes a few more notches up if you factor in the cloud compatibility of these technologies. This brings down cost of ownership and more importantly makes it measurable effectively. Now a mid-sized company can perform a round of analytics using cloud based infrastructure and calculate the dollars spent versus dollars made or saved and see for themselves what worth it is. Moreover traditional analytics tools tend to be very offline and batch oriented and the vendors of these technologies successfully sold this limitation of high lead times to get the results to their clients.
Lot of new generation frameworks are challenging and successfully countering the batch and offline assumption and providing value that the previous generation of million dollar tools failed to provide. Now businesses have to take a call on whether to continue on the same route or take a detour using the new toll-free highway to analytics.
What do you feel is the right path for businesses? Share your thoughts !
Take charge of your enterprise app strategy ! Download this eBook with the 10 best practices and get started