Anjan Srinivas, senior director of Product Management at Enterprise Cloud Platform company Nutanix, explains why algorithms are the next big thing for data centre infrastructure and cloud computing. While there are obvious challenges ahead for companies looking to embrace what Gartner calls the "algorithm economy", the rewards are significant for those who are ready to ride the next wave of disruption.
Algorithms are everywhere. These computing processes, or sets of rules to be followed in calculations or other problem-solving operations, are here to help us. They can assist us, for example, in finding the optimal route while we are on the road, or to pick a movie or TV series, based on our past viewing preferences.
Consumers are now spoiled with helpful recommendations, proactively pushed to them on popular websites or mobile applications such as Amazon Retail and Netflix. They no longer need to spend time looking for new things to try, as the options are just there presenting themselves in an easily consumable format. Without doubt, algorithms are adding efficacy to our daily lives, with computers performing a set of steps in order to find the most effective outcome.
By now, most companies are familiar with the term "Big Data", which is often used to describe extremely large sets of data being generated for analysis to reveal patterns or trends. These patterns are designed to spot human behaviours so businesses can turn them into meaningful and actionable intelligence for better business planning.
However, with these data sets shifting towards zetta byte territory, Big Data is likely to lose its relevance in its current form. This large growth in unstructured data has reached a stage where it is harder to digest and consume by any application. The unstructured data has led to numerous complexities in understanding, at a level which is beyond one's imagination.
The Big Data revolution undoubtedly comes with various issues including volume and complexity. But beyond volume, another key issue is the lack of ability to understand and make optimal use of this massive resource. Just having a lot of data is not necessarily sufficient to improve the quality of decisions that enterprises need to make in a timely manner. Combining these issues once again serves to disrupt the IT landscape and our approach to technologies.
Size matters, but it isn't everything
On close observation you can find that the real driving force behind the Big Data revolution is the enhancement of our ability to combine advanced statistical models and new computing technologies. New concepts like the Internet of Things (IoT) and highly scalable databases in the cloud have resulted in a vast amount of unstructured data, contributing to the adoption of Big Data technologies. This overall analytical capability helps us to derive benefits from the data resources and also to understand the content a lot better.
Take for example the way that new generation apps, such as intelligent personal assistant Apple Siri or Amazon Echo, have revolutionised human interaction with computers. These algorithms have provided a quantum leap in machine learning, which enables us to build constructive insights with our big data resources. This is what is driving the interest in advanced algorithm development.
Google's search engine is driven by algorithms. Algorithms power Netflix recommendations, voice driven assistants, driverless cars, next day parcel deliveries, high speed trading and a constantly growing number of services and technologies that are already taken for granted by every one of us. These sets of algorithms have not only made their product experience simpler and seamless, but have now also opened up new revenue streams and competitive differentiation.
In the examples above, it is not only the data that is of significance, but also the intelligence that the algorithms are able to provide. This intelligence enables the machines to reflect the context of the data and to suggest ways to make better use of it. As companies look at ways to monetise algorithms, their position as the next big thing in big data science is further validated. These will then be developed, traded and exploited just like mobile applications in the years to come, and will ultimately contribute to the developers' intellectual property.
Balancing the business
While all this sounds logical and makes the adoption of algorithms a no brainer, there are some factors that may slow down their adoption. Algorithms, as discussed earlier, depend on machine learning and then being able to process seamlessly to reveal meaningful predictions.
The algorithm will face the inevitable flipside of greater pressure on data storage and processing resources. CIOs and IT directors will have to look to enhance their computing capabilities with both on-premise cloud experience and public cloud services, in order to meet the ever-growing demand of their customers. Additionally, there is the possibility that financial institutions or government departments would prefer to keep their algorithms along with other business-critical data behind the corporate firewall for reasons of security. But algorithms will likely have their biggest impact by giving enterprises the power to dip in and out of the public cloud on demand and on their own terms.
The rise of algorithms will undoubtedly compound data storage and processing issues, but it will also play an essential role in solving the same issues by leveraging insights of big data in order to manage the originating source.
The same is being witnessed, in the form of machine learning tools automatically balancing storage demands and computing workloads across a mix of public and private platforms, allowing enterprises to make informed decisions on resource consumption. Consequently, the machines need to balance the workload with respect to subtle changes in demand profiles.
While analyst firm Gartner predicts the "algorithm economy" as the next big thing in big data, algorithmic business provides the speed and scale to accelerate digital business to deliver even greater impact. One can expect the familiarity of algorithms and machine learning as essential to the ongoing digital transformation process. The arrival of algorithms is going to be much faster than you could have ever imagined.