Ravi Shankar, Senior VP and Chief Marketing Officer at Denodo discusses the significance of automating data management with the help of Artificial Intelligence
In its May 6, 2017 edition, The Economist published an article entitled “The world’s most valuable resource is no longer oil, but data.” Today, many recognize that this was a correct assessment, especially if you consider the role that data now has in creating products, running business operations, and making decisions. Unfortunately, data integration and data management technologies, which are used for harnessing data and making use of it, still require an inordinate amount of manual labor. As always, the panacea is automation, and fortunately, artificial intelligence (AI) promises to automate many of the manual functions of data integration and data management. Leveraging the power of AI, organizations gain the maximum value from their most precious resource, their data.
Organizations have actually been leveraging data integration and data management technologies since the early 90s.At that time, the prevailing strategy was to copy data from multiple source systems and establish the copied data in a single repository, such as a data warehouse, via extract, transform, and load (ETL) processes. Such processes worked extremely well for replicating large volumes of structured data. However, any change in the process required a change in one or more of the governing scripts, and this meant that every change required rescripting, retesting, and redeployment, which was a highly manual and time-consuming process. Over the years, these traditional, ETL-based data integration vendors have made considerable improvements to their operations, but they still rely on replication scripts that must be tested, and with any change of the systems, such as data formats or server location, they will still break and inhibit automation.
Unfortunately, organizations still rely on these labor-intensive data integration and data management technologies to gain a unified view of their data. Large international enterprises, which generate billions of dollars a year in revenue and employ thousands, have extremely complex businesses, and technology landscapes with data that is spread across hundreds of disparate systems such as on-premises data centers and cloud repositories. Their data spans multiple formats—structured and unstructured—and is comprised of data that is both at rest and in motion. Given the volume, variety, and velocity of data that organizations must harness and manage on daily basis, there is no viable way to collect all of this data into a single central repository.
Instead of collecting the data, organizations need a way to connect to the data, wherever it may reside. Data virtualization (DV) is a modern data integration and data management technology that provides this capability. In addition, it enables seamless automation through AI.
Capitalizing on the Future: The Promise of Real-time Virtual Views Fulfilled
Rather than physically copying data and moving it to a new location, data virtualization enables real-time, virtual views of the data in its existing location. The strategy is elegant and powerful in its simplicity: Rather than investing the time, effort, and hardware to collect data and place it in a monolithic repository, DV enables data consumers and applications to connect to all data, remotely and in real time, establishing a data fabric across the enterprise.
Advanced data virtualization platforms will leverage AI and machine learning (ML) to automate such functions as inferring data changes in the sources and providing sophisticated recommendations to users to boost the efficiency of their operations. These platforms will also enable data scientists to rapidly build multiple-dimension data models to run statistics using languages like R, and combine queries, results, and notes into a data science notebook from which to deliver the results in a self-contained module to business users.
Organizations will be able to leverage data virtualization to easily adapt to technological change, as it can support virtually any data source, such as streaming, transactional, or cloud-based sources. They will also be able to leverage DV to navigate changing business conditions, such as those created by the pandemic, as access to real-time data will enable executives to make quick decisions to ensure continuous operations. Finally, business users will have the ability to quickly react to the changing competitive landscape, and they will be able to leverage data virtualization to gain self-service access to data, reports, and applications without having to ask IT for assistance.
For more such updates and perspectives around Digital Innovation, IoT, Data Infrastructure, AI & Cybsercurity, go to AI-Techpark.com.
Currently, we can ask Alexa about the weather, but in future, we will be able to ask, “What was our most profitable product last year?” and get a ready answer. But to enable this, AI, ML, voice-to-text conversion, natural language processing, SQL-ization of the textual query, and text-to-voice conversion, will all need to come together. Though this will take a few years, many companies are already laying the foundation.
We all know that data is our most valuable asset, but organizations still spend too much time and labor to gain this value.However, AI-enabled data virtualization will streamline, accelerate, and simplify data access, making it accessible in real time to business users in a self-service capacity, enabling organizations to gain progressively more value from their data.
