The volume of streaming data is overwhelming IT systems, and many businesses are struggling to update, organize and index their data so they can query it in real-time. Traditional systems simply cannot cope and some organizations are already leveraging hybrid transactional/analytical processing (HTAP) to support transformative business process innovation.

What is HTAP?

HTAP combines transactions with analytics and an HTAP database supports both workloads in a single database, offering simplicity and speed. This is causing an upheaval in established architectures, technologies and skills enabled by in-memory computing technologies.

Drawbacks of traditional approaches

Architectural and technical complexity

In traditional approaches, data must first be extracted from the operational database and then transformed before it is loaded into the analytical database.

Analytic latency

In a traditional setting, it can take hours, days or even weeks from the moment data is generated by the transaction processing application to when it can be used for analytics. This may be adequate for certain types of analytics and even processes, but not for others.

Synchronization

If analytical and transactional data storage is separated, when business users want to go from a point-in-time aggregate into the details of the source data in the operational database, they often find the source of data “out of sync” because of the analytic latency.

Data duplication

Using traditional architecture requires administering, monitoring and managing multiple copies of the same data and keeping it consistent, which can lead to inaccuracies and timing differences.

Development of HTAP

The idea of running transactions and analytics on the same database has been around for some time but did not materialize due to a number of issues. Transaction processing and analytical systems were therefore based on distinct architectures, which added complexity and introduced delays in data analysis.

Transactional systems enable fast access to data to support business processes, such as e-commerce, banking operations, order entry, and various other daily activities in every industry sector.

Analytical processing systems support efficient analysis of data for reporting, business intelligence etc. that require the fast scanning of large databases to create data summaries. They address the need to monitor and measure performance and identify trends by combining many sources of data from multiple applications.

Technology advances like in-memory computing (IMC) and HTAP architectures now allow applications to analyze “live” data as it is created and updated by the transaction processing functions.

Most HTAP implementations are IMC-enabled. IMC technologies, like in-memory DBMSs and in-memory data grids (IMDGs) support a single, low-latency, access, in-memory data store that can process high transaction volumes.

Adopting HTAP

The emergence of HTAP means organizations need to understand its fundamental tenets, identify the value of advanced real-time analytics, and where and how these enable process innovation.

Organizations will need to overcome the challenges of adopting this new approach and figure out how HTAP can help process innovation. It will entail an understanding of how the architecture can transform processes rather than just provide existing styles of analytics without latency and with more speed.

Organizations don’t necessarily need to take an all-or-nothing approach to HTAP. Using in-memory computing technologies offers them opportunities to adopt and build HTAP architectures incrementally. They can take a trial-and-error approach. In many cases, it is possible to initially implement a core set of capabilities and then add new analytics at a later stage.

There are SaaS providers and packaged application vendors already using HTAP architectures. These are in their early stages and adoption is often limited but customer success for SaaS companies is crucial and marketing efforts are drawing more attention to this approach.

Impact on small and large organizations

The degree of simplification enabled by HTAP will be different depending on the size of an organization. HTAP could make it much simpler to meet reporting and analytical needs based on operational data due to no need for a data warehouse or data mart.

However, HTAP may not have such a beneficial impact on large and complex organizations that need to aggregate data from multiple sources. However, even in these organizations, data marts that support analytical and reporting needs from a single application could be eliminated, and this could simplify some information management.

Benefits of HTAP

• HTAP creates a simple architecture because it replaces two separate types of databases and the ETL process with a single database.
• HTAP eliminates analytic latency and data synchronization issues.
• The HTAP database eliminates the need to create multiple copies of the same data (or reduces it). It makes operations simple and easy because only one system is running.
• It is easier to make a single database secure than when multiple data copies exist on different systems.
• As soon as data comes in for processing, it is available for analytics. There is no need to wait for it to go through OLTP or ETL before being able to use it for analytics.
• The simplicity of HTAP architecture and operations results in large cost savings. Higher performance offers more productivity of existing functions that produce revenue.
• The Internet of Things (IoT) means analysts, apps, and dashboards may need access to the same, updated data at the same time. HTAP makes this concurrent access possible without affecting performance.
• Without HTAP, artificial intelligence and machine learning are impractical. Businesses need to be able to learn from current and historical data.
• HTAP enables more traditional descriptive analytics and advanced analytics, such as forecasting. Predictive analytics allows organizations to determine factors such as whether business targets are likely to be met so they can take corrective action.

A final word

HTAP adoption is likely to grow significantly over the next few years because of its significant business impact. Hybrid data management is a concept that we’re likely to hear much more about in the future because it is more conducive to the data management needs of the future. Those organizations that are open to understanding integration challenges and the hybrid nature of modern data management will be better prepared for what’s to come.