TestProNews Q1 - 2007
Integrating your Enterprise Test Database with Other Enterprise Systems
Enterprise test data management (TDM) systems provide users with a centralized and unified method for collecting, organizing, and accessing data. Once the data is stored in a particular database, reporting tools and custom applications such as those provided by Arendar® enable you to mine the data to extract valuable information.
The goal of an enterprise system is to enable information sharing with multiple people and multiple departments across the enterprise. One of the biggest challenges of enterprise systems is not technical in nature, but rather has to do with achieving consensus on what tools to use. Different groups have different needs and sometimes they already have a lot of time and money invested in legacy systems that make it difficult to introduce a new approach to test data management. Many times a group will adopt a solution with multiple users sharing data via a database and a test data management system for their particular department, but the benefits they achieve are not leveraged in other departments across the enterprise. As a result, managers are unable to take advantage of test information in another department’s TDM system because the various TDM systems are unable to communicate with each other. Figure 1 shows this scenario.
|
Figure 1: Disparate departmental systems hinder data sharing. |
A heterogeneous mixture of systems, (e.g., multi-vendor hardware, software and other technology), like the one shown in figure 1 are very common in enterprises, small, medium and large. Fortunately, database technology provides a mechanism called replication that allows the sharing of data across multiple heterogeneous systems. Replication is simply a way of keeping multiple databases synchronized at all times. It consists of a set of tools and methodologies that enable databases from different vendors to talk to each other. There are two main types of replication: server-to-server and server-to-client replication. This article focuses on server-to-server replication.
Server-to-server replication can be defined as the process of consolidating and integrating data from multiple servers into one. Server-to-server replication is useful in the following scenarios:
|
Data from multiple sites or multiple departments within one site needs to be consolidated for data warehousing and reporting
Data from heterogeneous systems need to be made accessible via a single interface
Increased performance, scalability, and reliability |
One of the goals for adopting an enterprise test data management system is to keep data centralized and easy to access. The previous diagram shows that this might be true for a particular group or site, but is often not true for the whole organization. An alternative solution might be to use replication to create a single virtual database that serves as a central data repository for multiple databases. Figure 2 shows this concept.
|
Figure 2: Replication enables efficient data aggregation and sharing. |
By using replication, we can create a system that is capable of retrieving data from various databases and presents it as a unified and coherent set of data via a single interface. The replication model used by most commercial enterprise-class databases is based on a “publish-and-subscribe” approach. Databases that make their data available to others are called publishers. They make available several “publications” or groups of data via a distributor. Subscribers then connect to these publications via the distributor to retrieve the data.
There are several approaches to replicating data between databases. Many factors including data quantity, frequency of data updates, and physical replication environment, among others, come into play when selecting a replication approach. Most databases support the following replication approaches: snapshot replication, merge replication, and transactional replication. The best of all is that some of these approaches work smoothly between databases from different vendors. Let’s take a look at each individual replication approach.
|
Figure 3: Snapshot replication approach |
Snapshot Replication
Snapshot replication, as the name implies, consists of taking a snapshot of data in the publisher’s database and copying it into the subscriber’s database. During this operation, all objects on the publisher’s database are copied into the subscriber database. This form of replication is often the first step of all other methods of replication because it provides the subscriber with all the necessary tables and fields (database schema) so that future replications can occur. This method of replication is useful when when the publisher’s data is not constantly changing or when the subscriber doesn’t need to be synchronized with the publisher at all times to get the latest data. Both SQL Server and Oracle natively support this replication approach. The diagram shown in figure 3 summarizes the snapshot replication approach.
|
Figure 4: Merge replication approach |
Merge Replication
Merge replication consists of aggregating data from multiple databases into a single database. This method of replication is useful when data from multiple sources needs to be aggregated for processing, analysis, and reporting. A typical use case is the concentration of data from multiple departments or sites into a centralized database.
Merge replication starts with a snapshot replication as preparation for the subscriber to receive additional data. After the initial snapshot, subsequent data updates and database schema modifications at the publisher are tracked with triggers that force the subscriber to update all the rows that were updated since the last synchronization. The diagram in figure 4 summarizes the merge replication approach.
Transactional Replication
Transactional replication provides the subscriber with constant data updates. This approach also starts with a snapshot replication. In this scenario, data from the publisher arrives at the subscriber almost in real-time. Changes are sent to the subscriber in the same order and kept with the same transactional integrity as they were in the publisher.
This guarantees that only completed transactions will ever make it to the subscriber. This is very useful in applications where the subscriber needs to be constantly updated with new data. As with snapshot replication, both SQL Server and Oracle provide native support for transactional replication. The diagram in figure 5 summarizes the transactional replication approach.
|
Figure 5: Transactional replication approach |
Summary
Enterprise test data management software such as Arendar takes advantage of the flexibility and proven reliability that commercial database systems such as SQL Server offer in terms of topologies, scalability and accessibility to data. By utilizing these replication techniques, we can create enterprise TDM systems that bridge the gap between different department, geographical regions, and even different vendor’s database platforms while maximizing the security, availability, and accessibility to test data across your enterprise.
Daniel Elizalde
Arendar Integration Team Manager
VI Technology
|
|
|
|
Read about the latest information for integrating automated test systems throughout the enterprise. |
|
|
|
 |
Download PDF |
|
|
