Demystifying Data Federation for SOA
Case Study: Financial Services Using Data Services
One global financial organization struggled with the continuous flux of merger and acquisition activity in which data integration was a perpetual challenge. In this example, disparate subsets of information prevented the organization from achieving a single, organized customer view. This impacted their intimacy with the customer and slowed development of improved financial service products from being developed. Because their customer insights were fragmented, consequently their responsiveness to customers diminished as well.
The organization was challenged to correct these gaps by establishing a data-services layer to reconcile data, alleviate the demand on database administrators (DBAs) and developers, and reduce time to value of new financial service offerings. This organization had started down the path of simple data access using home grown mechanisms but quickly realized that the solution wouldn’t scale, was hard to manage and didn’t perform. In addition the organization didn’t want to re-invent the wheel of their data-tier and leverage existing data hubs that had already consolidated most of the company data.
Their goal: Develop highly flexible, scalable single views of the customer, based on a data across fragmented data sources.
Their methodology: Federate data within a Web services abstraction layer for better views of customer information. Leverage the existing data mart as the source.
The following issues were considered:
|•||Physical changes to the databases have a dramatic impact on Web services, these needed to be abstracted out so that they didn’t impact the consumers of the service.|
|•||Implement the data services as an abstraction, not as a replacement of their existing data marts and data hubs.|
|•||Data services must scale without performance degradation.|
|•||The data services solution must preserve or improve on current service-level agreements (SLAs).|
|•||The original data services had originally been ineffective because each line of business treated a customer as separate customers with separate interface requirements. Multiple customers and partners needed to be supported in a single interface.|
|•||Other business units had refused to adopt the original Web services interfaces because the data implementations were rigid and inflexible. These implementations needed to offer agile service bindings that could change quickly, offer new data signatures (the technical contract), be quick to model and implement, while still conforming to different groups, and business requirements.|
Data Services using the federation technique reduced overall complexity with greater data consistency and reusable data services. These services initially became extensions to the data tier implementation and later became adopted as new models within the data warehouse itself. This type of hybrid model reduced development from 12 weeks to 80 hours, and now requires half the number of development managing and maintaining the implementations. No changes to the database were required, freeing the DBA teams to address other tasks. User access to data no longer requires communication with back-end systems, preserving performance within SLA restrictions. The ROI extends beyond this individual project, freeing this growing financial organization from constraints that might otherwise affect its customer intimacy and its overall future business.
There are different approaches to building data services. Data federation provides an important option that results in a rich abstraction layer aggregated across multiple sources. Nevertheless, this approach shouldn’t be considered in a vacuum without weighing carefully the business objectives and the implementation requirements. Consider first the options of consolidation, federation, or perhaps using both together for improved agility and manageability. Regardless of your endeavor you’ll most likely need to consider a data management strategy for ensuring consistency, accuracy, and better control of your data implementations. Data services are likely to continue to evolve as data warehousing, SOA, and business intelligence technologies continue to mature.
(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)