Demystifying Data Federation for SOA

Turning Bad Data Good

Bad data puts enterprise software projects at risk. Because inaccurate and inconsistent data exists pretty much everywhere, the demand for trusted data continues to spiral upward, driven by investments in packaged applications and business intelligence applications. Strategic IT initiatives like MDM also add additional pressure. Further complicating the matter, regulatory compliance initiatives (such Sarbanes-Oxley, U.S. Patriot Act, and Basel II) require tracing the source of the data used in financial reports, as well as examination, tracking (through snapshots), and certification of the state and quality of the business data.

Figure 5

Technology alone doesn’t deliver trusted data. Information managers need to define what data quality means to their organizations. These can be implemented through data governance programs that can define data quality rules and their respective processes for how they are maintained, approved and iterated to improve benefits.

Data quality generally requires data movement. Often we see data cleansing examples that are part of data hub approaches, as shown in Figure 6. That is not to say that data federation cannot implement a more real-time style of data quality during the process of data access, but the cleansing actions must be architected carefully for performance when they are implemented through the query access step of federation. For example, many MDM systems require a cleansed customer record to be presented on the application screen, if that record must be recovered from the source system without any cleansing; it can be performed in real-time once the data is accessed from the source. This type of real-time data cleansing is tricky to optimize correctly, but entirely possible in practice.

Regardless of the technology, without a fundamental data governance strategy at the core, these initiatives may suffer, just as many SOA implementations languished without a strong SOA Governance foundation to give greater visibility and control.