Informatica Versus DataStage

[Steve Bennett]

These traditionally are the two major dedicated ETL platforms. I have a question for the CORTEX:

If you accept that one of these two tools is the right choice, then what are the factors that would influence your choice? Is it factors like:

• Feature set
• Price
• Incumbency (if you already have one product, stick with it)
• Vendor support
• Corporate standards

I have recently completed an evaluation for a client and made a decision to move from one tool to the other. The main factors were price and vendor support in that case, but I would be interested to hear from other people of their experiences.

[admin]

Informatica Customers And Partners Endorse Benefits Of Real-Time Data Integration

24 June, 2008 12:13:00



Informatica Corporation (NASDAQ: INFA), the leading independent provider of data integration software and services, today announced a groundswell of customer and partner endorsement of Informatica’s vision and capabilities for real-time data integration.

Introduced on June 3, Informatica 8.6 provides a single platform for supporting mission-critical projects within the firewall and beyond with cloud computing, including real-time data warehousing, real-time synchronisation of data between operational systems, operational data hubs, and data services. Informatica 8.6 supports the complete continuum of data integration latency, or timeliness, requirements from batch to real time.

“Businesses need to react faster than ever to meet escalating customer requirements and compete successfully in today’s increasingly dynamic global markets,” said Chris Boorman, chief marketing officer, Informatica. “As evidenced by the recent release of the Informatica 8.6 platform, Informatica is putting the full weight of its real-time data integration technology and expertise into enabling customers to gain a competitive advantage.”

PowerCenter Real-time Edition is optimised for real-time analytic and operational data integration use cases. In addition to bringing right-time data integration spanning the latency spectrum, PowerCenter Real-time Edition includes integrated data quality, scalable data services, orchestration, universal data transformation, streaming changed data capture, data synchronisation and replication.

“LinkShare plans to increase its competitive advantage as the largest online pay-per-action marketing network. Part of our strategy is to deliver timely data feeds and data services to our customers and partners,” said Jonathan M. Levine, chief technology officer, LinkShare. “By leveraging the flexible and scalable data services and orchestration capabilities of the Informatica PowerCenter Real-time Edition, LinkShare will be able to consistently roll out new products faster than before, meet customer demands for reporting and analysis, and grow our business by supplying customers and partners with the real-time delivery of holistic, accurate, and secure data.”

According to recent research published by Ted Friedman, VP and Distinguished Analyst at Gartner, “Most important for organisations to recognise is that their data integration will require a mix of latencies — while real-time activity is on the increase, there will always be a need for higher-latency data integration work, since not all data in the architecture changes frequently, and not all processes, teams and roles are capable of harnessing real-time data.” (“Survey on Data Integration Practices Shows Move Toward Strategic Initiatives,” June 6, 2008)

With Informatica 8.6, customers can leverage one solution, and one set of developer skills, for accessing and delivering complete, accurate, high-quality data wherever, however, and at whatever speed the business requires.

“Our commitment to our customers is 100% on-time, damage-free service; accurate and timely information about their shipments; and the highest transportation value. Informatica plays an important role in this commitment,” said Jeff McIntyre, associate vice president, BSNF Railway. “PowerCenter Real-time Edition and PowerExchange Change Data Capture enable us to capture and deliver real-time transportation information to our customers and internal-facing web applications with improved application performance and reduced costs. The result is real-time information on shipments and logistics available to our customers and our operations.”

“Customers can accelerate business responsiveness and mitigate risk with real-time operational business intelligence with high-quality data,” said Carl DCosta, Worldwide Director of Business Intelligence Alliances, HP. “Designed for operational business intelligence, the HP Neoview enterprise data warehouse solution, used in conjunction with Informatica PowerCenter Real-time Edition, provides customers real-time insight for their business with enterprise-class functionality and scale, combined with simplicity. This results in fast implementations with higher reliability, lower cost and better information, faster than current business intelligence solutions.”

For more information about real-time data integration, please visit http://www.informatica.com/realtime.

About Informatica Informatica Corporation (NASDAQ: INFA) is a leading provider of enterprise data integration software and services. With Informatica, organisations can gain greater business value by integrating all their information assets from across the enterprise. More than 3,200 companies worldwide rely on Informatica to reduce the cost and expedite the time to address data integration needs of varying complexity and scale. For more information, call 650-385-5000 (1-800-653-9871 in the U.S.), or visit www.informatica.com.

[zeyadsweidan]

It is a hard decision to make and I can see how it will be different for each client.

I would say price was a key factor for one of my client while the other was the vendor support and long term existance.
In my opinion, the main thing the client needs to be aware of that you don't need to buy a "Ferrari" to perform your basic transportation activities. It is important to know what is the current requirements and potential "real" future requirements.
But I think it is still a hard decision...

Anyway, I have attached a research from Forrester which may help in determining the areas to be considered when evaluating a tool. Hope it helps.
Regards,
ZS

[Doug Heywood]

Originally Posted by zeyadsweidan

It is a hard decision to make and I can see how it will be different for each client.

Good points. I just posted something relevant in another forum here.

Informatica and DataStage are serious commitments for any organisation - both in terms of dollars and skills. You don't need an 800 pound gorilla for all ETL problems, but it's nice to have if you can afford to feed it!

Doug

[Doug Heywood]

Just out on TDWI:

Open source data quality (DQ) tools are here and more are on the horizon.

For example, consider open source data integration (DI) specialist Talend. It made a splash at last week's TDWI World Conference in San Diego, announcing Talend Data Quality, a DQ complement to its flagship ETL software that the company claims is comparable to alternatives from Business Objects (an SAP company), DataFlux (a SAS Institute Inc. company), IBM Corp., and Informatica Corp.

DQ has been on Talend's roadmap for some time. Just two months ago, the company unveiled what it billed as the industry's first open source data profiling tool, too -- Talend Open Profiler.

There's a clear sense in which DQ is an established concern in the open source software (OSS) community. In fact, Talend Data Quality isn't strictly the first OSS data quality tool. Already available are the Open Source Data Quality and Profiling project at Sourceforge, along with DataCleaner and Mural, two newer OSS data quality efforts. There's also the obliquely-OSS OpenDQ 2.0 from InfoSolve Technologies, though it does not include built-in service or support, and its source code is available only to customers, an arrangement that some folks claim violates the spirit -- if not the letter -- of the GPL.

Even so, Talend Data Quality is the first OSS tool to deliver both commercial support and integration as part of a larger suite

Data quality is a surprisingly involved proposition, given both the complexity of its underlying technology (best-of-breed data quality assumes the use of speedy deduplication, matching, and other specialty algorithms) and the extreme breadth of its reach (best-of-breed data quality requires the development and maintenance of multi-lingual dictionaries, for example). Talend may be up to the challenge. It recently demonstrated its innovative open source DI platform: an ETL engine that produces extracted, cleansed, and transformed data in the form of an executable binary (http://www.tdwi.org/News/display.aspx?ID=8967).

"What we're offering is a complete data quality suite that … is composed of two different modules: a data profiler, which is a [more advanced] version of Talend Data Profiler, and a cleanser. This is a complete solution. With it, we are addressing most of the requirements [typically addressed by] standalone data quality tools," says Yves de Montcheuil, vice-president of marketing with Talend.

Talend Data Quality fills in quite a few DQ checkboxes, boasting -- among other features -- data identification (i.e., the ability to determine if data is reliable or unreliable on a record-by-record basis); data cleansing (i.e., the ability to clean incorrect, incomplete, or inconsistent data, either by using built-in routines or by cross-referencing it against masters databases or reference data); and data enrichment (i.e., the ability to flesh out data with "nice-to-have" information -- including latitude and longitude information, census data, or credit scores) capabilities.

In the last scenario, organizations are actually consuming data from external -- or supplementary -- sources. It's a fast-emerging requirement, according to Montcheuil.

"[Customers] are starting to include mapping information [in their data integration feeds]. When you do that, you need to have accurate geographical coordinates if you're going to include this [mapping] information. Some [customers] want to use [i.e., incorporate] credit scores, or information from the U.S. Census Bureau," he says. "Most of those feeds are going to go through Web services, although it will vary, based on the industry you're looking at. In some cases, it will require a subscription with the delivery of files. Because we have an architecture based on a SOA stack, it's very, very easy for us to incorporate third-party data through integrated components."

Although it boasts capabilities that are "comparable" to commercial best-of-breed offerings, Talend Data Quality is a work in progress, Montcheuil concedes.

"We have a very aggressive release cycle, and [customers] need to understand that because of [the nature of] our platform, there is a lot less risk of serious problems than if we were launching new products entirely from scratch," he comments. "We'll continue to improve the platform. With ETL, we have a release every four months; [Talend] Data Quality will be on a similar cycle. One major direction we're working on now, for future release … is the SOA stack."

Industry veteran Mark Madsen is cautiously optimistic about the OSS community's first suite-centric DQ offering. He's intrigued by the Talend product, which (because it won't formally debut until next month) he hasn't yet had a chance to put through its paces. Madsen isn't alone: according to Montcheuil, Talend isn't releasing any reference or beta customer information.

Madsen also thinks that Talend Data Quality, considered alongside other open source DQ tools -- and in the context of several nascent relational database DQ projects -- could augur coming commoditization in the DQ segment.

"[Talend has] not released, so I can't say how much there is in the tools yet, but it will improve over time," he asserts. "DQ is the latest product to become a commodity in the integration space. Everyone has ETL, profiling and [data] quality now. At my vendor session at TDWI, all three [relational database] vendors [i.e., IBM Corp., Microsoft Corp., and Oracle Corp.] demoed data quality in the open slot where they had ten minutes to show something of interest."

As for the OSS data quality tools, Madsen thinks Talend -- with its services, support, community base, and built-in development environment -- could be the pick of the litter.

"Today there are only rudimentary open source offerings, but there are three others focused on this. They [the other OSS tools] just don't have a good set of features or come married to DI tools," he concludes.

[Doug Heywood]

This is old news but I can across it while surfing:

BO MD claims Informatica misleading media

By Sarah Falson 6 June 2007 02:24PM

SYDNEY - Local MD of Business Objects (BO) claims Informatica, has made false claims regarding a recent court case.
ITnews filed a story on 25 May which included quotes from Informatica Australia and New Zealand managing director, Laurie Newman, stating that Informatica is offering to substitute Business Objects customers’ data systems as part of a global initiative off the back of the company’s recent court-case win against its competitor.

US-based Informatica recently won an injunction against Business Objects that said the latter company was infringing on Informatica’s patented data transformation technology by incorporating the technology into its Business Objects Data Integrator product, which has been installed worldwide, according to Newman.

“The number one objective for large corporations is data integration. This injunction will severely impact Business Objects customers,” he told ITnews.

Business Objects must now remove the infringing code from its product before it is sold and installed, with already-established systems rendered ineligible for upgrades, he said.

However, Business Objects vice president and managing director – Australia and New Zealand, Rob Wells, would like the following information made available to ITnews readers.

“Existing Data Integrator customers can continue to use the feature in the product they currently have,” he told ITnews.

“There are many ways to get reusability within the Data Integrator, and it would be foolish to say that reusability is restricted to this particular use of embedded dataflows. You can continue to reuse jobs, workflows, dataflows, datastore definitions, custom functions and other objects as you have in the past. Even reusing embedded dataflows is fine as long as there is at most one input or one output port.”

Business Objects has shipped a new release of the product which has removed the “one small discrete feature at the centre of the lawsuit”, according to Wells.

“Removal of this feature does not impact customers reusablility of ETLs or Dataflows within the Data Integrator product – only one input and one output. There are many other ways to achieve this function.

“We have been transparent with our customers and partners on this issue,” said Wells, referring to two letters which were sent to Business Objects customers during the lawsuit.

Wells told ITnews that Business Objects will “continue the legal process and will fight vigorously to defend our technology”.

Has anybody got an update on this story? Any experience with losing functionality in BOBJ DI?

[zamir]

What about expanding this topic to considering open source options? I think that Eclipse is ready for prime-time enterprise ETL.

Take a look at the attached analysis if you want to know more. Here are the main findings of the Bloor research:

Key findings
In the opinion of Bloor Research the following represent the key facts of which prospective users should be aware:

• Talend Open Studio is a code generating product. That is to say, it generates either Java or Perl (plus SQL) that you can deploy as required (in parallel and/or across a grid, if appropriate). This means that Talend avoids the ‘black box’ approach of many other vendors that can result in the black box becoming a bottleneck.
• A major advantage of the code generating approach is that it is easy to encapsulate a job as a web service that can be invoked by third party applications, or to embed the generated Java into third party applications. This is more of an issue for conventional approaches because they require an engine (the black box) in order to run.
• Using a conventional ETL approach you would normally deploy your applications as close to the source as possible but ELT options allow you to deploy on the target. In the latest release, ELT options, based on SQL patterns (a combination of Java and SQL) that are specific to each database, have been significantly enhanced so that this approach can be used with any supported database environment.
• The recommended approach for development within Talend Open Studio starts with the Business Modeler before drilling down to developer functions. We particularly like this emphasis on the business analysts as a starting point for development.
• In most respects Talend Open Studio looks pretty much like other products: a graphical user interface, dragging and dropping onto a palette, a visual SQL builder, debugging facilities and so on. It actually uses the familiar Eclipse interface.
• There are data cleansing and data profiling capabilities built into the product.
• The latest release of the product supports significant parallelisation capabilities.
• Metadata management, based on a repository that supports reuse, is provided within Talend Open Studio though full multi-developer support with version control and check in and out is only available within Talend Integration Suite.
• As far as we know, Talend is the only open source vendor in this market to have offices outside its home market: in this case in China and the United States as well as France and Germany. This means that Talend can offer 24x7 support on all continents.

I haven't used Talend myself but I'm hearing good things about it on the net. Does anyone know more?

Is the Informatica v DataStage discussion irrelevant??!

[admin]

From the New York Times:

August 24, 2009, 1:49 PM
Informatica’s Software Glue Sells in the Recession
By ASHLEE VANCE

When the economy started to worsen, Sohaib Abbasi put his sales people at Informatica on the clock.

The chief executive threw out the annual sales targets used by most business software companies and set weekly targets for employees. The salespeople started hunting smaller deals that could come to fruition in the near-term rather than striving for mega-deals that may or may not materialize over many months. This strategy, Mr. Abbasi figured, would provide a more pragmatic and realistic structure during a time when customers were constantly reworking their technology budgets.

“It forced the salespeople to focus on what they could do this week as opposed to a huge deal that they could do next quarter or next year,” Mr. Abbasi said. “It let us figure out what could get approved or not approved much earlier.”

These types of maneuvers have helped Mr. Abbasi turn Informatica from a company spinning its wheels into a growing force in the software industry in the five years since he took over as chief executive. Informatica’s revenue doubled to $456 million. When losses have become common for business software makers, his company increased revenue at 5 percent in the first quarter of this year and 3 percent in the second quarter.

Data integration software provides a crucial role for businesses. It’s the code that links database, supply chain management, accounting, human resource management and marketing applications made by different companies.

There’s an argument to be made that Informatica’s technology will diminish in importance as companies like Oracle, I.B.M. and SAP digest their numerous acquisitions and create stronger ties between their own applications.

Mr. Abbasi, however, dismisses this notion. “Will the industry giants eventually eat our lunch?” he said. “The answer is no.”

While Oracle, for example, may shuffle data among its own applications more easily, it will remain reluctant to share information with competing software, Mr. Abbasi said. Along these lines, there will always be room for an independent company that can marry software from different companies as well as freeing up data from custom applications developed in-house by businesses.

“The majority of the data is in those custom applications,” Mr. Abbasi said. “Even if Oracle acquired every software maker, they still would not be able deal with that.”

In the past, Informatica made what appeared like a natural move into the data analytics business. But Mr. Abbasi shut down these efforts, arguing that it was more practical for Informatica to focus on what it does well and leave the analytics jobs to companies that had already developed leading technology.

But the company has identified some new areas where it would like to play and try and capitalize on its integration skills.

In June, it bought a company called AddressDoctor that helps government organizations and businesses check the validity of mailing addresses. The technology grew out of the direct-mail industry but now has broader applications like helping health care providers and retailers make sure they have the right addresses for their customers.

To make such software work, Informatica has agreements in place with the postal authorities in hundreds of countries.

Such technology fits into the company’s goal of validating the accuracy of data within organizations. For instance, Informatica has software that can hunt through databases in different languages to determine the connections between information stored on a single individual. This has proved a favorite among intelligence agencies.

Similar software can correct the spelling of someone’s name throughout various databases by running checks to make sure the databases are describing the same person.

More broadly, Informatica looks to benefit from the shift toward cloud computing where it can serve as a type clearing house for data by linking cloud systems from a variety of companies.

“The current control that the software industry giants have is getting compromised very quick by cloud computing,” Mr. Abbasi said.

While customers may have more freedom in the cloud model by avoiding being locked into a database or particular application, they must still deal with moving data between cloud systems owned by different companies. That’s where Informatica thinks it can help.

“Customer information is recorded in a different cloud in a slightly different way,” Mr. Abbasi said. “You have to figure out what the meaning of a customer is over here versus over there. Those are the things we know best.”

[Steve Bennett]

HP and Informatica Expand Alliance to Optimise Business Performance for Customers

iTWire, Wednesday, 09 September 2009

HP and Informatica today announced a new portfolio of integrated business intelligence solutions that help customers accelerate business decisions by giving them access to more timely and accurate information.

These offerings combine Informatica’s data integration capabilities with the HP Neoview enterprise data warehouse platform as well as HP’s business intelligence strategic consulting and implementation services. HP plans to sell these offerings through its Business Intelligence Solutions sales teams.

“Customers view information as a strategic asset, and yet they still struggle to quickly access their data to accelerate decision making,” said Thomas E. Hogan, executive vice president, Software and Solutions, HP. “These new solutions will empower our clients to reduce their information-related costs and gain competitive advantage.”
The portfolio of integrated data management solutions include:

- HP Neoview with Informatica Data Integration Platform provides customers with a consistent view of their business data. It allows different stake holders within a business ecosystem – business analysts, call center agents – to do analysis in their own context and make decisions with timely data.

- HP Master Data Management Services with Informatica Data Integration Platform provides a holistic and accurate view across various sources of an organisation’s core business entities, such as a customer or a product. This also improves efficiency of collaboration with partners in the supply chain.

- HP Information Quality Management Services with Informatica Data Integration Platform gives customers the confidence to go beyond data access and report generation to leverage information for high quality, actionable business and financial decisions.

“CIOs and IT organisations continue to realise the importance of managing data and delivering it in a consistent and high-quality manner to business users,” said Mark Smith, chief executive officer and executive vice president, Ventana Research. “Informatica and HP working more closely together to deliver technology and services that help reduce time and costs in advancing information architectures is an important step for IT organisations to increase their value and efficiency in data management.”

“Together HP and Informatica have provided comprehensive data integration solutions to customers for more than 10 years,” said Sohaib Abbasi, chairman and chief executive officer, Informatica. “The new combined offerings will allow organisations to derive even more business value from all of their information assets while improving its quality and IT efficiency.”

More information on HP Business Intelligence Solutions is available at Business Intelligence Solutions: Connected intelligence for better business outcomes.

About Informatica
Informatica Corporation (NASDAQ: INFA) is the world’s number one independent leader in data integration software. The Informatica Platform provides organisations with a comprehensive, unified, open and economical approach to lower IT costs and gain competitive advantage from their information assets. More than 3,700 enterprises worldwide rely on Informatica to access, integrate and trust their information assets held in the traditional enterprise and in the internet cloud. For more information, call +1 650-385-5000 (1-800-653-3871 in the U.S.), or visit Data Integration - Informatica.

About HP
HP, the world’s largest technology company, simplifies the technology experience for consumers and businesses with a portfolio that spans printing, personal computing, software, services and IT infrastructure. More information about HP (NYSE: HPQ) is available at HP United States - Computers, Laptops, Servers, Printers and more.

[admin]

Cloud computing intricacies to spawn more data fragmentation
Computerworld Singapore writer 19 November, 2009 11:37:00

Aware of the value of relevant and timely data in business processes, data integration software provider Informatica has recently released its unified and open data integration infrastructure platform, Informatica 9.

With user-friendly features such as browser-based analyst tools and auto-generation of implementation details, Informatica 9 was developed to facilitate the work among the business units needing the relevant data and IT.

"Informatica 9 enables organisations to gain the most business value from their information assets," said Sohaib Abbasi, chairman and CEO, Informatica. "With unique capabilities for business-IT collaboration, pervasive data quality and service-oriented architecture (SOA)-based data services, Informatica 9 helps deliver the most relevant, trustworthy and timely data."

Informatica 9 automates every step of the data integration lifecycle while maintaining the quality of data. Prior to Informatica 9, data quality was an issue that can be a misery to the IT department.

As a comprehensive platform, Informatica 9 also features B2B data exchange, application information lifecycle management, complex event processing and cloud data integration.

Tools

Because data relevance is a crucial issue that needs to be addressed, Informatica 9 provides browser-based analyst tools that can aid users in specifying business requirements.

To ensure that loss of details in the translation from requirements to actual implementation becomes negligible, Informatica 9 is capable of automatic generation of implementation details from business specifications.

For business analysts and IT developers to communicate and share their work with each other, Informatica 9 uses to its advantage a metadata repository of all specification and implementation artifacts.

The demand for comprehensive support continues to hound all solutions providers. For this reason, Informatica 9 has integrated the support for nearly all data and related purposes.

The presence of its unified role-based tools enables stakeholders to handle data quality requirements through data quality scorecards, simple analyst tools and productive developer tools.

Multi-modal

To save on time, the information catalog services of Informatica 9 strengthen user's capability to quickly find relevant data. Data discovery is further hastened using logical data objects that simplify data representation.

When it comes to data delivery, innovative multi-modal data provisioning services makes possible the transfer of data in various formats and in different protocols without additional IT development work.

"Combining data quality and data integration into a single platform and the delivery of multi-modal data provisioning, all with a single-user interface ensures that customers will be able to reduce risk and minimise costs, ultimately improving their business agility," said Carl Olofson, research vice president at IDC. "That's the message of Informatica 9, and it is setting the course that the industry must follow."

Informatica 9 can also help companies pursue the reduction of systems administration costs with its policy-based data services governance to centrally manage data rules only once.