Datastage Information Server Architecture

IBM Information Server 8.x.x and 9.x.x Architecture

IBM Information Server is a client-server architecture made up of client-based design, administration, and operation tools that access a set of server-based data integration capabilities through a common services layer as shown here in  below Figure 1-2. 

In this section, we briefly discuss the following topics:

1. Component overview

2. Topologies supported

1.2.1 Component overview

The main components shown in Figure 1-2 are briefly described here.

1.2.1.1 Client tier :

IBM Information Server provides a number of client interfaces, optimized to different user roles within an organization. The clients tier includes IBM InfoSphere DataStage and QualityStage clients (Administrator, Designer, and Director), IBM Information Server console, and IBM Information Server Web console.

There are two broad categories of clients — Administrative clients and User clients.

Both these types of clients have desktop and Web based interfaces.

1. Administrative clients

These clients allow you to manage the areas of security, licensing, logging, and scheduling.

   – Administration tasks are performed in the IBM Information Server Web console. The IBM Information Server Web console is a browser-based interface for administrative activities such as managing security and creating views of scheduled tasks. 

  – For IBM InfoSphere DataStage and IBM InfoSphere QualityStage project administration, you use the IBM InfoSphere DataStage Administrator client. It administers IBM InfoSphere DataStage projects and conducts housekeeping on the server. It is used to specify general server defaults, add and delete projects, and to set project properties. User and group privileges are also set using the Administrator client.

2. User clients

These clients help perform client tasks such as creating, managing, and designing jobs, as well as validating, running, scheduling and monitoring jobs. The IBM Information Server console is a rich client-based interface for

activities such as profiling data and developing service-oriented applications.

         * The IBM InfoSphere DataStage and QualityStage Designer helps you create, manage, and design jobs. You can also use the Designer client to define tables and access metadata services.

The Designer client allows you to move DataStage and QualityStage objects between projects on the same Information Server engine, or on different Information Server engines. You can also use the Information Server Manager client to move objects from one domain to another. The Information Server Manager supports the model of having separate systems for the developing, testing and running of DataStage and QualityStage jobs. It facilitates the model by providing secure and managed methods of moving objects between the different systems.

     The IBM InfoSphere DataStage and QualityStage Director client is the client component that validates, runs, schedules, and monitors jobs on the IBM InfoSphere DataStage Server.

Note: Clients are supported on 32-bit Microsoft® Windows XP Pro, Vista, Win7 and Server 2003.

1.2.1.2 Server tiers:

The server tiers of the Information Server Platform that includes the Services, Engine, Repository, Working Areas, and Information Services Director Resource Providers as follows:

   A) Services tier

IBM Information Server is built entirely on a set of shared services that centralize core tasks across the platform. Shared services allow these tasks to be managed and controlled in one place, regardless of which suite component is being used.

The Services Tier includes both common and product-specific services:

– Common services are used across the Information Server suite for tasks such as security, user administration, logging, reporting, metadata, and execution.

– Product-specific services provide tasks for specific products within the Information Server suite. For example, IBM InfoSphere Information Analyzer calls a column analyzer service (a product-specific service) that was created for enterprise data analysis. The shared service environment allows integration across IBM Information Server because they are deployed using common SOA standards.

IBM Information Server products can access three general categories of service:

1.Design :

Design services help developers create function-specific services that can also be shared.

2. Execution :

Execution services include logging, scheduling, monitoring, reporting, security, and Web framework.

3. Metadata :

Using metadata services, metadata is shared “live” across tools so that changes made in one IBM Information Server component are instantly visible across all of the suite components. Metadata services are tightly integrated with the common repository. You can also exchange metadata with external tools by using metadata services. The common services layer is deployed on the J2EE™-compliant application

server IBM WebSphere® Application Server, which is included with IBM Information Server.

Note: An Application Server is a high performance transaction engine that helps you build, run, integrate, and manage dynamic Web based applications typically involving HTTP protocol.

  B) Repository tier

The shared repository is used to store all IBM Information Server product module objects1 (including IBM InfoSphere DataStage objects), and is shared with other applications in the suite. Clients can access metadata and results of data analysis from the respective service layers.

Note: The repository supports DB2 for LUW 9, Oracle10g R2 or SQLServer 2005 as the underlying database.

  C) Engine tier

This is the parallel runtime engine that executes the IBM Information Server tasks. It comprises the Information Server engine, Service Agents, and Connectors and Packaged Application Connectivity Kits (PACKS2).

1. The IBM Information Server engine consists of the products that you install, such as IBM InfoSphere DataStage and IBM InfoSphere QualityStage. It runs jobs to extract, transform, load, and standardize data.The engine runs DataStage and QualityStage jobs. It also executes the parallel jobs for Information Analyzer tasks.

2. Service Agents are Java™ processes that run in the background on each computer that hosts IBM InfoSphere DataStage.They provide the communication between the Services and Engine tiers of Information Server.

3. Connectors and PACKS

IBM Information Server connects to a variety of information sources whether they are structured, unstructured, on the mainframe, or applications. Metadata-driven connectivity is shared across the suite components, and connection objects are reusable across functions. Connectors provide design-time importing of metadata, data browsing and sampling, run-time dynamic metadata access, error handling, and high functionality and high performance run-time data access. Prebuilt interfaces for packaged applications called PACKS provide adapters to SAP®, Siebel®, Oracle, and others, enabling integration with enterprise applications and associated reporting and analytical systems.

   Working areas

These are temporary storage areas used by the suite components.

Information Services Director (ISD) Resource Providers Information service providers are the (data) sources of operations for your services. Using IBM InfoSphere Information Services Director, you can create services from five sources — IBM InfoSphere DataStage and QualityStage, IBM DB2 for LUW, IBM InfoSphere Federation Server, IBM InfoSphere Classic Federation Server for z/OS, and Oracle Database Server.