Newly Added Features in InfoSphere
DataStage 11.7.x
New in 11.7 Base Version
1.
By using DataStage Product Insights, you can connect your
InfoSphere DataStage installation to IBM Cloud Product Insights. Such
connection gives you ability to review your installation details and metrics
like CPU usage, memory usage, active jobs, jobs that failed, and jobs that
completed.
2.
Data Masking stage supports Optim Data Privacy Providers version
11.3.0.5.
Connectivity Enhancements :
New in 11.7 Fix Pack 1
A. Cassandra connector
is supported. It has the following features:
1.
DataStax Enterprise (DSE) data platform built on the Apache
Cassandra is supported. DataStax Enterprise Java Driver is used to connect to
the Cassandra database.
2.
The connector reads data from and writes data to Cassandra
database in parallel and sequential modes.
3.
You can provide SQL statement (SELECT, INSERT, UPDATE, DELETE)
to read or write data.
4.
Reading and writing data of a single column in JSON format is
supported.
5.
Custom types of codecs are supported.
6.
You can specify a consistency level for each read query and
write operation.
7.
You can modify data by inserting, updating, deleting an entire
row or a set of specified columns.
B. Azure Storage connector is supported. You can use it to
connect to the Azure Blob storage and Azure File Storage and perform the
following operations:
1.
Read data from or write data to Azure Blob and File Storage.
2.
Import metadata about files and folders in Azure Blob Storage
and Azure File Storage.
C. HBase connector supports the following
features:
1.
Metadata can be imported on the table level and higher.
2.
You can run data lineage on a table level.
3.
ILOG JRules connector supports Decision Engine rules in all
engine modes (Core, J2EE, J2SE).
D. Kafka connector
supports the following features:
1.
Connector supports new secure Kafka connections, including SASL/PLAIN,
SASL/SSL and SSL with user authentication.
2.
Apart from String/Varchar, new message types are supported:
Integer, Small Integer, Double and Byte array.
3.
Kafka partitioning - fetching key and partition number from
Kafka - is supported. The partitioning type is preserved for the write mode.
E. Data lineage for the
Hive connector is enhanced in the following ways:
1.
Data flow to the column level is supported.
2.
The following URL formats of JDBC drivers are supported:
jdbc:ibm:hive (DataDirect) and jdbc:hive2. For the jdbc:ibm:hive driver
version, the database name is 'ibm'. For the jdbc:hive2 driver version, the
database name is set by using the entire URL, which is JDBC default behavior.
3.
You can use the URL attribute Database to set the database
schema.
4.
You can use the URL path as the database schema.
F.
Db2 connector supports Db2 12 for z/OS.
G. SFDC API 42 is
supported.
H. Sybase IQ 16.1 is
supported.
I.
Greenplum connector supports Greenplum database 5.4.
New in 11.7 Base Version
:
1. HBase connector is supported.
You can use HBase connector to connect to tables stored in the HBase database
and perform the following operations:
A. Read data from or write data to HBase database.
B. Read data in parallel mode.
C. Use HBase table as a lookup table in sparse or normal mode.
Kerberos keytab locality is supported.
2. Hive connector
supports the following features:
A.
Modulus partition mode and minimum maximum partition mode during
the read operation are supported.
B.
Kerberos keytab locality is supported.
C.
Connector supports connection to Hive on Amazon EMR.
3.
Kafka connector supports the following features:
4.
Continuous mode, where incoming topic messages are consumed
without stopping the connector.
5.
Transactions, where a number of Kafka messages is fetched within
a single transaction. After record count is reached, an end of wave marker is
sent to the output link.
6.
TLS connection to Kafka.
7.
Kerberos keytab locality is supported.
8.
IBM MQ version 9 is supported.
9.
IBM InfoSphere Data Replication CDC Replication version 11.3 is
supported.
10.
SQL Server Operator supports SQL Server Native Client 11.
11.
Sybase Operator supports unichar and univarchar datatypes in
Sybase Adaptive Server Enterprise.
12.
Amazon S3 connector supports connecting by using a HTTP proxy
server.
13.
File connector supports the following features:
14.
Native HDFS FileSystem mode is supported.
15.
You can import metadata from the ORC files.
16.
New data types are supported for reading and writing the Parquet
formatted files: Date / Time and Timestamp.
17.
JDBC connector is certified to connect to MongoDB and Amazon
Redshift.