Datastage Version 11.3 job compatibility fixed : IBM DataStage version 11.3 introduce new features and functions that affect
existing Version 9.1 or older version of the jobs. This post will help you to fix those issues in Datastage 11.3.DK®
After importing jobs from IBM DataStage, Version
9.1 to Version 11.3, you need to recompile all jobs. In addition, you might
need to make changes to your parallel jobs to accommodate some changes in
Version 11.3. DK®
Changes to Parallel Jobs :
Below are the environment variables that you
might need to set to revert back to the behavior prior to version 11.3. DK®
*******************************************************************************
Problem
1 - In
InfoSphere Information Server Version 9.1 and earlier, the sequential file
stage used the value of the job NLS map as the character set
for the field names when writing the file. This could result in the column
names being in a different character set than the data. In Version 11.3, if the
output NLS Map setting is defined on the sequential file stage, it will use
that value as the character set for the field names.DK®
Solution:Set
the environment variable APT_EXPORT_FIELD_NAMES_USING_IMPEXP_ CHARSET to use
the job NLS map to define the character set for the field names in the output file
instead of the NLS Map setting on the sequential file stage. This variable
reverts to the behavior from 9.1 and earlier.DK®
*******************************************************************************
Problem
2 - Prior to InfoSphere DataStage
Version 11.3, aggregation reduce output fields were set as nullable fields. In
InfoSphere DataStage Version 11.3 aggregation reduce output fields are defined
such that their nullability matches the nullability of the input field that is
being reduced. Consequently, if in any of the upstream stages in the job the
aggregation reduced output field has been defined as nullable but the input
field nullablility of that column is not nullable, then that upstream stage
does not produce null output for that reduced column.
Solution: In Version 11.3, set the
APT_AGGREGATOR_NULLABLE_OUTPUT variable to cause the group operator to define
reduce fields in its output schema as nullable, regardless of the nullable
setting of the corresponding field. By default, the operator matches the null
setting of the output schema to the null setting of the input schema that is
being reduced.
*******************************************************************************
Problem
3 - When APT_NO_JOBMON is defined, job
monitoring is turned off. However, starting with release 11.3, this variable
will not disable the polling thread since doing so can cause job hangs for
operators like the surrogate key stage that needs this thread to run.
Solution: To revert to the same behavior from before
11.3, set both environment variables APT_NO_JOBMON and
APT_NO_POLLING_THREAD.
Setting both will turn off job
monitoring and all threads it creates including the polling thread. This
setting can cause job hangs if they have stages that are dependent on the
polling thread to run.DK®
*******************************************************************************
Problem
4 - In InfoSphere DataStage Version
11.3 the behavior of the Sequential File stage was changed so that a file
pattern is converted into a fileset before the files are imported. This change
allows the files to be processed in parallel rather than sequentially and also
support better data lineage.
Solution: Set the APT_IMPORT_PATTERN_USES_CAT environment
variable to force the behavior of prior versions of InfoSphere DataStage. This
prior behavior was to concatenate all the files that match a file pattern
before importing the files sequentially.
No comments:
Post a Comment