Datastage Scenario Based Question/Answer :1

Datastage Scenario Based Question/Answer : 1

Question1 : 

Input :
source,destination,km
hyd,bang,1000
delhi,chennai,1500
chennai,bang,600
bang,hyd,1000
bombay,pune,1000
bang,chennai,600

Output :
source,destination,km
hyd,bang,1000
delhi,chennai,1500
chennai,bang,600
bombay,pune,1000

see here hyd to banglore distance is 1000km
another rows is also banglore to hyd 1000km
so need to delete rows like this.

Solution :

Kindly find the solution for this problem.There are so many different ways. This is, one of the way to get the 

Source=>Sort=>RemoveDup=>Transformer=>Lookup=>Filter=>Target

Steps:
1. Read the Source Data.
2. Sort the Data on initial 2 columns.
3. Remove duplicate values (If Any) on initial 2 columns.
4. Pass 2 output links from transformer.
One Link-> Master Link
Col1=> Source
Col2=> Destination
Col3=> Distance

2nd Link => Refrence Link 
Col1=> Destination 
Col2=> Source
Col3=> LkpChk ( With Default Value = 1)

5. Lookup the master link with the reference link.
with Col1=Col1 and Col2=Col2

Output from lookup is 
One Link-> Master Link
Col1=> Source
Col2=> Destination
Col3=> Distance
Col4=> LkpChk( If Lookup found =1 Else you will get Null)

6. Filter the data only for (LkpChk <> 1)

7. You will get your required values.