Concatened csv files with tMap conditions

[gadje1]

Hello, i would like be able to generate a csv file from concatened 2 csv files but with conditions in a tMap.

In my iMap, if a test on "rub.CDGRP" with a blank value or null is true then the out value is web.produit_code_article value, else it will be bac.produit_code_article value.

here is my tMap :

But there is far too many lines in result :

I join an output file example with a lot occurrences for the same article number.

If somebody knows.

Thanks a lot.

[Anonymous]

Hi,

Because your 'rub' file is not joined to any other inputs, I'm afraid you're going to create a cross join.

Is there any logic that ties 'rub' to the rest of your data? If so you'll need to join the dataset to avoid having this behaviour.

[gadje1]

Hi,

Thanks for your reply.

For a better understanding, i give you the structure of my 2 csv files.

web.csv :

To see the whole post, download it here
OriginalPost.pdf

[nivedhitha]

@gadje , What is it that you are trying to do here?

In the first post i see tat you are trying to check if the code exists in "rub" table and if it doesn't you are bringing in the code from  "web" table.

Where is the base list of codes that you want to check in the "rub" table? Can you explain the requirement and the output a bit more?

NullPointerException means there is some null value flowing into tMap on which you are trying to do some operation.

Also for checking null condition can you try using Relational.ISNULL(rub.CDGRP)?x:x instead of column.equals(null) ?

[gadje1]

Hi Nivedhitha,
Thanks for your reply.
in fact i would that my 2 csv files are concatened in one. But if an article code is present in "bac" and in "oWeb" datasets, the out data must be from the "bac" dataset in the output csv file. Else if the article code is only present in the "oWeb" dataset it will be the oWeb value in the output csv file.

In this first example the article code "001521" is present in both csv files, the out data must be the lines from BAC :

in BAC CSV file :

001521|image|PHOTO_VIGNETTE|https://media-orcab.azureedge.net/bac/GFD01/L150/GFD01[PP[156008[01.jpg|JPG|IMAGE_VIGNETTE|
001521|image|LOGO_FABRICANT|https://media-orcab.azureedge.net/bac/GFD01/L1200/LOGO_GFD01.jpg|JPG|IMAGE_SECONDAIRE|
001521|image|PHOTO_PRODUIT_1|https://media-orcab.azureedge.net/bac/GFD01/L1200/GFD01[PP[156008[01.jpg|JPG|IMAGE_FICHE|
001521|image|PHOTO_PRODUIT_2|https://media-orcab.azureedge.net/bac/GFD01/L1200/GFD01[PP[156008[02.jpg|JPG|IMAGE_SECONDAIRE|
001521|document|SCHEMA_1|https://media-orcab.azureedge.net/bac/GFD01/GFD01[MB[156008[01.jpg|PDF|DOC_TECHNIQUE|

===============================================

in OWEB CSV file :

001521|document|SCHEMA_1|http://www.placo.fr/Solutions/Catalogue-produits/Profiles-et-accessoires-pour-plaques/Fixations2/Fixations/Vis-TTPC-45|PDF|DOC_TECHNIQUE|
001521|image|PHOTO_VIGNETTE|/media/produit/vignette/VisTTPC.gif|JPG|IMAGE_VIGNETTE|
001521|image|PHOTO_PRODUIT_1|/media/produit/zoom/VisTTPC.gif|JPG|IMAGE_FICHE|

In this second example the article code "001518" is only present in OWEB csv files, the out data must be the lines from OWEB :

in BAC CSV file :



===============================================

in OWEB CSV file :

001518|image|PHOTO_VIGNETTE|/media/produit/vignette/VeluxGHL.gif|JPG|IMAGE_VIGNETTE|
001518|image|PHOTO_PRODUIT_1|/media/produit/zoom/VeluxGHL.gif|JPG|IMAGE_FICHE|
001518||SCHEMA_1||PDF|DOC_TECHNIQUE|

i guess it's not necessary to use the "rub" table entry. It's possible to do just with my 2 csv files.

Thanks in advance.

[gadje1]

I tried with "Relational.ISNULL" syntax instead of ".equals(null)". There is no anymore null pointer problème but it's like if the test dosn't work in tMap component.

The output just contain the same lines count as return of the sql query from BAC_ORCAB.

I dont understand why the test dosn't work in my tMap.

[nivedhitha · Accepted solution]

@gadje , If im understanding it right, this should work. I dont think "RUB" has to be brought in this scenario.

I have created sample BAC and OWEB tables. 

BAC Data

code	desc
a	yes
b	yes

OWEB Data

code	desc
b	yes
c	no

So the code 'a' is found in both the table sand hence has to come from BAC. 'b' is found in BAC but not in OWEB and hence comes from BAC. 'c' is found in OWEB but not in BAC and hence comes from OWEB

This is how the job has to be designed with 2 subjobs, one with BAC as the main and OWEB as lookup and the other woth OWEB as main and BAC as lookup.

The first tmap has BAC as main and OWEB as lookup. An inner join is made between these two tables. There are two output groups, out1 which gets the results of the inner join (so these are the records found in both the tables), and the 'rejectsBAC' collects the rejected records from BAC (this means that these records are only found in BAC and not in OWEB). -> shown below

the tMap of the second subjob is shown below. This as OWEB as main and BAC as lookup and an inner join is mad enad only the rejects are collected here (all records from OWEB that are not found in BAC)

All thse outputs are sent to a tBufferOutput and hence they are all unioned and the final output from the tBufferOuptut is

The common records, records found only in BAC and records found only in OWEB.

Let me know if it helps

[gadje1]

Hello Nivedhitha,

Thanks a lot for your help.
You just save my life.
It's exactly what i want to do.

It work perfectly.

[nivedhitha]

Glad that worked. You are welcome

[gadje1]

Merci Madame.