topic Re: delete rows according to criteria in Talend Studio

delete rows according to criteria

gadje1 — Wed, 11 Mar 2020 09:55:29 GMT

Hello everybody,

Here is my problem of the day.

I want to recover only certain lines depending on the presence of other lines in my data flow.
In my csv output file, for the article code "008969" for example, I must not retrieve the 2 highlighted lines having the column "prix_quantite_minimum" with values ("50,000" or "100,000") if "Promo" is indicated at the first line of "008969" article, in the "prix_type_promo" field. I tested several solutions but when I manage to delete these lines, I lose others which I must keep.

Here is my csv structure :

Here is my job structure :

It is the framed flow which is concerned.

And here is my tMap structure :

Thank you to those who are willing to look into my problem.

Re: delete rows according to criteria

Anonymous — Wed, 11 Mar 2020 15:15:32 GMT

Hi,

I assume you want to make a join between your different data flows and keep the article having a "promo" field and exclude the duplicates.

You can bind your flows to your tmap, make joins on article code and write your specific use cases like

! row1.prix_type_promo.equals("promo")

More clear explainations are also welcome to help us understanding what's really your problem.

Re: delete rows according to criteria

gadje1 — Wed, 11 Mar 2020 16:12:05 GMT

Hi Pierre and thanks for your reply,

As i explained it in my firs post, i want to test if the field "prix_type_promo" have the value "Promo".

If it have this value ("Promo") i dont want to keep the lines of the same article (field "produit_code_article") that have any values in field "prix_quantite_minimum", like those that have some values like "50.000" or "100.000" for example.

In the case of article N° "008969", i want to drop only the lines N°11 and N°12. and i would like to keep all the rest for this article number.

"prix_type_promo" : 2

"prix_quantite_minimum" : 1

"produit_code_article" : 3

I don't want to test the lines values one by one but test if an article number(1) have a value "Promo" in field "prix_type_promo"(2) of my csv file. If it's the case, i don't keep the lines that are differents of null in field "prix_quantite_minimum"(1).

I tried different things but nothing works.

Re: delete rows according to criteria

Anonymous — Wed, 11 Mar 2020 16:46:53 GMT

OK i assume "promo" articles may come from your 3 db sources.

1/ Unite your 3 DB data and filter on "prix_type_promo".equals("Promo") and write them to a temporary file (Promo.csv)

2/ Redirect the reject flow (from your filter component) to a temporary file (notPromo.csv)

3/ Main flow is NotPromo.csv, use tmap and lookup on Promo.csv, make the good join in order to exclude the articles Matching the Promo articles and having prix_quantite_minumum not null.

i will try to give you explicit screen shots later.

Re: delete rows according to criteria

gadje1 — Thu, 12 Mar 2020 07:18:33 GMT

Ok Pierre,

Thanks for your reply.

I try what you tell me to do.

If you can give me some screens shots, it will be helpfull for me.

Thanks a lot.

Re: delete rows according to criteria

gadje1 — Thu, 12 Mar 2020 09:11:19 GMT

This is what i did fro trying :

My tMap structure is like that :

I add filter "out10.prix_type_promo.equals("Promo")".

But when i run my job there's a pointer exception error.

It's the filter that causing problem, i assume.

I continue my searchs.

Re: delete rows according to criteria

gadje1 — Thu, 12 Mar 2020 14:26:06 GMT

for those that will be interrested ...
I remaked all the job from start, it was somebody else that did it. I splited the main jobin three subjobs, like that :

1°) subjob tarifsDeBase :

2°) subjob quantitatif :

3°) subjob adherents :

This is my tMap structure for the output filter "!Var.PROMO.equals("Promo")" :

And this is my main job that use the subjobs :

If this can be usefull.