topic Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression. in Talend Studio

Need help to use globalMap's column name's VALUES in tMap Filter Expression.

DHazari1598547528 — Sat, 16 Nov 2024 01:34:34 GMT

We are using tMap for the following:

1. We are joining source and target table on key columns source.id= target.id

2. We are using globalMap to store the name of some columns for both the source and target.

3. Global Map contains the following values:

source_hash_column > hash_key

3. We want to use the column name stored in the globalmap, in the Filter Expression of tMap output.

(source.((String)(globalMap.get("hash_key")) == null)

4. Now issues are:

a) If we use it the way mentioned in point 3, it gives compilation error: Syntax error on token ".", Identifier expected after this token.

b) If we use concatenation like below, then its treating column names itself as String and Not the actual value stored in that column.

(source + "." + ((String)(globalMap.get("hash_key")) == null)

Can someone please suggest, how can we use globalMap's column name's values in tMap Filter Expression?

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

manodwhb — Fri, 28 Aug 2020 06:22:26 GMT

@Deepti Hazari , can you show your job design where are you setting a global variables and read it?

Thanks,

Manohar

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

DHazari1598547528 — Fri, 28 Aug 2020 15:14:56 GMT

Thanks @Manohar B for your reply. Here is a detailed explanation with the snapshot.

Since I can't share the confidential Project Design, here is sample example to replicate the issue.

In this snapshot, there are :

a) 2 inputs row1 and row2 for tMap

b) a left join between them.

c) tMap join output"out1" with filter expression where target key column is null.

d) globalMap is already populated with the values : "source_key_column" >> EMP_ID and "target_key_column" >> EMP_ID.

Now, in usual case, to achieve this, we would have just dragged the key column from row2 to the expression filter with null check such as

row2.EMP_ID == null

But, we are trying to parameterize the column name "EMP_ID" here by substituting it from globalMap. Now there are 2 options here:

1. If we are using it as below, it is throwing compilation error "it gives compilation error: Syntax error on token ".", Identifier expected after this token."

row2.globalMap.get("target_key_column") == null

2. If we are using it as below, then it is actually comparing the String value "row2.EMP_ID" and not the value stored within the EMP_ID column of row2.

row2 + "." + globalMap.get("target_key_column") == null

so please suggest, how should I use the parameterized column name in the tMap filter expression?

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

manodwhb — Fri, 28 Aug 2020 15:18:22 GMT

@Deepti Hazari ,Where are you put that variable in global map? since your getting the variable in tMap

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

DHazari1598547528 — Fri, 28 Aug 2020 15:20:16 GMT

@Manohar B before this tmap, I'm using tJavaRow to populate the values in globalMap.

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

manodwhb — Fri, 28 Aug 2020 15:23:14 GMT

@Deepti Hazari ,in tJavarow you can print that global variable and check what are you getting?

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

DHazari1598547528 — Fri, 28 Aug 2020 15:27:13 GMT

@Manohar B , I have already tried that and I am getting the correct value "EMP_ID" when I print globalMap.get("target_key_column"), but he problem is that when I am using it with row2.globalMap.get("target_key_column") in the Filter expression , then it is not treating it the same way as it would treat row2.EMP_ID.

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

manodwhb — Fri, 28 Aug 2020 15:33:49 GMT

@Deepti Hazari , When you have multiple records you need to iterate record by record using tFlowtoIterate.

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

manodwhb — Fri, 28 Aug 2020 15:40:03 GMT

@Deepti Hazari ,And one more thing that when you put you need to use the below way right.

((String)globalMap.put("myObject", row2.col)

((String) globalMap.get("myObject"))

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

DHazari1598547528 — Fri, 28 Aug 2020 15:42:48 GMT

ok @Manohar B . I will try this option of having "row2" concatenated while putting the value in the globalMap itself

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

DHazari1598547528 — Fri, 28 Aug 2020 15:45:29 GMT

Not sure though if it would just take that as a String or substitute it with the actual value within that col in the Filter Expression.

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

DHazari1598547528 — Fri, 28 Aug 2020 16:22:03 GMT

@Manohar B : I tried using the approach by having row2 concatenated in the globalMap value itself, but still it is just treating that value as a String in the Filter Expression and not substituting the actual value within that column.

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

manodwhb — Sat, 29 Aug 2020 02:34:30 GMT

@Deepti Hazari , I really not sure how you have designed and expecting to work. but i have create a simple job to check the globalvar filter in tMap. check the below one. it was working as expected.

Thanks,

Manohar

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

DHazari1598547528 — Sat, 29 Aug 2020 03:08:23 GMT

Thanks @Manohar B , but unfortunately i cant see the screenshot..can you please try to attach it again

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

manodwhb — Sat, 29 Aug 2020 03:34:08 GMT

@Deepti Hazari ,here is the job attached.

Please like and select as best the correct replay.

Thanks,

Manohar

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

DHazari1598547528 — Sat, 29 Aug 2020 06:15:29 GMT

Thanks a lot @Manohar B

I think I got an idea from your attachement to fix my issue. But can you please suggest for this as well.

I can see in your tJavaRow code, you have used the below so that 1 row from the input to tJavaRow is mapped to its output.

"output_row.col1 = input_row.col1;"

So, what if I have many input columns and I want to do this for all the columns and get all the input columns mapped to the tJavaRow's output columns without writing the mapping for each column individually.

Can you please suggest a short design/ code statement for this?

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

manodwhb — Sat, 29 Aug 2020 07:01:06 GMT

@Deepti Hazari , Once you connect from previous component to tJavarow. click on Sync columns and then click on Generate code.what ever the columns you have in tJavaRow_1 output schema will generate automatically.

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

DHazari1598547528 — Sat, 29 Aug 2020 07:15:13 GMT

Thank you @Manohar B , yup even I thought about syncing col. Thanks for your time 🙂.

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

manodwhb — Sat, 29 Aug 2020 07:17:04 GMT

You are welcome @Deepti Hazari .

Re: Need help to use globalMap's column name's VALUES in tMap Filter Expression.

madhukolli2003 — Tue, 11 May 2021 14:17:50 GMT

Hi @Deepti Hazari ,

I am having same issue to pass column as global variable in tmap and filter source data.

Can you please share how you have composed global variable and used it in tMap ?

@Manohar B -- Can you please share your job if you still have it, looks like it's delete from this ticket.

Thanks & Regards,

Madhu Kolli