topic Re: tREST http body problems using global variable in Talend Studio

tREST http body problems using global variable

Anonymous — Sat, 16 Nov 2024 09:29:10 GMT

Hello everyone

I have looked at previous questions on this topic but I suspect I may have a different issue with a global variable I am trying to pass into tREST http Body.

I am definitely doing something wrong in my Talend DI job.

The following fails:

http body in tREST calling ElasticSearch set like so:

((String)globalMap.get("myscrollid"))

Error from ElasticSearch results:

{"error":"ElasticsearchIllegalArgumentException[Failed to decode scrollId];

nested: IOException[Bad Base64 input character decimal 46 in array position 4]; ","status":400}~400

When I use the hard-coded value of the ScrollID in http Body and surrounded by double-quotes like so:

http Body:

"c2NhbjsxMDsxODQ1OTM5ODE5OndCZktsWUo1VHYta3RvdWZiUmMzWFE7MTg0NTkzOTgyMDp3QmZLbFlKNVR2LWt0b3VmYlJjM1hROzE4NDYyNzg2ODk6bkE2T0ZVVWtRcFdNMktLT0tmaTRUBBISHMjc4NjkwOm5BNk9GVVVrUXBXTTJLS09LZmk4M1E7MTg0NDQ0NjcyNzo4cE9xcjB0S1NfQ195M0VmdWpSMlFnOzE4NDQ0NDY3Mjg6OHBPcXIwdEtTX0NfeTNFZnVqUjJRZzsxODQ0MTA3MzM4OkZIdnkyaXEzUXMtSS1RR0htRTlnRlE7MTg0NDMwNjE1MTpTelRndzJmN1J2Q0JTWlRNdUpRaVNBOzE4NDU2MDkyOTI6Q0xzTllsVl9TdmF5em9RVW1lOWhfdzsxODQ0ODkzMDg5OjJTTkpRNUd0U1MyWlg3ekNQWmFMd0E7MTt0b3RhbF9oaXRzOjEzNzIwNTs="

Then it works.

So I know the software on client and server is OK and it must be my error in Talend DI job.

I am doing something wrong passing the value stored in global variable to http body.

I have verified the value of global variable by using tLog and writing the value to csv file, so I am sure the value itself is OK, since I used it as hard-coded value to prove it works.

Somehow the way I extract that value programmatically into http body upsets ElasticSearch !

Thanks for any tips and clues!

Re: tREST http body problems using global variable

Anonymous — Wed, 26 Jul 2017 08:46:29 GMT

I suspect that this may be relatively straight forward, but we will need more info to give you an answer. Can you post a couple of screenshots showing your job, where you set the globalMap variable and you tRest config please?

Re: tREST http body problems using global variable

Anonymous — Wed, 26 Jul 2017 08:50:52 GMT

I think I solved it, but need to confirm with you.

in a Key - Value for the global variable I had this in Value: "row5.scroll_id" - i.e. surrounded by double quotes "".

Because it is expression I had to remove those double quotes like so: row5.scroll_id

It seems to be working now, but please confirm this for me so others may also benefit from my Learner mistakes ! 8^) LOL

I was passing scroll_id string value of "row5.scroll_id" to the http body of tREST, it seems.

Re: tREST http body problems using global variable

Anonymous — Wed, 26 Jul 2017 09:01:27 GMT

I'm not sure what you have done here. Are you still referring to a globalMap variable or are you just referring to the row value? Can you take a screenshot please?

Re: tREST http body problems using global variable

Anonymous — Wed, 26 Jul 2017 09:09:42 GMT

in designer in Key-Value
I have "gbl_varname" in Key (in double-quotes) and "row5.scrollid" in Value in double-quotes.
I should have had rows5.scrollid in Value *without* double-quotes

Re: tREST http body problems using global variable

Anonymous — Wed, 26 Jul 2017 09:20:41 GMT

I think I see. So you went from this....

globalMap.put("gbl_varname", "row5.scrollid")

....to this....

globalMap.put("gbl_varname", row5.scrollid)

If so, that explains everything. Putting quotes around text makes it a String Literal in Java.

Re: tREST http body problems using global variable

Anonymous — Thu, 27 Jul 2017 00:27:28 GMT

yes, exactly, and I did not see this until I examined the generated Java code

which means that even if you do not write any Java code to be able to work with Talend you still need to be able to read and understand Java code.

Re: tREST http body problems using global variable

Anonymous — Thu, 27 Jul 2017 00:39:42 GMT

Yes, you do need Java.....which is no bad thing in my opinion. While some people wishing to sell development products like to be able to claim "my grandmother can use this", it really helps nobody when you end up with a "development team" who do not know how many bits are in a byte 🙂