topic How to handle errors with tHttpRequest in Talend Studio

How to handle errors with tHttpRequest

JamesR1 — Tue, 15 May 2018 00:58:03 GMT

I am posting to a URL that returns an error 400 Bad Request however I cannot seem to get access to the actual http response body.

If I send tHttpRequest to a tLogRow, it just logs 400 Bad Request.

=== begin console output

Starting job HandlingErrors at 16:46 14/05/2018.

[statistics] connecting to socket on port 3742

[statistics] connected

Exception in component tHttpRequest_1

java.lang.RuntimeException: Request failed:400 Bad Request

at thrust2bq.handlingerrors_0_1.HandlingErrors.tHttpRequest_1Process(HandlingErrors.java:673)

at thrust2bq.handlingerrors_0_1.HandlingErrors.runJobInTOS(HandlingErrors.java:1011)

at thrust2bq.handlingerrors_0_1.HandlingErrors.main(HandlingErrors.java:868)

[statistics] disconnected

Job HandlingErrors ended at 16:46 14/05/2018. [exit code=1]

==== end console output

Using Postman

If I use postman, it returns the body of the response.

{
"status": "SM_ERROR",
"errors": [
{
"error": "BAD_REQUEST",
"message": "Friendly Error Message Here"
}
],
"requestkey": ""
}

I have even tried putting a tExtractJSONFields after the tHttpRequest but the ResponseContent is never filled in because of the "error" condition.

So the question is: How can I get the Response Body content from the tHttpRequest regardless if there is an error condition (non-200 Http Status)?

Ultimately, what I am trying to do is:

I have a file thousands of rows of post data that I need to send one at a time to a URL (API)
Parse the JSON response and IF error, write the response body to a log file. If success, extract a key value from the response body and write to a success file.

If there is NO error, then everything works fine. But as soon as the error happens, processing stops. FWIW I did uncheck the "die on error" box but is seems that as soon as an error occurs, the tExtractJSONFields stops working and I no longer get any successes written to my success log file.

I am using Talend Open Studio for BigData 6.3.0 on macOS 10.12.6

Anywho, thank you for any assistance you can provide.

Re: How to handle errors with tHttpRequest

markdjenkins — Wed, 16 May 2018 18:36:26 GMT

I'm facing the exact same issue. I get an error 400, but I'd like to see the actual response so I understand why.

Re: How to handle errors with tHttpRequest

Jesperrekuh — Wed, 16 May 2018 21:01:38 GMT

Could you make a screendump of your job flow, the url itself and the data.

A common mistake is due to:
1 - incorrect headers. your endpoint provider should give you the specs to make correct call.

2 - GET just to get a response based on endpoint with formvars to be able to query data. often mistaken with POST

3 - PUT (filedata), when you want to upload a lot of shizzle.

4 - POST (like web-form vars ?startDate=2015-01-02 ) , make sure your headers are set ok!

5 - Incorrect url and data ... spaces not escaped special chars etc etc etc...

On server side its trying to parse your request (your composed url). And with a 400 bad request make sure step 1 is correct.

Use a tRestClient which is really flexible in handling response, the good and the bad.

Good luck!

Re: How to handle errors with tHttpRequest

JamesR1 — Wed, 16 May 2018 21:31:47 GMT

Thank you for responding.

Screenshots added. I obfuscated the URL.

FWIW, calls to the API work fine until there is a non-200 http status code.

The postdata looks like:

userId=38340824&streamId=6481006509&rateId=8135506&chargeSource=HybridClientFix&amount=300

No spaces in the POST parameters

Complete URL is:

https://api.url.com/v1/charge/charge?

userId=38340824&streamId=6481006509&rateId=8135506&chargeSource=HybridClientFix&amount=300

WorflowHttpRequest headers

Re: How to handle errors with tHttpRequest

Jesperrekuh — Wed, 16 May 2018 22:38:19 GMT

Could you please provide the wrong url which result in the 400 bad request?
The url you post looks allright to me.

Although your job ... I suggest change :

[write post data] ------ ROW2 -----> [Call billing api]

into

[write post data] ------ On Subjob Ok -----> [Call billing api]

Maybe the post data file isn't ready and sends no parameters

Last suggestion because of money .... make sure the amount is in the correct notations decimals. thousands etc... as clean as possible

Re: How to handle errors with tHttpRequest

JamesR1 — Wed, 16 May 2018 22:46:43 GMT

the post data I identified in the previous comment contains the data that produces the 400. With the JSON in my original message in the HTTP response.

FWIW, I even tried manually creating the postdata.txt file and feeding that the tHttpRequest and get the same error.

Amount field is purposefully sent as cents (no decimal).

I will try your suggestion on the workflow --> onSubJobOk but I am thinking the issue lies in the tHttpRequest component treating any non-200 http status code as an error and NOT making the http response body available.

Have you successfully been able to retrieve the HTTP response body when the http status != 200?

Re: How to handle errors with tHttpRequest

markdjenkins — Wed, 16 May 2018 23:18:41 GMT

I agree. I played with it for a few hours and I could never make the response body available when the status code was non-200.

I eventually gave up and migrated to using the tRest component to do my POST request. It provides a <null> status code when a 200 is given and a valid status code when anything other than 200 is given. This in turn allowed me to use an 'if' statement against that value to determine success or failure (yet still having the response body to log or handle as necessary).

Re: How to handle errors with tHttpRequest

Jesperrekuh — Wed, 16 May 2018 23:38:47 GMT

tRestClient is a better solution and provide this ... but however... I did more or less the same as you, look at attachment

REST.jpg