topic Re: [resolved] Need to convert JSON string to document for use in tRESTClient in Talend Studio

[resolved] Need to convert JSON string to document for use in tRESTClient

Anonymous — Tue, 24 Nov 2015 16:42:43 GMT

Hello,
In Talend Data Integration 5.6, I get the following error on compilation:
"Type mismatch: cannot convert from String to Document"
The context is this: I need to upload a JSON file with the following schema (note that the array "data" will contain up to 500 elements):
{
"data":
}
I was unable to get the tWriteJSON field component to format the array correctly, so I was using that to define the subdocument, then passing that into tDenormalize & tMap to generate a string containing the full, correctly formatted JSON.
However, I need to sent this to a REST API, but the tRESTClient component seems to expect a document type. How can I convert my string to a document?

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Anonymous — Tue, 24 Nov 2015 16:48:36 GMT

Send the JSON string to the "string" input and not the "body" of the tRestClient. JSON is not a Document.

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Anonymous — Tue, 29 Dec 2015 15:18:07 GMT

Thanks, this seems to have worked.

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Mahboob — Tue, 03 Jul 2018 10:59:14 GMT

@rhall suggestion worked for me. thanks.

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

LuciusHunt — Wed, 09 Sep 2020 15:27:56 GMT

This works for me:

tFixedFlowInput -> tXMLMap -> tRESTClient

And be aware: Never press "Sync Columns" when you have a tRESTClient Component nearby! This destroys the schema of the tRESTClient and you have to replace it with a new one!

tRestClient: Set Content Type to JSON and in Advance Settings make sure "Drop JSON Request Root" is selected!

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

sam2000 — Wed, 09 Dec 2020 13:56:02 GMT

Thanks @Holger Damczyk for the descriptive answer. It worked for me.

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Reda_Sdn — Mon, 16 May 2022 07:23:24 GMT

Hi @Richard Hall ,

I tried your solution, it works for me but I have a problem when the request sent in the body is too long (more than 4050 characters) the trestClient does nothing and returns an empty response.

thanks in advance

Reda

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Anonymous — Mon, 16 May 2022 09:36:34 GMT

Hi @Reda Saidani ,

Are you saying that when you send a JSON String to the "string" input column of the tRestClient it fails if it is larger than 4050 characters? Or are you saying that it fails if the URL is greater than 4050 characters? I've never seen either of these limitations, but I may not have tested them. So I'd like to try this to see.

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Reda_Sdn — Mon, 16 May 2022 09:43:41 GMT

when my String is less than 4050 characters it's work and i get the response but if it's greater than 4050 i have un empty response and any message and nothing happen in the API

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Reda_Sdn — Mon, 16 May 2022 09:47:03 GMT

this is the response for the 2 query

response just for 1 query wich is less than 4050 characters

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Reda_Sdn — Mon, 16 May 2022 12:06:45 GMT

@Richard Hall

do you have a solution for the trestClient to process all requests regardless of the size of the String (i'm using PATCH methode in trestClient)

thank's

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Anonymous — Mon, 16 May 2022 16:25:18 GMT

I have just carried out a test and have not seen the data being truncated. I have been able to send 1023878 characters using a tRestClient component. What web service are you using where you see this issue?

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Reda_Sdn — Mon, 16 May 2022 16:46:17 GMT

thanks for your feedback @Richard Hall

I shared my job above in the chat, im using an API rest to Patch data, in trestClient input I have a string in the format expected by the API.

the Strings are in a tfileDelimited, each line represents a request that I send to the API using the PATCH method to update the data.

Unfortunately the trestClient only executes requests that are less than 4050 characters long and for the longest nothing happens on the API side.

above an example of execution of my job with two string which exceeds 4050 characters

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Anonymous — Mon, 16 May 2022 21:57:32 GMT

Hi @Reda Saidani,

I'm not sure that this limitation is down to the tRestClient. As I said, I built a job using a tRestClient and sent a massive file as a String to another web service I had created using Talend. It accepted it. My suspicion is that the service you are calling has a limit on size. Obviously I cannot prove this conclusively without seeing the API specification of the service you are calling. This may be due to something else in your job. For example, your json string may have a character that is expected as a "new row" character or a column separator character in your tFileInput Delimited component. Have you checked this?

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Reda_Sdn — Tue, 17 May 2022 07:13:44 GMT

HI @Richard Hall

I tested the same query in postman and it works fine.

do you think that the trestClient can't read my string correctly?

I'm really stuck because of this problem

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Anonymous — Tue, 17 May 2022 21:41:22 GMT

You say that it errors with JSON that is over 4050 characters. Is that every JSON string OR is it one JSON string? Can you print the JSON string to the output window before you send it, copy it and then test it here....

https://jsonpathfinder.com/

If the JSON is good, it will reveal a breakdown of the JSON in the window on the right.

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Reda_Sdn — Tue, 17 May 2022 21:51:57 GMT

Hi @Richard Hall

I used this link to validate my JSON and it displays them in the desired format on the right.

each line of my file represents a request in json format which allows to update the data of a product.

unfortunately the trestClient does not return any response when the request is too long and it does not perform any update

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Anonymous — Wed, 18 May 2022 00:04:26 GMT

I'm not saying that you are not having issues somewhere, the problem is that I have no way of debugging at the moment. I tested sending JSON strings much greater in length than 4050 chars to a service and I couldn't see this issue. Maybe you can send me the JSON string that is causing this issue and I can take a look? I will private message you with my email and if you can share it, I might be able to see where the problem may be. It would also be good to know the API you are using.

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Anonymous — Wed, 18 May 2022 15:29:54 GMT

Thanks for sending me your example. Can you let me know more about your tFileInputDelimited? What are your settings there? Have you specified a suitable encoding (advanced settings)? It looks like you will need UTF-8. This could cause your service some issues.

Re: [resolved] Need to convert JSON string to document for use in tRESTClient

Reda_Sdn — Wed, 18 May 2022 15:38:42 GMT

hi @Richard Hall

here is my tFileInputDelimited

I can't move forward because of this problem.