[resolved] Need to convert JSON string to document for use in tRESTClient

[Anonymous]

Hello,
In Talend Data Integration 5.6, I get the following error on compilation:
"Type mismatch: cannot convert from String to Document"
The context is this: I need to upload a JSON file with the following schema (note that the array "data" will contain up to 500 elements):
{
 "data":
}
I was unable to get the tWriteJSON field component to format the array correctly, so I was using that to define the subdocument, then passing that into tDenormalize & tMap to generate a string containing the full, correctly formatted JSON.
However, I need to sent this to a REST API, but the tRESTClient component seems to expect a document type. How can I convert my string to a document?

[Reda_Sdn]

@Richard Hall​ 

do you have a solution for the trestClient to process all requests regardless of the size of the String (i'm using PATCH methode in trestClient)

thank's

[Anonymous]

I have just carried out a test and have not seen the data being truncated. I have been able to send 1023878 characters using a tRestClient component. What web service are you using where you see this issue?

[Reda_Sdn]

thanks for your feedback @Richard Hall​ 

I shared my job above in the chat, im using an API rest to Patch data, in trestClient input I have a string in the format expected by the API.

the Strings are in a tfileDelimited, each line represents a request that I send to the API using the PATCH method to update the data.

Unfortunately the trestClient only executes requests that are less than 4050 characters long and for the longest nothing happens on the API side.

above an example of execution of my job with two string which exceeds 4050 characters

[Anonymous]

Hi @Reda Saidani​,

I'm not sure that this limitation is down to the tRestClient. As I said, I built a job using a tRestClient and sent a massive file as a String to another web service I had created using Talend. It accepted it. My suspicion is that the service you are calling has a limit on size. Obviously I cannot prove this conclusively without seeing the API specification of the service you are calling. This may be due to something else in your job. For example, your json string may have a character that is expected as a "new row" character or a column separator character in your tFileInput Delimited component. Have you checked this?

[Reda_Sdn]

HI @Richard Hall​ 

I tested the same query in postman and it works fine.

do you think that the trestClient can't read my string correctly?

I'm really stuck because of this problem

[Anonymous]

You say that it errors with JSON that is over 4050 characters. Is that every JSON string OR is it one JSON string? Can you print the JSON string to the output window before you send it, copy it and then test it here....

https://jsonpathfinder.com/

If the JSON is good, it will reveal a breakdown of the JSON in the window on the right.

[Reda_Sdn]

Hi @Richard Hall​ 

I used this link to validate my JSON and it displays them in the desired format on the right.

each line of my file represents a request in json format which allows to update the data of a product.

unfortunately the trestClient does not return any response when the request is too long and it does not perform any update

[Anonymous]

I'm not saying that you are not having issues somewhere, the problem is that I have no way of debugging at the moment. I tested sending JSON strings much greater in length than 4050 chars to a service and I couldn't see this issue. Maybe you can send me the JSON string that is causing this issue and I can take a look? I will private message you with my email and if you can share it, I might be able to see where the problem may be. It would also be good to know the API you are using.

[Anonymous]

Thanks for sending me your example. Can you let me know more about your tFileInputDelimited? What are your settings there? Have you specified a suitable encoding (advanced settings)? It looks like you will need UTF-8. This could cause your service some issues.

[Reda_Sdn]

hi @Richard Hall​ 

here is my tFileInputDelimited

I can't move forward because of this problem.