curl (method: GET) performed in Talend: is possible to pass "-data" parameter with a tREST or tRESTClient component?

[Lorenzo5]

Hi,

I have to perform a cURL with Talend, and I would know if tREST or tRESTClient are useful for that, instead of a generic tSystem Component.

curl -X GET <url>/<path> -d '<json-data>'

My curl is aimed to perform a GET request, pass a "data" (json) parameter and retrieve a json response.
It needs also to be authenticated.

Well, speaking about authentication, tREST / tRESTClient are great, the work fine (tREST permith to specify headers good for authentication, and tRESTClient works perfectly because is fully configurable in terms of authentication method).

My problem is that my curl have to pass json data:
curl -X GET <url>/<path> -d '<json-data>'
and I don't know how to pass this data (body) since I have to perform a GET (not a POST).

Any help?

Thank you,
Lorenzo

[Anonymous]

If you pass the JSON Body as a payload document to tRestClient, does it not work?  You will pass the body as a row from a previous component.

[Lorenzo5]

Hi @iburtally

Unfortunately it doesn't work.

I put a tFixedFlowInput as previous (of tRESTClient) component, to pass the body, but the tRESTClient seems to ignore it.

Since the Input Schema of the tRESTClient has originally two fields, "body" of type Document and "string" of type string, I have tried to:

- fill the "string" input field with my json string body and leave empty the "body" field

- fill the "string" input field with my json string body and delete the "body" field

but both of the attempt failed (tRESTClient component does not consider the input field "string", when calling the REST server)

Any idea?

[Lorenzo5]

Mmmmm...

I noticed that when you perform a GET with tRESTClient component you are not able to specify the ContentType of your request (the field disappear).

That probably means the component isn't able to send a payload in GET, is it?

So, maybe the focus should be on my original question: how to perform a curl, in GET, specifing the "data" (-d or -data, for linux) parameter? 

For this purpose tSystem component is very unfriendly because any complex command need to be managed with the "Use Array Command" (tSystem Component option) that, unfortunately, split on spaces and my json payload is full of spaces 😞

[Anonymous]

Normally, when you do a GET, you will pass query parameters.  I have not come across the ones that require a body data.  If it doesn't work, then yes, tSystem is your best bet right now.  Generally when passing JSON data like this, I have seen other services looking for a Base64 encoded string.  You encode your json as base64 and the service decode it, just to avoid that spaces problem etc.  Maybe you should try a URL Encode on the JSON data.

[Anonymous]

Use the HTTP Headers to set this on the Advanced Settings tab of the tRESTClient. The name will be "Content-type" and the value will be "application/json".

[Lorenzo5]

Hi @rhall

as you can see, the problem is not the content-type itself.

The problem is that when you use tRESTClient with HTTP Method = GET you have no way to specify a payload (the fields that comes from the Input Link, "body" and "string", seem to be ignored by the tRESTClient).

So, my question is:

is it possible to use tRESTClient to perform the following curl, with GET http method and with a specific "data"?

curl -X GET <url> -d "<json-payload>"

Thank you,

Lorenzo

[Lorenzo5]

Hi @iburtally

thank you for your answer. I'm sending request to an ElasticSearch service.

ElasticSearch, as most of the other Restful Services, require to use GET to get some data ("_search" api) and PUT, POST, ... to write some data.

And yes, when you search for some data you have to specify query-constraints and/or specific aggregation rules, you want Elasticsearch applies.

On the other hand, infact, the unix "curl" let you specify "--data" (or "-d") parameter also when you perform a GET request.

Thank you for your help. tSystem is also a little complicated to configure, I'm trying and I'm getting some trouble. There is not, unfortunately, a good documentation (with various use-cases) on Talend Components.

Lorenzo

[Anonymous]

@lorenzolucioni I was answering your sub question about the Content-Type as @iburtally had answered your main question....or at least attempted to.

For your main question, maybe you can try using something like below....

http://Your_uri?data={your: "json"}

You will probably have to encode your JSON to be compatible with a URL.

I haven't tested this since I don't actually use GET REST services that require a body instead of params, but this *may* work.

Unfortunately with web services, each implementation is usually subtley different to the next. You cannot rely on ANY tool catering for all of the cases out of the box and there is usually some jiggery pokery involved in trying to get them to work. I am currently playing around with the Spotify API using OAuth authentication. I've used dozens of OAuth secured services, but this one is proving to be a real pain. 

[Anonymous]

Hello,

Please could Talend community can answer to this question ?

 How can we query elasticsearch from Talend, no way to do that on tRest or tRestCleient ?

My simple query is like that: curl -XGET "http://localhost:9200/_search" -H 'Content-Type: application/json' -d'
{
"query": {
"multi_match" : {
"query": "VIEILLES",
"type": "best_fields",
"fields": ["doc.*_NORMALISEE"]
}
}
}'

I cross that I will pluck my hair !!