topic Re: How to get response header in tRestClient in a GET response in Talend Studio

How to get response header in tRestClient in a GET response

EspenH — Fri, 21 Mar 2025 15:18:38 GMT

We use tRestClient to connect to an api.

In the response we get statusCode, body and string, but the source uses pagination and this is returned in the respose hearder.

In Posteman in the console i find the response header:

▶Response Headers

Date: Fri, 21 Mar 2025 15:03:28 GMT

Content-Type: application/json

Content-Length: 3562

Connection: keep-alive

x-amzn-RequestId: xxxxxxxxxxxxxxxx

Access-Control-Allow-Origin:

Content-Encoding: gzip

x-amz-apigw-id: xxxxxxxxxxxxxxxxxxxx

X-Trace-Id: xxxxxxxxxxxxxxxxxxxxxxx

link: <https://rest.api.xxxxxxxxx.com/v1/transactionlines?dateFrom=2024-02-01&dateTo=2024-02-02&page=2>; rel="next", <https://rest.api.xxxxxxxx.com/v1/transactionlines?dateFrom=2024-02-01&dateTo=2024-02-02&page=82>; rel="last", <https://rest.apixxxxxxxxxxcom/v1/transactionlines?dateFrom=2024-02-01&dateTo=2024-02-02>; rel="self"

X-Amzn-Trace-Id: Root=xxxxxxxxxxxxxxx;Sampled=1;Lineage=1:c1d6dcf4:0

Access-Control-Allow-Credentials: true

In this response header i need to get information in the "link" part and more specific information about next page and total number of pages (both in red text above).

Do any of you experts know how to do this in a Talend? Is it possible through a tRestClient or do I need to user another component?

Re: How to get response header in tRestClient in a GET response

quentin-vigne — Mon, 24 Mar 2025 09:10:59 GMT

Hello @EspenH

We implemented a tRest component (but it might also work with a tRestClient) then in the job we linked it to a tExtractJSONFields

Depending of the structure of your response header you'll have to either use Xpath or JsonPath

In our case the response header is a body without structure so we use Xpath without a loop.

The mapping of tExtractJSONFields looks like this :

Then for your problem I guess you'll have to do string extraction to get the number after "page=".

To do that you will have to split the link in 2 columns (one for current link and one for link with max pages)

and for each you'll use String.lastIndexOf("page=") which will give you the index of where to start the substring and then you can use String.substring("yourstring", yourstring.lastIndexOf("page=") +5, 7)

The only problem I can see is if you don't know how many characters to get ? It can be 1, 2 or 3 so you'll have to probably get the index of ">;" which is just after your number and then substract the index this give you (for example 50) with the index you got before (for example 48) which in the end will give you only what you need : the page number. The end function should look like this

row1.link.substring(row1.link.lastIndexOf("page=")+5, row1.link.lastIndexOf(">;")-row1.link.lastIndexOf("page=")+5)

Tell me if you need more help

- Quentin

Re: How to get response header in tRestClient in a GET response

EspenH — Thu, 27 Mar 2025 07:31:59 GMT

Hi Quentin

Thank you for feedback, your code was very helpful, but I choose a slightly different approach.

I created a subjob to "get number of pages". I used a tJaveRow component connected to the tRestClient and got the header information. I splitt this in two and used part of your code to get the total number of pages.

Code in tJavaRow:

// Get header

context.vRestHeader = ((java.util.Map<String,java.util.List<String>>)globalMap.get("tRESTClient_2_HEADERS")).get("link").get(0).toString();

// Splitt header to get the part of the string with total number of pages

context.vRestHeader2 = context.vRestHeader.substring(context.vRestHeader.indexOf("rel")+10, context.vRestHeader.length());

// Get number of pages

context.vLastPage = Integer.valueOf(context.vRestHeader2.substring(context.vRestHeader2.indexOf("page=")+5, context.vRestHeader2.indexOf(">;")));

I stored this in context variables.

I connected this subjob to a new subjob "Get Rows" where i started with a tLoop to loop all values from 1 to total number of pages (= context.vLastPage). I connected the tLoop to a tJava to get the current iteration and stored this in a new context variable and then connected a new request to the api through a tRestClient for each iteration.

Code in tJava:

context.vNextPage = ((Integer)globalMap.get("tLoop_1_CURRENT_VALUE"));

System.out.println("Page : "+context.vNextPage);

The context variable context.vNextPage is used in the connected tRestClient to request eache iterated page.