How to loop the trestclient to reach the end of the api page

[Anonymous]

when i start with the url:"https://flow.pluralsight.com/v3/customer/core/repos/?offset=0"

it gives the data of 100 records like

{

     "count" :   1584 ,

     "next" :   "https://flow.pluralsight.com/v3/customer/core/repos/?limit=100&offset=100" ,

     "previous" :   null ,

     "results" :   [

         {

             "id" :   1 ,

             "project_id" :   1 ,

             "added_by" :   {............... and so on

                                }

                       ]

         }

}

in this it also gives the next page value as next

untill next page value becomes null i want to loop my trestclient wit the next url

currently i am just incrementing my offset value in the url

like given in the image

[Anonymous]

You can do this with a tLoop component. I will give you the high level solution as I don't actually have access to the API. But even though this isn't an actual solution, it is very easy to implement if you understand my description. Any questions, just come back to me.

First of all, you need to set a globalMap key/value pair to be your initial URL. You can do this in a tJava that is linked to your tLoop via an OnSubJobOK link. The code you will use will be something like this....

globalMap.put("myUrl","https://flow.pluralsight.com/v3/customer/core/repos/?offset=0");

Then in the tLoop, set it up as a WHILE Loop. Set the following as your condition. Do not worry about the numeric parameters...

globalMap.get("myUrl")!=null

Then connect your tLoop via the iterate link to a tJava component. This is a dummy component to allow you to link to the tRestClient. The tRestClient will not allow you to connect it via an iterate link. Then connect your tRestClient to the tJava using a OnComponentOK link.

When you get to this point you set the URL of the tRestClient to be the "myUrl" value. This can be done using the code below...

((String)globalMap.get("myUrl"))

Now the clever logic comes just before your tFileOutputExcel component. You need to make sure you have extracted the "next" URL value from your JSON first. If you have done this, then you know whether there is another URL. What you do is add a tJavaFlex before the tFileOutputExcel and use some Java to either set the "myUrl" value to the next link OR to null. If the link is not null, the loop will fire again. If the link is null, then the loop will stop.

Set the "Data Auto propagate" to be ticked and add code similar to below to the Main Code....

String url = row1.next;

if(url!=null){
       globalMap.put("myUrl", url);
}else{
       globalMap.put("myUrl", null);
}

This should work for you. It may need some tweaking depending on how your API works, but it is a high level solution that I have used on many an occasion.

[Anonymous]

unable to connect tRestClient to the tJava using a OnComponentOK link.

thats why i passed as main component

trestclient gives (status ,body and string) and in the string i have my next url value how to extract it and pass it to the tjavaflex

tjava component:

 and it is also showing  an error that duplicate local variable url

[Anonymous]

Do you need to pass the string or body column values to the tREstClient? If not, you can do it as I suggested. That is perfectly OK. If you do need to pass either of those column values all you need to do is use a tFixedFlowInput between the tLoop and the tRestClient. I wasn't thinking that you might be passing some sort of data to your tRestClient.  

Regarding your error, it is very difficult to tell from the image. Can you click on the "Code" tab in the bottom left corner of the screen and look for where the red blocks are on the right of the screen? If you click on those, it will take you to the Java that is incorrect. That's a really nice way of finding issues.

[Anonymous]

in the below image

next cannot be resolved or is not a field

error2:

Duplicate local variable

[Anonymous]

The code in your first image is erroring because there is no column called "next" coming from the tJava. You do not need this code and you do not need the tJava component. Simply simply link from the tLoop to the tRestClient as I suggested. 

The code you have in the tJava should be in the tJavaFlex after you have extracted the "next" value from the JSON. There should be a column called "next" which will hold this value. However, if you called it something different, you can simply change the name of the variable.

The other error I am not certain about. It looks like it is coming from your tRestClient config. Can you show me a screenshot of that please?

[Anonymous]

i have put the tjava code to the tjavaflex but also the same error

and here is the output in which i have the next data field

in this image i wanted to show you that i am getting the next value in the output

the out put of the trestclient

where status code is 200

body is null

and the string data contains the json data

like :

{

"count":1584'

"next":"https://flow.pluralsight.com/v3/customer/core/repos/?limit=100&offset=100",

"previous":null,

"results":[{.....}}

}

[Anonymous]

Can you show me your new job. I will need to see the configuration of the tRestClient, the tJava and the tJavaFlex please.

[Anonymous]

hi have deactivaed and unlinked the tloop and tjavaflex then i am able to get for single component for the code that you have given previously

and i think the problem may be in the tjavaflex component

and the tloop component i havent given any declaration and iteration only the condition is given as

globalMap.get("myUrl")!=null

in the tjavaflex i have given the same code of yours but the row number is only changed

String url = row37.next;

if(url!=null){
       globalMap.put("myUrl", url);
}else{
       globalMap.put("myUrl", null);
}

[Anonymous]

@mukeshmsp96 I'm sorry to be difficult, but it is very important that I see the job in it's failed state with all connections, no deactivated components and all of the code in the tJava and tJavaFlex. Things like row names and column names are important here.

Also, I notice that you don't appear to have a tJava linked by OnSubJobOK to the tLoop. It should be....

tJava

|

|
tLoop

In the tJava you need to set your initial url in the globalMap used in the tLoop.