Hi Talend experts
I have got below job which reads next API URL until it finds one also it iterates for different course_ids. I have tweaked the design a bit to run parallel execution(the idea is to use parallel execution option of iterate) to execute multiple calls at a time.
StoreCourseID:
//globalMap.put("canvas_id", row1.canvas_id);
globalMap.put("V_API_URL" + row1.canvas_id, "
https://swinburneonline.instructure.com/api/v1/courses/"+ row1.canvas_id +"/analytics/student_summaries?per_page=100");
globalMap.put("V_LOOP"+ row1.canvas_id, true);
tLoop:
((Boolean) globalMap.get("V_LOOP"+ row1.canvas_id))
tRestClient:
((String) globalMap.get("V_API_URL" + row1.canvas_id))
GetNextUrl
System.out.println("Current URL IS: "+globalMap.get("V_API_URL"+ row1.canvas_id)); (prints correctly)
System.out.println("Rest URL"+ globalMap.get("tRESTClient_1_HEADERS")); (doesn't print and errors out to null)
java.util.List <STRING> strList = ((java.util.Map<STRING>>
globalMap.get("tRESTClient_1_HEADERS")).get("Link");
//System.out.println("Link Header URL is:"+ strList);
java.util.List <STRING> new_list = java.util.Arrays.asList(strList.get(0).split(","));
int foundIndex = -1;
String next_url="";
for(int i=0; i < new_list.size(); i++)
{
if(new_list.get(i).indexOf("rel=\"next\"") != -1)
{
System.out.println("Item Found...");
foundIndex = i;
break;
}
}
if (foundIndex != -1)
{
String[] found_item = new_list.get(foundIndex).split(";");
next_url=found_item[0].substring(1,found_item[0].length() - 1);
System.out.println("Next URL is: " + next_url);
globalMap.put("next_url"+row1.canvas_id,next_url);
}
else
{
System.out.println("No Next URL found...");
globalMap.put("V_LOOP"+ row1.canvas_id,false);
}
SetNextURlToCurrUrl:
if ((Boolean) globalMap.get("V_LOOP"+row1.canvas_id))
{
System.out.println("URL1 IS: "+globalMap.get("V_API_URL"));
globalMap.put("V_API_URL"+row1.canvas_id,globalMap.get("next_url"+row1.canvas_id));
System.out.println("URL IS: "+globalMap.get("V_API_URL"));
}
So it makes calls multiple times as an example below:
https://abc.com/api/v1/courses/course_id1/analytics/student_summaries?per_page=100&page=1 https://abc.com/api/v1/courses/course_id1/analytics/student_summaries?per_page=100&page=2 https://abc.com/api/v1/courses/course_id1/analytics/student_summaries?per_page=100&page=3 https://abc.com/api/v1/courses/course_id2/analytics/student_summaries?per_page=100&page=1 https://abc.com/api/v1/courses/course_id3/analytics/student_summaries?per_page=100&page=1 https://abc.com/api/v1/courses/course_id3/analytics/student_summaries?per_page=100&page=2 https://abc.com/api/v1/courses/course_id3/analytics/student_summaries?per_page=100&page=3 https://abc.com/api/v1/courses/course_id3/analytics/student_summaries?per_page=100&page=4
So in doing so I have run into problem where Rest URL is always getting NULL when I get it from GetNextURL (tjava) component. Not sure what's wrong. Any help is really appreciated!
Thanks
Harshal.</STRING></STRING></STRING>