Paginated API call and writing to DB

[Parikhharshal]

Hi there

I want to achieve something like this:

1. Read data from API using paginated URL (check all pages and keep reading)

2. Make URL dynamic by reading IDs (https://abc.com/api/v1/courses/<Course_id>/analytics/student_summaries)

3. Write to Redshift DB/File

I know there have been few threads regarding same but I want 2 loops. And would appreciate if someone can share some design code on this.

Urgent response is really appreciated!

Thanks

Harshal.

[Parikhharshal]

Hi there

I continued doing what was mentioned in the earlier post:

https://community.talend.com/t5/Design-and-Development/Iterative-Data-extraction-Pagination-and-Poll...

However, I seem to stuck at TRestClient where it keeps giving me nullpointerexception Java error.

Not sure if the problem is with this component though as I can see that in Tjava component I had println but nothing is getting printed there.

Is there anything I should be looking at?

In Tjava I got below code:

globalMap.put("VApiUrl",row4.NextUrl);

 System.out.println("Value of Globar Var: "+globalMap.get("VApiUrl"));

In TrestClient I have got config like this:

((String)globalMap.get("VApiUrl"))

Can you please share your job code if possible?

Thanks

Harshal.

[root]

Can u post the job? Otherwise its difficult to pinpoint the issue

[Parikhharshal]

@mbocquet: I have already posted job picture above. Please have a look. Thanks.

[Anonymous]

You are setting this key before ever calling the api.

globalMap.put("VApiUrl",row4.NextUrl);

 It's always going to be null for the first call if your job is designed in this fashion.

[Parikhharshal]

@evansdar: Thanks a lot for your reply. So can you please suggest how it should be? Would appreciate if you can guide me on this or share few screenshots/examples you might have.