Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi @all 🙂
I am new at talend and struggling with following issue. Please be lenient with me.
The main intention is to make multiple request to a graphql interface with different values (number of objects and cursor).
I created following jobs:
job for making chuncked requests
job to execute request (executeChunkedStationsRequests)
In my approach overhead I tried to execute the requests sequentially. After every request I return the cursor from child to parent. In the parent I update the cursor value in the global map. I am delivering the global map value back to the context of the executeChunkedStationsRequests-subjob.
The issue occurs when I make more then one requests. In the first request I use a initialized default value. This works fine.
Calling the executeChunkedStationsRequests-subjob multiple times doesn't work. The context var for the cursor is filled with 'null' and occurs an NullPointerException.
tJava: initProcessingVars
globalMap.put("query_stations_leftCount", context.totalCount); globalMap.put("query_stations_first", context.query_stations_firstDefault); globalMap.put("query_stations_cursor", context.query_stations_afterDefault);
globalMap.put("triggerChunkedStationsRequestsFlag", true);
tJava: updateProcessingVars
//---------------------------------------------------------------------- // Updating der Vars für das Processing der Chuncked Requests //---------------------------------------------------------------------- //---------------------------- // Updating query_cursor //---------------------------- globalMap.put("query_stations_cursor", row23.cursor); //---------------------------- // Updating query_stations_leftCount //---------------------------- int query_stations_first = ((int) globalMap.get("query_stations_first")); int query_stations_leftCount = (((int) globalMap.get("query_stations_leftCount")) - query_stations_first); globalMap.put("query_stations_leftCount", query_stations_leftCount); if ( query_stations_leftCount > 0){ if (query_stations_leftCount < query_stations_first){ //Make request with remaining amount of stations globalMap.put("query_stations_first", query_stations_leftCount); } //Make request with default amount of stations } else{ // Stop repuesting for stations globalMap.put("triggerChunkedStationsRequestsFlag", false); }
tLoop: triggerChunkedStationsRequests
tLoop
tRunJob: executeChunkedStationsRequests
tRunJob
Error message:
Starte Job OpenDataContentProvider am 14:12 20/05/2020. [statistics] connecting to socket on port 3980 [statistics] connected null null .--------------------. | #1. showTotalCount | +------------+-------+ | key | value | +------------+-------+ | totalCount | 1157 | +------------+-------+ ---------------------------------------------------
-- processChunkedStationsRequests (initVars)
|-> ((String) globalMap.get(query_stations_cursor)) -> none
---------------------------------------------------
---------------------------------------------------
--------------------------------------------------- -- processChunkedStationsRequests (updateVars) |-> row6.cursor -> null |-> ((String) globalMap.get(query_stations_cursor)) -> null |-> ((String) globalMap.get(query_stations_leftCount)) -> 1147 |-> ((String) globalMap.get(triggerChunkedStationsRequestsFlag)) -> true --------------------------------------------------- --------------------------------------------------- tRunJob_9 in processChunkedRequests_GraphQL call etl_opendata.executechunkedstationsrequests_graphql_0_1.executeChunkedStationsRequests_GraphQL with: [--father_pid=NkMjsk, --root_pid=QqmQ9G, --father_node=tRunJob_9, --context=Sandbox, --stat_port=3980, --parent_part_launcher=JOB:processChunkedRequests_GraphQL/NODE:tRunJob_9, --context_param query_stations_first=10, --context_param query_stations_after=null] Exception in component tJava_1 (executeChunkedStationsRequests_GraphQL) java.lang.NullPointerException at etl_opendata.executechunkedstationsrequests_graphql_0_1.executeChunkedStationsRequests_GraphQL.tJava_1Process(executeChunkedStationsRequests_GraphQL.java:497) at etl_opendata.executechunkedstationsrequests_graphql_0_1.executeChunkedStationsRequests_GraphQL.runJobInTOS(executeChunkedStationsRequests_GraphQL.java:4346) at etl_opendata.executechunkedstationsrequests_graphql_0_1.executeChunkedStationsRequests_GraphQL.runJob(executeChunkedStationsRequests_GraphQL.java:4177) at etl_opendata.processchunkedrequests_graphql_0_1.processChunkedRequests_GraphQL.tJava_19Process(processChunkedRequests_GraphQL.java:1266) at etl_opendata.processchunkedrequests_graphql_0_1.processChunkedRequests_GraphQL.runJobInTOS(processChunkedRequests_GraphQL.java:2120) at etl_opendata.processchunkedrequests_graphql_0_1.processChunkedRequests_GraphQL.runJob(processChunkedRequests_GraphQL.java:1953) at etl_opendata.opendatacontentprovider_0_1.OpenDataContentProvider.tRunJob_1Process(OpenDataContentProvider.java:767) at etl_opendata.opendatacontentprovider_0_1.OpenDataContentProvider.runJobInTOS(OpenDataContentProvider.java:1144) at etl_opendata.opendatacontentprovider_0_1.OpenDataContentProvider.main(OpenDataContentProvider.java:961) Exception in component tRunJob_9 (processChunkedRequests_GraphQL) java.lang.RuntimeException: Child job running failed. java.lang.NullPointerException: null at etl_opendata.processchunkedrequests_graphql_0_1.processChunkedRequests_GraphQL.tJava_19Process(processChunkedRequests_GraphQL.java:1283) at etl_opendata.processchunkedrequests_graphql_0_1.processChunkedRequests_GraphQL.runJobInTOS(processChunkedRequests_GraphQL.java:2120) at etl_opendata.processchunkedrequests_graphql_0_1.processChunkedRequests_GraphQL.runJob(processChunkedRequests_GraphQL.java:1953) at etl_opendata.opendatacontentprovider_0_1.OpenDataContentProvider.tRunJob_1Process(OpenDataContentProvider.java:767) at etl_opendata.opendatacontentprovider_0_1.OpenDataContentProvider.runJobInTOS(OpenDataContentProvider.java:1144) at etl_opendata.opendatacontentprovider_0_1.OpenDataContentProvider.main(OpenDataContentProvider.java:961) Exception in component tRunJob_2 (OpenDataContentProvider) java.lang.RuntimeException: Child job running failed. java.lang.RuntimeException: Child job running failed. java.lang.NullPointerException: null at etl_opendata.opendatacontentprovider_0_1.OpenDataContentProvider.tRunJob_1Process(OpenDataContentProvider.java:792) at etl_opendata.opendatacontentprovider_0_1.OpenDataContentProvider.runJobInTOS(OpenDataContentProvider.java:1144) at etl_opendata.opendatacontentprovider_0_1.OpenDataContentProvider.main(OpenDataContentProvider.java:961) [statistics] disconnected Job OpenDataContentProvider endet am 14:12 20/05/2020. [exit code=1]
My fundamental questions are:
I would be very grateful for your help! And excuse my bad english! 🙂
Hi
Using tJavaRow instead of tJava for updateProcessingVars, you can't access the value of input data flow on tJava, you can read the difference between tJava and tJavaRow in this topic.
https://community.talend.com/t5/Design-and-Development/What-is-the-difference-between-tJava-tJavaRow...
Regards
Shong
Hi
Using tJavaRow instead of tJava for updateProcessingVars, you can't access the value of input data flow on tJava, you can read the difference between tJava and tJavaRow in this topic.
https://community.talend.com/t5/Design-and-Development/What-is-the-difference-between-tJava-tJavaRow...
Regards
Shong
Using tJavaRow works for me fine! Thank you very much @shong! 🙂