Setting up a generic job to load snowflake table based on Context Param values

[vvazza10]

Hi ! Wanted to check if there is any way to create a generic job that does the following -

1. Parent job that contains 2 tRunJob components - One to load Employee table and one to load Department table
2. tRunJob component for Employee having Context Params Employee as target table and Emp as flow param and

   tRunJob component for Department having Context Params Department as target table and Dept as flow param

3. Based on the context param value, corresponding flow in tXMLMap should execute and load to target snowflake table. For example, if Emp is the flow param value, Emp flow in tXMLMap should execute and Employee table in snowflake table should get executed.

Please advise how to approach a scenario like this.

Parent job:

Context Param values that gets passed to child job for Employee:

Child job:

[vvazza10]

Hi ! Could someone please help? 🙂

[Anonymous]

Hi

Because the employee and department table have different structure and tXMLMap don't support different output flow based on variable, so you can't create a generic job flow for different target object. As a workaround, create different subjobs, and use runIf connector to trigger the subjob based on the context variable. For example:

tJava--runIf1--tFileInputXML--tXMLMap--tSnowFlakeOutput(for employee)

--runIf2---tFileInputXML--tXMLMap--tSnowFlakeOutput(for department )

on tJava:

if(context.tableName.equals("employee")){

globalMap.put("runEmployeeFlow",true);

}

if(context.tableName.equals("department")){

globalMap.put("runDepartmentFlow",true);

}

Set the condition of runIf1 as:

(Boolean)globalMap.get("runEmployeeFlow")

Set the condition of runIf2 as:

(Boolean)globalMap.get("runDepartmentFlow")

Hope it helps!

Regards

Shong

[vvazza10]

Hi @Shicong Hong​  ! I would want to load both Employee and department in a single load. XML file will contain both Employee and Department information. Running subjobs based on context variable would allow only one dataset, either Employee or Department, to be loaded to Snowflake table.

Tried testing with the below design, but the load seem to take 13-15 minutes to complete for a 2MB XML file

How can i design this better?