tPivotToColumnsDelimited is very slow

[Anonymous]

Hi,
When i have tried loading some 1  million data using tPivotToColumnsDelimited it's taking  around 7 hours.
I thing its not de normalizing when it receives any row.its collecting all the row first and de normalizing after collecting all.
Do you have any approach that will make the process faster.
My Scenario is:
I want to load about 1 month data(around 5 crores) from MongoDB to a table which has a de normalized format.
so i need to write it through a tPivotToColumnsDelimited file and then load it to the table.
is there any way that i can load all the one month data in 7 to 8 hours?
Is there any mechanism to use,so that it will use local-memory to store temporarily.
Any way will be appreciated....I need this urgently please.....as i am not find any way to make the load fast.
Thanks,
jyotiranjan

[Anonymous]

Hi jyotiranjan,

Can you provide some screenshots of your current job? It will be helpful for us to locate the problem.

Best regards
Sabrina

[Anonymous]

Hi jyotiranjan,

Can you provide some screenshots of your current job? It will be helpful for us to locate the problem.

Best regards
Sabrina

Hi Sabrina,
Please find below the screen shot of my job.

Here the taggregate is extracting a nested json value and copying into another file.
The tpivotcolumn.... is taking a lot of time

[Anonymous]

Hi jyotiranjan,
Can you upload again the screenshots you wanted to show, please? For some reason it didn't make it to your post.

Best regards
Sabrina