Outputting data to multiple files, but making sure that a group of records stays within one file

[Anonymous]

Hey. Has anyone an idea how to split output to separate files not more than a million records per file, but making sure that a group of records (for example grouped by a customer_id field) is not split? It should be moved whole to the next file. 

[Anonymous]

Hello,

The tFileOutputDelimited has the feature to split the result into mutliple files.

We would like to give you a precise answer if you could elaborate your case with an example.

Best regards

Sabrina

[Anonymous]

Thanks for quick response. 

There is an input file with many million rows of data. Data contains records for many users and each user may have multiple rows. I sort data by customer_id using tSortRow to group the records for each customer. Now what's needed is to output all the data into separate files, but to ensure that group of records is never split. So if the limit of 1 million records per file is reached, the group must be moved to the next file as a whole. So this is why the default feature to split files doesn't work in my case. 

Sorry I don't have a good picture example, tried to explain by words as best as I could.