Read x numbers of line from a csv and iterate on them

[Anonymous]

I need to query a large amount of data based on specific Ids from Salesforce.
Since I have too many records, the query "select id, name from account where id in ('1','2','3',...)" is too long to be executed (it would be over 20.000 chars), therefore I need to split it in multiple queries.

The starting Ids can be read from a csv file.

My idea is to read about 200 lines of ids from the file, create a String that concatenates the Ids (like '1','2','3', ... , '200') and then (like '201','202','203', ... , '400') and send each line of this type to a Salesforce input component (I imagine with a subjob).

Any idea on how to structurate this job? 

[David_Beaty]

Hi,

You'll need to pre-process through the one big file and write out X smaller files, on the file output component, Advanced, theres a s split function every 200 rows. Put these files out to another directory, then use a tFileList on that directory and read each file of 200 rows and build the SF query.

[Anonymous]

Hi @altibzz ,

Best way to do this would be,

Step 1 : Load the list of id files in a temporary table say named as "ID". So from ID File -> Table Load.

Step 2, your query can be written like :

"select columns from table where id in (select id from ID_TABLE)

Thanks and Regards,

Subhadip