Query condition for tSalesforceInput based on Ids in a CSV File

[Anonymous]

Hi,
We are running a talend (With a salesforce connection) job to Loads Task object-records in a .csv file (using tFileOutputDelimited).
We wanted to restrict the number of records queried for an Object based on Ids in a file delimiter (CSV file) . Could someone please help us here, as we are new to talend.

Thanks in advance!

[Anonymous]

Can you better explain your issue.
Are you reading or writing to Salesforce?
What exactly are you trying to achieve?

[Anonymous]

Hey Thanks for a prompt response!
We are reading and writing both to Salesforce.
We are supposed to load Tasks from an Old Production to a New Production instance and there are about 28 lakhs task records in old org but we need to query only around 2 lakh records. We have the set of IDs to be queried in a CSV file.
We need a query condition which looks like : ""what.type = 'case' AND whoid = null and createddate" AND Id = '' (this is to be taken with a tFileInputDelimited ... right?)
PS: We cannot use tMap and then match the ids because according to our understanding it queries all the 28 lakh records and then matches them; but with that size, its running to heap size issues (Even if we increase the heap size in the .ini config file) and then time-out errors consistently, so there itself is the glitch.
WE WANT TO FILTER THEM WHILE QUERING ONLY using the CSV file , PLEASE SUGGEST IF THERE IS A BETTER WAY !

[Anonymous]

In Short,
We thought we can reference the CSV file in the query condition of a tSalesforceInput to query only when the Task object Id = 'Ids in the CSV File', ONLY, we don't really know how to do that.
I hope I am clear now

[Anonymous]

Ah...lakh=100000
It's a tricky one as it's a 'cloud' service.
Couldn't you reapply the logic to Salesforce that gave you the CSV files of IDs in the first place?
If not, I'd break things down.
Export all contenders from Salesforce to a CSV file.
Join your two files using tMap.
You can increase HEAP - Run->Advances Settings
tMap supports "Store temp data" which conserves memory; but slows things down
Exporting large Salesforce datasets with Talend is a challenge, especially if you want to restrict in this way.

[alevy]

See https://community.talend.com/t5/Design-and-Development/How-to-perform-Dynamic-Lookup/td-p/104177.