Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us in Bucharest on Sept 18th for Qlik's AI Reality Tour! Register Now
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Loading .CSV file from SFTP Server to SQL Server

Hi everyone,

I'm a new bee to Talend. I'm asked to do a POC on Talend. Please help me with your suggestions.

 

The requirement follows like below.

 

Once the .CSV files(multiple files)are detected in SFTP Server, A control file is created as a file event file event for a polling mechanism in Talend. A Talend job, once triggered by the polling event, compresses the exported data, transports them to another S-FTP CN file loaction. From this new SFTP location,I have to use Talend to load the data to SQL server database.If successful it moves csv-files to a backup share and deletes the control.csv file from the old SFTP Loaction.

 

Now I need to prepare a Design document for this whole process. 

Labels (4)
16 Replies
Anonymous
Not applicable
Author

Where will the job be execute? remote server (Linux box) or local machine? if the files are on SFTP server and you can only access the files through FTP protocol, the answer is yes, you have to get the files to local machine where the job is executed, and read the file using the tFileInputDelimited component.

Regards
Shong
Anonymous
Not applicable
Author

The Job gets executed on Local machine since TOS is on my local machine.

I didn't download/install TOS on Linux Box.

 

Can we access files from Linux box If we install TOS on Linux box?

 

Thanks & Best Regards,

Reddaiah Pidugu

 

 

Anonymous
Not applicable
Author

Take a look at the tFTPGet component and learn how to get the files to from SFTP server to local machine.

Regards
Shong
Anonymous
Not applicable
Author

Thank you so much Shong for the solution.

I could solve the issue.

 

Best Regards,

Reddaiah Pidugu

Anonymous
Not applicable
Author

Great, thanks for your feedback!
Anonymous
Not applicable
Author

Hi Shong,

 

I'm facing an issue with my job design.

 

My job has to pick 3 files(same structure) from a local machine and load into SQL server.

But it is picking all the files and loading only one file data to SQL server.

 

Please find the attachment my job design and suggest me where I'm doing wrong.

 

 

Thanks & Regards,

Reddaiah Pidugu

 


TalendJobDesign.PNG
Anonymous
Not applicable
Author

Shong, Also please note that all the files are in same folder and today I have 3 files and tomorrow it may vary.

Thanks,