How to connect multiple FTP locations in the same job

[Anonymous]

I have one table which contains many distributors and retailers ftp connection details.

The table structure is as follows:

Dist_Id       Reta_Id        Ftp_host       Ftp_user        Ftp_pswd
------------------------------------------------------------------------------------
1                 a                   123               abc                 xyz
1                 b                   456               def                  mno
2                 c                   431               ghi                  p9r
3                 d                   789               jkl                   12e

So for getting all files for Dist_Id=1, I want to connect ftp locations of Retailers a and b. And download files in each ftp location to the same folder.
How can I connect to multiple ftp locations and download files in the same folder??

[fdenis]

create your ftp job for one connection one file (use context to store values of line 1)
create a job calling this job (trunjob) with context values from your table.

[Anonymous]

Thanks for your response.. There is a chance for coming new retailers for each distributor.. At that case, want to  connect to that retailer's ftp location automatically and download the files.. Could you please make it more clear with some examples...

[Anonymous]

Neena,

below is the example which Denis was trying to explain.

consider that the ftp details are there in a file. so use tfileinputDelimited and connect to tFlowToIterate which would iterate the tRunJob for every output row from the file. 

The advantage of tFlowToIterate is that it sets the inflow values in global variable as key-value pair. In tRunJob, pass all the parameters from tFlowToIterate to context variable of the child job.

In the child job you can have FTP component. use all the defined context variables in the ftp component.

Hope this helps!

[Anonymous]

Thanks for your reply..  It is very much helpful.. I Designed the job based on your reply.. But when connects to second ftp location it shows some exceptions :

Exception in component tRunJob_1
java.lang.RuntimeException: Child job running failed.
com.enterprisedt.net.ftp.ControlChannelIOException: Control channel unexpectedly closed ('' read so far)

Is there any solution??

[fdenis]

try the second job with second ftp context.

[Anonymous]

I think the second one is a SFTP protocol. So how to handle both FTP and SFTP in the same job??

[fdenis]

add a sftp subjob and switch defore or add sftp connection into subjob and switch inside subjob

[Anonymous · Accepted solution]

Neena,

You can do this many ways. Here is the easiest way I would recommend. First of all, you should send the FTP method (either FTP or SFTP) from the parent to tRunJob.  

Here is the screenshot of child job

tJava: just to print the FTP method

First order: connect tJava with runif to FTP component.

Second order: connect tJava with runif 

tFtp get with "SFTP support" enabled. depending on your design , you can choose the "Authentication method" to public key. provide key Passphrase and private key (if required).

The idea is to have runif and decide your flow based on the ftp method sent by your parent. 

Hope this helps!

[Anonymous]

Very Thanks for your reply... It works  ....