Upload CSV file to server, trigger ETL job to load the contents to DB
I am new to Talend Open Studio and have been playing with it for a week or so. Please excuse me for my lack of knowledge.
Problem Statement: I have a CSV file (~5GB) in my local drive. I have to load the contents of the CSV to a DB table sitting in a remote server. The access to the server is restricted via domain check, so I cannot run my job on my local machine to load the file. It will have to run on a job server in the trusted domain. Also, I won't be allowed to manually drop the file in the job server. I have to use a web interface to drop the file there. How can I achieve this?
CSV file needs to be uploaded to a server via http. And once the file upload is complete, I need to trigger an ETL job to load the uploaded file contents to DB tables. A good to have feature is to respond back with to the user (me) with the status of the job completion.
Many Thanks,
Jc.
Hi jc, i'm jcs, an upgrade of you lol (just kidding)
You can try to capture the HTTP request (with its parameters) then 'copy' it to tHttpRequest. maybe it contains a header key with the path of the file
to capture your request on google press ctrl + maj + I and go to network, or use wireshark.
Uploading a 5 GB file on a http protocol will takes some time...
