Dear all,
First of all, I'm already very sorry, as this might to be a very basic question... But I am starting to dig into Talend and still feel quite lost :-)
What I would like to do is (I guess) pretty simple: download an xml or csv file through http, map it against an existing PostgreSql table and populate the Postrgre table with the records of the input file.
My problem is that the component tFileInputDelimited seems to accept only local files and don't recognize my http string:
http://earthquake.usgs.gov/earthquakes/catalogs/eqs1day-M1.txt Is there a particular syntax? or did I forget to do something? a component that will download the file to feed the tFileInputDelimited component?
Thanks a lot in advance for your help!
Stéphane
Well... I have one already!
How to link the tFileFetch component to the tFileInputDelimited?
Should I "hard-code" in the parameters of tFileFetch the output path and name and then use this as an input for tFileInputDelimited? Or is there a way to link directly the 2 components? (so that, e.g., if I change that output path of tFileFetch, the input path of tFileInputDelimited is automatically updated).
And, in any case, will the output file of tFileFetch replace the previous output file everytime I run the job? Or is there a specific setting to setup to reach this behavior?
Thanks again!
Stéphane
Hello
The job looks like:
tFileFetch
|
onsubjobok
|
tFileInputDelimited--row1-->tLogRow
so that, e.g., if I change that output path of tFileFetch, the input path of tFileInputDelimited is automatically updated).
Define a context variable for the output path of tFileFetch and the input path of tFileInputDelimited, so that both the two component always use the same path.
And, in any case, will the output file of tFileFetch replace the previous output file everytime I run the job?
If you set a fixed file name, it will replace the previous output file everytime, so you need to set a dynamical file name, eg:
"test"+TalendDate.formatDate("yyyy/MM/dd_HH:mm",TalendDate.getCurrentDate())+".txt"
when you run the job, it generate a file called:
test_2011/01/20_13:27.txt
Best regards
Shong
As far as I understand, the OnSubJobOk will only check that the fetch was successful
Yes, you are right. Only the subjob, tFileFetch here works successfully will fire the next subjob begin to run, otherwise, it is not.
Best regards
Shong
It works perfectly!
Now just to improve a little bit, I would like to use what you wrote in your first post in order to keep tracks of all fetched files:
"test"+TalendDate.formatDate("yyyy/MM/dd_HH:mm",TalendDate.getCurrentDate())+".txt"
This used to work if I hard-code the path and file name but not if I enter it as default value in the context variable. Any advice on this?
Thanks!
Hello
Add a tJava at the begining of job, for example:
tJava
|
onsubjobok
|
tFileFetch
|
onsubjobok
|
tFileInputDelimited--main-->tLogRow
and type in the following code on tJava:
context.filename="test"+TalendDate.formatDate("yyyy/MM/dd_HH:mm",TalendDate.getCurrentDate())+".txt";
Best regards
Shong
