topic Re: Pass filename stored in CSV-File to tfileinputXML in Talend Studio

Pass filename stored in CSV-File to tfileinputXML

Anonymous — Sat, 16 Nov 2024 14:25:25 GMT

Hi,
I would like to compare the content of two files. I have a list of pairs stored in csv file:

Example of pairs.csv

Filename_CSV;Filename_XML
/path/to/file/1.csv;/path/to/file/1.xml
/path/to/file/2.csv;/path/to/file2.xml

How can I passed the filenames of the "Filename_CSV" column to the tfileinputdelimeted component and the filenames of "Filename_XML" column to the ffileinputxml component?

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Thu, 20 Mar 2008 11:25:36 GMT

I didn't understand what you exactly mean by compare the content of two file?
For the second part there is a solution but I want to be sure that you only want to get the information in each file.

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Thu, 20 Mar 2008 11:34:32 GMT

I want to compare the content of 1.csv to the content of 1.xml.
The comprehension already works manually, but I don't know how the tfileinputXML-component can get the filename stored in the pairs.csv file.
There is a solution?

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Thu, 20 Mar 2008 11:44:40 GMT

Hi,
you can try to use a tFileInputCsv (with two attributes) and a iterate link. In each iteration you can get the filenames from the globalMap.
Bye
Volker

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Thu, 20 Mar 2008 12:35:26 GMT

Hi Volker,
thanks for your replay, but I don't know how to use globalmap within a java job.
The component, which reads the pairs.csv is called "tFileInputDelimited_2" and the column is called "Filename_CSV".
I tried the following string, as input-flow:
((String)globalMap.get("tFileInputDelimited_2.Filename_CSV"))
But it doesn't work.

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Thu, 20 Mar 2008 13:27:09 GMT

tFlowToIterate is the component you need 🙂

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Thu, 20 Mar 2008 13:53:14 GMT

You are my personal hero...
Thanks

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Thu, 20 Mar 2008 19:34:21 GMT

Well it only works almost. My job just compares the last pair of files in my lookup filelist. I tried to setup an iteration connector to both tfileinput-components, but this seems to be forbidden. So what can I can do, so that my job reads the lookup filelist step by step.

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Thu, 20 Mar 2008 21:31:15 GMT

You should add a iterate link between "iterate lookup list" and "read current csv file". If you do not have a link you have two jobs with no connection and each job is executed only once. Therefore you get only one "output".
Bye
Volker

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Thu, 20 Mar 2008 21:42:14 GMT

Then I get this error message:

Starte Job compare_content am 22:41 20/03/2008.
Exception in component tFileInputXML_1
java.net.MalformedURLException
at java.net.URL.<init>(URL.java:601)
at java.net.URL.<init>(URL.java:464)
at java.net.URL.<init>(URL.java:413)
at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at geschaeftsmodell.compare_content.compare_content.tFileInputXML_1Process(compare_content.java:1029)
at geschaeftsmodell.compare_content.compare_content.tFileInputDelimited_2Process(compare_content.java:204)
at geschaeftsmodell.compare_content.compare_content.runJobInTOS(compare_content.java:1231)
at geschaeftsmodell.compare_content.compare_content.main(compare_content.java:1150)
Job compare_content endet am 22:41 20/03/2008.

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Thu, 20 Mar 2008 22:30:59 GMT

I can reproduce the problem. I'm not sure if this is a limitation or a bug:
The lookup row is evaluated before flow to iteration starts. So you cant get the value out of the global map.
Alternatively you can split your functions in two jobs. The first one handles the file list and will call the second job via tRunJob. You can define in your compare job both files as context variables and pass this variables in the definition of tRunJob.
Bye
Volker

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Fri, 21 Mar 2008 08:42:34 GMT

Its frustrating, I can't get this job synchronized. I tried your idea with trunjob and I added an addition tMap (tMap_3), which just copy the flow, to get the "XML-Path" longer. That helps, so that the first iteration runs, but in the second iteration the job dies ...

Starte Job rettung am 09:35 21/03/2008.
1|0.0|0.0|true
2|0.0|0.0|true
3|12.0|0.0|false
4|123235.0|0.0|false
5|168.5|0.0|false
6|168.5|0.0|false
7|168.5|0.0|false
8|168.5|0.0|false
9|166.3|0.0|false
10|166.3|0.0|false
11|164.8|0.0|false
12|164.8|0.0|false
13|164.8|0.0|false
14|164.8|0.0|false
15|166.3|0.0|false
16|166.3|0.0|false
17|166.3|0.0|false
18|166.3|0.0|false
19|166.3|0.0|false
20|166.3|0.0|false
21|168.5|0.0|false
22|168.5|0.0|false
23|168.5|0.0|false
24|168.5|0.0|false
Exception in component tMap_2
java.lang.NullPointerException
at geschaeftsmodell.compare_content.compare_content.tFileInputDelimited_1Process(compare_content.java:477)
at geschaeftsmodell.compare_content.compare_content.runJobInTOS(compare_content.java:1257)
at geschaeftsmodell.compare_content.compare_content.runJob(compare_content.java:1182)
at geschaeftsmodell.rettung.rettung.tFileInputDelimited_1Process(rettung.java:235)
at geschaeftsmodell.rettung.rettung.runJobInTOS(rettung.java:416)
at geschaeftsmodell.rettung.rettung.main(rettung.java:335)
Exception in component tRunJob_1
java.lang.RuntimeException: Child job running failed
at geschaeftsmodell.rettung.rettung.tFileInputDelimited_1Process(rettung.java:246)
at geschaeftsmodell.rettung.rettung.runJobInTOS(rettung.java:416)
at geschaeftsmodell.rettung.rettung.main(rettung.java:335)
Job rettung endet am 09:35 21/03/2008.

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Fri, 21 Mar 2008 22:38:46 GMT

Hi,
could this be a problem with your file or you data? Does it run if you use your job with the second file-pair?
If so, can you give us some more information about your job (tMap2)

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Tue, 25 Mar 2008 10:58:58 GMT

Thanks Volker, that you still helps me ...
Okay I check the second pair and it works without problem, so I added a screenshot of tMap2

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Tue, 25 Mar 2008 23:04:48 GMT

I'm not sure I could help you really :-(
I can't see a problem in your tMap. The exception is thrown if you work with null values. I can't see (at first view) where the problems could happen.
I've for testing purposes written a job to. And it works.
I think that this is a problem of the data (or the job) and not of the multiple execution as a subjob.
Because I can't reproduce the problem I could only say what I should do to isolate it:
- Check the generated source code (switch from the designer into the code view) and take a look at the given line. Or better use the debugger.
- Reduce the files to check if the problems exist anymore. (Test only one pair, than another isolated, then both together, ...)
Hope this helps.
Bye
Volker

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Wed, 26 Mar 2008 05:56:19 GMT

Hi Everyone,
I created one job in TALEND and its running fine.I parsed one csv file and put in database.
Now i have one issue is that if i need to run that job by help of bactch file without open TALEND, then it searching for csv file.
Can anyone guide me that how can i provide the csv file path through command line and run the by batch process?

Re: Pass filename stored in CSV-File to tfileinputXML

Anonymous — Wed, 26 Mar 2008 06:10:12 GMT

Hello

Now i have one issue is that if i need to run that job by help of bactch file without open TALEND, then it searching for csv file.
Can anyone guide me that how can i provide the csv file path through command line and run the by batch process?

Please read this 1615 to learn how to specify a variable value from the command line.
PS: Please create a new topic for new questions next time.
Best regards

shong