topic Re: [resolved] Execute python script using tSystem in Talend Studio

[resolved] Execute python script using tSystem

Anonymous — Sat, 16 Nov 2024 10:46:26 GMT

Hello,
I'm trying to execute a python script in Talend.
I just put a tSystem on my job and fill the 'command' field by : "cmd /c C:/Python24/python.exe Q:/Python/Test/myScript.py"
The script is supposed to read files, create 2 others one. All the files the script uses (or writes) are in the same folder than the script. Outside Talend, the script works well.
But when I run the job, my script doesn't work. I have the 'prints' on the console inside Talend, but the files are not detected and the others are not written.
Any idea to help me ?
Thank you !

Re: [resolved] Execute python script using tSystem

Anonymous — Sun, 28 Feb 2016 18:04:34 GMT

Can can you share your Python script?
It may be writing the output to the Talend job's current directory rather than where you expect it to be .
It might also not be reading the File based on the way the parameters are passed to the CMD

Re: [resolved] Execute python script using tSystem

Anonymous — Mon, 29 Feb 2016 13:12:02 GMT

Hi,
Thank you for your answer.
Sadly, I can't share the script.
But here is the structure :

from glob import glob
for file in glob("*.csv"):
    infile = open( file, 'r' )
    lines = infile.readlines()
    infile.close()
    for line in lines:
        // read the data
        // operations on data, result stored in a list

outfile = open( outputFileName , 'w' )
for outputLine in list:
    outfile.write(outputLine) 
outfile.close()

I checked the Talend job's current directory, nothing inside.
But actually you're right, none of the files are read.

Re: [resolved] Execute python script using tSystem

Anonymous — Mon, 29 Feb 2016 16:58:49 GMT

The script appears to make an assumption that the CSV files are in the current directory, based on the glob statement . For an easy mod, just specify an absolute path to glob ex
glob("/data/20160229/*.csv")
You could also add some debugging output lines For help in troubleshoouting
It's files might sometimes be in a different folder, then you can look at ways to perm parameterize The script or add a configuration file

Re: [resolved] Execute python script using tSystem

Anonymous — Mon, 29 Feb 2016 17:03:23 GMT

Of course, I should ask…
Why not use Talend components for your task? It is very easy to list files , read CSV files, transform, and write the files out in a different format

Re: [resolved] Execute python script using tSystem

Anonymous — Wed, 02 Mar 2016 09:27:37 GMT

I specified the absolute path to glob, and it seems it's working !
Thank you dbh !
To answer you, several things :

I've already written the python script
I'm a beginner in Talend (but I have to work with jobs already built)
The script is looking in different ways in the data already processed (comparison, eliminate duplications, insertions...)
It would be easier to upgrade the script (adding more error types/messages for instance) than modify Talend
But, if I hadn't spent hours on the script before, I certainly would have use Talend components.
Thanks again !