topic Re: Filename context in Talend Studio

Filename context

daez — Wed, 09 May 2018 10:41:43 GMT

Hello everyone,

I'm trying to get a file from FTP which is composed of a date looking like : yyyyMMdd_hhmm (exemple : *TEXT*_20180509_1030.xml) and then turn it to CSV.

Since there will be file everyday, i planned to add a context variable and use it in the file name.

Here is what it looks like :

New context : "FTP_GET "

value : "TEXT"+TalendDate.formatDate("yyyyMMdd_hhmm",TalendDate.getCurrentDate())+".xml"

My FTPGET is working fine, tho, my TFileInputXML doesnt understand my path, what i'm missing?

FileName : "C:/Users/......../"+context.FTP_GET.

Another question, must I write the file from FTP GET on my disk and then get it back with my tFileInputXML ? Can't i keep the data on Talend and not write then on my disk?

Thanks for the answer!

Re: Filename context

TRF — Wed, 09 May 2018 10:49:21 GMT

For sure, your tFileInputXML must contain the full pathname for your XML file (including directory).

For the 2nd question, no can't. You must get the file locally, then you can read it using tFileInputXML except if the file is stored on a shared drive (probably not if you use ftp).

Re: Filename context

daez — Wed, 09 May 2018 11:18:31 GMT

Hi @TRF and thanks for the reply !

Thanks for the second answer. About the first one, I don't get it. My tFileInputXML does contain my fullpathname, its just that it doesnt use my context result and returning it as an error (syntax incorrect, see screenshot). I must do something wrong when i use context in the settings ?

context : FTP_GET , value : "Text"+TalendDate.formatDate("yyyyMMdd_hhmm",TalendDate.getCurrentDate())+".xml"

Filename : " C:/"+context.FTP_GET "

Re: Filename context

TRF — Wed, 09 May 2018 12:07:17 GMT

Can you share your tFTPGet settings and the piece of code where you set the context variable value.

Re: Filename context

Anonymous — Wed, 09 May 2018 12:25:12 GMT

This is easily fixed. Your problem is caused by trying to use Java to set the context variable in your Contexts tab. You cannot do this. If you want to set a Context variable using code, you can add a tJava component before your tFileList, connect it via an OnSubJobOK link and add this code to it....

context.FTP_GET = "Text"+TalendDate.formatDate("yyyyMMdd_hhmm",TalendDate.getCurrentDate())+".xml";

This MUST be done in a prior subjob to the subjob you are going to use it for a filename

Re: Filename context

Anonymous — Wed, 09 May 2018 12:29:39 GMT

I just read the other part of your question. You can stream the file data from its source location, but it is trickier. You will need to the tFileFetch (https://help.talend.com/reader/9q55KsfASqX0qY4GVhEDNQ/pfuQtcwLBNAvHnxMDbe2ag) and make use of the "Use cache to save resource" make.

This tutorial might help (https://help.talend.com/reader/iYcvdknuprDzYycT3WRU8w/2kKyGJVxys~Zwh8HsNLtaQ)

Re: Filename context

daez — Wed, 09 May 2018 13:31:25 GMT

Thanks for the reponse to both of you, @rhall , @TRF

I had already tried the tJava component via OnsubJob Ok tho, I still have the same error where my tFileInputXML is not interpreting my context value.

As you can see it looks like he doesnt get the TalendDate fonction and use it as a string. (Basically : syntax wrong)

Re: Filename context

Anonymous — Wed, 09 May 2018 13:35:17 GMT

You are not showing enough info. If you tried my solution, it WILL have worked. So can you show me precisely how you tried it (hiding nothing.....there are no spies here trying to work out information about you or your organisation via filepaths) please?

Re: Filename context

daez — Wed, 09 May 2018 13:48:58 GMT

Edditing my response here, I tried to change some stuff & finally worked by linking the tJava to the tFileInputXML instead of tFTPGet ... No idea why but yet its working, resolved then, thanks for the tJava tricks .

Kindly

Re: Filename context

kynio — Wed, 27 Feb 2019 18:24:34 GMT

Another question, must I write the file from FTP GET on my disk and then get it back with my tFileInputXML ? Can't i keep the data on Talend and not write then on my disk?

I use tBufferOutput to store all data from a file, then use it in the same Job to perform operations. In this example I filter the data on Buffer several times to assign values in different columns depending on the filtered data.

You may want to set a generic schema on both tBufferOutput and tBufferInput to ensure your columns match.

tBufferOutput

o.png