topic dynamic file name in FTPget in Talend Studio

dynamic file name in FTPget

_AnonymousUser — Sat, 16 Nov 2024 13:20:46 GMT

Hi, Please help.
I need to get the files "fill_*_tst.txt" from sFTP server.
When I specify the file name, it works. but I tried to create a dynamic name, it is not working in tFTPget.
HEre is the code I input in 'filemask", which can be wrong.
"fill_" + TalendDate.formatDate("yyyyMMdd", TalendDate.getCurrentDate()) + "_*_tst.txt"
How can I do that?
Thank you.

Re: dynamic file name in FTPget

jkrfs — Wed, 21 Jul 2010 16:22:35 GMT

Im confused, you first say
"fill_*_tst.txt"
but then you add the date, is this correct? if you were going by the original pattern, shouldnt it be:
fill_" + TalendDate.formatDate("yyyyMMdd", TalendDate.getCurrentDate()) + "_tst.txt" ?
Remember, in filemask, using * is a wildcard character...

Re: dynamic file name in FTPget

_AnonymousUser — Wed, 21 Jul 2010 16:33:01 GMT

Sorry I wasn't clear.
Files in the server are named as "fil_20100721_2222_tst.txt" or "fil_20100721_2223_tst.txt".
So I want to get all the files for "fil_YYYYMMDD_*_tst.txt" with wild card in it.

Re: dynamic file name in FTPget

_AnonymousUser — Wed, 21 Jul 2010 16:34:24 GMT

So when I tested with a filename (fill_20100720_*_tst.txt), it worked throuhg tFTPget. but I need to generate the file name everyday.

Re: dynamic file name in FTPget

jkrfs — Wed, 21 Jul 2010 16:46:08 GMT

Personally, I like to set my dates in the beginning of the flow using tJava using a context parameter.
First, create a context parameter something like context.todaysDate
Then in the beginning of the flow add a tJava and connect it with SubJobOk to the next in the flow. Add this in the component code:
GregorianCalendar instance = new GregorianCalendar();
SimpleDateFormat formatter = new SimpleDateFormat("yyyyMMdd");
context.todaysDate = formatter.format(instance.getTime());
Then you will use this in the filemask:
"fill_" + context.todaysDate + "_*_tst.txt"
Its a little more work, but that way I can just use one set context parameter throughout the job, and its easier to handle overall.

Re: dynamic file name in FTPget

_AnonymousUser — Wed, 21 Jul 2010 16:49:58 GMT

Thanks a lot. I will try it.

Re: dynamic file name in FTPget

_AnonymousUser — Wed, 21 Jul 2010 22:13:53 GMT

Does it require any imports?

Re: dynamic file name in FTPget

_AnonymousUser — Wed, 21 Jul 2010 22:20:51 GMT

somehow it doesn't work. simply I tried to add String variable in tJava code as following.
String filename = "fil_20100720_2226_tst.txt";
And tFTPget, I put file mask 'filename' but I get an error "filename cannot be resolved". How can I call the variable from filemask? do you have an example? Thanks for your help.

Re: dynamic file name in FTPget

_AnonymousUser — Wed, 21 Jul 2010 22:29:40 GMT

Hi, I tried your code but I got this error.
context.todaysDate cannot be resolved or is not a field

Personally, I like to set my dates in the beginning of the flow using tJava using a context parameter.
First, create a context parameter something like context.todaysDate
Then in the beginning of the flow add a tJava and connect it with SubJobOk to the next in the flow. Add this in the component code:
GregorianCalendar instance = new GregorianCalendar();
SimpleDateFormat formatter = new SimpleDateFormat("yyyyMMdd");
context.todaysDate = formatter.format(instance.getTime());
Then you will use this in the filemask:
"fill_" + context.todaysDate + "_*_tst.txt"
Its a little more work, but that way I can just use one set context parameter throughout the job, and its easier to handle overall.

Re: dynamic file name in FTPget

Anonymous — Thu, 22 Jul 2010 14:48:23 GMT

Hi,
The FileMask value expected in your case is :
"fil_"+TalendDate.getDate("YYYYMMDD")+"_*_tst.txt"
Let me know if that resolve your issue.
The function TalendDate.getDate("DATE_PATTERN") return the current date of your machine/server and format it following the expected format.
For example you date is YYYYMMDD : It's return 20100722; if your date is YYYY-DD-MM : It's return 2010-22-07.
Using my FileMask in your tFTPGet will GET all the files start by : fil_20100722_ - and finishing by _tst.txt
Have a good test.
Best regards;

Re: dynamic file name in FTPget

_AnonymousUser — Thu, 22 Jul 2010 16:03:12 GMT

Thanks. I am going to try that way.
One quick question, If I want to get Yesterday's date, TalendDate.getDate("YYYYMMDD") - 1 will work?

Re: dynamic file name in FTPget

Anonymous — Thu, 22 Jul 2010 18:07:23 GMT

I haven't tried, am I'm not a java expert, but I suspect not.
I use TalendDate.addDate and a context variable to do this.
context.myDate = Calendar.getInstance().getTime();
context.myDate = TalendDate.addDate(context.myDate, -1, "dd");

Re: dynamic file name in FTPget

_AnonymousUser — Thu, 22 Jul 2010 18:31:11 GMT

TalendDate.addDate works. Thank you for all your help.

Re: dynamic file name in FTPget

_AnonymousUser — Fri, 17 Feb 2012 15:10:58 GMT

I am putting together a small POC using Talend toolkit (v5.0.1.r74687) and this example is is exactly what I'm looking to do, but when I run the job, it will not compile, giving me an error saying that "GregorianCalendar cannot be resolved to a type" and highlighting the tJava object (expectantly I guess) as to the source of the error.
Any pointers as to what I'm doing wrong would be most welcome as its the first hurdle in 3 days of playing with this software that I've been unable to resolve myself (with the help of talendforge and/or google anyway)

Re: dynamic file name in FTPget

Anonymous — Fri, 17 Feb 2012 16:05:47 GMT

Try fully qualifying it, eg:
java.util.GregorianCalendar = new java.util.GregorianCalendar();

Re: dynamic file name in FTPget

_AnonymousUser — Tue, 09 Oct 2012 08:48:04 GMT

hi Yunee,
Could u paste the screen shot ofyour job here.