topic Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1 in Talend Studio

tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

mkosater — Tue, 10 Sep 2019 15:14:39 GMT

When upgrading project from 6.4 to 7.2.1, a job started to fail due to arrayoutofbounds error. I isolated this to the tFTPFileList component. When reviewing the generation code and comparing it to the prior version it appears there is a bug in the logic.

Is this the proper venue to report this or is there a SOP for doing so?

6.4.0 code: The loop for pulling the data from the FTP dir method was slightly different.

for (String filemaskTemp_tFTPFileList_1 : fileList_tFTPFileList_1) {
if (fileNamePattern_tFTPFileList_1
.matcher(
filemaskTemp_tFTPFileList_1)
.matches()) {
fileListTemp_tFTPFileList_1
.add(filemaskTemp_tFTPFileList_1);
}
}
}

7.1.1 code: Issue is that the ftp_tFTPFileList_1.dir(null, true) and ftp_tFTPFileList_1.dir(null, false) bring back different sized arrays, and the looping logic is based off of the larger array.

String[] fileList_tFTPFileList_1 = ftp_tFTPFileList_1.dir(null, true);
String[] fileListwithNames_tFTPFileList_1 = ftp_tFTPFileList_1.dir(null, false);
ftp_tFTPFileList_1.chdir(rootDir_tFTPFileList_1);
List<String> fileListTemp_tFTPFileList_1 = new java.util.ArrayList<String>();
List<String> fileListTempWithNames_tFTPFileList_1 = new java.util.ArrayList<String>();

for (String filemask_tFTPFileList_1 : maskList_tFTPFileList_1) {
java.util.regex.Pattern fileNamePattern_tFTPFileList_1 = java.util.regex.Pattern
.compile(filemask_tFTPFileList_1.replaceAll("\\.", "\\\\.")
.replaceAll("\\*", ".*"));
for (int i_tFTPFileList_1 = 0; i_tFTPFileList_1 < fileList_tFTPFileList_1.length; i_tFTPFileList_1++) {
String filemaskTemp_tFTPFileList_1 = fileList_tFTPFileList_1[i_tFTPFileList_1];
String filemaskTempName_tFTPFileList_1 = fileListwithNames_tFTPFileList_1[i_tFTPFileList_1];

if (fileNamePattern_tFTPFileList_1.matcher(filemaskTempName_tFTPFileList_1)
.matches()) {
fileListTemp_tFTPFileList_1.add(filemaskTemp_tFTPFileList_1);
fileListTempWithNames_tFTPFileList_1
.add(fileListwithNames_tFTPFileList_1[i_tFTPFileList_1]);
}
}
}

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

Anonymous — Wed, 25 Sep 2019 03:30:24 GMT

Hi,

Please report the bug t Talend JIRA forum. The link is as shown below.

https://jira.talendforge.org

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

sam2000 — Fri, 31 Jan 2020 10:44:04 GMT

Talend Team,

Can you please let us know, if there is any patch available for the bug explained in this thread related to tFTPFileList in v7.2.1?

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

Anonymous — Wed, 05 Feb 2020 17:41:01 GMT

Didn't see JIRA so created ticket: TDI-43662 tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

mkosater — Wed, 05 Feb 2020 18:43:43 GMT

So I had put a ticket in with Talend support since the issue was in their enterprise version. Talend support created bug report and then a patch for the issue. I guess I expected that they would have been pushed back into the TOS repo as well. The relationship between the TOS and the for profit side of things is a bit confusing at times.

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

vyu — Tue, 11 Feb 2020 03:06:41 GMT

Please attach the job which exists bugs.

Here is the tutorial https://help.talend.com/reader/HqPypUO_cLctKv_dAWcgFw/fYXMbAVE56gH~wvSDjnxTw

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

mkosater — Tue, 11 Feb 2020 20:45:10 GMT

The internal case number was 00151288. Patch was Patch_20191015_TPS-3447. I don't have the test job any longer that reproduces. Here are some notes from the interaction with support.

Some details:

Our internal server is a windows based server. A public server I hit was Linux based, and my test job at the time does not fail. The FTP server is Windows 2016 running ipswitch wsftp server 8.5.2.
I suspect there MAY be a change in behavior in the ftp library that this component uses, in how it handles dir() (using LIST versus NLIST), but regardless, the component should handle it.
My test/job uses a file mask (*.*). However the error occurs with or without the mask.

Added notes about the folder I am trying to list:

happens o be one level of folder off of the root.
there multiple folders off of the root.
number of files in the folder does not change the behavior.

My work around was taking the 6.4 code and creating a custom object and I used that.

I put together a new test....

test_ftpfilelist_export.zip

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

vyu — Wed, 12 Feb 2020 07:00:02 GMT

Thank for your information. But sorry, I still couldn't find the problem.

It would be helpful to have the original job.

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

Anonymous — Wed, 12 Feb 2020 14:36:50 GMT

@yliu If you're testing with the patch installed, you won't be able to reproduce the error. The patch is only available to Enterprise Edition users. As I'm using the community edition, the patch is unavailable to me.

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

mkosater — Wed, 12 Feb 2020 15:34:31 GMT

That job I attached is the same that was submitted to Enterprise ticket and used for debugging. I want to emphasize that you need to connect to a FTP server that I mentioned above. I ran the job attached against an internal windows server, and was able to reproduce the error.

I think the enterprise engineer who worked the case installed the server mentioned above and was able to reproduce.

Re: tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

vyu — Fri, 14 Feb 2020 06:51:17 GMT

This issue will be fixed in 7.3.1 Release. Please wait for the new version.