Skip to main content
Announcements
Join us at Qlik Connect for 3 magical days of learning, networking,and inspiration! REGISTER TODAY and save!
cancel
Showing results for 
Search instead for 
Did you mean: 
mkosater
Contributor II
Contributor II

tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

When upgrading project from 6.4 to 7.2.1, a job started to fail due to arrayoutofbounds error. I isolated this to the tFTPFileList component.  When reviewing the generation code and comparing it to the prior version it appears there is a bug in the logic.

 

Is this the proper venue to report this or is there a SOP for doing so?

 

6.4.0 code: The loop for pulling the data from the FTP dir method was slightly different.

for (String filemask_tFTPFileList_1 : maskList_tFTPFileList_1) {
java.util.regex.Pattern fileNamePattern_tFTPFileList_1 = java.util.regex.Pattern
.compile(filemask_tFTPFileList_1
.replaceAll("\\.", "\\\\.")
.replaceAll("\\*", ".*"));

for (String filemaskTemp_tFTPFileList_1 : fileList_tFTPFileList_1) {
if (fileNamePattern_tFTPFileList_1
.matcher(
filemaskTemp_tFTPFileList_1)
.matches()) {
fileListTemp_tFTPFileList_1
.add(filemaskTemp_tFTPFileList_1);
}
}
}

 

 

7.1.1 code: Issue is that the ftp_tFTPFileList_1.dir(null, true) and ftp_tFTPFileList_1.dir(null, false) bring back different sized arrays, and the looping logic is based off of the larger array. 

 

String[] fileList_tFTPFileList_1 = ftp_tFTPFileList_1.dir(null, true);
String[] fileListwithNames_tFTPFileList_1 = ftp_tFTPFileList_1.dir(null, false);
ftp_tFTPFileList_1.chdir(rootDir_tFTPFileList_1);
List<String> fileListTemp_tFTPFileList_1 = new java.util.ArrayList<String>();
List<String> fileListTempWithNames_tFTPFileList_1 = new java.util.ArrayList<String>();

for (String filemask_tFTPFileList_1 : maskList_tFTPFileList_1) {
java.util.regex.Pattern fileNamePattern_tFTPFileList_1 = java.util.regex.Pattern
.compile(filemask_tFTPFileList_1.replaceAll("\\.", "\\\\.")
.replaceAll("\\*", ".*"));
for (int i_tFTPFileList_1 = 0; i_tFTPFileList_1 < fileList_tFTPFileList_1.length; i_tFTPFileList_1++) {
String filemaskTemp_tFTPFileList_1 = fileList_tFTPFileList_1[i_tFTPFileList_1];
String filemaskTempName_tFTPFileList_1 = fileListwithNames_tFTPFileList_1[i_tFTPFileList_1];

if (fileNamePattern_tFTPFileList_1.matcher(filemaskTempName_tFTPFileList_1)
.matches()) {
fileListTemp_tFTPFileList_1.add(filemaskTemp_tFTPFileList_1);
fileListTempWithNames_tFTPFileList_1
.add(fileListwithNames_tFTPFileList_1[i_tFTPFileList_1]);
}
}
}

Labels (2)
10 Replies
Anonymous
Not applicable

Hi,

 

    Please report the bug t Talend JIRA forum. The link is as shown below.

 

https://jira.talendforge.org

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

sam2000
Contributor II
Contributor II

Talend Team, 

 

Can you please let us know, if there is any patch available for the bug explained in this thread related to tFTPFileList in v7.2.1?

 

Anonymous
Not applicable

Didn't see JIRA so created ticket:  TDI-43662 tFTPFileList ArrayIndexOutOfBounds exception in 7.2.1

mkosater
Contributor II
Contributor II
Author

So I had put a ticket in with Talend support since the issue was in their enterprise version. Talend support created bug report and then a patch for the issue. I guess I expected that they would have been pushed back into the TOS repo as well. The relationship between the TOS and the for profit side of things is a bit confusing at times.  

vyu
Contributor
Contributor

Please attach the job which exists bugs.

Here is the tutorial https://help.talend.com/reader/HqPypUO_cLctKv_dAWcgFw/fYXMbAVE56gH~wvSDjnxTw

mkosater
Contributor II
Contributor II
Author

The internal case number was 00151288. Patch was Patch_20191015_TPS-3447.  I don't have the test job any longer that reproduces. Here are some notes from the interaction with support.

 

Some details:

  • Our internal server is a windows based server. A public server I hit was Linux based, and my test job at the time does not fail. The FTP server is Windows 2016 running ipswitch wsftp server 8.5.2.
  • I suspect there MAY be a change in behavior in the ftp library that this component uses, in how it handles dir() (using LIST versus NLIST), but regardless, the component should handle it.
  • My test/job uses a file mask (*.*). However the error occurs with or without the mask.

 

Added notes about the folder I am trying to list:

  • happens o be one level of folder off of the root.
  • there multiple folders off of the root.
  • number of files in the folder does not change the behavior.  

My work around was taking the 6.4 code and creating a custom object and I used that. 

 

I put together a new test....


test_ftpfilelist_export.zip
vyu
Contributor
Contributor

Thank for your information. But sorry, I still couldn't find the problem. 

It would be helpful to have the original job.

Anonymous
Not applicable

@yliu  If you're testing with the patch installed, you won't be able to reproduce the error.  The patch is only available to Enterprise Edition users.  As I'm using the community edition, the patch is unavailable to me.  

mkosater
Contributor II
Contributor II
Author

That job I attached is the same that was submitted to Enterprise ticket and used for debugging. I want to emphasize that you need to connect to a FTP server that I mentioned above. I ran the job attached against an internal windows server, and was able to reproduce the error.

 

I think the enterprise engineer who worked the case installed the server mentioned above and was able to reproduce.