List files in S3 bucket folder using wildmatch operator

prayner — Wed, 03 Jul 2024 09:41:31 GMT

I am trying to get a list of files from an S3 bucket using string matching operations. I tried using FileList(), but it doesn't work with the S3 bucket.

for each vFile in filelist('lib://S3Dir/'&'*'&'.qvd')

Load id from [vFile](qvd)
;

next vFile

Using absolute file paths works so I know it is not an issue with the connection. I found some Qlik forum articles discussing the use of tS3List, which seems to be a Talend data integration function that I cannot use it in Qlik load editor.

As a Qlik SaaS developer, how do I get this list of files from the S3 bucket using string matching operations or regex in the data load editor?

Re: List files in S3 bucket folder using wildmatch operator

prayner — Wed, 03 Jul 2024 11:54:34 GMT

Found the solution here: https://community.qlik.com/t5/New-to-Qlik-Analytics/Use-of-for-files-in-Amazon-S3/td-p/1718740

My final solution was:

SUB ConcatenateFiles(table_type)

for each vFile in filelist('$(vS3Dir)')

IF WildMatch('$(vFile)', '*$(table_type).qvd') THEN
Load * from [$(vFile)](qvd)
;
END IF

next vFile

END SUB

Call ConcatenateFiles('table_pattern');

topic List files in S3 bucket folder using wildmatch operator in Connectivity & Data Prep

List files in S3 bucket folder using wildmatch operator

Re: List files in S3 bucket folder using wildmatch operator