Loading Multiple csv file from amazon s3 bucket

[Dataguy46]

Hi,

I am trying to pull data from the Amazon S3 bucket, and the path is ibv2-live\activity\server1\,

there are multiple CSV files in this path with prefix activity,

Could you please help me with a script to load this?

Thanks in advance.

[datadiva · Accepted solution]

The best way to make this work without doing loops is to do to the Data Editor.  In there, you should connect to your S3 bucket.  Open the connection and navigate to the file path where the files you want to open are and select one file from the path and click "Insert Script".  

You'll get something like:

LOAD
"Column 1",
"Column 2",
"Column 3"
FROM [lib://<SPACE>:<BUCKET>/<FOLDER>/<FOLDER>/<FILENAME>.csv]
(txt, codepage is 28591, embedded labels, delimiter is ',', msq);

Change the columns listed to LOAD * and remove all of the <FILENAME> portion with the extension and leave the last line of code.  That way you end up with something like this:

LOAD * FROM [lib://<SPACE>:<BUCKET>/<FOLDER>/<FOLDER1>/]

(txt, codepage is 28591, embedded labels, delimiter is ',', msq);

That will pull all of the files.

Also for security it would be wise to not share your actual S3 bucket name and file path in questions.