topic Re: Spark MinIO connection possible? in Talend Studio

Spark MinIO connection possible?

mattg1221 — Wed, 26 Mar 2025 13:46:49 GMT

Is it possible to create a hadoop cluster connection to a MinIO s3 via Talend Studio?

Re: Spark MinIO connection possible?

quentin-vigne — Wed, 26 Mar 2025 15:53:48 GMT

Yes it is possible with the tS3 Components

Also here is the Talend documentation saying you can do it : https://help.qlik.com/talend/en-US/components/8.0/amazon-dynamodb/s3-compatible-solutions

But if it's not working as expected Talend support won't cover it or will only try since it's not the aim of those S3 components

Have a good day,

- Quentin

Re: Spark MinIO connection possible?

mattg1221 — Mon, 31 Mar 2025 17:01:10 GMT

That worked, thanks for the tips.

Re: Spark MinIO connection possible?

mattg1221 — Mon, 31 Mar 2025 17:30:24 GMT

one question, do you know if there is a way to populate a tdeltainput via a query of an s3 table instead of reading the files of the table? I cannot seem to figure it out using 'Query' instead of 'Folder/Files'