Unexpected behaviour with tS3[components] for working with Minio / AWS S3 Compatible Object Storage

IRe — Fri, 15 Nov 2024 21:28:13 GMT

Hi @Everybody,

i have a special use cases which contains connectivity and data transfer with a (private on-prem) minio object server. ( MinIO ) It is AWS S3 Compatible Object Storage.

Currently we try to connect to this object store via Talend Open Studio and the component tS3Connection which contains on tab <Adv. Settings> a field named <Custom Region Endpoint>. It is configured correct and works fine for minio data store. A listing of the buckets within this object storage via component tS3BucketList is also successful.

But every other component like:

tS3Get
tS3Put
tS3Copy
tS3Delete
tS3List
tS3BucketExist
tS3BucketCreate
and many more...

will fail with error <service not available>.

I am also confused that I can configure a custom endpoint in component tS3Connection, but when i want to configure it in each other component like tS3Get a field for entering a custom endpoint is missing.

Why we have here a difference?

Is it possible that all the other components will use just the standard AWS S3 endpoint and not the configured custom endpoint? It seems this is an error...

We can see in our logs that the action Bucket-Listing is always detected and successful, but all the other actions are not detected and seen within logs.

Could anyone help here with a hint?!

Greetings from Germany

In Re

Re: Unexpected behaviour with tS3[components] for working with Minio / AWS S3 Compatible Object Storage

IRe — Mon, 21 Aug 2023 10:59:58 GMT

Here is a part of the Talend console output of component <tS3List> for Listing all bucket objects:

You can see that in our own object storage is a bucket named <test>, but then it tries to execute the second action <list objects> and it fails.

topic Unexpected behaviour with tS3[components] for working with Minio / AWS S3 Compatible Object Storage in Talend Studio

Unexpected behaviour with tS3[components] for working with Minio / AWS S3 Compatible Object Storage

Re: Unexpected behaviour with tS3[components] for working with Minio / AWS S3 Compatible Object Storage