Skip to main content
Announcements
Introducing a new Enhanced File Management feature in Qlik Cloud! GET THE DETAILS!
cancel
Showing results for 
Search instead for 
Did you mean: 
IRe
Contributor III
Contributor III

Unexpected behaviour with tS3[components] for working with Minio / AWS S3 Compatible Object Storage

Hi @Everybody,

i have a special use cases which contains connectivity and data transfer with a (private on-prem) minio object server. ( MinIO ) It is AWS S3 Compatible Object Storage.

Currently we try to connect to this object store via Talend Open Studio and the component tS3Connection which contains on tab <Adv. Settings> a field named <Custom Region Endpoint>. It is configured correct and works fine for minio data store. A listing of the buckets within this object storage via component tS3BucketList is also successful.

But every other component like:

  • tS3Get
  • tS3Put
  • tS3Copy
  • tS3Delete
  • tS3List
  • tS3BucketExist
  • tS3BucketCreate
  • and many more...

will fail with error <service not available>.

I am also confused that I can configure a custom endpoint in component tS3Connection, but when i want to configure it in each other component like tS3Get a field for entering a custom endpoint is missing.

Why we have here a difference?

Is it possible that all the other components will use just the standard AWS S3 endpoint and not the configured custom endpoint? It seems this is an error...

We can see in our logs that the action Bucket-Listing is always detected and successful, but all the other actions are not detected and seen within logs.

Could anyone help here with a hint?!

Greetings from Germany

In Re

Labels (4)
1 Reply
IRe
Contributor III
Contributor III
Author

Here is a part of the Talend console output of component <tS3List> for Listing all bucket objects:

 

0695b00000nR59XAAS.pngYou can see that in our own object storage is a bucket named <test>, but then it tries to execute the second action <list objects> and it fails.