[Data Integration][Connection][S3 Target] curlCode: 6, Could not resolve hostname.

[HNAKAGAW]

I failed to create S3 target.

Test Connection failed with below message.
Status(StatusCode="Internal", Detail="Data movement test connection failed:SYS-E-HTTPFAIL, Failed to connect to bucket '<My bucket Name>' .. SYS,GENERAL_EXCEPTION,Failed to connect to bucket '<My bucket Name>'., Failed to stat S3 bucket '<MY bucket Name>' AWS get bucket location failed: curlCode: 6, Could not resolve hostname curlCode: 6, Could not resolve hostname. QEM-19, HTTP request failed, Failed to connect using Data Movement gateway local_repsrv.")

And below messeages are outputted in In Replication engline logs(repsrv.log)
----------------------------------------------------------
[FILE_FACTORY]E: curlCode: 6, Could not resolve hostname. [1001705] (anw_retry_strategy.cpp:170)
[FILE_FACTORY]E: AWS get bucket location failed: curlCode: 6, Could not resolve hostname [1001712] (s3_file_actions.cpp:424)
[FILE_FACTORY]E: Failed to stat S3 bucket '<MY bucket Name>' [1001712] (s3_file_actions.cpp.310)
[SERVER]E: Failed to connect to bucket '<MY bucket Name>'. [1020401] (file_imp.c:6096)
----------------------------------------------------------

This connection "use AWS privatelink" and "VPC Endpoint URL" is 'https://<My bucket Name>.<Region>.amazonaws.com'.
Data movement gateway server can reslove <My bucket Name>.<Region>.amazonaws.com' to IP Address.

I want to know what hostname must be resoloved.

If Qlik Replicate , I can check verbose repsrv.log by changing server logging level , I can't find same setting in Qlik Cloud Data Integration Data gateways.

How can I know what hostname must be resoloved?

Best Regards

Nakagawa

[Mohammed_Messaoudi · Accepted solution]

Hello Nakagawa,

When you use AWS PrivateLink with a VPC endpoint, the solution that uses PrivateLink will resolve the VPC endpoint’s DNS name, not the generic S3 regional endpoint.

• Generic S3 endpoint (what you tested manually):
  https://<bucket>.<region>.amazonaws.com
• PrivateLink endpoint DNS name :
  <endpoint-id>.<vpc-endpoint-service-name>.us-east-1.vpce.amazonaws.com (region & VPCE name depend on your AWS setup).

So the hostname that must be resolvable by the gateway is the interface endpoint’s private DNS name, not the public bucket.region.amazonaws.com.

You can determine the hostname from AWS:

1. In AWS console, go to VPC > Endpoints.
2. Find your interface endpoint for S3.
3. Note the DNS names listed:
   • They will look like:
     <endpoint-id>.vpce-<id>.<region>.vpce.amazonaws.com
     or
     <endpoint-id>.s3.<region>.vpce.amazonaws.com
     depending on your AWS configuration.
4. This is the hostname that your gateway must be able to resolve.

[HNAKAGAW]

Hello Mohammed_Messaoudi

I cloud avoid this error today. 

Thank you for your answer.

Best Regards

Nakagawa