AWS CLI Command not working while executing in Apache airflow.

Ganshyam — Fri, 15 Nov 2024 21:11:07 GMT

Hello All,

I am trying to execute the AWS CLI command to list the s3 folder file details through tsystem component and create the txt file from the output of the command.

aws s3 ls s3:.//bucket/bucket_path >> abc.txt

It is working fine in lower environment , where we are running in the Talend Execution servers.

But in higher environment jobs are containerized , where the job docker image are created and it is run via airflow , which hits the ECS fargate.

My doubt is whether the AWS CLI command is not getting executed in ECS fargate , any permission required to execute the command ?

topic AWS CLI Command not working while executing in Apache airflow. in Talend Studio

AWS CLI Command not working while executing in Apache airflow.