Hello, i'm trying to setup a big data job and i need to add some packages to spark. I'm trying to create an Iceberg data lakehouse on S3 tables. Exactly i need to define the equivalent for the following:
I've tried to configure it in the spark configuration tab adding the property spark.jar.packages and i've added the respective modules in talend, but when i run the job those packages are missing anyway. What's wrong? Is there something more or different i shall do?
Indeed a request, why don't you provide full support to iceberg either in normal jobs?