delta.columnMapping.mode

[MBourassa1682971203]

Hi,

I have created a big data batch in spark 3.3.

I use a TDeltaLakeOutPut to write in a Delta files.

How can I add this option "delta.columnMapping.mode" ?

Ref: https://community.databricks.com/t5/data-engineering/invalid-characters-in-column-names-quot-n-t-quo...

https://docs.delta.io/latest/delta-column-mapping.html

Thank you

[Anonymous]

Hello,

Please try to  spark properties in Spark Configuration UI and feel free to let us know if it works.

Best regards

Sabrina

[MBourassa1682971203]

Hi,

it works !!

Thank you very much

I'm looking for this since a long time.

Can you tell me where did you find theses properties ?

Thank you

[MBourassa1682971203]

It works but there is a litle problem with the column partition.

This is my configuration:

and this is the result:

instead of this:

[MBourassa1682971203]

this appen only if I add "spark.databricks.delta.properties.defaults.columnMapping.mode" "name" to the spark properties.

[Anonymous]

Hello,

I will check it with our big data experts and then come back to you as soon as possible.

Best regards

Sabrina