topic Re: parquet table error in Talend Studio

parquet table error

mshafeeq — Fri, 15 Nov 2024 22:00:57 GMT

hello ALl,

i'm facing this error while run job to insert data into parquet file format

org.apache.hive.service.cli.HiveSQLException: Error while processing statement: Cannot modify parquet.compression at runtime. It is not in list of params that are allowed to be modified at runtime

Re: parquet table error

Anonymous — Fri, 10 Mar 2023 05:20:32 GMT

Hello,

By default, the HDP cluster has been configured to set these values as "final" and not modifiable by a job, but a cluster administrator can change them to non-final. This can depend on many factors, such as their network configuration, their security policies, etc.

In this case as you can see in the error you are getting the cluster is thinking that the job is trying to modify a parameter hence the error you were getting.

Could you please post your job design here? Is it a hive job? Are you trying to pass that parameter(parquet.compression) within the hive job?

Best regards

Sabrina

Re: parquet table error

mshafeeq — Sun, 12 Mar 2023 11:32:14 GMT

@Xiaodi Shi hello ,

this is the job design , put file from local directory to hdfs and create table parquet and insert data from file to this table

Re: parquet table error

Anonymous — Mon, 13 Mar 2023 04:53:36 GMT

Hello,

My recommendation will be in the thivecreatetable component that you are using to uncheck the resource manager checkbox, the scheduler and also the namenode checkbox and that should make that error go away.

Also check and see if you are passing that property through hadoop properties.

Feel free to let us know if it helps.

Best regards

Sabrina