Hadoop Cluster Conn - Hadoop Properties to Context problems
Hi,
I'm using TOS BD version 6.0.1. I set up a Metadata Hadoop ClusterConnection with some Hadoop properties specified to support HA.
I then exported as a context. All that seems to work fine. The context definition looks correct except for a couple problems.
1. Some hadoop properties have a "-" (hyphen) in the name like "dfs.ha.automatic-failover.enabled.epsdatalake". When these properties get exported to a context they automatically get a name like context.ClusterName_dfs_ha_automatic-failover_enabled_epsdatalake. The "-" (hyphen) is illegal for Java variable names and will cause compiler errors when it's used. This is easy enough to work around by manually editing context variable name.
2. When creating a component with the Repository connection, like tHDFSConnection the Hadoop properties do automatically show up in the Hadoop Properties table as one would expect. However the all the context variables in the Value column are quoted. They are context variables at this point and not text so should not be quoted. Removing the quotes allows jobs to work correctly, however then the connection Property Type changes from Repository to Built-In. This negates much of the point of using a repository connection.
Thanks
Pete
When creating a component with the Repository connection, like tHDFSConnection the Hadoop properties do automatically show up in the Hadoop Properties table as one would expect. However the all the context variables in the Value column are quoted. They are context variables at this point and not text so should not be quoted. Removing the quotes allows jobs to work correctly, however then the connection Property Type changes from Repository to Built-In. This negates much of the point of using a repository connection.
Would you mind posting your tHDFSConnection component setting screenshot with the context variables into forum?
Best regards
Sabrina
Hi , I am using context parameters for both hadoop properties and value columns in hdfs connection component. using repository connection however hadoop properties column showing with double quotes.They are context variables at this point and not text so should not be quoted. I need to use only repository connection. could you please help on this. Thanks.
