Surrogate Key Generation in Talend Big Data Batch Spark Job

[ravi999]

Hi All , We were unable to use the Numeric Sequence (in tmap) in Talend Big Data Batch Spark Job as the spark job is running with multiple executors and is creating duplicate keys. We don't want to use UUID as we want to have a integer value to surrogate key column . Could you please let us know how to generate unique surrogate key in Talend Big Data Batch Spark Job ?

[manodwhb]

You can use tSqlrow and generate numeric sequence by writing similar way of oracle query form the input row.