tSqlRow - SQL Spark Context.

Anonymous — Sat, 16 Nov 2024 09:18:56 GMT

Hi,

I want to eliminate duplicate row using tSqlRow. I wrote SQL Hive Context query but want to convert into SQL spark context. Please find below hive query.

"Select B.* from (select row_number() over(partition by id order by acc_id desc, top_id desc) as row_numb, A.* from row1 A) as B where row_numb=1"

Can I use row_number using SQL Spark context. I can't use tUniqueRow component.

Thanks.

Re: tSqlRow - SQL Spark Context.

Anonymous — Fri, 22 Sep 2017 09:04:01 GMT

Hello,

if row_number() is a valid spark sql query, you can use it in your job.

Best regards

Sabrina

topic Re: tSqlRow - SQL Spark Context. in Talend Studio

tSqlRow - SQL Spark Context.

Re: tSqlRow - SQL Spark Context.