Generic Load is creating duplicate records, how to overcome this in Qlik Sense?

[Prabhu]

Hi All,

I am trying to create a generic load with the below-given script.  But the output table(Unique) is having duplicate records. Actually, it should have unique records. 

SchemaData:
LOAD distinct
"Case ID",
taskName,
taskStatus,
SchemaAge,
SchemaStartTimestamp,
SchemaAgeAtTimestamp
FROM [lib://BOB/qvd/SchemaData.qvd]
(qvd);


Unique:
LOAD Distinct
"Case ID"
RESIDENT SchemaData;

task:
GENERIC LOAD "Case ID", taskName &':Task Status', taskStatus
RESIDENT SchemaData;

drop table SchemaData;


FOR i = NoOfTables()-1 to 0 STEP -1
LET vTable=TableName($(i));
IF WildMatch('$(vTable)', 'task*') THEN
LEFT JOIN (Unique) LOAD * RESIDENT [$(vTable)];
DROP TABLE [$(vTable)];
ENDIF
NEXT i

@Saravanan_Desingh @vamsee 

[canerkan]

Hi Prabhu,

maybe try this change in your Script:

SchemaData:
LOAD distinct
"Case ID",
taskName,
taskStatus,
SchemaAge,
SchemaStartTimestamp,
SchemaAgeAtTimestamp
FROM [lib://BOB/qvd/SchemaData.qvd]
(qvd);

Unique:
LOAD Distinct
"Case ID"
RESIDENT SchemaData;

task:
GENERIC LOAD "Case ID", taskName &':Task Status', taskStatus
RESIDENT SchemaData;

drop table SchemaData;

FOR i = NoOfTables()-1 to 0 STEP -1
LET vTable=TableName($(i));
if SubField(vTable,'.',1) = 'task' THEN
LEFT JOIN (Unique) LOAD * RESIDENT [$(vTable)];
DROP TABLE [$(vTable)];
ENDIF
NEXT i

Maybe there are Tables with Names that also contain the word 'task*' and are therfore joined without being part of the generic load.

Let me know if it helped.

Regards,

Can.

[Prabhu]

Hi canerkan,
Thanks for the update. But it's not working. I am still getting duplicate
records.

[canerkan · Accepted solution]

Please check if there are any duplicates in the Table you are using for your generic load. It could be a Data Quality Issue.

[Prabhu]

Yes Canerkan, my data has duplicates that is causing the problem. There is no problem with generic load.