Skip to main content
Announcements
Qlik Introduces a New Era of Visualization! READ ALL ABOUT IT
cancel
Showing results for 
Search instead for 
Did you mean: 
Prabhu
Creator
Creator

Generic Load is creating duplicate records, how to overcome this in Qlik Sense?

Hi All,

I am trying to create a generic load with the below-given script.  But the output table(Unique) is having duplicate records. Actually, it should have unique records. 

 

SchemaData:
LOAD distinct
"Case ID",
taskName,
taskStatus,
SchemaAge,
SchemaStartTimestamp,
SchemaAgeAtTimestamp
FROM [lib://BOB/qvd/SchemaData.qvd]
(qvd);


Unique:
LOAD Distinct
"Case ID"
RESIDENT SchemaData;

task:
GENERIC LOAD "Case ID", taskName &':Task Status', taskStatus
RESIDENT SchemaData;

drop table SchemaData;


FOR i = NoOfTables()-1 to 0 STEP -1
LET vTable=TableName($(i));
IF WildMatch('$(vTable)', 'task*') THEN
LEFT JOIN (Unique) LOAD * RESIDENT [$(vTable)];
DROP TABLE [$(vTable)];
ENDIF
NEXT i

 

 

@Saravanan_Desingh @vamsee 

Labels (2)
1 Solution

Accepted Solutions
canerkan
Partner - Creator III
Partner - Creator III

Please check if there are any duplicates in the Table you are using for your generic load. It could be a Data Quality Issue.

View solution in original post

4 Replies
canerkan
Partner - Creator III
Partner - Creator III

Hi Prabhu,

maybe try this change in your Script:

 

SchemaData:
LOAD distinct
"Case ID",
taskName,
taskStatus,
SchemaAge,
SchemaStartTimestamp,
SchemaAgeAtTimestamp
FROM [lib://BOB/qvd/SchemaData.qvd]
(qvd);


Unique:
LOAD Distinct
"Case ID"
RESIDENT SchemaData;

task:
GENERIC LOAD "Case ID", taskName &':Task Status', taskStatus
RESIDENT SchemaData;

drop table SchemaData;


FOR i = NoOfTables()-1 to 0 STEP -1
LET vTable=TableName($(i));
if SubField(vTable,'.',1) = 'task' THEN
LEFT JOIN (Unique) LOAD * RESIDENT [$(vTable)];
DROP TABLE [$(vTable)];
ENDIF
NEXT i

 

 

Maybe there are Tables with Names that also contain the word 'task*' and are therfore joined without being part of the generic load.

 

Let me know if it helped.

Regards,

Can.

 

 

Prabhu
Creator
Creator
Author

Hi canerkan,
Thanks for the update. But it's not working. I am still getting duplicate
records.
canerkan
Partner - Creator III
Partner - Creator III

Please check if there are any duplicates in the Table you are using for your generic load. It could be a Data Quality Issue.

Prabhu
Creator
Creator
Author

Yes Canerkan, my data has duplicates that is causing the problem. There is no problem with generic load.