

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Duplicate rows when storing data into csv
Hi,
I'm storing fields from a table into a csv file in the following way:
Ads:
LOAD
LoginID,
LoginCreated,
LoginDeletionDate,
AdID,
AdCreated,
AdStartDate,
AdPublished,
SourceID,
.... (and some more)
Resident Dataset;
STORE * from Ads into $(pathQVDs)Seller\ads.csv(txt);
This works fine, but many rows in the csv are exactly the same but are duplicated several times (some 5 times, some 30 times).
Here's some sample lines:
LoginID,LoginCreated,LoginDeletionDate,AdID,AdCreated,AdStartDate,AdPublished
123456,28.09.2015,,134752458,28.09.2015,28.09.2015,,,12.11.2015,,6
123456,28.09.2015,,134752496,28.09.2015,28.09.2015,,,12.11.2015,,6
When I create a table object and add the fields in the same order, I have only one line! So I'm really sure that the lines are exact duplicates.
I have no idea why this happens. Has anyone experienced the same issue?
Many thanks for any hint!
KR, Nigel
- Tags:
- store csv
Accepted Solutions

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
u can try
Load
Disctinct...
.....
....
store (qvd)

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
sample:
Hi, use the keyword "DISTINCT" and RowNo() function.
I.E.
TABLE1:
LOAD DISTINCT
FIELD_01,
FIELD_02,
FEILD_03,
...
FIELD_09
;
with this keyword all rows with the same values on all fields are reduced to one row and the table are load with no duplicate rows.
TABLE2:
LOAD
RowNo() AS NUMBER,
*
RESIDENT TABLE1;
DROP TABLE1;
with RowNo() you can enumerate the rows in a table.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
In provide sample lines AdID different so those lines are not duplicate.
If it really contain duplicate check your value in DataSet.
Regards
Please appreciate our Qlik community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂


- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
The Table Object doesn't display the duplicate values (if all field values are Same)..
You can check it in the Table Viewer, It will shows how many rows in the table..
for eg. When i load the below 2 rows, the table object will display single row only.. see the screen shot.
LOAD * Inline [
Dim1,Dim2,Dim3
A,B,C
A,B,C
];
Table Viewer shows : 2 Rows


- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Chanty,
beginner's mistake I missed the DISTINCT
thanks!

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hmmm noprb....
u can make others as helpful. which is other try fr ur requirment.
Thanks
Suresh chanty
