topic Re: Data extraction from csv file in QlikView

Data extraction from csv file

Mon, 11 Jul 2016 18:06:52 GMT

Hello Qlik Sense Experts,

I am new to Qlik Sense and trying to extract data using Data Load Editor. I have a csv file with over 500K records but when I extract data using Data load editor, it only extract some 297K records. I checked to find common reason why Sense is missing so many records but could not find one. I created source file with ";" as well as "|" delimited but same results. If I save the file as csv after converting the data to respective columns then Sense loads all records. Am I missing any thing or there is a known issue with large data extraction?

Here is my sample data:

04-000680NM|E|DM0011154 WALL MOUNT BRACK/CNTV DS

04-000680US|E|J070 AL - TUBE BOOTH 84"

04-000681NM|E|PATCH CORD FIBER MM-25M LC/LC

04-000681US|E|TTY - CARD ONLY FIRMWARE CHIP 1.13

The first two records get loaded and the last two never get loaded even if I reduce the dataset to under 4 records.

The Qlik Sense version we are using is 2.2.3.

Any help is appreciated.

Thank you in advance.

~Kuljit

Re: Data extraction from csv file

MarcoWedel — Mon, 11 Jul 2016 23:15:44 GMT

please post a sample of your csv file you cannot load (meaning a file itself, not an extract copy/pasted into your post)

thanks

regards

Marco

Re: Data extraction from csv file

Tue, 12 Jul 2016 15:59:21 GMT

This seems like a bug in the Qlik Sense.

This is a problem with CSV with embedded " marks. Line # 2 has " at the end, but a csv always assumes a quote always comes in pairs. It does not find so it stops.

Easy way is NOT to pull is with delimiter of "|", but then use this character to parse the line.

Here is a copy & paste from Qli sense that I did and I got 4 lines.

Load

Subfield (@1, '|', 1) as colum1,

subfield (@1, '|', 2) as column2,

subfield (@1, '|', 3) as column3

;

LOAD

FROM [lib://Testcsv/Test.txt]

(txt, codepage is 1252, no labels);

Re: Data extraction from csv file

Anil_Babu_Samineni — Tue, 12 Jul 2016 16:21:53 GMT

Kuljit,

Is this file really in CSV, Because you are getting data from TXT(Here it showing)

FROM [lib://Testcsv/Test.txt]

Re: Data extraction from csv file

Tue, 12 Jul 2016 16:39:58 GMT

Hi Anil

I sample I loaded shows the txt but same applies for csv. I am using pipe as delimiter. I loaded data this way and it loads all records.

Re: Data extraction from csv file

Anil_Babu_Samineni — Tue, 12 Jul 2016 17:02:45 GMT

Is there any chance to post on wall that CSV file

Re: Data extraction from csv file

Tue, 12 Jul 2016 18:44:27 GMT

Hi Anil, You can copy the sample rows given into a csv file and try to load it.