unwanted incomplete load of a datafile

[#1]

Hello,

i have a very simple but large csv file to load, with around 48M lines.

The script below always stops at 25,380,219 with no error, saying load is complete, which is false.

LOAD @1,@2, @3, @4, @5 FROM lines.csv (txt, codepage is 1252, no labels, delimiter is ';', msq);

My QlikView installation is new, and no special options have been set.

Has anyone already faced this issue ?

Thanks

[#2]

Precision : QlikView personnal Edition v 9.00.7502.8 SR5

[rwunderlich · Accepted solution]

I've seen these problems before caused by data problems, usually improper quoting. For example a " mark appearing within a field. This is common in the US where " in used to mean "inches", such as " 19" monitor ".

Look at the data around line 25,380,219, even delete some lines before and after to see if you can isolate the bad line(s).

[#4]

Hi,

Easiest way to get rid of unbalanced quotes is to drop them from a batch file . Get the command line tools from GnuWin32 (or Cygwin)

cat file.csv | tr -d \042 | tr -d \047 > filterred_file.csv

-Alex

[#5]

You could also change the quoting options. Instead of:

LOAD @1,@2, @3, @4, @5 FROM lines.csv (txt, codepage is 1252, no labels, delimiter is ';', msq);

Use:

LOAD @1,@2, @3, @4, @5 FROM lines.csv (txt, codepage is 1252, no labels, delimiter is ';', None);

-Alex

[#6]

Hello Rob

I haven't found the extra '"', but by using "no quote" instead of MSQ, it works fine

Thanks for your help !

Bye

[#7]

Hello Alexandru

It works !

Thanks !