topic Re: Incomplete CSV Loading in QlikView

Incomplete CSV Loading

agni_gold — Fri, 27 May 2016 11:30:05 GMT

Hi All ,

I am trying to load CSV having 1 million records , but it stopes at 7 lacs , And my CSV is delimited by ~ .

I have tried many way to handle this issue by Qoutes none and ingnore end of file like that options , but when i am using it it is loading more than 1 million records.

Re: Incomplete CSV Loading

Peter_Cammaert — Fri, 27 May 2016 11:31:55 GMT

Can you post that part of the CSV where QlikView stops loading? Let's say 10 lines before and 10 lines after...

Re: Incomplete CSV Loading

Fri, 27 May 2016 11:32:26 GMT

Can you send a sample of the CSV file?

Re: Incomplete CSV Loading

agni_gold — Fri, 27 May 2016 11:42:04 GMT

Sorry csv i cant because it is in client area . But it is not giving any error while loading execution will sucessfully finished at 7 lac records.

Is there any way to find which row creating error?

Re: Incomplete CSV Loading

agni_gold — Fri, 27 May 2016 11:44:29 GMT

stalwar1‌

gwassenaar‌

Any suggestion on this

Re: Incomplete CSV Loading

muthukumar77 — Fri, 27 May 2016 11:44:50 GMT

Hi,

Can you send me error in screenshot?

Re: Incomplete CSV Loading

Fri, 27 May 2016 11:46:52 GMT

Use the First function to see where is breaking.

Try Half million if it works load more if not start to reduce until you have a clue where it is.

You might have a data error in the data and you need to make some data cleaning.

example: First 10 Load * from abc.csv;

Re: Incomplete CSV Loading

santiago_respane — Fri, 27 May 2016 11:49:26 GMT

Hi, always that i had this kind of problems is was because of a bad formed csv,

Have you tried adding the "no eof" specifier in your load?

LOAD * from myfile.csv (txt, utf8, embedded labels, delimiter is '~', no eof);

Hope this helps.

Kind regards,

Re: Incomplete CSV Loading

agni_gold — Fri, 27 May 2016 11:54:20 GMT

It is loading more that exact rows....

Total Records in CSV : 1 Million

But it is loading 1.8 million

Re: Incomplete CSV Loading

Peter_Cammaert — Fri, 27 May 2016 12:01:19 GMT

Your Op has a title that says "Incomplete CSV Loading", and now you say that almost double the amount of rows are loaded? That could be caused by Newline-characters (ASCII value 10) in text fields in your csv file. They will be treated as line separators, turning every row into two rows.

BTW what does "lacs" mean?

Peter

Re: Incomplete CSV Loading

agni_gold — Fri, 27 May 2016 12:02:58 GMT

So how can i handle this problem.

Re: Incomplete CSV Loading

marcus_sommer — Fri, 27 May 2016 12:10:32 GMT

I think you should look at the data within an editor like notepad ++ to see which character are really there then like Peter mentioned you will very probably have more delimiters as the "~" char.

An a small example will be really helpful - for 10/20 records you could manually scramble the data (I know it's an ugly job).

- Marcus

Re: Incomplete CSV Loading

santiago_respane — Fri, 27 May 2016 12:16:14 GMT

I agree with Marcus, you should verify also which characters do you have as end of line (carriage return and/or line feed), if this is misinterpreted by QV you will end up loading more records than you have.

Hope this helps.

Kind regards,

Re: Incomplete CSV Loading

Fri, 27 May 2016 13:28:08 GMT

By “lacs” my guess is that Agnivesh meant units of 100,000: Lakh - Wikipedia, the free encyclopedia