Qlik Community

Qlik Sense App Development

Discussion board where members can learn more about Qlik Sense App Development and Usage.

vasiliy_beshent
Contributor III

Count of symbols in one field?

Hello, everyone today!

I have a problem with loadind log file.

So first of all i have a big log file, about 4 mb.

It's got more than 16000 rows in.

2017-01-23 14_47_27-Qlik Sense Desktop.png

I'm loadind data like this, 16000+ rows in one field.

2017-01-23 14_54_47-Qlik Sense Desktop.png

I'm already write handler for this 1 row.

So here is my question: WHY QLIK SENSE CUTTING MY FILE?!

It's not full load of my file! I'm write that 1 record is 16000+ lines but after loadind it's NOT!

How i get the point? Take a closer look.

My last record after working handle is

2017-01-23 15_01_12-Qlik Sense Desktop.png

Let's take a look where is this record in file in NOTEPAD:

2017-01-23 15_03_36-.png

Why it's not full load of file?

1 Solution

Accepted Solutions
vasiliy_beshent
Contributor III

Re: Count of symbols in one field?

Thanks again, folks!

The right solution is:

2017-01-24 15_01_23-Qlik Sense Desktop.png

12 Replies
vasiliy_beshent
Contributor III

Re: Count of symbols in one field?

I found that in my 1 row it load only 680992 symbols.

vasiliy_beshent
Contributor III

Re: Count of symbols in one field?

All file got 2 725 462 simbols (looking in MS Word)

vasiliy_beshent
Contributor III

Re: Count of symbols in one field?

I found that if copy/paste all data from log file into txt file it become less weight (2 mb) and now count of symbols in 1 row is became 1 000 001

Re: Count of symbols in one field?

Could you explain what you mean by "symbols"? Do you mean distinct values? can you post your complete script?

If you are loading all rows into a single field and then processing that field, you may be running into some length restrictions depending of what statements you are using to process it.

-Rob

vasiliy_beshent
Contributor III

Re: Count of symbols in one field?

Hello, Rob, thanks for join discuss!

When i say "symbols" i mean count of chars in 1 row (in this example).

Or when you use Len(Field) - you can get this count.

So, when i load data from file i use this scripts:

1) It's the last, it paste into 1 row 1 000 001 chars.

2017-01-24 09_25_18-Qlik Sense Desktop.png

2) It's the first load script, that i use, it give me 680 992 chars in 1 row.

2017-01-24 09_31_14-Qlik Sense Desktop.png

(Do not be confused by filenames, i just copy all from log into txt)

Re: Count of symbols in one field?

Try adding "no eof" to your format specification and see if that makes a difference.  I've never tried to read that many ( 2 725 462) bytes into a single field, so you may have other problems as well.  Can you process this as individual rows? If you are trying to tie multiple lines together, you can do that by assigning a logical record number to each and use concat() to group them back together,

-Rob

vasiliy_beshent
Contributor III

Re: Count of symbols in one field?

Thanks, no eof does nothing.

How to assign this logical number to each row?

This what i get when load as individual rows:

2017-01-24 10_42_30-Qlik Sense Desktop.png

I need to concat by this groups, how to do that?

vasiliy_beshent
Contributor III

Re: Count of symbols in one field?

Made this:

2017-01-24 10_50_34-Qlik Sense Desktop.png

So what next? Please, help

Re: Count of symbols in one field?

I think Rob is right and you shouldn't load everything within a single field-value else loading them row by row, maybe with an included recno(), rowno() and maybe some further counter per peek() or previous() in a following ordered resident load or if the datastructure is consistent (or could be mode) with something like:

mod(rowno(), 12) as FieldID,

ceil(rowno() / 12) as RecordID

which I used for a logfile which had a stream-format and each 12 rows belonged to one record. If this isn't helpful in any way please elaborate what is the intention to keep the whole content within a single field-value.

- Marcus

Community Browser