topic Version 9 - What's new - Store INTO plain text file in QlikView

Version 9 - What's new - Store INTO plain text file

Tue, 17 Nov 2009 23:03:49 GMT

Hello !

I've seen that on version 9 we can dump a table onto a plain text file.

Always wanted that mostly to check how the load process was.

But I noticed that there are weird characters on the beginning of the output file.

On the attached kit you can see the problem by running the load process and then checking the 'test122.txt' output file.

Thanks !

Version 9 - What's new - Store INTO plain text file

hector — Wed, 18 Nov 2009 00:01:45 GMT

It works fine to me

NAME_TEST;DATE_TEST;TIME_TEST;VALUE_TEST
MARY;01/01/2008;00:12:14;200
MARY;01/02/2008;00:12:15;200
ANN;04/01/2008;15:12:25;300
MARY;01/01/2009;00:20:20;210
JOHN;31/12/2008;14:20:30;100

rgds

Version 9 - What's new - Store INTO plain text file

Wed, 18 Nov 2009 00:02:25 GMT

on my machine, the output file is :

NAME_TEST;DATE_TEST;TIME_TEST;VALUE_TEST
MARY;01/01/2008;00:12:14;200
MARY;01/02/2008;00:12:15;200
ANN;04/01/2008;15:12:25;300
MARY;01/01/2009;00:20:20;210
JOHN;31/12/2008;14:20:30;100

it looks fine

Version 9 - What's new - Store INTO plain text file

Wed, 18 Nov 2009 00:08:01 GMT

Please open the file NOT ON NOTEPAD !

LOL

Open the file on an editor that SHOWS you the weird characters !

I use TSE/32 but one can use TextPAD or whatever.

I can see that the file begins with character # 239 from ASCII table then followed by character # 187 and then char # 191. Then comes the header...

Version 9 - What's new - Store INTO plain text file

hector — Wed, 18 Nov 2009 00:16:40 GMT

hi again, i've opened it with, notepad, notepad++, wordpad and i can't see nothing!! LOL

But, with EDIT from DOS, i see this image

do you suggest any other text editor?

rgds

Version 9 - What's new - Store INTO plain text file

Wed, 18 Nov 2009 00:17:53 GMT

oh ok

it's simply not an ASCII text file

sometimes the first bytes in text files are used to specify used encoding

unfortunately i'm far from being an expert in text encoding, so i can't give you any details

Version 9 - What's new - Store INTO plain text file

Wed, 18 Nov 2009 00:23:55 GMT

you can view the weird characters in notepad++ in hexadecimal mode

Version 9 - What's new - Store INTO plain text file

Wed, 18 Nov 2009 00:26:14 GMT

So, by simply using EDIT, you can see the weird characters.

AFAIK that is not a plain ASCII file.

Or is that part of the UTF-8 specification ?

Version 9 - What's new - Store INTO plain text file

rwunderlich — Wed, 18 Nov 2009 02:45:56 GMT

The x'EFBBBF' at eh beginning of the file if the UTF-8 "Byte Order Mark" (BOM) http://en.wikipedia.org/wiki/Byte_order_mark .

I'm not clear on exactly how the new STORE to txt feature is supposed to work, but I would guess that if you specify ANSI or OEM in the format spec there would be no BOM. Don't know that for a fact though. You may want to run this by support if the BOM is causing you problems.

-Rob

AW:Re: Version 9 - What's new - Store INTO plain text file

rbecher — Thu, 19 Nov 2009 02:00:05 GMT

Rob,

I'm facing the same problem. Where do I specify ANSI or OEM in the format spec?

- Ralf

AW:Re: Version 9 - What's new - Store INTO plain text file

rwunderlich — Thu, 19 Nov 2009 03:01:35 GMT

In the format-spec parameter.

STORE TEST INTO TEST122X.TXT (txt, oem, delimiter is '|');

What I didn't make clear in my last post was that oem/ansi do not produce the desired effect. The STORE doc does not show options beyond txt/qvd, but based on Adriano's example, the format spec can include all the parms used in a LOAD format-spec. For example, "delimiter" is accepted and affects the output. However, oem or ansi seem to have no effect as the ouptut always has the utf-8 BOM.

I'm hoping that someone who is experiencing the problem will contact support to work this through.

-Rob

AW:Re: Version 9 - What's new - Store INTO plain text file

Thu, 02 Dec 2010 15:28:45 GMT

Hi!

Last week I run into that problem. Trying to store a txt-file with codepage = 1252 (ANSI).

Exemple:

store Data into Data.txt (txt, codepage is 1252, embedded labels, delimiter is '\t');

I needed to store the file in encoding = ANSI. When I loaded the file in to a SQL-database the result was not the expected. I then open the txt-file with NotePad and I saw that Encoding was UTF-8 not ANSI. I now store data in txt and then open it with Notepad and save it as Encoding = ANSI.

I use QV9.0 SR5. Have someone contacted support to work this through? Can you in SR6 store a txt-file with something else then UTF-8?

Regards
Kristofer

Re: AW:Re: Version 9 - What's new - Store INTO plain text file

gussfish — Tue, 17 Sep 2013 23:26:35 GMT

This post gives a workaround and a long-term solution to this problem.

Background

I've just run into this problem, trying to output a text file for loading into a downstream system. That system is rejecting the file because of that 3-character so-called UTF-8 BOM. As others have discussed in prior posts, these characters aren't shown in the standard Windows GUI text tools e.g. Notepad. They are displayed, though, by using TYPE at the Windows command line.

I contacted QlikView Support, but as far as they're concerned, it's working as intended.

Rob Wunderlich's hopeful hypothesis sadly fails: STORE's format-spec ignores character-encoding parameters.

Workaround
1. If you have the option and skill, write a program in an appropriate system-level programming language (e.g. C or PERL) that strips-out the 1st 3 characters, and invoke that program via the EXECUTE command in your QlikView script. (Sadly, my environment doesn't let me readily install new executables (i.e. the stripping script) on our servers, so this isn't actually an option for me).

2. Otherwise, manually strip-out the characters using Notepad: open the file in Notepad; then Save As, selecting Encoding: ANSI in the Save As dialog.

Long Term Solution

Vote for my Idea at http://community.qlik.com/ideas/3071 'STORE to provide flexible character encoding options'.

Angus.