Qlik Community

New to QlikView

Discussion board where members can get started with QlikView.

Not applicable

upload only new csv

Hi to all,

I’m Cesare from italy and I’m new on qlikview.

I need support on new project I’m working on

I have a folder where weekly I deposit a csv file

I have built a simple script that allow me to upload all file present and store them in a QVD.

Something like this

Data:

LOAD
CONN_ID,
[CONCETTI ESTRATTI],
Cluster,
AgentGroup,
Team,

CHANNEL,


FROM

(txt, codepage is 1252, embedded labels, delimiter is ';', msq);

STORE Data INTO data.qvd (qvd);

The problem is when I deposit a new file, there are a lot of csv and the upload process is becoming too long.

I try to test some script in qlikview forum to upload only new file  on qvd but I failed everytime.

Someone can help me about that?

Thanks

Cesare

7 Replies
laddu_927
Valued Contributor

Re: upload only new csv

Hey Cesare,

Welcome to Qlik,

Please use below like to do the incremental load.

Incremental Load of New Files Only

You can use filename related functions (find here) to traverse your file name and proceed.

annafuksa1
Contributor III

Re: upload only new csv

add a date into your csv file and in a code add condition where for example date > WeekStart(today())

so only this week data

Not applicable

Re: upload only new csv

thank,

can you help me with a script that I can use for upload new csv data on QVD from a desktop folder.

Thank you

laddu_927
Valued Contributor

Re: upload only new csv

Can you provide the filename formats ?? or screenshot of the files in your folder?

Not applicable

Re: upload only new csv

Hi,

the file are

OLD FILE ON FORLDER

sales_01022017.csv

sales_01012017.csv

NEW FILE ON FOLDER

sales_01032017.csv

and the folder is    'C:\Users\baronices\Desktop\Sales'

I write successfully a script that load all files into qvd file.

When I deposit a new file (sales_01032017) I want to store the new file data on qvd without reload old files

// Load QVD

[DATA_TABLE]:

LOAD

     Datarif,

     Nome,

     Amount

FROM

(qvd);

thanks very much

laddu_927
Valued Contributor

Re: upload only new csv

Do something like

set vFilePath = 'C:\Users\baronices\Desktop\Sales\';

For Each vFile in FileList ('$(vFilePath)sales_*.csv')

if Date(Date#(KeepChar($(vFile),'0123456789'),'MMDDYYYY'),'DD/MM/YYYY') > $(vLastReadFileDate) then

Data:

LOAD

CONN_ID,

[CONCETTI ESTRATTI],

Cluster,

AgentGroup,

Team,

CHANNEL

from $(vFile)

(txt, codepage is 1252, embedded labels, delimiter is ';', msq);

endif

next

LET vLastReadFileDate = Date(Date#(KeepChar($(vFile),'0123456789'),'MMDDYYYY'),'DD/MM/YYYY');

rahulpawarb
Valued Contributor III

Re: upload only new csv

Hello Cesare,

Please refer below sample script logic:

If IsNull(QVDCreateTime('C:\Users\baronices\Desktop\data\QVD\'))

Then

Data:

LOAD

  CONN_ID,

  [CONCETTI ESTRATTI],

  Cluster,

  AgentGroup,

  Team,

  CHANNEL

FROM

(txt, codepage is 1252, embedded labels, delimiter is ';', msq);

STORE Data INTO C:\Users\baronices\Desktop\data\QVD\data.qvd (qvd);

End if

Hope this will help.

Regards!

Rahul