Skip to main content
Announcements
See what Drew Clarke has to say about the Qlik Talend Cloud launch! READ THE BLOG
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

load new data from last reload

hi guys,

I have SQL script, where I get respondentID and key variable "TrackId" which I load every morning.

Then I have another query where I load data based on "TrackId".  Then I store the second data to qvd.

The problem is the second query has more "TrackId" value.

If I use max(TrackId) from qvd file, I skip some numbers.


Is it possible, when I load the first query, store last or maximum value of "TrackId" and then use this variable to load new data from the second query.

The new max variable should be from my last reload, not today.

Any ideas?

Thanks,

15 Replies
Anonymous
Not applicable
Author

it works well, if there is Id grater than vmaxTrackId.

alex_millan
Creator III
Creator III

Hi Vladimir,

Seems to me that this error occurs when vValue is empty (at least in the screenshot seems to be empty).

If so, perhaps you can implement an if before loading your 'COMMENT_URLs' table.


If len('$(vValue)') > 0

Then

     COMMENT_URLs:

          SQL SELECT [Id] as paramValue, [SessionId], [TimeStamp], [Ip], [Url]

          FROM [SEAC].[dbo].[UserTrack] where Id in ($(vValue));

End if

I'm not sure if it would work but perhaps it's worth to try.

Good luck

alex_millan
Creator III
Creator III

Hi Vladimir,

did you finally manage to solve it?

Regards,

Anonymous
Not applicable
Author

I have an idea how it might work.

From my main file after each reload I create vMaxUserTrackId in Temp table. (last keyID from the main file).

Then I added new temp table to find vLastNumber, which take only 1 max(keyID) from the main file

SELECT TOP 1 paramValue FROM [cRespondentData] as cRD ...

....

Let vLastNumber = Num(Peek('paramValue', 0, 'MaxValue'));

Here I'm stuck

I need a condition

IF $(vMaxUserTrackId)>$(vLastNumber)

THEN 

should be load all scripts

ELSE should be load only my qvd file.

When I added to my QV scripts the condition below

IF $(vMaxUserTrackId)>$(vLastNumber)

THEN

USERTRACKID:

SQL SELECT paramValue, respondentID

FROM cRespondentData3rdInfo WHERE surveyID=$(surveyID)  and isnumeric(paramValue)=1

and paramName='UserTrackID' and paramValue>$(vMaxUserTrackId);

Starting from THEN QV reads like wrong expression.

Any suggestion, I'd be appreciated.

Thank you again,

Anonymous
Not applicable
Author

I found out where the problem is

IF $(vMaxUserTrackId)>$(vLastNumber) THEN

IF and then should be in 1 row

but when I load only qvd file

LOAD AutoNumber(paramValue) as Auto_Id,*

FROM (qvd)

  ORDER BY SessionId, [TimeStamp] ASC;

It works, but gave me script error

Capture.PNG

how can I avoid it?

Anonymous
Not applicable
Author

It looks like qvd file can't use order by.

I used resident load