Hello colleagues!
I’m trying to load data from Elasticsearch (for one particular day) by the loop. But each time I get the same part of data. Could you please advise what I am doing wrong?
I do the following steps:
1. Declare variable Today:
vToday = today()
2) Load MasterTable.
Filters in With connection: @timestamp:asc, @timestamp:$(vToday)
3) Load Table_1 with fields where Resident - MasterTable
4) Load table Max_timestamp where Resident - Table_1: max(@timestamp) as max_timestamp
Declare variable : Max_timestamp = peek(‘max-timestamp, 0, ‘Max_timestamp’)
5) Drop Tables MasterTable, Max_timestamp
6) Make a loop:
Let j=0
for j=0 to 10:
7) Load MasterTable.
Filters in With connection: @timestamp:asc, @timestamp:{$(vToday) TO *), @timestamp:{$(vMax_timestamp) TO *}
😎 Repeat steps 3 to 5.
9) Next j
So I try to concatenate several loads by 10000 rows starting from max(@timestamp) of last loading.
