topic Re: Automatically retrieve all data rows in Integration, Extension & APIs

Automatically retrieve all data rows

Anonymous — Wed, 29 Oct 2014 13:36:25 GMT

Hi all

I am currently putting together my first extension - a mapping extension that will draw a polyline from the latitude and longitude coordinates taken from a GPX file. I'm using the leaflet JS library (http://leafletjs.com/) in order to do this. I have hit a bit of a stumbling block due to the fact that the initial data fetch (qHeight) is limited to 1000 which will only plot part of the route - a 3 hour ride could result in up to 10800 points, for example. I have this currently:

return {

initialProperties : {

version: 1.0,

qHyperCubeDef : {

qDimensions : [],

qMeasures : [],

qInitialDataFetch : [{

qWidth : 2,

qHeight : 1000

}]

}

The values retrieved are then being added to a JavaScript array ready to plot using the following code:

var dimensions = layout.qHyperCube.qDimensionsInfo,

qData = layout.qHyperCube.qDataPages[0].qMatrix;

var coords = [];

$.each(qData, function(rownum, column){

coords.push(L.latLng(column[0].qText,column[1].qText));

});

A colleague has worked on an algorithm that will reduce the number of data points to be plotted (to speed up the visualisation by removing points when stopped at traffic lights, etc) once the selections are passed to the extension but, as before, I can't figure out how to get +1000 rows. I notice that the SimpleTable example extension is using the backendApi.getData to get any additional 50 rows each time the more button is clicked so it certainly seems possible.

So, would it be possible to automatically (without the user having to click a more button) retrieve all data points (latitude and longitude coordinates) to add to the JavaScript array ready for creating the polyline using the backendApi? If so, any hints on how I could do so?

My thanks in advance for any assistance - any input will be greatly appreciated.

Re: Automatically retrieve all data rows

ErikWetterberg — Thu, 30 Oct 2014 10:17:55 GMT

Hi Luke,

You can use the getData method to get more data. Check out this thread for an example.

Erik

Re: Automatically retrieve all data rows

Anonymous — Fri, 31 Oct 2014 16:37:45 GMT

Many thanks, Erik.

Would I be correct in saying that the only way of plotting 3500 location points then would be to call the paint function four times - the first three plot 1000 points each and the fourth 500?

Re: Automatically retrieve all data rows

ErikWetterberg — Mon, 03 Nov 2014 08:51:10 GMT

Hi Luke,

The maximum number of cells you can get in one go is 10.000 (se this thread) - note that the limit is on cells, not rows. If your qWidth is 2 you should be able to get all 3500 rows in one go.

You don't actually need to call the paint method again, but you need to call getData, which will run asynchronously and add the new values to your visualization in the callback.

Erik

Re: Re: Automatically retrieve all data rows

ErikWetterberg — Mon, 03 Nov 2014 14:07:59 GMT

Another point to consider: if you are after the numeric value you probably should use qNum and not qText:

var coords = [];
       $.each(qData, function(rownum, column){
            coords.push(L.latLng(column[0].qNum,column[1].qNum));
 });

qText will be formatted for display: use it for labels etc. qNum is the numeric value, use for all kinds of calculation etc.