Upper bound on num rows for bundle load?

[Anonymous]

Hi,

I am trying to do a bundle load, associating png images with some field values. When I test on a small toy application everything works fine, both when there is one image per possible field value, when there are more images in the bundle load statement than values in the original table, when there are more values than images --- and also for different widths of the table containing the field in question.

So far, so good.

In my real application, however, I have a table with 6 columns and about about 100 000 rows.  I want to associate two images with (two values of) a particular field, and these two values each occur about thirty times in the table.

In this case the bundle load statement simply seems to be ignored. So is there an upper bound on the number of rows when it comes to bundle loads?

[Anonymous]

Update - I came up with a workaround using a data island, so my application now works as it should. Nevertheless, if anyone knows the answer to the original question it would be very interesting to know!

[#3]

I'm new on this.

I have the same problem.

Can you explain me how to work with data island.

Thanks a lot!!!

[Anonymous]

Hey there!

I'm writing this partially from memory, so hopefully I won't say anything which is wrong

What you would usually do is first to load your data, e.g.:

Data:
LOAD Field1, Field2
From some_data_source;

and then add images with a bundle load, in this case for Field2:

BUNDLE LOAD * Inline [
Field2, second_name_can_be_arbitrary
1,path_to_image1.jpg
2,path_to_image2.jpg
3,path_to_image3.jpg
4,path_to_image4.jpg
];

And then you would refer to your images like so: qmem://Field2/1 --- and so on.

However, this doesn't work for large data sets, as we know. So instead, I constructed a table which has no link to my original table, but which contains the (few) values which should have images.

So, as before, we do

Data:
LOAD Field1, Field2
From some_data_source

Now we make a copy of some of the rows, namely those which are to have images:

Island_with_image_rows:

NOCONCATENATE LOAD Field2 as copy_of_Field2
Resident Data
where some_clause_that_picks_the_image_rows;

Then we do a bundle load on this new table, namely

BUNDLE LOAD * Inline [
copy_of_Field2, second_name_can_be_arbitrary
1,path_to_image1.jpg
2,path_to_image2.jpg
3,path_to_image3.jpg
4,path_to_image4.jpg
];

So, now when you want to refer to your image, you need to write qmem://copy_of_Field2/1 instead of qmem://Field2/1, but as long as this string is generated using only() and whatnot, it should be doable --- at least it was in my case.

Hope you figure it out!