article Missing fields when reading MongoDB table in Official Support Articles

Missing fields when reading MongoDB table

Sonja_Bauernfeind — Tue, 01 Jun 2021 16:09:53 GMT

When selecting data from MongoDB table using MongoDB connector (Qlik ODBC Connector package), some fields are not recognized by the connector. These fields do exist in certain documents of the table, but does not show up in the "Select data to load" wizard.
This happens when loading the whole table. When separating the sample problematic entries to a new MongoDB collection, the issue does not happen.

Cause:

For better performance, Qlik MongoDB does not scan all documents in the source table to build up table metadata. Instead, it follows a sampling strategy to pick up a sample set of documents. Default sampling strategy is to take the first 100 documents.

If a filed does not appear in the sampled documents, it will not be captured in metadata of the table in and thus does't show up in "Select data to load" wizard.

Environment:

, all versions

Resolution:

Check if the missing fields exists within the first 100 documents of the source table.

If it is not, configure the following Advanced parameters appropriately to make sure at least one document containing the missing fields is captured in the samples.

SamplingLimit

Default value: 100
The maximum number of records that the driver can sample to generate a temporary schema definition.
When this option is set to 0, the driver samples every document in the database.

SamplingStrategy

Default value : Forward
This options specifies how the driver samples data when generating a temporary schema definition.
Possible values:
- Forward: The driver samples data starting from the first record in the database, then samples the next record, and so on.
- Backwards: The driver samples data starting from the last record in the database, then samples the preceding record, and so on.
- Random: The driver selects sample records from the data source at random until the sampling limit is reached.

Note: The random sampling strategy is only supported by MongoDB Server 3.2 or higher.

SamplingStepSize

Default Value : 1
The interval at which the driver samples records when scanning through the database to generate a temporary schema definition. For example, if you set this option to 2, then the driver samples every second record in the database.
If the Sampling Method is set to Random, this setting is ignored.

You can find more details about advanced parameters in Simba' help site (the third party provider for Qlik ODBC Connector package) at Simba MongoDB ODBC Driver - Schema Definition Driver Configuration Options

Re: Missing fields when reading MongoDB table

vmontano — Thu, 13 Apr 2023 12:07:16 GMT

Hi!

It does not works for us. Seems that Qliksense always set Forward method even setting Backwards. Could be a kind of issue of the software?

Any alternatives?

It occurs using:

Mongo version 3.7.1
ODBC connector 6.166.1
QlikSense Descktop February 2022 (14.54.2)

Many thanks!
Regards

Re: Missing fields when reading MongoDB table

Sonja_Bauernfeind — Thu, 20 Apr 2023 07:41:46 GMT

Hello @vmontano

In order for us to investigate this further, a support case may be necessary. Please contact support and report your findings, providing the information you gave in this article.

All the best,
Sonja

Re: Missing fields when reading MongoDB table

vmontano — Thu, 15 Jun 2023 08:39:46 GMT

Hi @Sonja_Bauernfeind

First of all, thanks a lot for your response. I've found that support was moved to community. What is the process to open a ticket and where, it is, could you share the link?

Regards

Re: Missing fields when reading MongoDB table

Sonja_Bauernfeind — Thu, 15 Jun 2023 09:37:31 GMT

Hello @vmontano

The easiest way to contact support is to use our chat bot. You can see it in the bottom right corner, the green bubble with CHAT NOW 🙂

All the best,
Sonja

Re: Missing fields when reading MongoDB table

mikecrengland — Thu, 28 Dec 2023 21:45:14 GMT

@vmontano , did you get resolution on the Forward/Backwards issue?

thanks,