CSV to MongoDB - XML-Tree problem

Anonymous — Sat, 16 Nov 2024 09:14:20 GMT

Hi Community!

I have a really small problem with loading my structured csv to a MongoDB and achieving the right format.

What I currently have:

My Talend process loops through a directory of csvs, loads them, uses a tMap component to add some information, gives it to a tMongoDBOutput.

The input data is structured like like this:

The columns "info", "sensorbasis" and "filename" are added within the tMap component and come either user input into a context variable, or (the filename) from the tFileList component.

What I want to achieve:

In the MongoDB there is a database called "mobidic" and a collection called "sensordata". I want to load all information from the CSVs to this exact collection, but, in a very specific structure that we need to work with at later stages. The format should look like this:

So basically, the result should be grouped by the fixed information, such as "sensorbasis", "info" and "filename", but the other columns should be in an array within the object "measurements". I understand, that I somehow have to set something as group, and something as a looping element.

This is what I have achieved so far:

Data is loaded to the MongoDB. I have configured the tMongoDBOut component, but not as intended:

And this is how the json tree within the component looks like:

What am I missing? I have searched through the board and did find some posts regarding the same challenge, but none of them brought me any further.

I hope somebody here can give me a hint on what I am missing.

Tanks in advance!

Freddy

Re: CSV to MongoDB - XML-Tree problem

Anonymous — Mon, 02 Oct 2017 07:56:01 GMT

Nevermind. As long as Talend does not know what column to group by, I definitely won't be able to get the grouping I want.

Therefore, this is my working design now:

Result is as intended.

Closed.

Re: CSV to MongoDB - XML-Tree problem

Anonymous — Tue, 03 Jul 2018 06:48:10 GMT

Hi,

Could you please show me the tMap component. I am also trying to create a nested document, but loop element is not creating a array, it is simply creating a sub element.

topic Re: CSV to MongoDB - XML-Tree problem in Talend Studio

CSV to MongoDB - XML-Tree problem

Re: CSV to MongoDB - XML-Tree problem

Re: CSV to MongoDB - XML-Tree problem