Using postgres input in tMap

[Anonymous]

Hello,

I'm currently loading data from ELK to postgres, everything works fine using tRESTClient, but in tMap i have to check if IP of event exists in postgres table and do something with it. I've writen my function to do it, but I have no idea how to use all ip's from postgres table for each row from ELK.

My job contains : tREST -> ExtractJsonFields , PostgresInput -> tMap -> LogRow

In tMap's output I'm calling function with arguments : Field from ELK Json and as a second argument I need to pass "array of ips from postgres"

How can i achieve it?

Thanks in  advance.

[Anonymous]

The tMap has the join feature. You can join 2 or more separate input flows.

How to join: There are basically 2 load lookup data options and 2 join options:

Load all at once: The means at the beginning of the flow (containing your tMap) all data from the lookup (your IP numbers) will be loaded and kept in the memory.

Reload each row: Here you can load the lookup data for every incoming row and you can set variable (they will kept in the globalMap) to filter your lookup.

It depends on the amount of lookup data and if you always need up-to-date lookup data for every incoming record which load method you should use. IP addresses sound like a huge number of data, perhaps the reload each row strategy would be the best.

Now to the join options:

You can decide Inner Join and Left Outer Join and (unlike SQL) you can also decide if the match should happened if the IP number appears unique or multiple times in your lookup.

I think for a more complete answer we should know a little bit more about your use case. The tMap is greatly documented. Select the tMap in your job and hit F1.