tBufferOutput/Input components are changing the value of the byte array

[Metikosh]

Hi,

As described in the subject, I am having an issue with the tBufferOutput/Input components. I have a table in a MariaDB database which contains a column which is BINARY datatype (in this column I have a hash value calculated with SHA256). For this example I used a single row so it would be easy to notice the difference.

Here is a screenshot of my job:

Following are screenshots of the schemas of the job components:

And finally here is the result (testDataHash1 is the source table and testDataHash2 is the destination table - both have the same definition - only one column BINARY with length 64).

This is a simplified scenario to demonstrate the issue I am facing - I actually have complex parent-child jobs where I need to propagate data from the child to the parent job (including the DataHash column which I am later using in a tMap component as a lookup; the lookup fails because the values are different because of the tBuffer components).

There is no workaround in my job logic (in terms of not using the tBufferOuput/Input), so I would appreciate any advice on how to tackle this issue!

Thanks!

[gjeremy1617088143 · Accepted solution]

for example you can create a private static List<byte[]> in a routine.

on the child job you use tJavaFlex : in the begin part you instanciate the liste, in the main you add the byte[] of the current row in the ArrayList on the end part you set the private var with the list you filled.

on the father job you use another tJavaFlex with a foreach clause inside to send the list to a flow.

here for eg I use a list of string and globalVar for put or get the list :

 

you just have to transpose it to list<byte[]> and use getter setter of private variable in a routine instead of globalVar

[Metikosh]

Hi @guenneguez jeremy​ ,

Thank you for your input! I am sorry for the late reply, but priorities shifted on my side and I didn't have the time to test your approach.

So finally I have tested it and it works great! However, I would like to inform you that the mistake was on my side - the tBuffer components work as they should, my data was actually incorrect (I won't go into details, let's just say that it was incompatible...). But thank you for introducing the global routine approach to me, as I was unfamiliar with the usage of global routines.

I will select your answer as best because of your valuable input from which I have learned a lot, but since there is no problem with the tBuffer components - my question, and this whole discussion, is nullified.

Best regards!