topic Re: data masking / encryption in Talend Studio

data masking / encryption

Anonymous — Tue, 15 Nov 2016 04:10:07 GMT

Hey guys,
i have a requirement to mask data before i push it into a cloud database so in TOS i want to encrypt or hash ip address data and then after it has been encrypted or hashed compare the values to identify if two people have used the same ip address without decryption in in the destination database how can i do this is it possible in a tmap?

Re: data masking / encryption

Anonymous — Tue, 15 Nov 2016 05:17:03 GMT

Hi 2ndofafew,
Yes you can do that in TOS. Easiest way to encrypt is using the PasswordEncryptUtil.encryptPassword supplied by Talend natively. You can modify the code <main install folder>/ plugins/ org.talend.librariesmanager/ resources/ java/ routines/ system/PasswordEncryptUtil.java in order to get a unique key.
If you want something even simpler you can apply (int) row1.originatedIp.hashCode().
Then from your other process, just re-apply the same encryption/hashing and compare the two values with a lookup in your tMap. Since you database can be huge, I would probably do a Lookup Model = Reload at each row and applied the hash on the globalMapKey
After be aware that you can have in very rare scenarios some false positive / collisions. As well as people in large companies can be behind a proxy and then be originated from the same IP.

p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Monaco}

Re: data masking / encryption

Anonymous — Tue, 15 Nov 2016 05:30:00 GMT

Hey AdrienServian,
thanks for the help I've tried to use row1.IP_ADDRESS.hashcode() straight in a tmap with row1 as a string and the output as an int however this fails with the method hashcode() is undefined for the type string if its not too much trouble could you go into a bit more detail on usage or is there some documentation explaining this?

Re: data masking / encryption

Anonymous — Tue, 15 Nov 2016 08:00:02 GMT

Hi,
Could you please take a look at this KB article about:TalendHelpCenter: How to setup encryption of the passwords in Talend Studio? to see if it is what you are looking for?
Best regards
Sabrina

Re: data masking / encryption

Anonymous — Tue, 15 Nov 2016 22:39:28 GMT

Hey Xdshi, how could i apply this in a TMap on a data flow?

Re: data masking / encryption

Anonymous — Wed, 16 Nov 2016 09:52:04 GMT

Hi,
You can call your custom routine in tMap component.
Here is a KB article about:TalendHelpCenter:Creating a user routine and call it in a Job.
Best regards
Sabrina

Re: data masking / encryption

cterenzi — Wed, 16 Nov 2016 15:43:47 GMT

Hey AdrienServian,
thanks for the help I've tried to use row1.IP_ADDRESS.hashcode() straight in a tmap with row1 as a string and the output as an int however this fails with the method hashcode() is undefined for the type string if its not too much trouble could you go into a bit more detail on usage or is there some documentation explaining this?

Java is case sensitive. The method name is hashCode(). I put together a simple job that reads in a single column file and uses a tMap to add a column with the expression "row1.IP_ADDRESS.hashCode();" The job runs to completion and creates a different integer for each unique input value.

Re: data masking / encryption

Anonymous — Sun, 11 Nov 2018 22:07:43 GMT

hi ,

I am really new to the Talend open studio version 7.0 technology, and I am currently working on an project "anonymization" for the tables. In my STD they ask me to anonymize on several tables.
In the management rules
- Replace the string by 5 characters randomly generate followed by the string "archives"
ISO code case: Replace the string with "FR"
Case of the email: Replace the chain of characters email by "archives@archives.com"
IBAN code case: replaces the string with: "BL679999999999999999999934"
I do not know how and what component or routine does he do the job?
I thank you for your collaboration I would be very grateful
Thank you

Re: data masking / encryption

TRF — Sun, 11 Nov 2018 22:30:40 GMT

Based on your use case, all you need is a tMap to change the value for the desired fields