topic Re: Remove Duplicated from the record in Talend Studio

Remove Duplicated from the record

Vijay_K_N — Tue, 25 Feb 2020 04:35:43 GMT

I have source like this

record

1122

112233

11223344

i want output like

record

123

1234

Re: Remove Duplicated from the record

Anonymous — Thu, 27 Feb 2020 08:35:59 GMT

Create a routine and add this function to transform it.
public static String removeDupsInPlace(String word) {
final StringBuilder output = new StringBuilder();

for (int i = 0; i < word.length(); i++) {
String character = word.substring(i, i + 1);
if (output.indexOf(character) < 0) // if not contained
output.append(character);
}
return output.toString();
}

Read the data as a string and pass it to the function.

Regards
Shong

Re: Remove Duplicated from the record

Vijay_K_N — Fri, 28 Feb 2020 13:48:28 GMT

it showing the errors ,see in attachment

error.PNG

Re: Remove Duplicated from the record

uzix — Fri, 28 Feb 2020 15:02:52 GMT

hello,

you routine must be inside class

Re: Remove Duplicated from the record

Vijay_K_N — Fri, 28 Feb 2020 15:25:16 GMT

sorry i have zero knowledge in java can u clarify with screenshot

Re: Remove Duplicated from the record

uzix — Fri, 28 Feb 2020 16:17:03 GMT

in talend

see routine.jpg

-> code

-> routines

--> "create new "

name it whatever you want

in my case is "test"

inside class "test"

paste the function that @shong as made available to you_:

see routine2.jpg

you can use your function in a tjavarow .

example:

if data source is file , and you want to use the function in 3 cloumns of input data source:

see routine3.jpg

the ouptut will be the value of input data transformed by the function.

routine3.jpg
routine2.jpg
routine.jpg

Re: Remove Duplicated from the record

akumar2301 — Fri, 28 Feb 2020 17:55:03 GMT

Array.from(new Set(yourString)).join('')

If Array class not available try below regex

yourString.replace(/(.)(?=.*\1)/g, "")

Re: Remove Duplicated from the record

Vijay_K_N — Tue, 03 Mar 2020 05:50:01 GMT

Thanks for ur reply...its working

Re: Remove Duplicated from the record

Vijay_K_N — Tue, 03 Mar 2020 05:50:59 GMT

thanks for ur neat explanation

Re: Remove Duplicated from the record

Vijay_K_N — Tue, 03 Mar 2020 05:52:59 GMT

i dont have knowledge on regular expressions bro