How to implement MD5 mechanism ?

[Anonymous]

I have a set of csv files which load in daily but do not have an audit date columns and a Primary key as well to identify a new record.
I want to implement MD5 mechanism to help distinguish the new records. I am using the Talend Big Data v6.2
I tried to implement the logic given in this link  : https://community.talend.com/t5/Design-and-Development/Getting-the-MD5-hash-of-a-database-record-fie...
and also imported tMD5Sum.zip
When I import the routine using (Select archive file option) it  says  'No valid item to import'. 
I copy pasted the syntax from tMD5Sum_main.javajet file but to no avail.

Is there any way I can link this routine from the exchange and pull it direcly to my workspace or
is there any easy way out to solve a problem of this sort without implementing the Hash key mechanism?
My target dB is Cassandra 2.0.


Code :



<%@ jet 

imports="

    org.talend.core.model.process.INode

    org.talend.core.model.process.ElementParameterParser

    org.talend.core.model.metadata.IMetadataTable

    org.talend.core.model.metadata.IMetadataColumn

    org.talend.core.model.process.IConnection

    org.talend.core.model.process.IConnectionCategory

    org.talend.designer.codegen.config.CodeGeneratorArgument

    java.util.List

    org.talend.core.model.metadata.types.JavaTypesManager

    org.talend.core.model.metadata.types.JavaType

"

%>

<%

CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;

INode node = (INode)codeGenArgument.getArgument();

String cid = node.getUniqueName();



String fileName = ElementParameterParser.getValue(node, "__FILENAME__");

   

String outputConnName = null;

List< ? extends IConnection> conns = node.getOutgoingSortedConnections();

if (conns!=null) {

if (conns.size()>0) {

for (int i=0;i<conns.size();i++) {

IConnection connTemp = conns.get(i);

if (connTemp.getLineStyle().hasConnectionCategory(IConnectionCategory.DATA)) {

outputConnName = connTemp.getName();

break;

}

}

}

}



if (outputConnName != null){

%>     



<%=outputConnName %> = new <%=outputConnName %>Struct();



<%=outputConnName %>.filename = <%=fileName %>;

<%=outputConnName %>.md5_sum = com.ostermiller.util.MD5.getHashString(new java.io.File(<%=fileName %>));



<%

}

%>

[Anonymous]

Hi,
Are you looking for a custom component tMD5Sum on talend exchange portal?
https://exchange.talend.com/#marketplaceproductoverview:gallery=marketplace%252F1&pi=marketplace%252...
Have you tried to install it into studio to see if it works?
Here is a KB article about: TalendHelpCenter:Installing a custom component
Best regards
Sabrina

[Anonymous]

thank you , it worked like a charm,
 Are there any solved examples on tMD5Sum component as I couldnt find any online.