topic Re: iteration on component execution in Talend Studio

iteration on component execution

Anonymous — Sat, 16 Nov 2024 13:58:45 GMT

Hi everyboby,
I'm trying to create a new component that get an ArrayList of information. My schema is fixed (user can't change column). But with this code I get only the last set of value of my ArrayList...
Does anyone can help me?
In my Component_begin I've got this kind of code :

<%@ jet 
imports="
    	org.talend.core.model.process.INode    
		org.talend.designer.codegen.config.CodeGeneratorArgument
		org.talend.core.model.process.ElementParameterParser 
		java.util.List
		java.util.Map
		org.talend.core.model.process.IConnection
		org.talend.core.model.process.IConnectionCategory
		org.talend.core.model.metadata.IMetadataTable 
		org.talend.core.model.metadata.IMetadataColumn
		" 
%>
<%
	CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
	INode node = (INode)codeGenArgument.getArgument();
	String cid = node.getUniqueName();
	String username=ElementParameterParser.getValue(node, "__USERNAME__");
	String password=ElementParameterParser.getValue(node, "__PASSWORD__");
%> 
String user = <%=username%>;
String pass = <%=password%>; 
int nb_line_<%=cid %> = 0;
//connect to Twitter
winterwell.jtwitter.Twitter twitter = new winterwell.jtwitter.Twitter(user,pass);
//get information from Twitter
java.util.List<winterwell.jtwitter.Twitter.User>  userlist_<%=cid %> = new java.util.ArrayList<winterwell.jtwitter.Twitter.User>();
userlist_<%=cid%> = twitter.getFollowers();

<%
List< ? extends IConnection> conns = node.getOutgoingSortedConnections();
	List<IMetadataTable> metadatas = node.getMetadataList();
	if ((metadatas!=null)&&(metadatas.size()>0)) {
		IMetadataTable metadata = metadatas.get(0);
		List<IMetadataColumn> columnList = metadata.getListColumns();
		if (metadata!=null && conns.size()>0) {
           		for(IConnection conn:conns){
		%>
		for(int i_<%=cid %>=0; i_<%=cid %><userlist_<%=cid %>.size();i_<%=cid %>++){
		<%=conn.getName() %>.ID = userlist_<%=cid %>.get(i_<%=cid %>).getId();
		<%=conn.getName() %>.NAME = userlist_<%=cid %>.get(i_<%=cid %>).getName();
		<%=conn.getName() %>.SCREEN_NAME = userlist_<%=cid %>.get(i_<%=cid %>).getScreenName();
		<%=conn.getName() %>.DESCRIPTION = userlist_<%=cid %>.get(i_<%=cid %>).getDescription();
		<%=conn.getName() %>.LOCATION = userlist_<%=cid %>.get(i_<%=cid %>).getLocation();
		<%=conn.getName() %>.PROFILE_IMAGE_URL = userlist_<%=cid %>.get(i_<%=cid %>).getProfileImageUrl().toString();
		<%=conn.getName() %>.WEBSITE = userlist_<%=cid %>.get(i_<%=cid %>).getWebsite().toString();
		<%=conn.getName() %>.PROTECTED_USER = userlist_<%=cid %>.get(i_<%=cid %>).isProtectedUser();
		}
		
		<%
		
		}
		}
		}
%>

Re: iteration on component execution

_AnonymousUser — Mon, 27 Apr 2009 14:06:48 GMT

You should put this part of your code in Component_main part

<%
List< ? extends IConnection> conns = node.getOutgoingSortedConnections();
    List<IMetadataTable> metadatas = node.getMetadataList();
    if ((metadatas!=null)&&(metadatas.size()>0)) {
        IMetadataTable metadata = metadatas.get(0);
        List<IMetadataColumn> columnList = metadata.getListColumns();
        if (metadata!=null && conns.size()>0) {
                   for(IConnection conn:conns){
        %>
        for(int i_<%=cid %>=0; i_<%=cid %><userlist_<%=cid %>.size();i_<%=cid %>++){
        <%=conn.getName() %>.ID = userlist_<%=cid %>.get(i_<%=cid %>).getId();
        <%=conn.getName() %>.NAME = userlist_<%=cid %>.get(i_<%=cid %>).getName();
        <%=conn.getName() %>.SCREEN_NAME = userlist_<%=cid %>.get(i_<%=cid %>).getScreenName();
        <%=conn.getName() %>.DESCRIPTION = userlist_<%=cid %>.get(i_<%=cid %>).getDescription();
        <%=conn.getName() %>.LOCATION = userlist_<%=cid %>.get(i_<%=cid %>).getLocation();
        <%=conn.getName() %>.PROFILE_IMAGE_URL = userlist_<%=cid %>.get(i_<%=cid %>).getProfileImageUrl().toString();
        <%=conn.getName() %>.WEBSITE = userlist_<%=cid %>.get(i_<%=cid %>).getWebsite().toString();
        <%=conn.getName() %>.PROTECTED_USER = userlist_<%=cid %>.get(i_<%=cid %>).isProtectedUser();
        }
        
        <%
        
        }
        }
        }
%>

Re: iteration on component execution

Anonymous — Mon, 27 Apr 2009 14:52:30 GMT

I've split the code into two different parts by it doesn't work
begin :

<%@ jet 
imports="
    	org.talend.core.model.process.INode    
		org.talend.designer.codegen.config.CodeGeneratorArgument
		org.talend.core.model.process.ElementParameterParser 
		java.util.List
		java.util.Map
		org.talend.core.model.process.IConnection
		org.talend.core.model.process.IConnectionCategory
		org.talend.core.model.metadata.IMetadataTable 
		org.talend.core.model.metadata.IMetadataColumn
		" 
%>
<%
	CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
	INode node = (INode)codeGenArgument.getArgument();
	String cid = node.getUniqueName();
	String username=ElementParameterParser.getValue(node, "__USERNAME__");
	String password=ElementParameterParser.getValue(node, "__PASSWORD__");
%> 
String user = <%=username%>;
String pass = <%=password%>; 
int nb_line_<%=cid %> = 0;
//connect to Twitter
winterwell.jtwitter.Twitter twitter = new winterwell.jtwitter.Twitter(user,pass);
//get information from Twitter
java.util.List<winterwell.jtwitter.Twitter.User>  userlist_<%=cid %> = new java.util.ArrayList<winterwell.jtwitter.Twitter.User>();
userlist_<%=cid%> = twitter.getFollowers();

main :

<%@ jet 
imports="
    	org.talend.core.model.process.INode    
		org.talend.designer.codegen.config.CodeGeneratorArgument
		org.talend.core.model.process.ElementParameterParser 
		java.util.List
		java.util.Map
		org.talend.core.model.process.IConnection
		org.talend.core.model.process.IConnectionCategory
		org.talend.core.model.metadata.IMetadataTable 
		org.talend.core.model.metadata.IMetadataColumn
		" 
%>
<%
	CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
	INode node = (INode)codeGenArgument.getArgument();
	String cid = node.getUniqueName();
%> 

<%
List< ? extends IConnection> conns = node.getOutgoingSortedConnections();
		if (conns.size()>0) {
           		for(IConnection conn:conns){
           		if (conn.getLineStyle().hasConnectionCategory(IConnectionCategory.DATA)){
           		
		%>
		for(int i_<%=cid %>=0; i_<%=cid %><userlist_<%=cid %>.size();i_<%=cid %>++){
		<%=conn.getName() %>.ID = userlist_<%=cid %>.get(i_<%=cid %>).getId();
		<%=conn.getName() %>.NAME = userlist_<%=cid %>.get(i_<%=cid %>).getName();
		<%=conn.getName() %>.SCREEN_NAME = userlist_<%=cid %>.get(i_<%=cid %>).getScreenName();
		<%=conn.getName() %>.DESCRIPTION = userlist_<%=cid %>.get(i_<%=cid %>).getDescription();
		<%=conn.getName() %>.LOCATION = userlist_<%=cid %>.get(i_<%=cid %>).getLocation();
		<%=conn.getName() %>.PROFILE_IMAGE_URL = userlist_<%=cid %>.get(i_<%=cid %>).getProfileImageUrl().toString();
		<%=conn.getName() %>.WEBSITE = userlist_<%=cid %>.get(i_<%=cid %>).getWebsite().toString();
		<%=conn.getName() %>.PROTECTED_USER = userlist_<%=cid %>.get(i_<%=cid %>).isProtectedUser();
		}
		
		<%
		}
		}
		}
		
%>

Re: iteration on component execution

_AnonymousUser — Mon, 27 Apr 2009 15:20:47 GMT

I have a look to tRowGenerator and all is done in begin part. So forget what i said before.
What i think is :
your first loop should be :
for(int i_<%=cid %>=0; i_<%=cid %><userlist_<%=cid %>.size();i_<%=cid %>++){
and then :
for(IConnection conn:conns){
Try it and tell me if it works

Re: iteration on component execution

Anonymous — Mon, 27 Apr 2009 15:27:50 GMT

No it doesn't work :'(
I had a look to most of components and I can't understand where the problem comes from.
Thanks a lot for your help, If you've got any other suggestions...

Re: iteration on component execution

_AnonymousUser — Tue, 28 Apr 2009 08:56:38 GMT

If you've got any other suggestions... or THE solution.
When you look at tRowGenerator component, there is a trick : (see comments)
tTest_begin.javajet

<%@ jet 
	imports="
		org.talend.core.model.process.INode 
		org.talend.core.model.process.ElementParameterParser 
		org.talend.core.model.metadata.IMetadataTable 
		org.talend.core.model.metadata.IMetadataColumn
		org.talend.core.model.process.IConnection
		org.talend.core.model.process.IConnectionCategory
		org.talend.core.model.metadata.types.JavaTypesManager
		org.talend.core.model.metadata.types.JavaType 
		org.talend.designer.codegen.config.CodeGeneratorArgument
		java.util.Map
		java.util.List
	" 
%>
<%
	CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
	INode node = (INode)codeGenArgument.getArgument();	
	String cid = node.getUniqueName();
%>
java.util.List<String> test = new java.util.ArrayList<String>(); // replace with your ArrayList from Twitter (I'm not register so i cannot test)
test.add("test1");
test.add("test2");
test.add("test3");
test.add("test4");
test.add("test5");
int nb_max_row_<%=cid%> = test.size();
<%
	List<IMetadataTable> metadatas = node.getMetadataList();
	if ((metadatas!=null)&&(metadatas.size()>0)) { // if1
		IMetadataTable metadata = metadatas.get(0);
%>
	for (int i<%=cid %> = 0; i<%=cid %> < nb_max_row_<%=cid%>; i<%=cid %>++) { // for2 row loop
<%
		List< ? extends IConnection> conns = node.getOutgoingConnections();
		for (int i = 0; i < conns.size(); i++) { // for3 connection loop
			IConnection conn = conns.get(i);
			if (conn.getLineStyle().hasConnectionCategory(IConnectionCategory.DATA) 
			   && (!conn.getLineStyle().hasConnectionCategory(IConnectionCategory.USE_HASH))) { // if2
				for (IMetadataColumn column: metadata.getListColumns()) { // for4 metadata columns loop
%>
		<%=conn.getName() %>.<%=column.getLabel() %> = test.get(i<%=cid %>);
<%
				} // end for4
			} // end if2
		} // end for3
	} // end for2	
%>
// here there is no end for "if1"

tTest_end.javajet

<%@ jet 
	imports="
		org.talend.core.model.process.INode  
		org.talend.designer.codegen.config.CodeGeneratorArgument
	" 
%>
<%
	CodeGeneratorArgument codeGenArgument = (CodeGeneratorArgument) argument;
	INode node = (INode)codeGenArgument.getArgument();
%>
} // end if1

What i get with a tFileOutputXML :

<?xml version="1.0" encoding="ISO-8859-15"?>
<root>
<row><test>test1</test></row>
<row><test>test2</test></row>
<row><test>test3</test></row>
<row><test>test4</test></row>
<row><test>test5</test></row>
</root>

In tTest_java.xml : there's just one parameter which is a schema type parameter
What i don't understand is why first "if" begins between <% %> and ends out of <% %> ? and why in end.javajet templates ? How does it really work ?
If there is a TalendTeam who can explain that ...

Re: iteration on component execution

Anonymous — Tue, 28 Apr 2009 09:27:30 GMT

Yes It works!!!!
Thank you very much.
I don't understand the } thing. I hope that someone from the Talend Team will explain us this fonctionnality.
I will publish my components soon

Re: iteration on component execution

Anonymous — Fri, 10 Dec 2010 12:05:58 GMT

Hi cahsohtoa,
How did this component development go ?
Did you finally manage to create a twitter component ?
I am quite interested in your work, please tell me if I can give you a hand in something.
Best regards,
Maxime.