Run-if connectors not working even on condition fulfillment

[Anonymous]

I am working on this Talend job. The objective is to upload the contents of all the excel files from a particular folder to a database table. The files may be huge (containing 5000 or more rows). In "tJava_3", I have set the value of a context variable called "count1" to the number of rows to be inserted in the current excel file. "tFileInputExcel_1" filters some of these rows. After mapping the columns from the excel sheet to the appropriate columns in the database table, I use "tMysqlOutputBulkExec_1" to write the contents of the excel sheet to a file before inserting these rows to the database table. I count the number of rows in the file outputted by "tMysqlOutputBulkExec_1" using "tFileRowCount_2", and set this value to another context variable called "count2". I now compare the context variables "count1" and "count2" in "tJava_4". I have two run-if connectors from "tJava_4" with the conditions "context.count2<context.count1" and "context.count2==context.count1" respectively. For small files, containing around 100 rows, this job works perfectly fine. But, for larger files, containing, say 2000 rows, none of the if-conditions is met (both are shown false), even when "tMsgBox_2" shows the value of both the context variables - "count1" and "count2" to be the same. Why is the control not flowing through the If (order:2) connected to "tJava_4" even when both the context variables are equal?

[Anonymous]

It is difficult to know what is going on, from looking at your screenshot.
Personally, I never have multiple inbound OK connectors, as you have done with tFileCopy_1.
I also never have both OK and IF outbound connectors, as you have done with tJava_4.
I think it leaves the execution path open to interpretation and makes it difficult to understand the intent.

[Anonymous]

Tal00000 is right, using both OK and IF connectors between components is dangerous if you are not entirely sure of timing implications. However, I came across an issue with setting context variables in tJava components and this *may* be your problem. Have a read of this post here ( https://community.talend.com/t5/Design-and-Development/resolved-The-correct-way-to-assign-the-value-...). Essentially using.....

context.setProperty("myContextName", "ContextValue");

....is better than simply assigning the context value like so....

context.myContextName = "ContextValue";

....in a tJava component.

[Anonymous]

RH is right, I have seen this issue too. There is a method that you can call - context.synchronizeContext() - which should also sort this out for you, if this is your issue; however, I think you need to look at your Job architecture too.

[Anonymous]

Hi, guys! Thanks for the replies.
tal00000 & rhall_2.0 : I am using this line now to set the context variable "count2" in "tJava_4" -

context.setProperty("count2",Integer.toString((Integer)globalMap.get("tFileRowCount_2_COUNT")));

instead of simply doing -

context.count2 = ((Integer)globalMap.get("tFileRowCount_2_COUNT"))

It gives me a null pointer exception at "tJava_4" because this is the only point in the job I try to set the value of this context variable. Also, I have no problem removing "tMsgBox_2". It was only meant to serve as a check for getting the values of the two context variables.

[Anonymous]

Here is my job with a slight modification as per the suggestions you have given:
Now, I am using "tContextDump_1" and "tContextLoad_1" before comparing the values of the two context variables in the run-if connectors. In "tJava_4", I have -

context.setProperty("count2",Integer.toString((Integer)globalMap.get("tFileRowCount_2_COUNT")));
context.synchronizeContext();

I don't receive the null pointer exception error now, and the job works perfectly, again, for smaller files. But, the run-if connectors are still not behaving as expected for larger files-

tLogRow_1 output-
Observe that the values of both "count1" and "count2" are, indeed, the same. Still, both the if-connectors from "tContextLoad_1" show "false" on a run for relatively large excel files.

[Anonymous]

This is clearly a timing issue which is caused by the mixing of Row, RunIF, OnSubJobOK and OnComponentOK connectors on single components and the creation of "loops" in the subjob. Talend flows should be straight lines that may go off on tangents, but flows should never reconnect in the same subjob. While you *can* do this using RunIf connectors, it is extremely bad practise unless you have complete control of the timing.
In order to solve this, you need to approach the problem in steps. Break your process flow down into individual, logical and ordered tasks, then create a SubJob for each task. Any data that needs to be stored between subjobs can be held in tHash components.  

[Anonymous]

Okay, rhall_2.0! I believe this is as linear as it gets, right? 

"tMsgBox_1" does not even execute for big files. The job just sticks, and I have to manually kill the job after some time. It works perfectly for smaller ones showing the values of both the context variables correctly. What could be the possible explanation?

[Anonymous]

OK. I understand the flow better now. I have some questions?
1) Why is the tFileList not connected by the iterate link to the tFileInputExcel? Having two iterates between the tFileList and the tFileInputExcel could be causing issues.
2) Are you aware that you can get a rowcount from the tFileInputExcel? If you need the count for comparisons later one, you will need to make sure that the comparison number is in the same subjob as the tFileInputExcel.
3) The msgbox will be useless unless you always intend to run this job within the Studio environment. 
4) Are you using the tMySQLOutputBulkExec for performance reasons? The tMySQLOutput may be a better component. You can get throughput metrics from that. Or maybe use the tMySQLOutputBulk with the tMySQLBulkExec for the same reason. I suspect that iterating over this component might be causing you timing issues as well, but I have not tested that.
It looks to me like this job could be run with the following flow.....
tFileList-iterate->tFileInputExcel--Row-->tMap--Row-->tMySQLOutput--Row-->tJavaFlex
The tJavaFlex could be used to do the comparison between the ((Integer)globalMap.get("tFileInputExcel_1_NB_LINE")) variable and the ((Integer)globalMap.get("tMysqlOutput_1_NB_LINE")) in the "End Code" section of the tJavaFlex.
Simple comparison code could be something like this....

if(((Integer)globalMap.get("tFileInputExcel_1_NB_LINE")).intValue()==((Integer)globalMap.get("tMysqlOutput_1_NB_LINE")).intValue()){
         System.out.println("Rows match");
}else{
         System.out.println("Rows do not match");
}