Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik and ServiceNow Partner to Bring Trusted Enterprise Context into AI-Powered Workflows. Learn More!
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

tPOP3 too much slow!

Hi all,
in a job of mine I use tPOP3 component to retrieve mail having some excel files in attachment.
The email box contains (at today) about 1000 email and each day the job retrieves 100 email and filters them as showed in the attached images.

But, tPOP3 is too much slow to retrieve email!
In fact, it requires about 20 minutes to complete.
I perfomed a lot of ping against the pop3 server and it is ok (response time is about 40-50 ms)
Do you think I need to edit any settings or is this the normal behavior of the component?
0683p000009MDws.png 0683p000009MDoz.png
Labels (2)
13 Replies
Anonymous
Not applicable
Author

Moreover, I found this following (old) post:
https://community.talend.com/t5/Design-and-Development/Variable-in-tpop-filter/td-p/111673
where at the last the user  wvaibhav  describes just what I wrote above.
I'm still trying different configuration to understand the problem related to slow performance, at the moment, without success.
Rosanero4Ever
Contributor
Contributor

Today, after about 3 months I haven't found a solution able to speed up my job in which I use tPOP component with filters as described in my previous posts.
Can anybody who use tPOP component with filters can share its own configuration and experience, please?
This strange behavior is killing me... 0683p000009MACn.png
Many thanks in advance for your contribution
Anonymous
Not applicable
Author

I have to admit that I have never had this issue, so can't give you any advice on this. However, if you want to prove this is a Talend issue and not something related to your email account (it is possible), you can do a comparison with Talend of using their component and using some example Java to do the same thing.
1) Create a new email account where you can share the credentials.
2) Create a job using the Talend component.
3) Create a similar job using some example Java code for retrieving emails (there is loads on the net) in a tJava or tJavaFlex
Then compare. If both examples perform in the same way, then there is clearly something else wrong (your network, your computer, your email service provider, something Java related). However, if you can clearly see a performance improvement when using the job that uses the example Java, then you can send it to Talend to prove the issue.
The other advantage of doing this is that if the Java version does work better, you can use that until Talend fix this (.....if it is their fault). 
Rosanero4Ever
Contributor
Contributor

Hi,
i verify the email retrieving behavior using  custom code (using tJava and in a main method in Eclipse) 
I also read the code generated in Talend and I noticed that, because POP3 limitations, all INBOX folder are loaded each job execution.
So, at the beginning of the "mailbox life" all is fast. After some mail (my inbox folder contains about 3500 mail)  and the code
javax.mail.Message[] msgs_tPOP_4 = folder_tPOP_4.getMessages();

cause an array filled with 3500 messages
Each element of this array is processed by a loop as follows
for (int counter_tPOP_4 = 0; counter_tPOP_4 < msgs_tPOP_4.length; counter_tPOP_4++) {

where filters are checked.
Once the array is filled all mail server are equal, so I put my attention to the loop because the array loading is fast
I noticed that this loop are executed differently in different servers!
In a server, in fact, a loop over about 800 messages is completed immediately, while a loop over about 3500 messages is completed after 15 minutes!
what do you think? Shall I check JVM options?