tPOP & Microsoft Basic Authentication Retirement

[JBristow]

We have a job that extracts emails from an account using the IMAP format - and then we consume those emails and perform some automated processing. We've been notified that the service accounts we use are accessing their mail accounts via Basic Authentication (User Name & Password) - and Microsoft will be retiring Basic Authentication in October of 2022.

Looking at the tPOP component, there are no security options - with the exception of "use SSL" - which we have checked. Thus my assumption is that the component uses Basic Authentication.

Are there any plans to address this - or anyone else who might have a similar issue to extract emails for processing and uses a different approach?

Thanks in advance for any direction/response.

[JBristow]

We are just now getting resources to start this. We also have issues with 8.0.1 and cannot go "live" at this time so we are having to re-design our 7.2.1 job to grab the Token and then access the mailbox in customized java code. We'll grab the token via an API call (Microsoft Graph API) with an application key and a secret key - and once we have the access token we'll extract the emails from the INBOX we need. Then when we finally get 8.0.1 working and can go "live" with it - we'll convert the job to use the new tPOP component.

Complicated process change as Basic Authentication was easy. lol.

I'll keep the tread updated once we have it working.

Thanks.

[Anonymous]

Did you know that the component is open source so you can take a look how it will look with 8?

https://github.com/Talend/tdi-studio-se/tree/master/main/plugins/org.talend.designer.components.localprovider/components/tPOP

[DOSI_ALKOR]

Will these changes be in R2022-09?

If so, do you have a release date ?

Thanks.

[Anonymous]

Hello,

Yes these will be included in both 7.3 and 8 releases of September. 7.3 is already released and the component help page is reflecting the new parameters:

https://help.talend.com/r/en-US/7.3/pop/tpop-standard-properties

And the step by step guide for configuring the AD client is also available:

https://help.talend.com/r/en-US/7.3/pop/configuring-an-oauth2-application-for-pop-and-imap

Cloud customers can already download the 7.3 patch from the Downlaod sections of TMC, and the Talend 8 patch will be available in the update page soon: https://update.talend.com/Studio/8/updates/

[Anonymous · Accepted solution]

UPDATE 2023-02-17: Feature request for tSendMail + Exchange Auth: https://feedbackportal.microsoft.com/feedback/idea/c343ff42-a6ae-ed11-a81b-000d3a0450e3

Hello,

I'd like to provide an official update to this question / thread.

Microsoft is going to deprecate basic Auth (announcement , feedback) in the beginning of this October. This is done as Basic Auth is considered insecure. Talend mail components only support Basic Authentication, and while some providers provide Application passwords this functionality (feedback/feature request) is not available for Microsoft.

tSendMail

Talend jobs and components should be considered a daemon/service like application. For this it’s essential that there’s a non-interactive option for authentication. Microsoft recently made available Client Credential Flow support (announcement) for POP/IMAP. With this functionality it become possible to read e-mails. However in order to send e-mails one would traditionally rely on the SMTP protocol. As of the middle of September 2022 there’s still no support for SMTP with Client Credential flow. 

This means that starting from October Microsoft will seemingly disable Basic authentication without providing a proper secure solution that can be used from daemon/service like applications. (No Non-interactive flow for SMTP , Confirmation from Microsoft Exchange team member )

The tSendMail component uses SMTP protocol and won't be affected by this change of Microsoft.

UPDATE 2022-10-06: To our current knowledge there's no non-interactive flow available for the SMTP protocol. This means that the same Microsoft Exchange auth type that is available for tPOP won't work with SendMail hence it wasn't added. In case there'll be a flow that can be used to generate tokens the token can be passed via the OAuth2 auth type as an Access Token. It might be necessary to enable 2 line auth under the Advanced settings.

It is also possible to add more dropdown options to make the token generation easier but these options need to support scheduled task executions where human interaction is not possible.

tPOP

The tPOP component uses POP / IMAP and will be impacted. Both of these components got their Authentication options modified and now have Oauth access token available next to the Basic Auth. If a token is presented the component can send/read e-mails. Such token can be generated via routines / external applications. This was introduced with Talend 8 R2022-08 and 7.3.1 R2022-09 releases. This should be a universal solution that can be used with any e-mail provider, and Oauth workflow.

Due to high demand we’ve also added support for Client Credential Flow in the tPOP component available as Microsoft Exchange auth. This will make the component to negotiate / retrieve an access token using the Microsoft Secure Authentication Library (MSAL). This was/going to be introduced Talend 7.3.1 R2022-09 and 8 R2022-09 releases. The necessary configuration steps can be found here:  https://help.talend.com/r/en-US/8.0/pop/registering-microsoft-azure-application-for-pop-imap

Debug logs (UPDATE 2022-10-18)

Under the Advanced settings you can specify Custom properties. Adding the following entries will generate more logs about the debug steps. This will include the token value generated during the process.

"mail.debug" "true"

"mail.debug.auth" "true"

I hope this can be accepted as an answer to this question.

Regards,

Balázs

[NAndrade1645029267]

Has anyone been able to successfully set up the authentication mode for Microsoft Exchange or OAuth? We have a Talend case open but its been confusing and still isn't working. We were told to create an app registration in Microsoft Azure. Anyone using Microsoft Azure and can assist with your settings?

[Anonymous]

Did you follow the official guide? https://help.talend.com/r/en-US/8.0/pop/registering-microsoft-azure-application-for-pop-imap

[NAndrade1645029267]

Yes I have followed these instructions. However when running the job in Talend we get an Authenticate failed error. Please see my attached image showing our setting in Talend. Does this look correct? I am still waiting for someone from Talend to assist.

[NAndrade1645029267]

[Anonymous]

This setup looks correct. Did you also do the PowerShell steps? Without that you'll never be able to access the e-mails.