topic Re: Talend Open Studio for Data Quality in Data Quality

Talend Open Studio for Data Quality

JuttuSenthil — Fri, 15 Nov 2024 22:50:46 GMT

Dear Community,

Request your kind help asap with the below query:

We are exploring TOS for Data Quality tool for our DQ profiling. We have our source data in Azure Data Lake Storage (ADLS) for whom profiling need to be done.

Is there a way to connect to ADLS directly/indirectly from TOS for Data Quality?

Also do we have an option to load parquet/orc/avro files in TOS for Data Quality? If yes, kindly help with some documentation

Please feel free to revert should there be any queries.

Thanks,

Senthil

Re: Talend Open Studio for Data Quality

Anonymous — Fri, 10 Jun 2022 04:22:50 GMT

Hello,

I'm afraid that this feature is not available in talend open studio for data quality.

For your issue "How to connect azure storage from a data profiling perspective"

After checking with DQ experts, they recommended trying to create an HD insight cluster to connection azure storage from the DQ side.( solution required HD Insight installation on Azure storage)

Profile parquet files in “profiling perspective” is a new feature for us.

In addition to that, you are able to read parquet and CSV file from Azure storage in Talend Studio integration perspective in Standard job and Big Data Batch job and "tFileInputParquet" component, is available for both DI and BD (Batch/Streaming) jobs.

Hope it helps.

Best regards

Sabrina

Re: Talend Open Studio for Data Quality

JuttuSenthil — Fri, 10 Jun 2022 11:04:13 GMT

Thank You very much for the response. This was really helpful to understand the availability of the feature.

Re: Talend Open Studio for Data Quality

Anonymous — Mon, 13 Jun 2022 05:18:45 GMT

Hello,

Feel free to let us know if there is any further help we can give.

Best regards

Sabrina

Re: Talend Open Studio for Data Quality

msjian — Tue, 30 Jan 2024 04:38:43 GMT

hello
We can support profiling ADLS gen2 file by jdbc driver see ~~TDQ-20315~~ and ~~TDQ-18068~~

Profiling ADLS gen2: we have doc https://help.talend.com/r/en-US/Cloud/studio-user-guide-api-services-platform/profiling-adls-databricks-file
thanks