Comparing the data from the delimited file and data from the Table
Hi, I have been exploring the options from Open Source Talend Data Quality tool to compare the data from the uploaded delimited excel file with the data from the Table from the connected database. I navigated through Column set analysis option and able to select the required columns from the table but the same column details from the delimited excel file could not be selected . Please let me know if this functionalty is not activated in open source and only will be avaible in Enterprise edition. Thanks Mallikharjuna Pagadala
I navigated through Column set analysis option and able to select the required columns from the table but the same column details from the delimited excel file could not be selected .
Could you set an example for us, we don't understand your requirement very well. Which component do you want to use? more info will be appreciated.
Best regards
Sabrina
Hi,
I have data from two different sources. One is from the table connected through DB connections and the second one is a excel file which is uploaded through file File delimited connections option.
My requirement is to compare the data from the above two source ( Table Vs Excel file)
Below are the steps I have followed to compare the data as explained above
1.Select new Analysis under the Data Profiling tab.
2. Selected Column content comparison under the Redundancy Analysis Tab
3. Select the connection from the database under the Microsoft SQL server after that the columns are selected from the table to be compared. Then all the selected columns are displayed in the box under the Left columns box as expected
4. Select the excel file located under File delimited connection. The selected columns are NOT displayed in the box under the right columns box.
If the above mentioned steps are not correct. Please let me know the procedure to compare the data as per my requirement. Also let me know if this functionality is available only in the Enterprise version.
Please let me know in case of further questions.
Thanks
Mallikharjuna P
Could you please provide the response as per the details provided. Thanks Mallikharjuna Pagadala
Hi
No, it is impossible to select columns from file, it only support the selection from database. The solution can be, use Talend open studio for data integration to migrate data from delimited file to the same database, and then compare the datas with Talend open studio for data profile.
HI,
Thanks You for the clarification for my query. I wonder if this option is avaible in the enterprise verison. then we would be ready to buy the enterprise verison of Data Quality tool.
Thanks
Mallikharjuna Pagadala
Hi
This is a feasibility technical problems, it does not exist both in community version and enterprise version. It can't compare two data from file and database.
As I suggested, create a DI job to move data from file to database, and use tLaunchDQReport run this analyse. The job looks like:
tFileInputDelimited--main--tMysqlOutput
|
onsubjobok
|
tLaunchDQReport (run DQ report)
Hi Shong,
I am comparing the same data from two different sources . One id from the SQL server 2008 database and the other is from the MS access database. Below are the detaild steps that I have done.
1. Created a new DB connection to access MS SQL server 2008 ( let us consider this as data source 1)
2. Created MS ACCESS databse and access the same through ODBC connection and imported one of the table data from the MS SQL server 2008 databse (Let us consider this as Data source 2).
3. Create new Analysis and select column content comparison.
4. Select columns for setA from table from the data source 1 and select columns for setB from table form the Data source 2.
5. When I tried to save this analysis, I am getting an error 'Only elements from the same Schema (or Catalog) can be comapred.
Could you please let me know if Talend DQ can compare the tables from the different databases.
Also let me know in case of any questions.
Thanks
Mallikharjuna P
Hi Shong, 5. When I tried to save this analysis, I am getting an error 'Only elements from the same Schema (or Catalog) can be comapred. Could you please let me know if Talend DQ can compare the tables from the different databases.
hi
I am sorry to tell you that Talend DQ can not compare the tables from different database, as the error showed, the tables must be in the same schema (or catalog). The workaround is still to migrate data from one database to anther database.
Shong
Hi,
I want to compare the free text columns(name,address,pobox, etc) in the table are populated correctly as like source table. Data gets populated to the IQ table by using Talend jobs only.
is there any predefined component available to compare the same or only by sql statement only possible.
Please advice.
by
Vims
