Reading input files from directory and inserting their records in table

[RSA1598789664]

Hello Everyone,

Appreciate if some one could assist me in the below requirement.

Requirement :

Create Table : country_talend_xslx

Table Structure:

xslx_id xlsx_name csv_file

xslx_id = incremental value

xlsx_name = name of XSLX file

csv_file = names of CSV files generated for each tab, data should be comma separated.

I need to create a Talend job wherein Job will read all the xlsx files from a directory and Convert all xlsx files individually to csv files .if there exists a tab in excel then each tab should get converted to a csv [ job for creating csv files from xlsx file is done now]

and insert into the above table - country_talend_xslx.

Column "Xslx_id" will be incremental value.

Column "xlsx_name" will contain all the names of XSLX file within a directory

Column "csv_file" will contain names of CSV files generated for each tab, data should be comma separated

Thanks & Regards,

Anil Rawat

[Anonymous]

Hi

You need to use tFileList to iterate each file in a directory, there are some global variables pre-defined on tFileList can be used directly, such as:

current file path:

((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"))

current file name:

((String)globalMap.get("tFileList_1_CURRENT_FILE"))​

Please refer to the component documentation and learn this component, let me know which step you have troubles.

Regards

Shong

[JohnRMK]

Hello,

As Shong said, you have to use tFileList and these global variables to get the filenames.

You can run a query at the start of processing with SELECT MAX (xslx_id) FROM ... to retrieve the last value then +1 for the next file.

I advise you to use tSetGlobalVar in your iterations.

Can you give us an example of processing (a flow ), tell us how many tables and the structure of the files. If you have the paid version of Talend, you can use the dynamic schema so not a lot of jobs otherwise if it's the tos , it will be a job for each table with the different schema