Skip to main content
Announcements
Introducing Qlik Answers: A plug-and-play, Generative AI powered RAG solution. READ ALL ABOUT IT!
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

load data with dynamic schema

Hi,

 

we have requirement where we need to load data file(.dat) into Oracle database. However data file structures will not same(no of columns and data type may differ) for all the files. All file will have same data delimiter.  We need to design a standard graph which should process all input files even with different schema. Thus graph should read file and create dynamic schema based on input file and load into database. Looking for an assistance to design this graph in Talend. 

 

Regards,

Vivek

Labels (3)
4 Replies
fdenis
Master
Master

read line 1 who must have columns names, create or alter table (All columns as String), then load your data.
fdenis
Master
Master

use dynamic Types to load Data.

If you did not have dynamic type it's because you are in free version.

in this case you can load data using a dynamic insert query.

good luck.
Anonymous
Not applicable
Author

There is no column heading in input file. Can you please elaborate your solution so we can visualize better and look for possibility. We are using license version software.

fdenis
Master
Master

if you have no header it's the same, but your columns name will be Col1, Col2....
read file line by line. by splitting first line, count the number of columns.
after you just have to generate Create Table ....
read the file line by line and générât one insert statement by line. Insert into (Col1,..) values ('val1...).

Good luck