Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
See why IDC MarketScape names Qlik a 2025 Leader! Read more
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

Retrieve dynamically schema columns from CSV

Hi to all
I have this problem. I need to create a job that get data from a CSV and load it inside a Excel file. This job will run a lot of time but the data inside CSV will not be the same: I write down 3 scenario:
SCENARIO 1) The data inside the CSV
colA;colB;colC;colTemp1;colTemp2
"...";"...";"...";"...";"..."
"...";"...";"...";"...";"..."
SCENARIO 2) The data inside the CSV
colA;colB;colC;colTemp1;colTemp2;colTemp3; colTemp4
"...";"...";"...";"...";"...";"...";"..."
"...";"...";"...";"...";"...";"...";"..."
SCENARIO 3) The data inside the CSV
colA;colB;colC;colTemp1
"...";"...";"...";"..."
"...";"...";"...";"..."
So the problem is that the first 3 columns (colA, colB, colC) are fixed colums while the next colums are dynamic (one time could be two colums like SCENARIO 1, another time could be 4 colums like SCENARIO 2, another time could be 1 colum like SCENARIO 3, and so on).
How can I make the schema dynamic so I retrieve in correct way the data from the CSV and make the right table inside the Excel? I read a lot of posts about the dynamic schema. Is there a way to make it in TOS DI?
I thought to get the first line from CSV using tFullInputRow-> tNormilize and then make a XML schema with the right number of columns. After, use this xml schema inside the tFileInputDelimited. It's an idea but I don't know how to make the XML Schema dynamically.
Thanks in advance,
Tommaso
Labels (3)
17 Replies
Anonymous
Not applicable
Author

Hi,
I have created the custom components folder and point it in Talend and did as told by jloling "to delete the configuration/ComponentCache.javacache and restart TOS".
Now it worked, the missing jar message is not there, but subsequent login to TOS now it prompted the new error msg (attached).
Then I tried to delete the componentcache file again and it can login, but next login will prompt the same error msg.

If you don't delete the component cache file, does this error pop up?
Best regards
Sabrina
Anonymous
Not applicable
Author

Yes it will pop up again, that's why I need to delete the comp.cache file to be able to login.
Thanks
Erjan
Anonymous
Not applicable
Author

Hi,
I think it is the problem as jlolling said " if you install an update of a component, the files of the old component will be overwritten and new files will be added BUT files which does not exists anymore in the new component (like old versions of jar or not used javajet files) will NOT removed. That caused often problems and the user currently has only the chance to fix that locally in his plugins directory ".
Our colleague will report an issue on talend bug tracker.
Best regards
Sabrina
Anonymous
Not applicable
Author

Hi cyberjan
I don't get any trouble to install this custom component and make this component work just follows the guides, I have tried this on two version: 502 and 511. There may be something wrong in your studio, I suggest you to install the custom component in a new studio, and see if the problem still exists.
Shong
Anonymous
Not applicable
Author

Note that to clean automatically the component cache, if need you can simply start the studio with the additional parameter: --clean_component_cache
(You can create a new shortcut on windows with this parameter added...)
This might be helpfull in some case with custom components, even if here in fact it seems more like a problem of installation.
Anonymous
Not applicable
Author

Hi All,
Thanks for the help, think some problem on my Studio installation components.
I will try nrousseau suggestion
Thanks
Erjan
Anonymous
Not applicable
Author

I am trying the similar scene as to use Dynamic datatype for list of tables integration using single job..
Its really helpful, thanks Talend.
But saying that, I need a little help to get this working well. My date values from the source are inconsistent, I mean they are 'mm/dd/rr' or 'mm/dd/yyyy' or 'yyyy-mm-dd'. as these are written in same fashion of string value to target, its not working out for me.
My source is teradata, so I tried setting the session dateform to ANSIDATE, which may help here. I am not able to configure the session using tTeradataInput.
Please help to setup values for the session at runtime, as we do in Teradata SQLA. I guess I can use that in different scenes as well.
Appreciate quick reply or any generic job available for the similar purpose.
Thanks,
Rahul
Anonymous
Not applicable
Author

Hi Rahul
This topic was a little old, can you please report a new topic for your question, it is easy for us to follow up your topic and manage the replies.
Thanks!
Shong