Use Variable Context to define File Delimited Metadata
Hello !
I am using File Delimited Metadata and I would like to use Variable Context in there path file definition.
In File Delimited creation window, I saw I could export it in metadata but that's not want to do. All my files are in a directory and I would say Talend to use a variable to define path in the file name.
For instance :
- define the context variable context.path_to_file_delimited = 'C:/CSV/import_1_0'
- then define filename = context.path_to_file_delimited+'state.csv' in file delimited metadata 'State',
- then define filename = context.path_to_file_delimited+'client.csv' in file delimited metadata 'Client',
- etc...
I don't want to create as many variable as I have files. I want to easily change path to my files because I have a new directory when I have a new set of files to import. And people in my team don't have the same path on their machine too...
Is it possible to do that with Talend ? I didn't find it in TOS documentation and in forum.
This is my environment : Windows XP / TOS 3.2.3 / Language : Java.
Best regards,
Aurélie
So you want to define one context variable to contain a certain path, and then use this variable in a bunch of filedelimited component? After you've declared your context variable you could give it a value in a tJava component and then use the variable like you did to define the filenames of your filedelimited component?
Hello Afrosimon and thank you for your response.
In fact I don't have any problem to create a context variable and give it a value but when I want to use it in my file delimited metadata creation I have an error because the variable is not yet interpreted and the file is not found so I can't create my metadata.
I hope my explanations are clear enough.
Regards,
Aurélie
I was thinking about this I realized a tFileList should be better for your use case : I just did a test job and I think it should resolve your problem. Also I think it would simpler for you, when you're creating your metadata files, to create them manually and to forget about the wizard (in any case that's what I did and I didn't had any error when running my job) - seems like the wizard is having a hard time dealing with the context variables. Well anyway, if there's anything, please let me know.
OK, thank you for your sample. I better understand what you are meaning.
I was thinking about this I realized a tFileList should be better for your use case : I just did a test job and I think it should resolve your problem.
For the tFileList, I can't use it because my jobs are very different a file to another.
Also I think it would simpler for you, when you're creating your metadata files, to create them manually and to forget about the wizard (in any case that's what I did and I didn't had any error when running my job) - seems like the wizard is having a hard time dealing with the context variables.
In fact I would prefer creating my file delimited resources in Repository Metadata because each file is used many times in many different jobs. I have about 40 file delimited and as many jobs, and any file appear many times in many jobs, so I wonder if this solution would not be worse than my problem because it would impact my jobs'reliability acquired using metadata.
I tried to save in metadata a component with built-in properties (as you advised me to do) by clicking on the "save" icon but I think you can't concatenate a context variable with a string in a field : the file name is ignored and only context variable is taken into account and it isn't a file so I've got an error.
I think the solution would be to define a context variable depending on another context variable. If it is possible, I could define a context variable that would be used in filename field. This context variable would be composed of context.my_directory+"filename.csv" (if you can concatenate a context variable with a string in another context variable). I didn't manage to do this but it would be a solution.
Regards,
Aurélie
Hi All,
I am facing the same problem and I am trying to use the solution proposed by afrosimon, but as I am a newbie on this tool I could not reproduce the image that you uploaded here. Could you help me how to link the components, a simple drag and drop is not working, thank you so much.
Regards,
Tomás
As you say, you can't use a context variable in a metadata file definition but you don't need to. What you need to understand is that the tFileInputDelimited component treats the file location and schema independently. So multiple jobs can use the same schema from the repository but read the file from a different location based on a context variable. In fact, once the Delimited File metadata has been created (from whatever actual file location you want), there is no need to have a file name in the metadata at all. See my screen-print.
