Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Showing results for 
Search instead for 
Did you mean: 
Anonymous
Not applicable

store the dynamic values in context variable

Hi team,

 

I have source oracle table and there is one column called txn_date. 

 

values are:

txn_date

02-06-2017

04-06-2017

07-06-2017

04-06-2017

 

Requirement: How to create folder based on date for examples:

folder 1: c;/Txn_dt_02-06-2017/

folder 2 :c:/Txn_dt_04-06-2017/

folder 3 :c:/Txn_dt_07-06-2017/

 

But no duplicate folder even if same records with same date arrive. 

 

and is possible to store single context variable  to store unique txn_dt values?

 

Thanks

Shree

 

Labels (1)
46 Replies
Anonymous
Not applicable
Author

Hi,

 

I am using below flow.

 

tfiledelimited_1 => tUniqueRow_1 => tFlowToIterate_1 => tFileTouch_1

 

tFileTouch_1:

Properties:
File: "C:/Users/DELL PC/Desktop/reports/SOURCE FILE 3/" + ((String)globalMap.get("row2.newColumn2"))
   Checked: create direcotry if does't exist

 

I am able to create file with empty records. but even folder also not creating.

 

Please let me know any changes required?

 

Thanks

Shree

Anonymous
Not applicable
Author

You need to set up in the following way....

 

tFileTouch_1:

Properties:
File: "C:/Users/DELL PC/Desktop/reports/SOURCE FILE 3/" + ((String)globalMap.get("row2.newColumn2")) + "/DEL.ME"

 

Anonymous
Not applicable
Author

Hi,

 

I am getting below error.

 

[statistics] connecting to socket on port 3710
[statistics] connected
Exception in component tFileTouch_1
java.io.IOException: The system cannot find the path specified
at java.io.WinNTFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(Unknown Source)
at recon_test.mutiple_folder_creation_0_1.mutiple_folder_creation.tFileInputDelimited_1Process(mutiple_folder_creation.java:1535)
at recon_test.mutiple_folder_creation_0_1.mutiple_folder_creation.runJobInTOS(mutiple_folder_creation.java:1951)
at recon_test.mutiple_folder_creation_0_1.mutiple_folder_creation.main(mutiple_folder_creation.java:1795)
[statistics] disconnected
Job mutiple_folder_creation ended at 16:17 09/08/2017. [exit code=1]

 

Could you please check your local talend and share the job? would be very helpful.

 

Thanks

Shree

Anonymous
Not applicable
Author

I don't have a local version of your job. Like before, can you print out the path that is created by the Java you are using to create your touch file. 

 

I am confused as to why you are using a different piece of code for this. Previously you were using.....

((String)globalMap.get("row2.newColumn1"))

Now you are using....

"C:/Users/DELL PC/Desktop/reports/SOURCE FILE 3/" + ((String)globalMap.get("row2.newColumn2"))

First, why is row2.newColumn1 being used when you were creating the folder using a tSystem component? Also, why when you are using row2.newColumn2, are you prefixing it with "C:/Users/DELL PC/Desktop/reports/SOURCE FILE 3/"? 

 

Anonymous
Not applicable
Author

Hi,

 

Sorry for that, Even i am also missing points in between.

 

My requirement is below.

Final Requirement: 

Source records:

Columns are:

txn_date, num 

12-06-2017,12

13-06-2017,24

12-06-2017,22

14-06-2017,24

12-06-2017,13

13-06-2017,24

 

 

tOracleInput_1 ==> tFileOutputExcel_1

 

1)  Create unique txn_dt directory. Ex:  12-06-2017, 13-06-2017 etc

2) Inside that directory create xlsx file. date_wise_load.xlsx ( like 12-06-2017)

3) Load that date records. 

Output File:date_wise_load.xlsx

txn_date,num

12-06-2017,12

12-06-2017,24

12-06-2017,13

 

Could you please tell the flow.  There is no confusion.

 

Thanks

Shree

Anonymous
Not applicable
Author

I think we are going around in circles a little. This was my first suggestion. I will explain.

 

1) Calculate your file path as a String (you were already doing this when you had the issue of trying to create the same path more than once).

2) Just create your xlsx file using a tFileOutputExcel component and the file path String from step 1.

 

That is it.

 

You do not need to care about creating the file path. Writing the file will take care of that.

You do not need to care about creating the file path more than once. That is not possible.

 

Simply query your data. Calculate your file path. Write to the Excel file.

Anonymous
Not applicable
Author

Hi,

 

Yes, Could you give me the job flow(components step by step).

 

Let me go through all before points discussed.

 

Thanks

Shree

 

Anonymous
Not applicable
Author

OK, I have been through the complete thread again and I have some questions.

 

1) Is your data coming from Oracle? If so, why have you been reading from a file in some of the screenshots?

2) Are you expecting to create multiple Excel files per job or just one? For example, will the data being returned from your query require more than one output Excel file?

 

 

Anonymous
Not applicable
Author

Thanks once again.

 

Please find below my answers.

 

1) Is your data coming from Oracle? If so, why have you been reading from a file in some of the screenshots?

Ans: I tried both source as oracle and excel file. 

Reason: 1) I am trying create folder initial process only then i need source as excel file and no target i want store txn_dt  in context variable.(why means i will use this context variable create folder based on date anywhere in the job)

               2) After some sub jobs, i am going to use oracle as source and target is excel files. 

 

Note: You consider Oracle as source(Final point)

 

 2) Are you expecting to create multiple Excel files per job or just one? For example, will the data being returned from your query require more than one output Excel file?

Ans: Yes per job.

 

flow:  tOracleInput_1(Source) ==> tExcelOutputFile_1(Target)   --- (multiple folder creations inside records based on unique txn_dt)

 

Please let me know if need any information.

 

Thanks

Shree

 

 

Anonymous
Not applicable
Author

OK. This is a bit more complex. I think this could be a good learning opportunity, so I will give you the components and the order.....but you will need to try it and debug it. This solution WILL work, but only once you have ironed out any bugs that are introduced while building it.

 

(1)tOracleInput -->(2)tJavaRow---> (3)tFlowToIterate ---iterate--> (4)tOracleInput -----> (5)tFileOutputExcel

 

1) Your first tOracleInput will return a distinct list of the data your files are grouped by. I believe that to be dates. Nothing else it required here.

2) Use the tJavaRow (or a tMap if your prefer) to create the file path from the group key. Output this to the tFlowToIterate.

3) Your tFlowToIterate will create a globalMap variable with that group key and the file path.

4) Connect your tFlowToIterate to another tOracleInput using an iterate link. This tOracleInput will return all of the data and be filtered by the group key derived in the first tOracleInput. You can use the globalMap variable in the WHERE clause of the tOracleInput. 

5) The tFileOutputExcel will need to use the globalMap value for the file path. 

 

The reason for doing it this way is that you will need to create a new file for each chunk of data requiring a new file. This cannot be done in one subjob. Therefore you need to iterate over that data. The tFlowToIterate will carry out the iteration and your second tOracleInput will supply the relevant data according to the key of the current iteration.