Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Join us to spark ideas for how to put the latest capabilities into action. Register here!
cancel
Showing results for 
Search instead for 
Did you mean: 
amit_n
Contributor
Contributor

how to provide dynamic file name to fetch the file from azure blob??

Please let me know how to give file name if tazurestroageList_1 . it should pick the file name with the pattern. 

 

Eg: filename

123_612673id778_student_plan_2018-20-08.csv

 

everytime 

123_612673id778 and 2018-20-08 will be changing..

Student_plan is constant.

how can i pick the file from the lis in blob.

 

Please find the below screen shot.

 

Let me know how to provide in the bob filter prefix.

 

 

Labels (6)
16 Replies
iamabhishek
Creator III
Creator III

For tAzureStorageList the "Blob filter" doesn't work like normal File List filters, and we don't have the option to use regular expression or string functions to match or filter out blobs.
SO, when you define a prefix, you are actually designating a directory level as the blob filter else you just pass null to bring all in.

 

One way to achieve what you have been trying is to bring all the blobs and then use tJavaRow to filter out the blobs using simple regular expression or any string matching functions.

 

Job Layout - 

0683p000009LzPL.jpgJava Code - 

String blobName = input_row.BlobName;

if(blobName.matches("(?i).*student_plan.*")) {
	System.out.println(blobName);
}
amit_n
Contributor
Contributor
Author

input_row cannot be resolved to be variable. M getting tis kind of error.

 

 

iamabhishek
Creator III
Creator III

Show us the job layout, highlighting the component and error.
amit_n
Contributor
Contributor
Author

Please find the attached screen shot.

 

i need to store the file name in the context variable like p_filename.


Capture3.PNG
Capture2.PNG
Capture1.PNG
iamabhishek
Creator III
Creator III

You could use 

context.p_filename = blobName;

to store the filename/blobname to context variable.

 

Looking at your job I didn't really get what you were trying to achieve here. You are trying to pull all the blobs from the container and trying to download them. It would download all the blobs - you see you have the java code after the tAzureStorageGet to filter out blobnames which would do nothing.

 

On top of this, the error you are getting because you have connected tAzureStorageGet with iterate on tJava which doesn't know about the input_row. If you notice I had the main connecting to tJavaRow directly from tAzureStorageList. Also, I am suspecting the way you have used the blob filter won't work - simply use 

"model_outputs/"

to get all the blobs under the directory "model_outputs"

amit_n
Contributor
Contributor
Author

Actually I am copying the file from bolb to local directory.

i need to store the file in local directory.

 

 

 

amit_n
Contributor
Contributor
Author

I am able to get the filename in the tjava. Whereit is having all the files in the model_output folder.

 Then i need to store that particuler file to local folder. 

 


Capture4.JPG
amit_n
Contributor
Contributor
Author

How can i pick the latest file from blob.?? 

iamabhishek
Creator III
Creator III

In that case do something like this - 

0683p000009LzaG.jpg

Basically, you get all the files and then put the filter condition in tJava and store the filtered file in global variable(always prefer to use Talend global variables instead of context variables if you are not using parent-child jobs and you have to pass values between those jobs). Then connect the tAzureStorageGet using "RunIf" trigger to download the selected blobs to local directory.

Java Code - 

String blobName = ((String)globalMap.get("tAzureStorageList_1_CURRENT_BLOB"));
//String p_filename = (String)globalMap.get("p_filename");
//System.out.println("Global variable: "+p_filename);

if(blobName.matches("(?i).*student_plan.*")) {
	//System.out.println(blobName);
	//context.p_filename = blobName;
	globalMap.put("p_filename", blobName);
}

RunIf trigger condition - 

!Relational.ISNULL((String)globalMap.get("p_filename"))