Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Want to see what's currently in public preview? We've pulled it all together in one place. Check it out here
cancel
Showing results for 
Search instead for 
Did you mean: 
Kartik2
Partner - Creator
Partner - Creator

Select Only the latest file from the Azure storage data connection

I want to automatically fetch only the first file into qlik sense from this path. How can do that? There is a date column in that excel file. 

Kartik2_0-1710217007831.png

 

Labels (5)
1 Solution

Accepted Solutions
Levi_Turner
Employee
Employee

Example logic (we need to use the Azure Storage Metadata Connector):

LIB CONNECT TO 'Azure_Storage_Metadata';
// Get a list of files, you will likely want to filter on prefix
[list_of_files]:
LOAD 
	Name as [file_name], 
	[Creation-Time] as [file_time_created], 
	[Last-Modified] as [file_time_modified];

SELECT Name,
	[Creation-Time],
	[Last-Modified]
FROM ListBlobs
WITH PROPERTIES (
prefix='',
includeUncommitted='false',
maxResults=''
);

// Get most recent file, we are ordering by the file_time_created field in order to ensure sort order for the later variable
[most_recent_file_list]:
NoConcatenate
LOAD
	[file_name],
    [file_time_created] AS [file_time_created],
    [file_time_modified] AS [file_time_modified]
RESIDENT [list_of_files]
ORDER BY [file_time_created] DESC;

// Drop initial load
DROP TABLE [list_of_files];

// Select first record in the sorted file list
LET v_file = Peek('file_name',0,'most_recent_file_list');

// Load the file, your load logic will need to be adjusted to reflect the schema of your file
[data]:
LOAD
    *
FROM [lib://Azure_Storage/$(v_file)] (txt, codepage is 28591, embedded labels, delimiter is ',', msq);

DROP TABLE [most_recent_file_list];

View solution in original post

1 Reply
Levi_Turner
Employee
Employee

Example logic (we need to use the Azure Storage Metadata Connector):

LIB CONNECT TO 'Azure_Storage_Metadata';
// Get a list of files, you will likely want to filter on prefix
[list_of_files]:
LOAD 
	Name as [file_name], 
	[Creation-Time] as [file_time_created], 
	[Last-Modified] as [file_time_modified];

SELECT Name,
	[Creation-Time],
	[Last-Modified]
FROM ListBlobs
WITH PROPERTIES (
prefix='',
includeUncommitted='false',
maxResults=''
);

// Get most recent file, we are ordering by the file_time_created field in order to ensure sort order for the later variable
[most_recent_file_list]:
NoConcatenate
LOAD
	[file_name],
    [file_time_created] AS [file_time_created],
    [file_time_modified] AS [file_time_modified]
RESIDENT [list_of_files]
ORDER BY [file_time_created] DESC;

// Drop initial load
DROP TABLE [list_of_files];

// Select first record in the sorted file list
LET v_file = Peek('file_name',0,'most_recent_file_list');

// Load the file, your load logic will need to be adjusted to reflect the schema of your file
[data]:
LOAD
    *
FROM [lib://Azure_Storage/$(v_file)] (txt, codepage is 28591, embedded labels, delimiter is ',', msq);

DROP TABLE [most_recent_file_list];