Select Only the latest file from the Azure storage data connection

Kartik2 — Tue, 12 Mar 2024 04:18:52 GMT

I want to automatically fetch only the first file into qlik sense from this path. How can do that? There is a date column in that excel file.

Re: Select Only the latest file from the Azure storage data connection

Levi_Turner — Tue, 12 Mar 2024 16:38:10 GMT

Example logic (we need to use the Azure Storage Metadata Connector):

LIB CONNECT TO 'Azure_Storage_Metadata'; // Get a list of files, you will likely want to filter on prefix [list_of_files]: LOAD Name as [file_name], [Creation-Time] as [file_time_created], [Last-Modified] as [file_time_modified]; SELECT Name, [Creation-Time], [Last-Modified] FROM ListBlobs WITH PROPERTIES ( prefix='', includeUncommitted='false', maxResults='' ); // Get most recent file, we are ordering by the file_time_created field in order to ensure sort order for the later variable [most_recent_file_list]: NoConcatenate LOAD [file_name], [file_time_created] AS [file_time_created], [file_time_modified] AS [file_time_modified] RESIDENT [list_of_files] ORDER BY [file_time_created] DESC; // Drop initial load DROP TABLE [list_of_files]; // Select first record in the sorted file list LET v_file = Peek('file_name',0,'most_recent_file_list'); // Load the file, your load logic will need to be adjusted to reflect the schema of your file [data]: LOAD * FROM [lib://Azure_Storage/$(v_file)] (txt, codepage is 28591, embedded labels, delimiter is ',', msq); DROP TABLE [most_recent_file_list];

topic Select Only the latest file from the Azure storage data connection in Integration, Extension & APIs

Select Only the latest file from the Azure storage data connection

Re: Select Only the latest file from the Azure storage data connection