Suggest an Idea

[simonaubert]

Hello all,

A very simple need : I would like to delete some files in my libraries. Some use cases :

1/I don't need it anymore
2/I changed the name
3/I miswrite the path to save the qvd.. ooopsi, doopsie, a 10 GB file is generated.

I think being able to delete a file in script (without using a cmd.exe that require to be in legacy mode) would really improve my life but also my customers lives.

Best regards,

Simon

[Thomas_Hopp]

Thank you for sharing this idea with us. As it has been asked as well by other people via other channels, I'll put this into our backlog to investigate more in terms of how this can be achieved.

Thank you,

Thomas

[DataShark]

This would be a very useful feature, as our company policy requires us to remove data after a certain amount of time.

[sri_c003]

+1 vote... was looking for the feature and chanced upon this thread.

[Kristof_D]

+1

[mountaindude]

Agree, that would be useful.

There are security implications though.Deletion of files must be kept within the lib.

Specifically, relative paths must NOT be allowed, i.e. NOT ok to do something like "delete ../../../Windows/*.*"

Lacking this feature in the standard Qlik Sense Enterprise on Windows product, I added it to the Butler tool.
https://butler.ptarmiganlabs.com

Butler is an open source (free!) add-on to Qlik Sense Enterprise on Windows.
Butler itself runs on Windows, Linux, in Docker or Kubernetes.
It does various things, including file delete/copy/move using pre-approved directories.

The pre-approval is important - it means that sys admins in advance (in Butler's config file) can specify in which directories files can be deleted, or between which directories files can be moved.

I realized just now that there is no good examples for file copy/move/delete at butler.ptarmiganlabs.com... Apologies for that - the feature is there though. Will try to update the website during next week. Will add a few utility QlikScript functions too that make it easy to use the Butler API calls. Similar to what already available here for Butler's key-value store.

Meanwhile feel free to take a look at the API spec, available here.

The general way to use Butler is like this:

1. From your reload script you call a Sub, for example "CALL DeleteFile(path_to_file)"
2. That Sub is simply a wrapper around a call to Butler's REST API.
3. You'll only be able to act on files in directories that have been specified in Butler's config file.

Enjoy!

[simonaubert]

Hello @mountaindude 

About security : you can already modify modify a file, I don't see any reason why it's ok to modify but not delete. And God, it's seems sooooo simple and so game-changing.

But thanks for the great tip about Butler, I think I will have a look and maybe implement it at work.

Best regards,

Simon

[mountaindude]

Right, but the whole concept of lib:// is largely about limiting what can be done to the file system in general. With lib:// you can only act on files in that part of the larger file system.

Btw, my personal favorite and reason in itself to use the Butler, is the failed reload alerts.

Getting comprehensive (what failed, when, and even select parts of the script log) alert emails is gold..

[mountaindude]

previous email was from mobile email client, here's one from GMail:

[mountaindude]

@simonaubert 

I've added some docs to butler.ptarmiganlabs.com, explaining how file copy/move/delete operations can be done from Qlik load scripts using the Butler open source service.

Basically, the needed steps are:

1. Install and set up Butler. Can be as native app in Windows, on Linux, in Docker - all works.
2. Configure which directories are approved for file deletes, and which directories are ok as source/dest directories for copy and move operations.
3. Call Butler's REST API, either directly or by using one of the subs that ship with Butler.

The configuration of directories in Butler's config file is pretty intuitive:

  fileCopyApprovedDirectories:
    - fromDirectory: e:\data\qvd
      toDirectory: e:\data\qvd_archive

  fileMoveApprovedDirectories:
    - fromDirectory: e:\data2\qvd
      toDirectory: e:\data2\qvd_archive

  fileDeleteApprovedDirectories:
    - e:\data\qvd_archive    
    - e:\data2\qvd_archive

In your load script you would then do something along these lines:

// Where is Butler running?
let vButlerHost = '10.11.12.13';
let vButlerPort = 8080;

// Delete files
Call DeleteFile('e:\data\qvd_archive\a.txt')

// Copy files with options overwrite-if-exists=true and keep-source-timestamp=true
Call CopyFile('e:\data\qvd\a.txt', 'e:\data\qvd_archive\a.txt', 'true', 'true')

// Move files with option overwrite-if-exists=true
Call MoveFile('e:\data2\qvd\a.txt', 'e:\data2\qvd_archive\a.txt', 'true')

[Vincentzjecksen2021]

I'm new to Qlik Sense Enterprise, I would like to know if Butler able to use in Qlik Sense Saas edition?