Skip to main content
Woohoo! Qlik Community has won “Best in Class Community” in the 2024 Khoros Kudos awards!
Announcements
UPGRADE ADVISORY for Qlik Replicate 2024.5: Read More
cancel
Showing results for 
Search instead for 
Did you mean: 
PGN
Creator II
Creator II

FileChannel Questions

First.  Can someone give me a 10K foot overview of what it is/does?  We're licensed for it, but I don't really know its functionality.  My searching is coming up pretty generic.  

Second, can this be used with AWS S3 storage, and is a separate license required for that?

Thanks!

 

Labels (2)
1 Solution

Accepted Solutions
Heinvandenheuvel
Specialist III
Specialist III

File-Channel usage is of course described in the Replicate Userguide under"11.4 Setting Up Qlik Replicate File Channel tasks".  It is a method to stretch tasks over time and space :-). Instead of a single replicate task both reading from a source and (synchronously) applying to a target a source task ('local') and target task ('remote) is used communicating using a staging area 'the file channel'. 

Local reads changes  from source and write to a proprietary flat file staging and loops for more,  NOT waiting for the apply to target. The Remote task(s) read from staging when it 'feels like it' and applies to target optionally deleting processed staging flat files then it loops back for more. 

Optionally, but really the primary reason for deployment, the local and remote task run on separate machines in different datacenter, possibly on the other side of a continent. That the staging area can be 'split' into a local storage storage on local and remote side with a the build-in "FTS" File transfer Service proving one-way transfer. A file-channel file-set can is written by a single (local) task and can be read and applied to targets by multiple  (remote) task. Thus offering a mechanism keep production an backup site in sync without cascading changes.

The File-channel setup works well as such, but it turns out to be relatively finicky, and needs more monitoring than most customers cared for.

Filechannel is largely replaced by LOGStream services - It provides multiple target tasks, and async processing very nicely and more robustly. It does not does not do anything special for long distance file transfer. But generally technology, and specifically network technology has changed, improved, since the early days of file channel now more than a decade ago. Network speeds allow direct connections now. And back in those days nobody considered 'cloud' solutions, whereas now it seems we do nothing but cloud technology making the 'distance' argument a moot point.

I suspect that nowadays FileChannel is really just there for backwards compatibility supporting existing solutions. It is unlikely FileChannel is the 'right' solution these days, but hey it is there and may well still serve a purpose in certain situations.

Clear?

Hein.

View solution in original post

2 Replies
Heinvandenheuvel
Specialist III
Specialist III

File-Channel usage is of course described in the Replicate Userguide under"11.4 Setting Up Qlik Replicate File Channel tasks".  It is a method to stretch tasks over time and space :-). Instead of a single replicate task both reading from a source and (synchronously) applying to a target a source task ('local') and target task ('remote) is used communicating using a staging area 'the file channel'. 

Local reads changes  from source and write to a proprietary flat file staging and loops for more,  NOT waiting for the apply to target. The Remote task(s) read from staging when it 'feels like it' and applies to target optionally deleting processed staging flat files then it loops back for more. 

Optionally, but really the primary reason for deployment, the local and remote task run on separate machines in different datacenter, possibly on the other side of a continent. That the staging area can be 'split' into a local storage storage on local and remote side with a the build-in "FTS" File transfer Service proving one-way transfer. A file-channel file-set can is written by a single (local) task and can be read and applied to targets by multiple  (remote) task. Thus offering a mechanism keep production an backup site in sync without cascading changes.

The File-channel setup works well as such, but it turns out to be relatively finicky, and needs more monitoring than most customers cared for.

Filechannel is largely replaced by LOGStream services - It provides multiple target tasks, and async processing very nicely and more robustly. It does not does not do anything special for long distance file transfer. But generally technology, and specifically network technology has changed, improved, since the early days of file channel now more than a decade ago. Network speeds allow direct connections now. And back in those days nobody considered 'cloud' solutions, whereas now it seems we do nothing but cloud technology making the 'distance' argument a moot point.

I suspect that nowadays FileChannel is really just there for backwards compatibility supporting existing solutions. It is unlikely FileChannel is the 'right' solution these days, but hey it is there and may well still serve a purpose in certain situations.

Clear?

Hein.

PGN
Creator II
Creator II
Author

Yes.  Thanks for the explanation!