What is the default user session timeout for Qlik Sense Business and Qlik Sense Enterprise SaaS? Can the session timeout for Qlik Cloud be changed?

The default (fixed) value is set to 30 minutes. This is controlled by SESSION_TTL.

It is not currently possible to adjust the session timeouts in the Qlik Cloud.

Environment:

Qlik Cloud 

Method 1: From the Data Sources

1. Open the Qlik Analytics Service
2. Select  Catalog 
3. Pick your Space 
4. Click Space Details then
5. Select Data Files from the context menu
   
   
   
   
   
6. Locate the data you wish to delete
7. Click the ellipses to open the next menu
8. Click Delete
   
   
   
   
   
9. Alternatively, add /data at the end of the URL.

Method 2: From the Data Load Editor

1. Open the App with a connection to the data files you wish to edit
2. Go to Data Load Editor
3. On the Right Side of the Data Load Editor under add data section select your data space.
4. Click on the table icon under datafiles.
   
   
    
5. select the Delete option on the right side of each file that you want to delete.
   
   

Environment

• Qlik Sense Business 
• Qlik Cloud 

The information in this article is provided as-is and to be used at own discretion. Depending on tool(s) used, customization(s), and/or other factors ongoing support on the solution below may not be provided by Qlik Support.

This Techspert Talks session covers:

- What to plan for
- Migration Pathways
- Cloud Best Practices

Chapters:

• 01:20 - Cloud Tenant Locations
• 01:47 - Qlik Cloud Architecture
• 03:08 - Planning the migration
• 03:50 - Migration Journey
• 05:13 - Licensing Guardrails
• 06:39 - Migrating users, groups, and permissions
• 11:11 - Migrating apps
• 14:44 - Migrating the data connections
• 21:07 - Completed Migration
• 21:35 - 5 Key Take Aways
• 22:48 - Q&A: How to license sync with hybrid?
• 23:59 - Q&A: Best practices for 1 app migration?
• 24:47 - Q&A: What's included with license cost?
• 26:03 - Q&A: How to customize the UI?
• 26:45 - Q&A: How safe is the data in Cloud?
• 27:37 - Q&A: What are common stumbling blocks?
• 28:41 - Q&A: How are reload monitored?
• 29:28 - Q&A: What is the data size cap?
• 30:15 - Q&A: What cannot be migrated?

Resources:

• Qlik Cloud Best Practices
• Troubleshooting Qlik Data Gateway Direct Access
• Qlik Cloud Migration Center
• Qlik Professional Services
• Space-aware data source syntax examples
• Trust and Security at Qlik
• The Reload Analyzer for Qlik Cloud Customers
• Qlik Licensing Service Reference Guide
• Migrating NPrinting to Qlik Cloud Reporting

This video will demonstrate how to use Qlik-CLI for SaaS to migrate Apps from your on-premises Qlik Sense environment to your Qlik Sense SaaS edition. Qlik-CLI makes it possible for scripting and automating App migration.

Environment

• Qlik Cloud 
• Qlik Sense Enterprise on Windows 

Related Content

• How to install and start using Qlik-CLI for SaaS editions of Qlik Sense 
• Qlik Sense: How to set up JWT authentication 
• Using Qlik Sense on Windows Repository API (QRS) with qlik-cli 
• NodeJS download
• JQ download
• Qlik Sense Business and Enterprise SaaS: App Size, Storage Size and Size for Data File 
• Migrate Apps from Qlik Sense on Windows to Qlik Sense SaaS 
• Migrating to Qlik Sense SaaS editions 
• How to Generate API Keys in Qlik Sense SaaS using APIs 

Transcript

Note: These instructions apply to Qlik Cloud. Qlik Sense Enterprise on Windows functions similarly. See How To Edit Published Qlik Sense Apps on Qlik Sense Enterprise on Windows.

Qlik Cloud offers two different spaces to work in. Shared Spaces and Managed Spaces. Apps previously published to a Managed Space e cannot be edited in place, and they will need to be replaced by an app modified on a personal or shared space.

This article covers publishing to both. For additional information on the available space types, see Working in spaces.

• Shared Spaces
• Moving an App to a Shared Space
• Editing in a Shared Space
• Managed Spaces
• Publishing an app to a Managed Space
• Editing a published app in a Management Space
• Replace an app in a Managed Space
• Related Content
• Environment

Shared Spaces

Apps are not published to Shared Spaces. They are moved to or uploaded to them. Apps in shared spaces can therefore be edited freely by anyone that has the correct permissions, like any other unpublished app. Please, check Developing and sharing apps with shared spaces on the help site.

Moving an App to a Shared Space

1. Click the ellipses menu on the app you wish to move
2. Click Move
   
   
   
   
3. Choose the Shared Space you wish to use and click Move
   
   

Editing in a Shared Space

1. Open the app inside the Shared Space
2. If the correct permissions are applied to your account (see Permissions) you will have access to all edit options, such as the Data load editor and Sheet modifications.
   
   You can also publish the App to a Managed Space from inside the Shared Space.
   
   
   

Managed Spaces

Apps cannot be edited directly in managed spaces. The developer will have to modify the original app which is present in their own personal space, or in a shared space. The app is always left in its original place when the app gets published: the publishing process will make a copy of the app, and keep the original where it was.

Publishing an app to a Managed Space

1. Click the ellipses menu on the app you wish to publish
2. Click Publish
   
   
   
   
3. Choose the Managed Space and click Publish
   
   
   
   At this stage, the app is copied. A copy of it will remain in the personal (or shared) space from which you are working. Do not delete this copy.
   

Editing a published app in a Management Space

1. Once published to a Managed Space, the app edit possibilities are reduced to sheet and object creation.
   
   You will not have access to the Data Load Editor, but can still create new sheets, publish/unpublish these sheets, and copy existing sheets.
   
   
   
   
2. If you need to edit the app directly, work on the copy of the app in your personal or shared space and then publish and replace the existing published app.
   
   

   If the original app was deleted at some stage, a published app can be unpublished using the REST API. The developer will then be able to edit the unpublished app.

Replace an app in a Managed Space

1. Click the ellipses menu on the app you wish to publish
2. Click Publish
   
   
   
   
3. Leave Publish new and choose your app. Then click Republish.
   
   
   
   
4. The next step will depend on whether the system is able to identify an existing connection between an unpublished and a published app.
   1. If the system was able to identify a connection, the app will be replaced. If multiple ones are available, choose the app you wish to replace. 

   2. If the system cannot identify a connection (for instance, if the modified app was actually redeveloped from scratch) you will need to manually the space and the exact name of the published app and prompt to replace the app.
      
      

       

       

       

Related Content

Republishing apps

Environment

• Qlik Cloud 

You can upload QlikView applications (.qvw files) to your Qlik Sense SaaS hub.

1. Log in to your Qlik Sense Enterprise SaaS tenant
2. Click on Add new (fig 1)
   Fig 1
   
   
3. Click Upload app (fig 1)
4. Drag and drop the file into the window or Browse for it. 
5. Decide what Space the App should be added to and if any tags are necessary. 
6. Click Upload.

Uploaded apps cannot exceed 1 GB in size. If you are licensed for expanded apps, you can upload apps up to 2 GB in size. If you are licensed for dedicated capacity, you can upload apps up to 10 GB in size.

Environment

• Qlik Cloud 
• Qlik Sense Business 

Qlik Cloud is a modern analytics and data platform built on the same software engine as QlikView and Qlik Sense Client-Managed and adds significant value to empower everyone in an organization to make better decisions daily. Qlik Cloud allows you to use one common platform for all users – executives, decision-makers, and analysts.

Navigation:

• Recommended Resources
• Migrating from Client-Managed Qlik Sense
• Migrating fromQlikView

Migrating to Qlik Cloud can help your organization:

• Expand your analytics possibilities.
• Reduce infrastructure costs and the total cost of ownership of your Qlik solutions.
• Automate entitlement management and upgrades.

Recommended Resources

Q&A with Qlik: Qlik Sense Migration to Qlik Cloud 
Articles relevant to migration and new Qlik Cloud users

Migrating from Client-Managed Qlik Sense     

This site provides you the tools to monitor, manage, and execute a migration from Client-Managed Qlik Sense to Qlik Cloud.

Index and home
Planning your migration
Setting up the Qlik Cloud migration tools
Create and configure a cloud tenant

Migrating from QlikView

This site provides you the tools to begin your journey to move from QlikView to Qlik Sense and Qlik Cloud.

• Index and home
• Comparing QlikView and Qlik Sense
• Comparing QlikView and Qlik Cloud for administrators
• Concepts and best practices 
• Migrating QlikView to Qlik Cloud

This Techspert Talks session covers:

• What Qlik Cloud Reporting can do
• Alerts and Subscriptions
• Migration process

Chapters:

• 01:29 - Qlik Cloud Reporting strategy
• 02:13 - How to Monitor Charts
• 04:18 - How to set Data Alerts and Subscriptions
• 06:32 - Shared Subscriptions with Section Access
• 07:49 - How to use Automation with Reporting
• 09:23 - Using Themes
• 11:08 - Distributing to Recipient Lists
• 13:18 - Excel Tabular Reports with the Qlik Add-in
• 16:58 - How to use Reporting Templates
• 18:53 - Using Filters
• 20:30 - Using NPrinting Excel report templates
• 22:36 - Q&A: How to trigger a monthly report?
• 23:11 - Q&A: How many reports sent for free?
• 24:36 - Q&A: Can NPrinting templates be used in Qlik Cloud?
• 25:21 - Q&A: Available Report format options?
• 26:12 - Q&A: Can selections be applied dynamically?
• 27:05 - Q&A: A separate license for Cloud Reporting?
• 27:48 - Q&A: What Role is needed for Alerts/Subscriptions?
• 28:24 - Q&A: Is App Access required?
• 28:53 - Q&A: Will Alerting and NPrinting be deprecated?
• 29:53 - Q&A: Is the EAP still open?

Resources:

Qlik Ideation

Qlik Cloud Reporting – Improving Table Presentation

Reporting Service Forum

How to send an Excel report as an attachment using Qlik Application Automation

Creating an Automation from Templates

Tips For Designing Application Automation Reporting Reports

Click here to see video transcript

Qlik Cloud apps thumbnail's size is not limited and this could lead to undesired behaviours if such is not taken into account when adding the thumbnails to your apps.

Such undesired behaviour could be end users facing slowness when navigating trough the Catalog or the Home section. 

Environment

• Qlik Cloud

Resolution

The bigger is the size of your thumbnail on the app the longer it will take to be loaded, this could be easily seen in the Developer tools of the browser as below:

It is strongly recommended to use such images as optimized as possible ensure end users performance when navigations trough such sections.

Cause

The main contributor on the loading time of an app thumbnail will be its size (e.g 5B vs 5MB) resulting  in different range of times.

A multi-cloud deployment allows you to distribute Qlik Sense apps to the cloud for consumption. You can set up your deployment to connect Qlik Sense Enterprise on Windows to Qlik Cloud. For more information on multi-cloud deployments, see: Multi-cloud deployments to Qlik Cloud 

In these multi-cloud environments, Qlik Cloud can be considered an expansion of the Qlik Sense Enterprise on Windows platform, whereas all app development is carried out on the client-managed platform, and the apps can be made available for further consumption in Qlik Cloud.

Any additional editing (modifications of the Data Model) or duplication of the distributed apps is not possible directly on Qlik Cloud and needs to be carried out in Qlik Sense Enterprise on Windows.

Limited options are available on the Qlik Cloud hub for apps distributed from a client-managed platform:

In this article, we outline the ideal workflow used when distributing apps.

This Techspert Talks session covers:

• Converting Complex QlikView documents to Sense Apps
• Qlik Cloud Capabilities
• User Experience in Sense Cloud
• Performance Evaluation Tools

00:00 – Intro

00:51 - Migrating Strategy

02:17 - QlikView Governance Sense Profile Score

04:11 - SaaS Readiness Tool

05:37 - Hybrid Readiness Application

06:54 - Manual App Migration

10:05 - Qlik Data Access Gateway for local data

11:33 - Semi-Automated App Conversion

12:47 - QlikView Converter tool

13:37 - Master Items

17:59 - Migration Tools

18:41 - Refactoring stage

21:36 - Performance Evaluation

22:09 – Lineage

22:29 - Reload Schedule

22:45 - Completed 20 min Refactoring

24:33 - Cyclic Groups and Sense solutions

26:07 - Q&A: What if you can’t find a solution?

27:09 - Q&A: How can we do Cyclic Groups?

28:14 - Q&A: How to connect to On-Prem data?

29:01 - Q&A: How to get parameterised variables?

29:41 - Q&A: Where is the QlikView Converter tool?

30:08 - Q&A: Is there a license conversion tool?

30:46 - Q&A: Is there an NPrinting migration tool?

31:40 - Q&A: Can large QVWs perform in Cloud?

33:12 - Q&A: Is Qlik Sense Desktop still available?

33:36 - Q&A: Is there a Governance Dashboard for Cloud?

34:18 - Q&A: What Performance Evaluation tools?

Resources

QlikView Converter Tool

Qlik Sense Migration Tools

Qlik Sense Visualization Best Practices

Qlik Sense Best Practices for Choosing Individual Visualizations

What if no standard chart suits my purpose?

Moving from QlikView to Qlik Sense SaaS

Qlik Sense Visualization Showcase

Optimizing Qlik Sense SaaS Apps with App Analyzer

Click here for video transcript

If your tenant is using the new navigation experience, Activity centers replace the Hub. For more information, see New platform navigation.

Welcome to Getting started with the Qlik Sense Cloud Analytics Hub and how to interact with an app. In this article, we'll cover the basics of how to navigate the Qlik Sense Cloud Hub and how to interact with the data provided in an app.

The example App used in the video and article (Beginner's Tutorial) can be downloaded from the Qlik Help site.

Environment

• Qlik Cloud 
• Qlik Sense Business 

Click here for video transcript

Here is a link to the referenced Qlik Help site page in the video:

Tutorial - Beginning with the Basics 

Opening the Qlik Sense Hub

1. You access the Qlik Sense SaaS hub through your tenant URL. Example: Tenantname.eu.qlikcloud.com.
2. Once successfully logged in, the hub will be presented:
   
   
   
   Note that you can customize your home view, but we will focus on opening apps and navigating an app in this article. 

Filter on the Hub and find your App

By default, the hub will show you all content available to you. This includes apps, charts, data, notes, and links. To begin filtering;

1. Click Catalog. 
2. Choose from Spaces available to you.
   
   

    

3. Open a space. In our example, we select Managed Spaces (available in Qlik Sense Enterprise on SaaS).
   
   
4.  Filter the space by using the Filters available in the top menu, or Search content. 
   
   You can also sort by Last modified, Last created, or in Alphabetical order, and switch between List view and Tile View.
   
   The filters allow you to filter by Types (Apps, Charts, Data, Notes, Links) and Owners.
   
   

   

5. Open your App

Navigating the App

1. The app will display a list of sheets available to you in the Public sheets section and My sheets section. 
   
   Right-clicking on sheets shows a number of options available to you (depending on your permission) level. Such as Make private, Copy, Paste, and Duplicate. 
   
   Note that in this article, we will not cover creating, copying, or customizing sheets. 
   
   

    

2. Open a sheet (we will open the Dashboard sheet in our example.)
3. You can navigate between sheets from inside an active sheet by either using the sheets menu or the navigational arrows to page forward and back.
   
   

    

   
   
   
4. Return to the App overview from the top-right menu.
   
   

    

5. Return to the Hub by clicking on the Qlik Logo 
   
   

    

Making selections

When you make selections, the colors of the values change accordingly. The characteristic Qlik Sense colors are green, white, and gray, and they represent the basic states: selected, possible, and excluded. To learn more about selections and their different states, see Selection states.

Depending on the visualisation used, different selection methods can be used, such as Click selection, Draw selection, Range selection, Lasso selection, Legend selection, and Label selection. For more information on each selection method available, see Make Selections.

Let's test some selections:

1. We select 2014 from a Filter Pane and Japan from the Pie Chart and then confirm by clicking the green checkmark. 
   
   This will select Sales Per Region in the Year 2014. 
   
   

    

2.  The Selections Toolbar allows you to remove, edit, and step through (forward and back) your selections. 
   
   For more information see Exploring with selections and the Learning Material Searching & Selecting Data.
   
   

    

3. Another way to select and explore data is to use the Smart Search available in the Selections Toolbar.
   
   Smart search is the global search tool in Qlik Sense, enabling you to search the entire data set in your app from any sheet in the app. 
   
   For more information, see Using smart search and the Learning Material Searching & Selecting Data.
   
   

This concludes the basics of navigating the Qlik Sense hub and how to interact with visualisations.

Related Content:

Advanced material is available on Help.qlik.com, as well as in our Qlik Continuous Classroom offering:

Navigating in the user interface 
Getting Started with Qlik Sense (Free Training)
Searching & Selecting Data (Free Training)
Create and Apply Bookmarks  (Free Training)
Build and Play Stories (Free Training)
Creating analytics and visualizing data 

Can QlikView Object Migration for Cloud connect through a proxy?

Environment

• QlikView 
• Qlik Cloud 
• Qlik Sense Business 

There is currently no support in the bookmark migration tool for a proxy. Make sure to set an exclusion allowing direct connection on port 443 to Qlik Cloud.

A Talend Administration Center execution plan allows you to execute multiple Job Conductor tasks in a predefined workflow. The workflow is presented in a hierarchical view, where each task can execute a set of child tasks in parallel or sequentially.

The parallel approach executes all tasks simultaneously. Whereas in the sequential approach, the end status of the previous task determines when each task launches. You can add tasks in sequential order based on the following conditions

1. OnOk: Launches a child task if the parent task finalizes without error.

2. OnError: Launches a child task if the parent task finalizes with error.

3. After: Launches a child task after the parent task finalizes, regardless of its error status.

In this example, the execution plan consists of five tasks, including parallel and sequential tasks:

1. The root of the plan has two parallel tasks: “test_users” and “test_address”.

2. If the execution of parallel tasks is successful, “test_names” task executes.

3. If the execution fails, the “test_error” task launches.

4. Finally, the “test_emails” task launches after the “test_names” task, irrespective of the error status.

The plan also has a CRON-based trigger defined, trigger_plan, which executes every quarter on the 5th at 12:37.

How does Talend Cloud Migration Platform help with the migration of Talend Administration Center execution plans?

Talend Cloud Migration Platform collects and aggregates on-premises assets from Talend Administration Center and migrates them to Talend in the cloud. As a result, you can get a holistic view of all of the on-premises data from different Talend Administration Center environments in a centralized location.

To do so, you must register Talend Administration Center servers in Talend Cloud Migration Platform by providing their Talend Administration Center web URL endpoint and database connection details. Once the servers are registered, Talend Cloud Migration Platform collects all the assets, including execution plans and their respective tasks. The aggregated list, across all of the registered Talend Administration Center servers, is then displayed in Talend Cloud Migration Platform.

You can view the execution plans and tasks in their respective Talend Cloud Migration Platform UI pages.

Migrate Talend Administration Center tasks and execution plans from on-premises to Talend in the cloud by performing the following five steps:

1. Select the source Talend Administration Center environment, in this case, TacDev, and then click the Cloud button.

    

   

   
    

2. Select the destination Talend Management Console Environment and its respective Workspace from the drop-down menu. The destination environment is where you will migrate and execute the assets.

   1. Specify a target runtime engine. You can deploy a Talend Management Console plan on three types of runtime engines:

      The lists of Remote Engines and Remote Engine Clusters are refreshed based on the selected Talend Management Console environment and workspace.

      • Cloud Engine

      • Remote Engine

      • Cluster of Remote Engines

   2. Click the TASKS button to proceed to the next step.
       

      

      
       
3. Select the execution plan and the corresponding tasks.
    
   The left side of the UI represents Talend on-premises plan and tasks tree from Talend Administration Center, and the right side represents the plan and tasks tree in Talend Management Console.
   
   The execution plans and tasks are sorted alphabetically in the Talend Administration Center tree. The tasks in the Talend Administration Center tree are grouped by project name. Each task contains an artifact name with a version and any defined triggers.
   
   In the Talend Administration Center tree, only tasks of the type ARTIFACT_TASK are listed. These task types are the only ones suitable for migration as they are assigned with Artifact GAV information. Artifact GAV (GroupId, ArtifactId, Version) is necessary for Talend Cloud Migration Platform to locate the task artifact from Artifact Repository for downloading from Talend Administration Center and later to upload to Talend Management Console.
   
   You can select the plan by clicking its name in the Talend on-premises tree (the Talend Administration Center execution plans and tasks). With this selection, the corresponding tasks and triggers get automatically selected for the migration.
   
   Talend Management Console supports only one trigger or schedule per task, so if a task defines multiple triggers, not all of them will migrate. Instead, multi-trigger tasks migrate with the trigger set as manual, and a warning message "More than one trigger defined" displays after the migration. 

    

    

   

    
   1. Click the arrow facing right between the two trees to move the execution plans and tasks.
   2. Click the Review button to continue with the migration
      
      
4. Review the assets you are migrating, Talend Management Console environment, Workspace, and Runtime details.
   
   By default, Talend Cloud Migration Platform migrates the artifact and trigger definitions along with execution plan definitions and the tasks.
   
   If the Artifact toggle is On and an artifact with the same name already exists in Talend Management Console, a new version of the artifact is created. You can disable this type of occurrence (that is, disable migration of artifacts with the same name) by toggling the option Off. A similar option is also available for the migration of trigger definitions.
   
   You can also add comments about the migration.
   
   When you finish reviewing the information, click the MIGRATE button to start the migration process.
   
   
   
   If multiple tasks share the same artifacts in the execution plan, you cannot proceed with the migration, and you'll see the following message.
   
   
   
   Use one of the following workarounds to bypass this limitation:
   
   

   • Toggle the Artifact option to Off. However, this setting applies to all the tasks and will not migrate artifacts with the same name even if multiple tasks did not share them.

   • Migrate the affected tasks before the plan, and then migrate the plan with the Artifact option toggled Off.

5. Notice that the migration occurs simultaneously for all the selected Plans and Tasks. The migration process might take some time if the size of the artifact is large.
   
   A green check box displays to the right of each successful asset migration. The left panel of the UI displays the completed percentage of the migration process and summarizes the results.
   
   The migrated artifact of the task is replaced with the artifact name and version in the Talend Management Console tree.
   
   The execution plan in Talend Management Console is created as steps, and the Talend Administration Center plan trigger is mapped to a Talend Management Console trigger.
   
   Click the CLOSE button to redirect to the Timeline UI, which displays the history of all migrations in chronological order.
   
   

Talend Cloud Portal

You can view the newly migrated execution plan and tasks in the Talend Management Console UI, and easily confirm that they migrated successfully.

The tasks in the plan are arranged sequentially and represented in the form of steps:

• Step 1 consists of parallel tasks (test_users, test_address) and failure handler (test_error).

• Step 2 launches the test_name task only if parallel tasks in Step 1 are executed.

• Step 3 executes test_emails after Step 2, irrespective of the error status.

Talend Administration Center plan CRON trigger, which executes Every quarter on 5th at 12:37 is mapped to TMC MONTHLY trigger which executes Every 3 month, on 5th at 12:37.

Talend white paper Self-Service Talend Migration shows you how to migrate your Talend on-premise to Talend Cloud. Highlights include; ensuring success by planning and assessment, Talend project audit, and potential pitfalls.

Content:

• Cloud upgrade best practices
• Upgrade process
• Jobs
• Remote Engines
• Differences in on-premises Talend and Talend Cloud
• UNC paths
• Context variable naming
• Connection parameters naming
• Context variables
• Double quotes
• Migrating your on-premises project to Talend Cloud
• Talend project upgrade
• Talend Cloud Remote Engines and Runtime sizing
• Alternative

Cloud upgrade best practices

The best practice depends on if you’re already using Talend Cloud extensively.

If you have developed a lot of Jobs, the best practice is not to do anything - at least in the short term. Talend Cloud is backward compatible with most Studio versions (currently to 6.5). So even if Studio is on an older version, Jobs will work.

The same applies to Remote Engines. For example, you can still use the old 2.5 Remote Engine though the latest one is version 2.8. However, Talend recommends that you upgrade. Look for the times when the project is stable or when you're starting a new project.

If you haven’t developed a lot of Jobs and are at release 6.5 or newer, or if you really want to use a new feature, you're welcome to upgrade Studio and Remote Engines.

For more information, see Talend Support Statements.

Upgrade process

Jobs

After you upgrade the Git/GitHub repository, it cannot revert back to the old version, so you need to duplicate (copy or clone) your Git/GitHub repository to a new repository. Then, if there are any issues, you can always fall back to the old project and on-premises Talend.

Use the repository copy to create a new project in Talend Cloud TMC. The Git/GitHub URL will be downloaded to Studio at the first connection. The repository will be automatically upgraded at this time. Individual Jobs may need a tweak or two. For example, any deprecated components should be replaced.

Remote Engines

To upgrade a Remote Engine, simply install and pair it. The new Remote Engine can be installed on the same compute resource as the existing resource. However, if any changes to the default configurations (memory, parallel executions) have been made, they need to be applied to the new Remote Engine.

Differences in on-premises Talend and Talend Cloud

UNC paths

When using UNC paths in Talend Cloud, observe the following:

• When hard-coding the UNC path within a file component, the format is "\\\\networkdrive\\folderA\\folderB\\file.txt" (quoted-string)

• When using Context Variable and setting the type to Directory, then the path format is \\networkdrive\folderA\folderB\file.txt (backslashes do not need escaping)

• When using Context "connection _" Variable, the corresponding connection in the cloud environment is \\\\networkdrive\\folderA\\folderB\\file.txt (no quotes) for the value

Context variable naming

Connection parameters naming

To facilitate a smooth migration from on-premises to Talend Cloud, Studio context variables should conform to Talend Cloud connection parameter naming standards. This is for context variables used for connections to external data systems. These can then be stored in context metadata and shared with multiple Talend Jobs.

Context variables to be used for connections should use the following pattern:

connection_<conntype>_<param1..n>

Where connection_ is a fixed string, <conntype> is the type of connection, and <param> is the parameter that matches one of the connection component parameters (for example, userid, password, and server). <conntype> can be anything.

For more information, see the list of Talend Cloud out-of-the-box supported connections.

Context variables

Double quotes

Jobs that run in TAC require double quotes for string values in context variables. However, Talend Cloud strings must not use double quotes. The following example shows the two sets of values for the context variable fileName. One conforms to Jobs run in TAC, the other to Jobs run in Talend Cloud.

Double-quote errors in the Talend Cloud execution log are similar to:

Failed
tFileInputDelimiter_1 - "C:\IntroTalendStudio\prod\"refProducts.csv (The filename, directory name, or volume label syntax is incorrect)

Migrating your on-premises project to Talend Cloud

Talend project upgrade

If your on-premises project is several releases behind the latest Talend Cloud version, you may want to compile all your Jobs to uncover any migration issues. If you have many Jobs in your project, Talend recommends that you consider installing and deploying Continuous Integration for Talend Cloud to automate the compile. Note any compile failures and take corrective action.

Common causes of compile failures in an on-premises migration include:

• Deprecated components

• Incompatible Java versions

• Old database drivers

• External sourced components

Talend recommends that you perform analysis on your on-premises Talend projects and Jobs to anticipate any migration challenges, and to reduce risk.

Talend Cloud Remote Engines and Runtime sizing

Remote Engine and Runtime servers can be sized very similarly to Remote JobServers. However, you have to take the following into consideration:

• Memory and CPU requirements of the Jobs you're going to run

• All are Java, so there there can be great variety in run unit size

• Simultaneous Job execution requirements, such as how many Jobs are required to be running at the same time

Based on this: (highest Job RAM requirement X max count of simultaneous Jobs) + 25% (or 33% or 50% -- pick a fudge factor) = initial sizing.

Be sure to account for the overhead of the Operating System, plus the memory requirements of the Talend Cloud Remote Engine and optionally the Talend Runtime.

Alternative

An easier alternative, and recommended best practice is to resize your compute resource on the fly. As compute resources are practically a commodity, changing size after the initial go-live is easy.

1. Start with a 4 CPU and 16 GB RAM configuration.

2. Monitor your Jobs.

3. Scale the compute node up or down from there as your load requires.

Historically born for on-premises systems, Talend started as an open-source data integration company with unique data quality features and quickly grew the product to make big data integration easy for customers. Talend Cloud on AWS, the first iteration of a cloud data integration platform, was introduced in 2015. Since then, Talend has expanded the cloud value, providing a multi-cloud integration platform with unique features.

Now Talend Cloud platform, with hybrid deployment capabilities, is mature enough to cover all cloud, as well as historical on-premises, use cases. By moving to Talend in the cloud, customers can do even more using cloud-only services and capabilities, such as Talend managed Execution Engines, Pipeline Designer, Data Inventory with the Talend Trust Score, API Services, and Stitch Data Loader.

For more information on the benefits of Talend Cloud platform, see Moving to cloud? What are the benefits of Talend Data Fabric in the cloud.

There are multiple paths to move to the cloud and the path you choose depends on two main factors:

• Skill set that you have in the company

• Modularity within your existing workloads

Moving to Talend in the cloud, choose your strategy

In general, when moving to the cloud, organizations follow one of three main strategies: Lift and shift, re-architect and refactor. Each of these strategies require different considerations when considering a move to Talend in the cloud.

Lift and shift

As the name implies, this method involves lifting your entire stack and shifting it from an on-premises environment to hosting it in the cloud. In other words, you will move a copy of your current environment, without making extensive changes, to the cloud. Once this is done, you no longer have to manage or maintain on-premises hardware for your Talend deployment.

Source code management

Talend in the cloud supports Git based source control only. If you are currently using SVN or other non-Git based source code management, you’ll need to migrate. You can find tutorials on Qlik Community to help with this process.

By moving to Git based source code management you’ll be able to use source code management (SCM) services. A key advantage of this is that hosted SCM services usually have DevOps capabilities that can be easily integrated with the continuous integration capabilities of Talend Cloud platform.

Development tools

Talend Studio is your unified development environment whether you are using Talend on-premises or Talend in the cloud. Moving to Talend Cloud platform gives you the opportunity to use an innovative set of development tools such as: Pipeline Designer, Stitch Data Loader, and API Services (API Designer and API Tester).

Execution engines

If you are using JobServers in your current on-premises environment, Talend Remote Engine is the recommended execution engine for Talend in the cloud. Like JobServers, remote engines are installed and managed by you, have unlimited resources, and are ideal for hybrid architecture deployments.

If your data sources or targets are behind a firewall or if the data that needs to be processed by Talend Jobs must stay in its original location – for example due to data jurisdiction, security or compliance requirements – remote engines are ideal. This is because remote engines only send status and execution logs to Talend Management Console with no data being transited through Talend Cloud platform.

Admin assets, projects, and tasks

To support your move to the cloud, Talend has developed a number of tools to facilitate your migration, which are included in Talend Cloud Migration Platform (TCMP). TCMP, is available in the download section of Talend in the cloud or from the Talend Community Move to Cloud > Cloud Migration Tools page. It moves your on-premise Users, Groups, Projects, Tasks, and Execution plans to Talend Cloud platform. With TCMP you’ll be able to easily migrate admin assets from multiple Talend Administration Console environments and different versions simultaneously from a single application.

Replatform

As a variation of lift and shift, replatforming involves making a few adjustments to your existing environment to optimize your landscape for the cloud. This allows workloads to take advantage of cloud functionality and cloud cost optimization without the high level of resources you had on-premises. The core architecture of applications stays the same, while allowing you to take advantage of these cloud capabilities for better scaling and resources management.

Managed execution engines

A managed execution engine or Talend Cloud Engine is a compute resource managed by Talend in the cloud. It runs on demand to executes Job tasks. It has fixed physical specification (2 vCPU, 8 GB Ram, 200 GB), can run up to three tasks in parallel, and requires sources and targets to be visible through the internet.

Talend Cloud Engines are completely preconfigured, making it easy to start right away in the cloud with just a couple clicks in Talend Management Console. If you move/synchronize data from SaaS applications or cloud platforms and if the cloud engines' fixed resources are sufficient for your workloads, this could be the right choice for you.

Infrastructure as Code

Infrastructure as Code (IaC) is the process of managing and provisioning resources (compute, storage, and others) through code rather than through physical hardware configuration or interactive configuration tools. IaC can greatly simplify and accelerate your infrastructure provisioning processes, help you avoid mistakes and comply with policies, keep your environments consistent, and save your company a lot of time and money.

IaC should be a crucial aspect of your move to the cloud. If you think of cloud computing as the first step to solving many of the problems caused by manual IT management, then it’s fair to say the IaC is the next logical step. It takes cloud computing to its fullest potential, and it frees developers and other professionals from performing manual, error-prone tasks.

Talend Remote Engines being available on Azure and AWS marketplaces allows you to programmatically scale, provision and deploy, and start and stop your Remotes Engines. The Talend Management Console API has Remote Engine management automation methods in order to create, delete and pair Remote Engines programmatically (TMC API). This allows you with unfettered elasticity – the ability to easily scale up or down your usage of Talend in the cloud and the needed Azure or AWS resources.

DevOps culture and Zero Install Continuous Integration (CI)

If your organization does not currently ascribe to the Software Development Life Cycle methodology, moving to the cloud is the right time to make the move and adopt the DevOps culture. Talend is compliant with Java and Maven standards, meaning that you can use a classic CI server application such as Jenkins and a common artifact repository like Nexus or Artifactory. In the cloud, a Talend Job can be built, delivered and deployed in two different ways: you can either build an Artifact, deliver it to a Talend Management Console repository, and deploy it to a Remote Engine, Cloud Engine, or build a Docker image to be delivered on a Docker Registry and deployed to container services.

When implementing CI/CD, Talend recommends dedicating a CI secondary machine that contains prerequisites to build a Talend Job, such as Talend CommandLine and Maven plugins (CI builder and Cloud Publisher). 

Refactor

The Refactor approach is driven by a strong desire to improve your environment and represents the opposite of a lift-and-shift migration. Under this approach, it is assumed that the business needs to leverage cloud capabilities that are not available in its existing environment. In some instances, this requires re-engineering the application logic completely and developing cloud-native versions from scratch. In the long run, refactoring can be more cost-efficient because of the additional features added to particular applications.

Adopt cloud-only applications

The main value pillars for Talend Data Fabric in the cloud are innovation, lower total cost of ownership, scale, and security. Innovation is always a step ahead on Talend Cloud platform with monthly releases of new features, automatic upgrades, and most importantly, cloud-only capabilities and services. To illustrate this, take a look the Talend Willamette Dental Group case study.

Stitch Data Loader makes loading data to your cloud data warehouses and data lakes simple, secure and frictionless:

Pipeline Designer is the next generation self-service cloud data integration application:

Talend API Services provides you with aa comprehensive API design and delivery platform.

Data Inventory with the Talend Trust Score maintains an inventory of data assets managed within Talend Cloud platform:

Leverage new personas in your company

Talend cloud-only capabilities and services will help more non-technical persons to use data to bring value to your organization. New data user personas are emerging in most organizations, with many being subject matter experts with little or no technical background. Talend Cloud platform brings the right set of tools for them.

Data owners can govern their data in the cloud using Data Inventory, leveraging the Talend Trust Score to ensure that the data is of high quality and relevant. Citizen integrators no longer have to go to IT to run a new report. They can easily create a data integration or a data quality pipeline using Pipeline Designer on a web browser, even using their tablet. Data stewards can run a data quality campaign using Data Stewardship and business analyst can leverage Talend Data Preparation in the cloud to easily standardize data from multiple sources to help them make crucial decisions. All of these capabilities are available in a governed self-service fashion on a single cloud platform.

Conclusion

Which one is the best approach?

There is no absolute answer to this question, mainly because different use cases require different things. Picking one of the three approaches is a matter of finding the one that best suits your specific needs. Your organization's needs may vary and delve you into adopting a combination of these strategies