If you have been building custom web applications or mashups with Qlik Cloud, you have likely hit the "10K cells ceiling" when using Hypercubes to fetch data from Qlik.(Read my previous posts about Hypercubes hereandhere)
You build a data-driven component, it works perfectly with low-volume test data, and then you connect it to production; and now suddenly, your list of 50,000+ customers cuts off halfway, or your export results look incomplete.
This happens because the Qlik Engine imposes a strict limit on data retrieval: a maximum of 10,000 cells per request. If you fetch 4 columns, you only get 2,500 rows (4 (columns) x 2500 = 10,000 (max cells)).
In this post, I’ll show you how to master high-volume data retrieval using the two strategies: Bulk Ingest and On-Demand Paging, usingthe@qlik/api library.
What is the 10k Limit and Why Does It Matter?
The Qlik Associative Engine is built for speed and can handle billions of rows in memory. However, transferring that much data to a web browser in one go would be inefficient. To protect both the server and the client-side experience, Qlik forces you to retrieve data in chunks.
Understanding how to manage these chunks is the difference between an app that lags and one that delivers a good user experience.
Step 1: Defining the Data Volume
To see these strategies in action, we need a "heavy" dataset. Copy this script into your Qlik Sense Data Load Editor to generate 250,000 rows of transactions (or download the QVF attached to this post):
// ============================================================
// DATASET GENERATOR: 250,000 rows (~1,000,000 cells)
// ============================================================
Transactions:
Load
RecNo() as TransactionID,
'Customer ' & Ceil(Rand() * 20000) as Customer,
Pick(Ceil(Rand() * 5),
'Corporate',
'Consumer',
'Small Business',
'Home Office',
'Enterprise'
) as Segment,
Money(Rand() * 1000, '$#,##0.00') as Sales,
Date(Today() - Rand() * 365) as [Transaction Date]
AutoGenerate 250000;
Step 2: Choosing Your Strategy
There are two primary ways to handle this volume in a web app. The choice depends entirely on your specific use case.
1- Bulk Ingest (The High-Performance Pattern)
In this pattern, you fetch the entire dataset into the application's local memory in iterative chunks upon loading.
The Goal: Provide a "zero-latency" experience once the data is loaded.
Best For: Use cases where users need to perform instant client-side searches, complex local sorting, or full-dataset CSV exports without waiting for the Engine.
2- On-Demand (The "Virtual" Pattern)
In this pattern, you only fetch the specific slice of data the user is currently looking at.
The Goal: Provide a near-instant initial load time, regardless of whether the dataset has 10,000 or 10,000,000 rows as you only load a specific chunk of those rows at a time.
Best For: Massive datasets where the "cost" of loading everything into memory is too high, or when users only need to browse a few pages at a time.
Step 3: Implementing the Logic
While I'm using React and custom react hooks for the example I'm providing, these core Qlik concepts translate to any JavaScript framework (Vue, Angular, or Vanilla JS). The secret lies in how you interact with the HyperCube.
The Iterative Logic (Bulk Ingest):
The key is to use a loop that updates your local data buffer as chunks arrive.
To prevent the browser from freezing during this heavy network activity, we use setTimeout to allow the UI to paint the progress bar.
qModel = await app.createSessionObject({ qInfo: { qType: 'bulk' }, ...properties });
const layout = await qModel.getLayout();
const totalRows = layout.qHyperCube.qSize.qcy;
const pageSize = properties.qHyperCubeDef.qInitialDataFetch[0].qHeight;
const width = properties.qHyperCubeDef.qInitialDataFetch[0].qWidth;
const totalPages = Math.ceil(totalRows / pageSize);
let accumulator = [];
for (let i = 0; i < totalPages; i++) {
if (!mountedRef.current || stopRequestedRef.current) break;
const pages = await qModel.getHyperCubeData('/qHyperCubeDef', [{
qTop: i * pageSize,
qLeft: 0,
qWidth: width,
qHeight: pageSize
}]);
accumulator = accumulator.concat(pages[0].qMatrix);
// Update state incrementally
setData([...accumulator]);
setProgress(Math.round(((i + 1) / totalPages) * 100));
// Yield thread to prevent UI locking
await new Promise(r => setTimeout(r, 1));
The Slicing Logic (On-Demand)
In this mode, the application logic simply calculates the qTop coordinate based on the user's current page index and makes a single request for that specific window of data (rowsPerPage).
const width = properties.qHyperCubeDef.qInitialDataFetch[0].qWidth;
const qTop = (page - 1) * rowsPerPage;
const pages = await qModelRef.current.getHyperCubeData('/qHyperCubeDef', [{
qTop,
qLeft: 0,
qWidth: width,
qHeight: rowsPerPage
}]);
if (mountedRef.current) {
setData(pages[0].qMatrix);
}
I placed these two methods in custom hooks (useQlikBulkIngest & useQlikOnDemand) so they can be easily re-used in different components as well as other apps.
Best Practices
Regardless of which pattern you choose, always follow these three Qlik Engine best practices:
Engine Hygiene (Cleanup): Always call app.destroySessionObject(qModel.id) when your component or view unmounts.
Cell Math: Always make sure your qWidth x qHeight is strictly < 10,000. For instance, if you have a wide table (20 columns), your max height is only 500 rows per chunk.
UI Performance: Even if you use the "Bulk" method and have 250,000 rows in JavaScript memory, do not render them all to the DOM at once. Use UI-level pagination or virtual scrolling to keep the browser responsive.
Choosing between Bulk and On-Demand is a trade-off between Initial Load Time and Interactive Speed. By mastering iterative fetching with the @qlik/api library, you can ensure your web apps remain robust, no matter how much data is coming in from Qlik.
💾Attached is the QVF and here is theGitHub repository containing the full example in React so you can try it in locally - Instructions are provided in the README file.
(P.S:Make sure you create the OAuth client in your tenant and fill in the qlik-config.js file in the project with your tenant-specific config).
Thank you for reading!
...View More
As we enter the last month of the year, let’s review some recent enhancements in Qlik Cloud Analytics visualizations and apps. On a continuous cycle, features are being added to improve usability, development and appearance. Let’s’ look at a few of them.
Straight Table
Let’s begin with the straight table. Now, when you create a straight table in an app, you will have access to column header actions, enabled by default. Users can quickly sort any field by clicking on the column header. The sort order (ascending or descending) will be indicated by the arrow. Users can also perform a search in a column by clicking the magnifying glass.
When the magnifying glass icon is clicked, the search menu is displayed as seen below.
If a cyclic dimension is being used in the straight table, users can cycle through the dimensions using the cyclic icon that is now visible in the column heading (see below).
When you have an existing straight table in an app, these new features will not be visible by default but can easily be enabled in the properties panel by going to Presentation > Accessibility and unchecking Increase accessibility.
Bar Chart
The bar chart now has a new feature that allows the developer to set a custom width of the bar when in continuous mode. Just last week, I put the bar chart below in continuous mode and the bars became very thin as seen below.
But now, there is this period drop down that allows developers to indicate the unit of the data values.
If I select Auto to automatically detect the period, the chart looks so much better.
Combo Chart
In a combo chart, a line can now be styled using area versus just a line, as displayed below.
Sheet Thumbnails
One of the coolest enhancements is the ability to auto-generate sheet thumbnails. What a time saver. From the sheet properties, simply click on the Generate thumbnail icon and the thumbnail will be automatically created. No more creating the sheet thumbnails manually by taking screenshots and uploading them and assigning them to the appropriate sheet. If you would like to use another image, that option is still available in the sheet properties.
From this To this in one click
Try out these new enhancements to make development and analysis faster and more efficient.
Thanks,
Jennell
...View More
Regex has been one of the most requested features in Qlik Sense for years, and now it’s finally here.
With this year's May 2025 release, Qlik added native support for regular expressions in both load scripts and chart expressions. That means you can validate formats, extract values, clean up messy text, and more, all without complex string logic or external preprocessing.
In this post, we’ll look at what’s new, how it compares to the old workarounds, and a practical example you can plug into your own app.
The New Regex Functions
Regex (short for Regular Expression) is a compact way to define text patterns. If you’ve used it in Python, JavaScript, or other programming languages, the concept will feel familiar.
Qlik now includes native support for regular expressions with functions that work in load scripts and chart expressions:
MatchRegEx() – check if a value matches a pattern
ExtractRegEx() – extract the first substring that matches
ReplaceRegEx() – search and replace based on a pattern
SubFieldRegEx() – split text using regex as the delimiter
There are also group-based versions (ExtractRegExGroup, etc.), case-insensitive variants (MatchRegExI), and helpers like CountRegEx() and IsRegEx().
🔗 Help Article
Replacing Old Patterns
Here's where regex saves time:
Format validation:Replace nested Len(), Left(), and Mid() with a single pattern.
Substring extraction:Skip the manual slicing; let the pattern do the work.
Pattern-based replacements:Clean or reformat values without chaining multiple functions.
With regex:
If(MatchRegEx(Code, '^[A-Z]{2}-\d{5}$'), 'Valid', 'Invalid') // check format
ExtractRegEx(Text, '\d{5}') // get first 5-digit number
ReplaceRegEx(Field, '\D', '') // strip non-digits
Cleaner logic. Fewer steps. Easier to maintain.
Use Cases That Just Got Easier
If any of the following sound familiar, regex will help:
Format checks: postal codes, product SKUs, ID numbers.
Data extraction: get domain from email, number from notes, etc.
PII masking: hide parts of a SSN or credit card.
String cleanup: strip unwanted characters, normalize spacing.
Splitting tricky fields: CSV lines with quoted commas, mixed delimiters.
Keep in mind that these functions can be used directlyin chart expression, so you can build visuals or filters based on pattern logic, not just static values.
Example: Clean and Validate Phone Numbers
Let’s say you’ve got a bunch of phone numbers like this:
(312) 678-4412
312-678-4412
3126784412
123-678-4412 // invalid: area code starts with 1
312-045-4412 // invalid: exchange starts with 0
312-678-441 // invalid: too short
You want to:
Validate that it’s a proper 10-digit North American number
Standardize the format to (###) ###-####
Here’s how to do it with regex in your load script:
LOAD
RawPhone,
// 1. Strip out anything that's not a digit
ReplaceRegEx(RawPhone, '\D', '') as DigitsOnly,
// 2. Validate: 10 digits exactly, starting with 2–9
If(MatchRegEx(RawPhone, '^\(?[2-9]\d{2}\)?[-.\s]?\d{3}[-.\s]?\d{4}$'),
'Valid', 'Invalid') as Status,
// 3. Standardize format to (###) ###-####
ReplaceRegEx(
ReplaceRegEx(RawPhone, '\D', ''),
'(\d{3})(\d{3})(\d{4})',
'(\1) \2-\3'
) as FormattedPhone
INLINE [
RawPhone
3025557890
(404) 222-8800
678.333.1010
213 888 9999
1035559999
678-00-0000
55577
];
Result:
One pattern replaces multiple conditions and formatting is consistent. This is much easier to maintain and easy to expand if the rules change.
Final Thoughts
Use regex where it adds value.For simple cases like Left() or Trim(), stick with built-in string functions.
When you're working with inconsistent inputs, embedded formats, or anything that doesn’t follow clean rules, regex can save a lot of time.
If you're applying regex across large datasets, especially in charts, it’s better to handle it in the load script where possible.
Not sure how to write the pattern?
Tools like regex101.comor regexr.comare great for testing and adjusting before you build in Qlik.
With native regex in Qlik Sense, you can now clean, validate, extract, and transform text with precision without convoluted scripts or third-party tools. It’s a quiet but powerful upgrade that unlocks a ton of flexibility for real-world data.
...View More
Several years ago, I wrote a blog post on how to create a profit and loss statement in QlikView. @Patric_Nordstromhas built upon this method and built a financial statement in Qlik Analytics with a straight table and waterfall chart using inline SVG. In this blog post, I will review how he did it.
Here is an example of the financial statement structure.
There are plain rows, such as gross sales and sales return where the amount is the sum of the transactions made against the accounts. There are subtotals such as net sales and gross margin which are a sum of the previous plain rows. And there are also partial sums such as total cost of sales that is a sum of a subset of the previous plain rows, but not all the previous plain rows.
Patric identified two functions, RangeSum() and Above() that are suitable for calculating the subtotal and partial sums in a table. The RangeSum function sums a range of values, and the Above function evaluates an expression at a row above the current row within a column segment in a table. The above function can be used with 2 optional parameters – offset and count to further identify the rows to be used in the expression.
The layout table below is used as a template for the financial statement.
The RowStyle column is used for styling the rows making text bold or underlining it. Currently, we cannot use tab or blank tags but hopefully in the future this will become available in the straight table.
The RowTitle is the account category that is to be used in the financial statement.
RowType is used to as input to calculate the offset and count for the above function.
AC is the actual amount.
The AC column is included here in the layout file for demo purposes but could be calculated from the accounts and transactions in the data model as well.
In the script, the layout table was loaded, and additional fields were created to support the waterfall chart, specifically offset and count fields to be used with the above function.
Here is a view of the layout table with the new fields that were created in the script.
After the layout table is loaded and the new fields are created, some master measures can be created to be used in the inline SVG expression. Here are the 3 master measures Patric created:
mBar is the bar length with an offset that is always 0.
mStart is the starting position of the bar in the waterfall chart and for subtotals, this is always 0.
mMax is the max bar length which is used to scale the bars in the waterfall chart.
Now the straight table can be created. The RowNr field is added for sorting purposes. The RowTitle field and the AC fields are added to show the account groupings in the financial statement along with their value. The below inline SVG expression for the waterfall chart is the last column added to the straight table. It is made up of 3 parts:
A plain line with an offset
A thin gray line where x=0
A label
The if statement on line 1 will determine if a bar is displayed. Bars will only be visible if the RowStyle is not blank.
Line 2 has the viewBox settings and sets the 0 for the x-axis.
On line 3 is the light gray line where x=0 and it is displayed on all non-blank rows of the financial statement.
Lines 4 and 5 in the yellow box is the plain line with the offset, scaled using the mMax measure to control the length of the line.
Line 6 handles the bar color, light green for positive values and red for negative values.
On line 7, the if statement is used to set the stroke width of the bar. A thin line is used if the RowType is retrosum and a wider line is used for all other bars.
In the red box, on lines 10 through 13, the label text is set and placed either to the left or right of the bar depending on the value. Positive values are placed to the right of the bar and negative values are placed to the left of the bar.
The result of the financial statement looks like this:
To add the text styling (bold and underline) from the layout table, the RowStyle field was added to the text style expression in the RowTitle and AC columns.
Indentation is added by using the repeat function in the RowTitle column. It will repeat a non-breaking space 6 times if there is a tab tag in the RowStyle field. Otherwise, no indentation is done.
If the RowStyle is not blank, a bar is displayed for the waterfall chart and the sum value for the actual amount (or mBar) in this case is displayed.
The chart column representation is set to Image in the properties of the straight table.
While this method looks complex, it is a simple and clean solution for adding a waterfall chart to a financial statement using straight table features and inline SVG. Using the layout table and inline SVG provides room for customization so that the financial report meets the needs and requirements of the user or customer.
Thanks,
Jennell
...View More
In this blog post, I will illustrate two ways that an image in an analytics app can be changed dynamically based on selections. In one example, a layout container and the show condition is used to control which image is visible. In a second example, an IF statement is used to determine which image URL to use. In the second example, I will also review how to obtain the URL for an image in the media library of an analytics app.
Let’s begin by looking at the first example. In this example, we have an app on automobiles. When the user selects one of the automobile names, a picture of the car will be visible. If no cars are selected, then an image of all 4 automobiles should be displayed. Below is the image with no selections and all 4 cars visible.
When the Nebula model is selected, the sheet updates and only the Nebula model is visible.
In this example, a layout container has 5 Text & image objects – one image for each automobile model and one image for all cars.
A show condition like this is used to determine if the Text & image object should be visible. Only one image will be visible at any given time.
Now, let’s look at another example of how the background image can be dynamically displayed using an URL. In this example, one Text & image object is needed. In the properties of the object, under Presentation > Styling, a background image can be set. When URL is selected from the drop-down list, the option to enter an expression is available. Note that you want to edit the Background image setting that is in Presentation > Styling, not the Background image section in the properties panel.
The IF statement below checks to see which automobile model is selected to determine which image to show. If there is not a selection on the model, then the image of all the automobiles is displayed. In this example, the URLs for the media library images were used, but you could also link to images in an external source like an Amazon s3 bucket.
To find the URL for images in the media library, you can open the media library or use a sheet in the app that has the image on it. In my Chrome browser, I opened the Developer tools to inspect the image. In Chrome, the Developer tools can be opened by using Ctrl-Shift-I or from the three-dots icon in the upper right corner of the browser window and selecting More tools > Developer tools. Once the Developer tools window is open you can inspect an element by clicking on the Elements tab and then selecting the image in the highlighted box below. This will allow you to select elements on the page.
In the analytics app, click on the image in the library so that it appears in the preview window. When you click on the image in the preview window, you will see that a line in the html is highlighted. If necessary, expand the code to find the URL for the image.
Copy the URL that is listed in the src attribute for the img tag (see below). In the expression, you can use the full URL that includes https:// and the tenant name or just what is displayed in the src tag. Repeat these steps for each image.
I found this insight useful, and I hope you do, too. I would like to thank my colleagues @Charles_Bannon for the blog idea and use of his app to provide examples and @Ouadie for the tip on how to capture the media library URL.
Thanks,
Jennell
...View More
When I’m exploring a Qlik app, I often want to show exactly how I reached a view—without typing instructions or jumping on a call.
Qlik Trail is a small extension I created that does just that: Click Record, make your selections, and you’ll get a tidy list of steps that you can replay from start to finish, or jump to a specific step. Then, when you’re happy with the journey, you can export the trail as JSON and hand it to someone else so they can import and run the same sequence on their side.
Why not just use bookmarks?
Bookmarks are great for destinations (one final selection state). Qlik Trail is more about the path:
Multiple steps in a defined order (A → B → C)
Replay from start for narration orReplay a specific step
Export/import a whole journey, not a bunch of one-offs
You can use bookmarks to save a point in time, and use Qlik Trail when the sequence matters more.
What can you do with the Qlik Trail extension?
Record steps manually, or toggle Auto to capture each change
Name, duplicate, reorder, delete, or ungroup steps
Organize steps into Groups (Executive Story, Sales Ops, Training, …)
Replay to here (apply one step) or Replay from start (play the group)
Export / Import trails as JSON
Demo
Before we go into how the extension was built, let's see a demo of how it can be used.
I’ll share two practical trails an Executive Storyand Sales Opswith selections you can record and replay on your end, for this demo I'll use the Consumer Goods Sales app (which you can get here: https://explore.qlik.com/details/consumer-goods-sales)
Setup
Upload the extension .zip file in the Management Console, then add Qlik Trail (custom object) to a sheet.
Optional: enable Auto-record in the object’s properties if you want every change captured while you explore.
Toolbar (left to right): Record, Replay from start, Delete, Delete all, Auto, Group filter, New group, Export, Import.
Trail 1 — Executive Story
Create Group: Executive Story
Step 0 — “US overview: top categories”Selections: (none)
Step 1 — “Northeast · Fresh Vegetables”Selections: Region = Northeast, Product Group = Fresh Vegetables
Step 2 — “Northeast · Cheese (A/B)”Selections: Region = Northeast, Product Group = Cheesequick A/B inside the same region.
Step 3 — “West vs Northeast · Fresh Fruit”Selections: Region = West, Northeast, Product Group = Fresh Fruitsame category across two regions to see differences.
Step 4 — “Northeast focus: PA · Fresh Fruit”Selections: Region = Northeast, State = Pennsylvania, Product Group = Fresh Fruitdrill to a particular state.
Presenting: select the group → Replay from start. If questions land on Step 4, use Replay to here.
Trail 2 — Sales Ops Story
Create Group:Sales Ops Story
Step 1 — “South · Ice Cream & Juice (summer basket)”Selections: Region = South, Product Group = Ice Cream, Juice
Step 2 — “Central · Hot Dogs (promo check-in)”Selections: Region = Central, Product Group = Hot Dogs
Step 3 — “West · Cheese (margin look)”Selections: Region = West, Product Group = Cheese
Tips when building trails
One idea per step. (Region + Category) or (Region + a couple of States)
Duplicate then tweak for fast A/B comparisons
Group by audience. Exec, Ops, Training
Ungrouped steps = scratchpad. Move into a group when it’s ready
After recording, drag to reorder so replay tells a clean story
Sharing your trails
Click Export to download qliktrail_export_YYYY-MM-DD_HHMM.json.
Your teammate can Import, pick a group, and hit Replay from start. Same steps, same order—no instructions needed.
Tech notes
1) Module & CSS injectionLoad CSS once at runtime so the object stays self-contained:
define(['qlik','jquery','./properties','text!./style.css'], function(qlik, $, props, css){
if (!document.getElementById('qliktrail-style')) {
const st = document.createElement('style');
st.id = 'qliktrail-style';
st.textContent = css;
document.head.appendChild(st);
}
// …
});
2) Capturing selections per stateRead SelectionObject, group by qStateName, then fetch only selected rows via a lightweight list object. Store both text and numeric to be resilient to numeric fields:
app.getList('SelectionObject', function(m){
// build grouped { state -> [fields] }
});
...
app.createList({
qStateName: stateName || '$',
qDef: { qFieldDefs: [fieldName] },
qInitialDataFetch: [{ qTop:0, qLeft:0, qWidth:1, qHeight:10000 }]
}).then(obj => obj.getLayout() /* collect S/L/XS states */);
Snapshot (what we export/import):
{
"ts": "2025-03-10T14:30:00Z",
"states": [
{
"state": "$",
"fields": {
"Region": { "text": ["Northeast"], "num": [] },
"Product Group": { "text": ["Fresh Vegetables"], "num": [] }
}
}
]
}
3) Replaying selectionsClear target state, then select values. Prefer numbers when available; fall back to text:
app.clearAll(false, state);...
field.selectValues(pack.num.map(n=>({qNumber:n})), false, false)
// fallback to qText
field.selectValues(pack.text.map(t=>({qText:String(t)})), false, false);
4) Auto-record with a quiet windowAvoid recording during replays to prevent ghost steps:
ui.replayQuietUntil = Date.now() + (layout.props.replayQuietMs || 1200);
if (Date.now() >= ui.replayQuietUntil && ui.autoOn) record();
5) Groups, ordering & persistenceTrails live in localStorage per app key. Groups look like { id, name, members:[stepId,…] }. Ordering is a simple seq integer; drag-and-drop reassigns seq inside the open list.
That’s Qlik Trail in a nutshell. It lets you hit record, name a few steps, and replay the story without hand-holding anyone through filters. Use Bookmarks to keep states, but Trail will help you keep tack of the sequence.
P.S: this extension is a work-in-progress experiment and not prod-ready, I'm planning to add more features to it and make it more stable in the future.🖇 Link to download the extension:https://github.com/olim-dev/qlik-trail
...View More
Delivering a multilingual Qlik Sense app shouldn’t mean forcing users to pick their preferred language every time. In this post, I explore how to streamline the experience by automatically detecting a user’s locale from their Qlik Cloud profile and instantly adapting the app’s interface - no dropdowns, no extra clicks.
Building on the well-known community approach shared by Jennel, this method keeps the flexibility of translation tables while removing friction for end users. The result? A truly seamless multilingual experience that feels natural and effortless.
