Issue with ClickUp API Pagination in Qlik (Rest Connector)

[cyrine]

I'm working with ClickUp's API using the REST connector and facing an issue with pagination. The ClickUp API provides only a page query parameter for pagination, and the response includes a last_page boolean in the JSON. However, I couldn't use the pagination feature provided in the Qlik REST connector since it only supports next_page and last_page, and I only have access to the page parameter.

To handle pagination manually, I implemented a DO WHILE loop in the script. Despite this, I still face redundancy issues with many tasks having the same key_task, resulting in duplicates + im not really getting all the tasks from the 9 pages some are still missing. Here’s a snippet of my code:

LET page = 1; // Start with page 1

//***************************************************************************************
LIB CONNECT TO 'Dev_Tasks';

// Loop to fetch all pages
DO WHILE page <= 9
// Chargement des données depuis le connecteur REST
RestConnectorMasterTable:
SQL SELECT
"__KEY_root",
(SELECT
"id" AS "id_u9",
"custom_id",
"custom_item_id",
"name" AS "name_u4",
"text_content",
"description",
"orderindex" AS "orderindex_u1",
"date_created" AS "date_created_u1",
"date_updated",
"date_closed",
"date_done",
"archived",
"parent",
"top_level_parent",
"priority",
"due_date",
"start_date",
"points",
"time_estimate",
"team_id",
"url",
"permission_level",
"__KEY_tasks",
"__FK_tasks",
(SELECT
"status",
"id",
"color",
"type",
"orderindex",
"__FK_status"
FROM "status" FK "__FK_status"),
(SELECT
"id" AS "id_u0",
"username",
"color" AS "color_u0",
"email",
"profilePicture",
"__FK_creator"
FROM "creator" FK "__FK_creator"),
(SELECT
"id" AS "id_u1",
"username" AS "username_u0",
"color" AS "color_u1",
"initials",
"email" AS "email_u0",
"profilePicture" AS "profilePicture_u0",
"__FK_assignees"
FROM "assignees" FK "__FK_assignees")
FROM "tasks" PK "__KEY_tasks" FK "__FK_tasks")
FROM JSON (wrap on) "root" PK "__KEY_root" FK "__FK_tasks"
WITH CONNECTION (
URL "https://api.clickup.com/api/v2/list/901800546921/task?subtasks=true&include_closed=true&page=$(page)"
);

// Increment the page number
LET page = $(page) + 1;

// Trace the current page number for debugging
TRACE Page Number: $(page);

LOOP;