Skip to main content

Connectivity & Data Prep

Discussion board where members can learn more about Qlik Sense Data Connectivity.

Announcements
CUSTOMERS ONLY: Now accepting customer applications for the 2023 Luminary Program: SUBMIT NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
fashid
Specialist
Specialist

Issue in Extracting Data from Elastic Search through rest connector

Hi Experts,

I have been trying to extract data from the Elastic search through rest connector but i have been able to fetch only 10 records . 

Below is the script that i am using 

let url = "http://10.35.12.5:9200/prem_reg_final_4_test_1_201903/_search?";

LIB CONNECT TO 'testconnection_heah (conn_qlikdv1)';

NoConcatenate

 

scrollTable:

SQL SELECT

"_scroll_id" AS "scrollid",

(SELECT

"total" AS "total_u1"

FROM "hits")

FROM JSON (wrap on) "root"
WITH CONNECTION(Url "$(url)");

 

Let total = Peek('total_u1',0,'scrollTable');
Let scrollid = Peek('scrollid',1,'scrollTable');

 

for startAt = 0 to total step pageSize
RestConnectorMasterTable:
SQL SELECT
"__KEY_root",
(SELECT
"__FK_hits",
"__KEY_hits",
(SELECT
"__FK_hits_u0",
"__KEY_hits_u0",
(SELECT
"NUM_REFERENCE_NO",
"__FK__source"
FROM "_source" FK "__FK__source")
FROM "hits" PK "__KEY_hits_u0" FK "__FK_hits_u0")
FROM "hits" PK "__KEY_hits" FK "__FK_hits")
FROM JSON (wrap on) "root" PK "__KEY_root"
WITH CONNECTION(url "$(url)");

scrollid = lookup('_scroll_id', '__KEY_root', 1,'RestConnectorMasterTable');

Trace $(scrollid);

url = 'http://10.35.12.5:9200/prem_reg_final_4_test_1_201903/_search/scroll?scroll=1m&size=100&scroll_id=$(...)';

 

NEXT startAt;

When i am trying to run this script i am getting the below error 

The following error occurred:
QVX_UNEXPECTED_END_OF_DATA: HTTP protocol error 400 (Bad Request): {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/prem_reg_final_4_test_1_201903/_search/scroll] contains unrecognized parameters: [scroll], [scroll_id], [size]"}],"type":"illegal_argument_exception","reason":"request [/prem_reg_final_4_test_1_201903/_search/scroll]
 
Also if i just load the hits table to get total and scroll id then my scroll id field is blank . i do have values in the total field .
 
If anybody could guide or throw some light into this . 

 

 

Regards,

Nadeem Shaikh

 

 

5 Replies
Anil_Babu_Samineni

For me this sounds, there might be some filter condition in the URL white creating the connection. 

Before develop something, think If placed (The Right information | To the right people | At the Right time | In the Right place | With the Right context)
fashid
Specialist
Specialist
Author

Hi,

thank you for replying could you please help out in understanding what you mean by filter condition in the URL 

Regards,

Nadeem 

fashid
Specialist
Specialist
Author

Any Experts out there who can help on this 

fashid
Specialist
Specialist
Author

Bump.....

martinpohl
Partner - Master
Partner - Master