topic Re: Publish to QS Issues in Catalog and Lineage

Publish to QS Issues

daniel_dalnekoff — Sat, 16 Nov 2024 03:15:23 GMT

Hi,

Trying to get this working, but I am stuck...and hopeful this community of experts might be able to point me in the right direction!

I went through the steps to setup this feature (in the pdf) and get the Publish option from the cart, but getting some errors:

If I choose publish to Qlik Sense I get the following:

"2020-02-16 11:57:34.269: Task OnDemandQSEntityPromotionTask failed: PodiumFault: core.error.code.USER_CANNOT_PROMOTE_ENTITY_FOR_PUBLISH - Selected entity is not in a managed state and doesn't carry any data. The user requested the publish cannot promote entity to Managed."

If I choose publish advanced I get:

"There was an error fetching the app list"

I would appreciate any guidance!

thanks,

Re: Publish to QS Issues

ThiebaudS — Mon, 17 Feb 2020 08:29:25 GMT

Hi there,

You have 3 data management levels in QDC, and you have to choose one when you onboard data into QDC.

Adressed, Registered or Managed.

Only the "Managed" option will copy the entire data into QDC storage platform.

As your error message says "Selected entity is not in a managed state and doesn't carry any data", I guess the "publish to Qlik Sense" action doesn't work with "Adressed" or "Registered" data, because QDC is not managing the data, only the technical schema and some profiling, or maybe your user doesn't have sufficient privileges as mentioned in "The user requested the publish cannot promote entity to Managed".

You should try to onboard data with the "Managed" option.

Best regards,

Thiebaud

Re: Publish to QS Issues

daniel_dalnekoff — Mon, 17 Feb 2020 19:58:11 GMT

Thank you for the info - this is helpful, but I should have been more clear:

I am using data catalyst for QVDs, and when creating my connection to the QVD folder my only option for default entity level is Registered. Is there a way in the product to promote a given entity once it has been loaded?

Interesting thought on the user, but they are setup as an admin in both QDC for QVD and also in QS.

Re: Publish to QS Issues

daniel_dalnekoff — Thu, 20 Feb 2020 00:56:16 GMT

ok - getting a new error now after working through this a bit more -

Failed to create Qlik app via process [QlikSenseAppGeneratorSvcImpl[http-bio-8080-exec-3]]com.podiumdata.base.error.PodiumFault: core.error.code.UNABLE_TO_EXTRACT_APP_ID_FROM_QLIK_SCRIPT - Unable to extract app id from qlik script call return.

Re: Publish to QS Issues

ThiebaudS — Thu, 20 Feb 2020 10:16:50 GMT

Hi again,

Could you share the content of the "QlikSense Publish Capability" section in the /usr/local/qdc/apache-tomcat-7.0.94/conf/core_env.properties file ?

And the content of your createQlikApp.js file ?

Best regards,

Thiebaud

Re: Publish to QS Issues

daniel_dalnekoff — Thu, 20 Feb 2020 11:27:20 GMT

Thank you Thiebaud,

I attached both of those (text combined in a single Word doc or 2 files zipped together), replacing some entries of internal info with <_EXPLANATION_>

I greatly appreciate any thoughts you could share.

Re: Publish to QS Issues

daniel_dalnekoff — Thu, 20 Feb 2020 11:32:42 GMT

If it is easier here - createApp.js

const enigma = require('enigma.js');
const WebSocket = require('ws');
const path = require('path');
const https = require('https');
const fs = require('fs');
const schema = require('enigma.js/schemas/12.20.0.json');
const engineHost ='<__HARD_CODED_PATH_TO_SENSE_ENGINE_HERE_FOR_TESTING__>';// process.argv[5];//'10.111.2.163';
const enginePort = process.argv[6];//4747;
const proxyPort = process.argv[7];//4243;
//const appId = '4cb2c5d0-b4d8-4805-8696-f255cbc14e4g'; moved to ramdom function
const XRFKEY = Math.floor(Math.pow(10, 16-1) + Math.random() * (Math.pow(10, 16) - Math.pow(10, 16-1) - 1));
//const XRFKEY = Math.floor((Math.random() * 10000000000000000) + 1);
//const userDirectory = 'EC2AMAZ-KKO6EGS';
//const userId = 'qlik_service';
const userDirectory = '<__DIRECTORY_NAME_IS_HERE__>'; //process.argv[8];//'EC'
const userId = process.argv[4];
const certificatesPath = './certs';
const readCert = filename => fs.readFileSync(path.resolve(__dirname, certificatesPath, filename));
//require("./console-file");
//console.file("./to.log");

var appId = '4cb2c5d0-b4d8-4805-8696-';
function createRandomAppId() {
var chars = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXTZabcdefghiklmnopqrstuvwxyz";
var randomstring = '';
for (var i=0; i<12; i++) {
var rnum = Math.floor(Math.random() * chars.length);
randomstring += chars.substring(rnum,rnum+1);
}
appId = appId + randomstring;
}
//this function set appId first before session initiate
createRandomAppId();

const session = enigma.create({
schema,
url: `wss://${engineHost}:${enginePort}/app/${appId}`,
createSocket: url => new WebSocket(url, {
rejectUnauthorized: false,
ca: [readCert('root.pem')],
key: readCert('client_key.pem'),
cert: readCert('client.pem'),
headers: {
'X-Qlik-User': `UserDirectory=${encodeURIComponent(userDirectory)}; UserId=${encodeURIComponent(userId)}`,
},
}),
});

var appname=process.argv[2];
var LOAD_SCRIPT = process.argv[3];
var appid='';
var sheetid='';
var scriptheader =`
SET ThousandSep=',';
SET DecimalSep='.';
SET MoneyThousandSep=',';
SET MoneyDecimalSep='.';
SET MoneyFormat='$#,##0.00;-$#,##0.00';
SET TimeFormat='h:mm:ss TT';
SET DateFormat='M/D/YYYY';
SET TimestampFormat='M/D/YYYY h:mm:ss[.fff] TT';
SET FirstWeekDay=6;
SET BrokenWeeks=1;
SET ReferenceDay=0;
SET FirstMonthOfYear=1;
SET CollationLocale='en-US';
SET CreateSearchIndexOnReload=1;
SET MonthNames='Jan;Feb;Mar;Apr;May;Jun;Jul;Aug;Sep;Oct;Nov;Dec';
SET LongMonthNames='January;February;March;April;May;June;July;August;September;October;November;December';
SET DayNames='Mon;Tue;Wed;Thu;Fri;Sat;Sun';
SET LongDayNames='Monday;Tuesday;Wednesday;Thursday;Friday;Saturday;Sunday';
SET NumericalAbbreviation='3:k;6:M;9:G;12:T;15:P;18:E;21:Z;24:Y;-3:m;-6:μ;-9:n;-12:p;-15:f;-18:a;-21:z;-24:y';`

var script=scriptheader+'\n\n'+process.argv[3];

var sheetId='';

session.open().then((global) => {
global.createApp(appname).then((appId) => {
setTimeout(function () {
console.error("REQUEST TIMEOUT");
session.close();
process.exit(1);
}, 90000);

appid=appId.qAppId
return global.openDoc(appId.qAppId)
})
.then(app => {
app
.setScript(script)
.then(() => app.createObject({
qInfo: {
qType: 'sheet',
},
qMetaDef: {
title: 'Podium_Sheet1' || '',
description: 'Podium Published Dataset' || '',
},
rank: -1,
thumbnail: { qStaticContentUrlDef: null },
columns: 24,
rows: 12,
cells: [],
qChildListDef: {
qData: {
title: '/title',
},
},
}))
.then((sheet) => {
sheetId=sheet.id;
//console.log('sheet id: ',sheet.id)
})
.then(() => app.doReload())
.then(() => { app.doSave()})
.then(() => {

// get the qlik ticket
const data = JSON.stringify({
UserDirectory: userDirectory,
UserId: userId
});
const options = {
hostname: `${engineHost}`,
port: `${proxyPort}`,
path: '/qps/ticket?Xrfkey=' + XRFKEY,
method: 'POST',
timeout: 8000,
headers: {
'X-Qlik-Xrfkey': XRFKEY,
'X-Qlik-User': `UserDirectory=${encodeURIComponent(userDirectory)}; UserId=${encodeURIComponent(userId)}`,
'Content-Type': 'application/json',
'Content-Length': data.length
},
key: readCert('client_key.pem'),
cert: readCert('client.pem'),
ca: [readCert('root.pem')],
rejectUnauthorized: false,
agent: false
};
const req = https.request(options, (res) => {

var body = [];
res.on('data', (chunk) => {
body.push(chunk);
})
res.on('end', () => {
body = Buffer.concat(body).toString();

if (body != "") {
var response = JSON.parse(body);
var ticketId = (response && response.Ticket || '');
if (ticketId) {
console.log(`[${appid}/sheet/${sheetId}/state/insight/theme/breeze?qlikTicket=${ticketId}]`);
} else {
console.error("Ticket not found in response.");
}
} else {
console.error("Response body was empty.");
}
session.close();
process.exit(1);
});

});
req.on('error', (error) => {
console.error(error);
session.close();
process.exit(1);
});
req.write(data);
req.end();
})

//.then(() => session.close())
//.then(() => { process.exit(1) })
.catch((exp) => {
//console.log(exp)
})

})
.catch((err) => {
console.log(err)
})
})
.catch((error) => {
console.log(error)
});

config Publish to Sense:

####################################
### QlikSense Publish Capability ###
####################################

# Setting this to true will start showing 'Publish to Qlik' option in Podium UI cart checkouts.
# Default: false
is.publish.to.qlik.enabled=true

# Name of Data connector in Qlik that will be used in generating the Qlik load script.
# Required property when 'is.publish.to.qlik.enabled' is set to true
podium.qlik.dataconnection.name=qdc <__this is the name of the data connection in sense to the QDC postgres db__>

# Quoting characters to use when creating Qlik load script - seem to vary between qlik connection types.
# Default: empty string (i.e. no quotes used)
#podium.qlik.dataconnection.sql.openquote.char=
#podium.qlik.dataconnection.sql.closequote.char=

# Command used to execute qsocks client code - externalized as in some environments might need a fully qualified path to node.
# Required property when 'is.publish.to.qlik.enabled' is set to true. Most installations will not require this value to be modified.
podium.qlik.appcreation.nodecommand=node

# Path to qsocks client script.
# Required property when 'is.publish.to.qlik.enabled' is set to true
podium.qlik.appcreation.qsockclient.script=/usr/local/qdc/qlik/createQlikApp.js

# Path to qsocks client script that is used when an existing QlikSense application is selected.
# Required property when 'is.publish.to.qlik.enabled' is set to true
podium.qlik.appUpdate.qsockclient.script=/usr/local/qdc/qlik/updateExistingApp.js

# URL to qlik sense server in following format: https://<put-qliksense-ip-or-machine-name-here>/sense/app/<podium-gen-app-id> .
# Required property when 'is.publish.to.qlik.enabled' is set to true
qlik.sense.url=https://<__URL TO SENSE SERVER HERE (ie sense.domain.__>/sense/app/<podium-gen-app-id>

# Engine Port to qlik sense server
# Required property when 'is.publish.to.qlik.enabled' is set to true
qlik.sense.enginePort=4747

# ProxyPort to qlik sense server
# Required property when 'is.publish.to.qlik.enabled' is set to true
qlik.sense.proxyPort=4243

# Publish to QS requires Active directory for authentication purposes. The following property
# stores the value ofthe user directory
qlik.sense.active.directory.name='<__DIRECTORY_NAME_IS_HERE__'

# User name to use for qlik application generation.
# Optional property. When absent or empty, Podium logged in user name will be used to generate the script.
podium.qlik.username=

#Name of the virtual proxy for QS which is used by QDC to make the REST API calls to QS
qlik.sense.proxy.name=qdc

#Port number for Publish to Qlik Sense Advanced feature
#This is used to develop the REST URL to grab the list of avaialble QS apps
qlik.sense.port=0

#QS JWT that is used to authenticate. Encrypted JWT can be used.
#example: qlik.sense.jwt=#0{ptErmgzvXnDZpq7ooq25jzRn1HqNqjHrMIvbbv3uaef2Jftdn82rap989LQbPLd27z54E8yuERvK8jXHsgHeP6KeOtDg5lwekwHZr87Oqz9je8iLvwhUSHKW4VuJIWewPZFlzSk+0FeYqtp8EAaQXkksBtR3fh19Nq/qz3T2AmBvpdkIzoZDnKF02pCfrvkogc1Qlb1pn7f9Jnd61G8TXPQ90H6eGBok02VtvlYj0IhljBff1f1w8VZr3lTVtdiBJr3YNNbA/SYRArywz/3peasvwa1rT8ytGp6pP6YnhEtASO6iAN9gJhjV7kq1QxiWlmwxuSY9YwiYjw/S8knqXeONoPHI9/OCu8DyB6+4WzM6EG6si4moLpQQAmqdavm1NfW6mlHVr+knKmtxK5kAD4XwVfbDIGY3FX6qnwrv0akVxIM31wZXS7Uuscc1gK5Uut5HB4vuftvcUe5Zkytt9aw73Xi+ulPpz/iBicwfOy2OMRYbbOsPMyH8hQedag1VDwGzcb/cUjrSIXfsy6iHdTqM4D3i5TSBuhmMs1V6+dt/G/V/4yr3tm4ibwavHYQ1V+9WQnebv8CwJcGGeU/lUg==}
qlik.sense.jwt=<__TOKEN_IS_HERE__>

# Qlik Sense target required for Publish to Qlik and Publish to Qlik Advance
qlik.sense.target.name=QLIK_TARGET

# Qlik Sense threshold required for Purging of Publish to Qlik and Publish to Qlik Advance
# Default is set to 24 hours
# Delete all P2Q data before 24 hours
qlik.logs.purging.threshold=24

Re: Publish to QS Issues

ThiebaudS — Thu, 20 Feb 2020 12:56:52 GMT

The only things that are differents on my side are in "Publish to Sense" config.

I have the line below that I don't see in your conf, and I think it's mandatory:

podium.qlik.appList.qsockclient.script=/usr/local/podium/qlik/applist.js

I don't have any line with:

qlik.sense.target.name=QLIK_TARGET

And I'm wondering why you have:

qlik.sense.port=0

as it should probably be 443 ?

Re: Publish to QS Issues

daniel_dalnekoff — Mon, 24 Feb 2020 11:13:50 GMT

Thank you. Ultimately we narrowed this down to a port issue - the QDC environment couldn't connect to the QS environment on 4747. I greatly appreciate the support in troubleshooting!

Re: Publish to QS Issues

CKM_35 — Mon, 25 May 2020 05:45:51 GMT

Hi @daniel_dalnekoff , @ThiebaudS

Hope you are doing good!

Reaching out to you as i am facing issue while performing integration between QDC (Qlik Data Catalyst)and Qlik Sense, I am following the steps mentioned in the attached document but i am not able to taste success, can you able to help me, Please let me know if you need any configuration files.

I have performed below steps:

Created a successful connection in Qlik Sense with PostgreSQL
Configured core_env.properties file on Qlik Data Catalyst machine as per instruction

Thanks in advance for your help and cooperation.

Thanks & Regards,

Chandan Mishra