PyTools - Unable to start service

[MarkJM]

We are trying to setup the Qlik-Py-Tools connector. To create an easy maintainable solution we use Miniconda and the setup instructions on the Github page.

With the installation manual below we have setup multiple virtual machines (local and Azure). But after a successful (no error) installation on our client's server we are unable to start the service.

Environment:

• OS: Windows Server 2016
• Qlik Sense Enterprise 13.82 (June 2020)
• SSE Version v.8.1

Installation manual
We installed multiple servers successfully with the manual below:

1. Install Miniconda (www.conda.io)
   1a. Install for all users
   1b. Install miniconda to the %PATH%
2. Open Anaconda Prompt and execute the following commando's in here.
3. Downgrade python to 3.6.8
   Command: conda install python=3.6.8
4. Update conda
   Command: conda update conda
5. Install C++ compiler (Python: mingw-w64) as describe in the PyStan documentation
   Command: conda install libpython m2w64-toolchain -c msys2
6. Build the script
   Command: Qlik-Py-Init.bat
7. Add PyTools to the Analytic Connections in Qlik Sense
8. Start the SSE Plugin
   Command: Qlik-Py-Start.bat

Error
After I execute Qlik-Py-Start.bat I simple get the error:
"Press any key to continue" and error message in the windows log (see attachments)

Question
What causes this issue and how to solve it.

[sheenkim1]

Still there are many customers requiring On Promise environment to run ML in Korea.

We're testing and provide services Qlik PyTools with Qlik Sense. As above symptom is same with Window Server 2016.

I found there is some bug issues when loading some added functions of PyTool Library in __main__.py. So I would share this info to you guys. 

 3 functionalities are as as follow. Pls check whether you can fix some codes in some where in python scripts.

# Import libraries for added functions
import numpy as np
import pandas as pd
from pandas.api.types import is_string_dtype
from pandas.api.types import is_numeric_dtype
import _utils as utils
from _prophet import ProphetForQlik

from _clustering import HDBSCANForQlik

""" Below three functions are  "not loading error" generated , so server does not respond. If excluded with references server would respond and load other functions accordingly in qlik sense.

""""

# from _sklearn import SKLearnForQlik
# from _spacy import SpaCyForQlik
# from _common import CommonFunction

[Sheen]

Finally I found root cause of above issues. By checking Python installation on window server related with sklearn and common features of PyTools. Even the python installation is well done, there is errors in imports of keras and tensorflow. There should be resolution to solve conflictsbetween Python and Window Server 2016 NOT PyTools. If Python works well then SSE will load without problem. I would upload the method and procedure to solve Python issue on Window Server 2016.

[sheenkim1]

We might have solution temporariliy. It might be related with AVX compatability of HW. In My Case Window Server 2016 and CPU X3470 (very very OLD CPU). They says it has compatability in AVX but it seems NOT. So installed with different weel of tensorflow it seems to run well and until now no problem. 

Below is procedure :

Visit https://github.com/fo40225/tensorflow-windows-wheel/blob/master/README.md 

Go to fo40225/tensorflow-windows-wheel and Download  1.12.0/py36/CPU/sse2/tensorflow-1.12.0-cp36-cp36m-win_amd64.whl and Install in PyTools virtual environment, which is AVX free version.

Test it if no problem in Python : just put import tensorflow and check console response as below :

C:\Users\qlik\AppData\Local\Programs\Python36\lib\site-packages\tensorflow\python\framework\dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
C:\Users\qlik\AppData\Local\Programs\Python36\lib\site-packages\tensorflow\python\framework\dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
C:\Users\qlik\AppData\Local\Programs\Python36\lib\site-packages\tensorflow\python\framework\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
C:\Users\qlik\AppData\Local\Programs\Python36\lib\site-packages\tensorflow\python\framework\dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
C:\Users\qlik\AppData\Local\Programs\Python36\lib\site-packages\tensorflow\python\framework\dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
C:\Users\qlik\AppData\Local\Programs\Python36\lib\site-packages\tensorflow\python\framework\dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])

Then it means everything is OK.

Next Step , yout shoud change some code in _sklearn.py and _utils.py as below ( Because tensorflow 1.12.0 has different logging syntax )

import tensorflow as tf
# tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)
# For version 1.12.0 below method
tf.logging.set_verbosity(tf.logging.ERROR)

No all is done. 

Go to Qlik-Py-Start.py run as administrator.

( The console can generate several rows of warning messages, but don't mind and just pass it, PyTools runs well. )