2 Replies Latest reply: Jul 23, 2018 3:43 AM by Andreas Kamber RSS

    Suddently unstable SAP connector - General advices?

    Andreas Kamber

      Hi,

       

      We're loading now since 2 years data out from SAP which worked quite well (maybe every 3th or 4th month a fail). But suddenly everything changed from one day to another. What happened? No idea! I guess you guys are our last chance before I just destroy the whole damn thing.

       

      SAP Team: "No change on our side"

      SAP Basis Team: "No change on our side"

      Network Team: "No change on our side"

      IT Team: "No change on our side"

      Our team: No change (that's the only thing I truly know )

       

      Our setup is the following:

       

      SQL Loads:
      A couple of tables (smaller & "bigger" like 2 millions), reading * and saving it in a qvd. We're using Generators for this, each of them will take table by table. We had 3 in parallel in the past, tried also to do only one a time.

       

      Extractor loads:

      Bigger tables (like JFRK, JKAP etc.). Also done by Generators, extractor by extractor. 3 in parallel (tried also with 1).

      Logic:

      - SQL connection for the status table, identify if the job still runs (from a previous failed load)

      - BAPI to abort the job

      - Extractor load (some with delta, some with *)

       

      Cause we were helpless we did a complete upgrade of all components:
      - Qlik V12 Nov. SR3

      - SAP Connector 6.6 (also updated the transports)

       

      But the jobs still failing.

       

      What happens?

      Sometimes it's doing smoothly load by load (however we didn't saw this since weeks ). Now most of the times it's starting to load some tables and suddently it hangs/freeze. And the timeout will appear. We also tried to work with retrys in the QMC, sometimes we're getting the data in the second load (therefore I assume the SAP service is still doing his job). Same with the SQL loads: It works, timeout, works again. SAP Service is running without any crash.

       

       

      Logs of some fails:

       

          2018-03-06 20:24:38 0137 SQL Select *  from JFPA

          2018-03-06 20:26:20      11 fields found: MANDT, VBELN, POSNR, PARVW, KUNNR, ADRNR, LAND1, ADRDA, XCPDK, ISPADRNR, ISPADRVAR,

          2018-03-06 20:26:20      Error: QVX_UNEXPECTED_END_OF_DATA:

          2018-03-06 20:26:20      LOCATION    CPIC (TCP/IP) on local host with Unicode

          2018-03-06 20:26:20      ERROR       connection to partner '[server]:3300' broken

          2018-03-06 20:26:20      TIME        Tue Mar 06 20:26:20 2018

          2018-03-06 20:26:20      RELEASE     721

          2018-03-06 20:26:20      COMPONENT   NI (network interface)

          2018-03-06 20:26:20      VERSION     40

          2018-03-06 20:26:20      RC          -6

          2018-03-06 20:26:20      MODULE      nixxi.cpp

          2018-03-06 20:26:20      LINE        4599

          2018-03-06 20:26:20      DETAIL      NiIWrite: P=145.234.71.231:3300; L=0.0.0.0:52418

          2018-03-06 20:26:20      SYSTEM CALL WSASend

          2018-03-06 20:26:20      ERRNO       10053

          2018-03-06 20:26:20      ERRNO TEXT  WSAECONNABORTED: Software caused connection abort

          2018-03-06 20:26:20      COUNTER     5

       

       

          2018-03-06 21:15:07 0225 [ZDS_JFRK_temp]:

          2018-03-06 21:15:07 0226

          2018-03-06 21:15:07 0227 SQL EXTRACTOR ZDS_JFRK

          2018-03-06 21:15:07 0228 TFRMETHOD T  

          2018-03-06 21:15:07 0229

          2018-03-06 21:15:07 0230 UPDMODE F   

          2018-03-06 21:15:07 0231 EXTRLANGUAGE E

          2018-03-06 21:15:07 0232 LOGSYS QTQVCEXTR1

          2018-03-06 22:20:32      98 fields found: AEDAT,[]

          2018-03-06 22:20:32      Error: QVX_UNEXPECTED_END_OF_DATA: Timeout while waiting for SAP data

          2018-03-06 22:20:32      Execution Failed

          2018-03-06 22:20:32      Execution finished.

       

       

       

      >> Info entry 2018-03-07 04:00:27.167 [Thread 1]

      Trace directory changed from .\ to C:\ProgramData\QlikTech\Custom Data\QvSAPSqlConnector\Log\

       

       

      **** Log file opened by process 8928 at 2018-03-07 04:00:27 (UTC+01:00 W. Europe Standard Time)

       

       

      SAP .NET Connector 3.0 with file version 3.0.18.0 running on 64-bit .NET Framework 4.0.30319.42000

      Program: C:\Program Files\Common Files\QlikTech\Custom Data\QvSAPSqlConnector\QvSAPSqlConnector.exe

      Working directory: C:\Windows\system32

      Operating system: Windows Server 2012 Standard Server 64-bit

      Processor: 14x AMD64 (or x64)

      SAP release: 721, Kernel release: 721, Kernel patch level: 713

      Hostname: [host], IP address: 145.234.72.76, IP_v6 address:

       

       

      Default trace level: None

       

       

      >> Info entry 2018-03-07 04:00:27.167 [Thread 1]

      Trace directory changed from .\ to C:\ProgramData\QlikTech\Custom Data\QvSAPSqlConnector\Log\

      >> Error entry 2018-03-07 04:00:28.761 [Thread 5]

      [SERVERTYPE=0 ASHOST=[server] CLIENT=100 SYSNR=00 USER=USR4QV PASSWD=******** SNC_MODE=0 SNC_QOP=9 ADVANCEDPROPERTIES=0 CONVROUTINE=0 CHECKSEPARATOR=0 NULLDATE=1 REMOVEALLBLANKS=0 REPLACENULLVALUE=1 TRACE=0 MIXEDMODE=0 NULLDATELEGACY=0 FLTPLEGACY=0 TIMSLEGACY=0 DATATYPELEGACYMODE=0 PACKETSIZE=20000 FETCHBUFFERS=99999999 BUFFERPERCENTAGE=10 TIMEOUTBATCH=600 TIMEOUTFETCH=1200 TIMEOUTSTARTBATCH=2400 BATCHJOBNAME=/QTQVC/READ_DATA NAME=DefaultDestination]

      SAP.Middleware.Connector.RfcAbapException: TIMEOUT_READ_DATA

         at SAP.Middleware.Connector.RfcConnection.ThrowRfcErrorMsg()

         at SAP.Middleware.Connector.RfcConnection.RfcReceive(RfcFunction function)

         at SAP.Middleware.Connector.RfcFunction.RfcDeserialize(RfcConnection conn, IRfcIOStream stream)

         at SAP.Middleware.Connector.RfcFunction.RfcCallReceive(RfcConnection conn, IRfcIOStream stream, RFCID rid)

         at SAP.Middleware.Connector.RfcFunction.Invoke(RfcDestination destination)

       

                 

        

        

       

           2018-03-07 02:00:20 0178 CUSTOM CONNECT TO ***

          2018-03-07 02:00:20 0181 JVTLFZUO:

          2018-03-07 02:00:20 0182 LOAD

          2018-03-07 02:00:20 0183 *

          2018-03-07 02:00:20 0184 SQL EXTRACTOR ZDS_JVTLFZUO

          2018-03-07 02:00:20 0185 TFRMETHOD T  

          2018-03-07 02:00:20 0186

          2018-03-07 02:00:20 0187

          2018-03-07 02:00:20 0188 UPDMODE F  

          2018-03-07 02:00:20 0189

          2018-03-07 02:00:20 0190

          2018-03-07 02:00:20 0191 EXTRLANGUAGE F

          2018-03-07 02:00:20 0192 LOGSYS QTQVCEXTR1

          2018-03-07 02:00:20 0193 WHERE

          2018-03-07 02:00:20 0194 ERSCHDAT I EQ '20180126'

          2018-03-07 04:00:30      17 fields found: ERSCHDAT, []

          2018-03-07 04:00:30      Error: QVX_UNEXPECTED_END_OF_DATA: Timeout while waiting for SAP data

          2018-03-07 04:00:30      Execution Failed

          2018-03-07 04:00:31      Execution finished.

       

       

      2018-03-07 02:00:20     Connector: C:\Program Files\Common Files\QlikTech\Custom Data\QvSAPConnector\QvSAPEXTRConnector.dll Ver: 6.6.0+Build:00257.origin/master

      2018-03-07 02:00:20     Using Server Host: [server] , Port: 8680

      2018-03-07 02:00:20     Socket timeout parameter (s): 900

      2018-03-07 02:00:20     Socket created OK

      2018-03-07 02:00:20     DNS: getaddrinfo

      2018-03-07 02:00:20     Connected to SAP (ASHOST): [server]

      2018-03-07 02:00:21     Making new SAP Request

      2018-03-07 02:00:21     EXTRACTOR = ZDS_JVTLFZUO

      2018-03-07 02:00:21     TFRMETHOD = T

      2018-03-07 02:00:21     UPDMODE = F

      2018-03-07 02:00:21     EXTRLANGUAGE = F

      2018-03-07 02:00:21     LOGSYS = QTQVCEXTR1

      2018-03-07 02:00:21     EXTRACTOR ZDS_JVTLFZUO#TFRMETHOD T#UPDMODE F#EXTRLANGUAGE F#LOGSYS QTQVCEXTR1#WHERE#ERSCHDAT I EQ 20180126#

      2018-03-07 02:00:21     Connection properties: ASHOST=[host], SYSNR=[00], CLIENT=[100], SRVHOST=[srv], SRVPORT=[8680], USERIP=[], XUSERID=[], XPASSWORD=[]

      2018-03-07 02:00:21     TimeoutSAP     : 14400

      2018-03-07 02:00:21     TimeoutInit    : 900

      2018-03-07 02:00:21     TimeoutActivity: 7200

      2018-03-07 02:00:21     TimeoutData    : 3600

      2018-03-07 02:00:21     TCPBufSize     : 1024

      2018-03-07 02:00:21     Received Meta Data, No of Fields: 17

      2018-03-07 02:01:45     Information from server: INFOMSG : Started job (INITRNR): REQU_QTQVCEXTR1_20180307020021

      2018-03-07 04:00:21     JOB STATUS: TIMEOUT TRIGGER (TimeoutActivity) SENT TO SERVICE TO ABORT THE SAP JOB

      2018-03-07 04:00:26 Err Timeout while waiting for SAP data

      2018-03-07 04:00:28     Disconnected

      2018-03-07 04:00:30     Closing the Socket

      END AT:

      2018-03-07 04:00:31

      Total run time: 02h 00m 11s