topic Re: help with running Qlik Replicate in docker in Qlik Replicate

help with running Qlik Replicate in docker

sgorey — Mon, 18 Apr 2022 16:10:49 GMT

starting replication task hangs.

I'm using areplicate-2021.11.0-266.x86_64.rpm.

The source endpoint is an Oracle 19 database and the target endpoint is Kafka. I'm fairly confident the endpoints are configured properly because we have a VM Qlik Replicate install working as desired.

When running in docker, I can verify the source and target endpoints. I can create the task, which involves selecting schema and tables from the oracle database.

The issue occurs when starting the replicate task.

the dialog box, "starting main loop ..." eventually times out.

Re: help with running Qlik Replicate in docker

KellyHobson — Mon, 18 Apr 2022 16:52:16 GMT

Do you have ports exposed in your docker run command?

Thanks,

Kelly

Re: help with running Qlik Replicate in docker

sgorey — Mon, 18 Apr 2022 17:08:03 GMT

yes, this is my run command

./run_docker.sh 3552 qlik Abcdef1234567890

it starts up fine. I'm able to navigate to https://localhost:3552/attunityreplicate/2021.11.0.266

logs are quite large as I've enabled verbose output.

I did deviate from the docker file a bit when it came to adding the Oracle instant client. I don't have the oracle zip it referenced. Instead, I've downloaded the instantclient rpms and install them that way in the Dockerfile (attached).

Thanks,

Steve

Re: help with running Qlik Replicate in docker

KellyHobson — Mon, 18 Apr 2022 19:39:14 GMT

Hey @sgorey ,

I wonder if deviating from the Oracle client steps may be causing the source connection to hang.

From create-dockerfile.sh

check_necessary_file_exists "oracleclient.rsp" "oracle$version" || return $?
filename=$1
version=$2
cat >> $dockerfile << EOF
ADD oracleclient.rsp /oracleclient.rsp
RUN yum install -y libaio
ADD $filename /
RUN unzip $filename
RUN usermod -G attunity attunity
RUN mkdir /opt/oracle && chown -R attunity:attunity /opt/oracle
USER attunity
RUN /client/runInstaller -silent -ignorePrereq -waitforcompletion -responseFile /oracleclient.rsp
USER root
RUN /opt/oracle/oraInventory/orainstRoot.sh
ENV ORACLE_HOME=/opt/oracle/$version/client
ENV LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:/opt/oracle/$version/client
RUN rm -rf $filename client oracleclient.rsp
EOF

Attunity user specifically does some steps which may set it up differently than applying the instant client .rpm.

Can you attach a log with SOURCE_CAPTURE and SOURCE_UNLOAD to trace when you try to run the task?

Thanks,

Kelly

Re: help with running Qlik Replicate in docker

sgorey — Mon, 18 Apr 2022 19:49:37 GMT

Thanks,
I was missing
RUN yum install -y libaio

I'll add that and report back.

Re: help with running Qlik Replicate in docker

sgorey — Mon, 18 Apr 2022 20:29:22 GMT

Log attached.

I see:

00000394: 2022-04-18T20:19:47 [SOURCE_CAPTURE ]T: Going to execute the statement 'drop directory attrep_dir_test' (without select) (oracle_endpoint_utils.c:3103)
00000394: 2022-04-18T20:19:47 [SOURCE_CAPTURE ]T: ORA-01031: insufficient privileges [1022317] (oracle_endpoint_utils.c:3113)

odd because I don't see this error when I run a simliar task on a Qlik Replicate server on a VM.

Re: help with running Qlik Replicate in docker

KellyHobson — Tue, 19 Apr 2022 13:43:45 GMT

Hey @sgorey ,

Thanks for sharing the logs. Can you confirm if the user specified in the Replicate Oracle source endpoint connection on docker instance is the same as your VM?

Thanks, Kelly

Re: help with running Qlik Replicate in docker

sgorey — Tue, 19 Apr 2022 15:09:17 GMT

Yes, the Oracle endpoints are the same.

I'm taking your suggestion of installing the oracle instant client via the
oracle zip versus just installing the instant client via rpms. I will
report back the status.

Thanks,
Steve

Re: help with running Qlik Replicate in docker

sgorey — Wed, 20 Apr 2022 20:39:36 GMT

hi Kelly,

It took some doing, but I was able to install the oracle client per the directions with an oracle response file.

However, tasks still do not start successfully, even though the source and target endpoints validate successfully.

I increased the logging to trace for all categories and compared with the working VM Qlik install.

There appears to be a timeout running the following sql statement. This error doesn't occur when running Qlik on a VM.

Going to prepare the statement 'select distinct(tzname), to_timestamp_tz('2011-12-21 12:00:00.0 ' || tzname, 'YYYY-MM-DD HH24:MI:SS.FF9 TZR'), to_timestamp_tz('2011-06-21 12:00:00.0 ' || tzname, 'YYYY-MM-DD HH24:MI:SS.FF9 TZR') from V$TIMEZONE_NAMES' (oracle_endpoint_capture.c:182)

more messages for the STREAM_COMPONEN in the attached log file.

Re: help with running Qlik Replicate in docker

KellyHobson — Thu, 21 Apr 2022 00:02:16 GMT

Hey @sgorey ,

Looks like could be from a timeout issue.

On your docker container running the Replicate Server/instance, can you do the following:

In the sqlnet.ora file on the Replicate Server in the Oracle client, set:

BREAK_POLL_SKIP=10000

DISABLE_OOB=on

From the log, does it seem to start the task successfully? or is [STREAM_COMPONEN ] the only output you are seeing in the logs?

Thanks,

Kelly

Re: help with running Qlik Replicate in docker

sgorey — Thu, 21 Apr 2022 13:41:29 GMT

Kelly,

your instruction to add the following lines to sqlnet.ora solved my issue.

BREAK_POLL_SKIP=10000

DISABLE_OOB=on

Thank you for your support!

Steve

Re: help with running Qlik Replicate in docker

KellyHobson — Thu, 21 Apr 2022 14:35:57 GMT

Hey @sgorey ,

Awesome! Thank you for letting us know.

Best,

Kelly

Re: help with running Qlik Replicate in docker

sgorey — Thu, 21 Apr 2022 14:52:06 GMT

for those interested, I also built a second image using the Oracle InstantClient rpms and it works as well.

I am going to test with this version as it's much small and faster to build.

Thanks again,

Steve