RunTask API Request stopped working giving status: '500' and 'error_code': 'SERIALIZED_OBJECT_NULL'

[PabloV]

Hi,

We have been using  API RunTask without any issues for the past 2 years to start tasks, however, it recently started showing this error:
{'error_code': 'SERIALIZED_OBJECT_NULL', 'error_message': 'Serialized object is null.'}

We haven't updated the code for this request and don't know what's causing this as there is no information online about this error. StopTask API works fine as usual but not RunTask. Other APIs work normally as well.

The request we send follows this URL format:
https://{host}/attunityenterprisemanager/api/v1/servers/{ServerName}/tasks/{TaskName}?action=run

Request:

Response:

Has anyone faced this issue and knows what's causing it? Any information will be appreciated, thanks in advance!

 Qlik Enterprise Manager 

[Steve_Nguyen]

@PabloV 

we have similar customer report the issue and below is what customer did to fix :

We made some tests, and it seems it's this line :
$AemRunTaskReq = New-Object Attunity.Aem.RestClient.Models.AemRunTaskReq

It's missing on our script and when I add it it works.
It seems this line secure the behavior of the runtask.

[tfarber]

We are having same issue.  I assume that something in our last OS patches applied last week caused the problem.  While we use Python to interface with the API we do not use the Python API instead using the REST API so the above line isn't in our code.  Are we basically needing to create a new session for the run task call?

[PabloV]

I agree. It is definitely not a coding issue, most likely it is in the server side. I receive the same error message when I test sending a http request using Python, PowerShell, and curl command. 

Curl example:

H:\>curl -i -k -X POST --header "EnterpriseManager.APISessionID: mT25kRJxnn-uwIocwYpTeA" --header "Content-Length: 0" https://Xxxxx-xxxxxx01.com/attunityenterprisemanager/api/v1/servers/Replicate^%20AWScc^%20^%20Dev^%2...

HTTP/1.1 500 Internal Server Error
Cache-Control: no-cache, no-store
Content-Length: 84
Content-Type: application/json
Server: Microsoft-HTTPAPI/2.0
Date: Mon, 12 Jun 2023 15:55:39 GMT

{"error_code":"SERIALIZED_OBJECT_NULL","error_message":"Serialized object is null."}


I don't know where that new session will need to be added per @Steve_Nguyen's comment. 
I'm currently working with someone from support on trying to identify the root cause by looking at AEM logs. I'll keep you posted if we find something or let me know if you find anything. Thanks.

[tfarber]

I have been testing today.  Creating a new session ID didn't help.  Here is AEM log file messages:

25 2023-06-12 21:07:12 [Host ] [DEBUG] request AemRunTask: /v1/servers/etcpdapp125/tasks/LRD_EMRF2PRD_LSS?action=run&option=RESUME_PROCESSING
25 2023-06-12 21:07:12 [WebLog ] [DEBUG] Dispatch REST request AemRunTask: /v1/servers/etcpdapp125/tasks/LRD_EMRF2PRD_LSS?action=run&option=RESUME_PROCESSING
25 2023-06-12 21:07:12 [Attunity ] [DEBUG] SYS-E-SEROBJNUL, Serialized object is null.
25 2023-06-12 21:07:12 [WebLog ] [DEBUG] REST AemRunTask response failed with status 500: SYS-E-SEROBJNUL, Serialized object is null.
25 2023-06-12 21:07:12 [Host ] [DEBUG] SYS-E-SEROBJNUL, Serialized object is null.
Attunity.Infrastructure.Globals.AttInfrastructureException: SYS-E-SEROBJNUL, Serialized object is null.
at Attunity.Infrastructure.Globals.JsonUtil.Serialize(Object inputObj, Type objectType, StreamWriter writer, SerializerSettings settings)
at Attunity.Infrastructure.Globals.JsonUtil.Serialize[T](T inputObj, SerializerSettings settings)
at Attunity.Infrastructure.HostManager.RestHandler.A(ApiMethodInfo , Stream )
at Attunity.Infrastructure.HostManager.RestHandler.A(ISession , RestHttpMethod , Stream , ReqData , D )

[tfarber]

Curious what OS your QEM server is running Pablo.  Our production QEM is Windows 2012.  I can start tasks using API from our non-production server running Windows 2016.  We are building new prod server running 2019 but it isn't ready yet.

[PabloV]

Our QEM server runs Windows 2016. It is weird that you are able to do runTask API requests on your non-production server running the same Windows but we can't. What was in your recent patch that caused the issue? 

[tfarber]

Well shoot.  I was hoping we'd find some similarity in our environments.  Being able to use the run task API in non-prod is confusing but patches applied were different from production server.  We applied the Windows security patches in mid-May then went into a change freeze for Memorial day.  We discovered our issue after the freeze ended and teams started using our scripts to bounce replication tasks.  The OS patches are the only change in our environment from when the API was working.  Do you know if your server had patches applied recently?  Below is list of patches we applied to each server.  Maybe there are similarities to your server....

NONPROD QEM SERVER
KB5026363
KB4589210
KB5012170

PROD QEM SERVER
KB5026415
KB5026366
KB4490128
KB4462901
KB4339284
KB4033428
KB4486459
KB4501226
KB4468323

[tfarber]

Hi @PabloV  have you resolved this yet?  We are still not working.  The work-arounds that Qlik have provided do not with with Python.  I am building a new QEM server without the May OS patches to see if I can get it to work.  We really need our automation...

[PabloV]

Hi @tfarber, we haven't fixed the issue yet. Qlik was able to replicate the error from their end but haven't provided a solution. They don't believe it is a OS patch but something that changed on their side in terms of the APIs. I'm also waiting for them to fix it. Something that worked for us was running "reptcl" from within the replicate server. For example run a cmd (as Administrator or it won't work):

C:\Program Files\Attunity\Replicate\bin>repctl connect; execute task="AnyTaskNameYouWantToTest" operation=3 flags=0; disconnect

More on this here:
https://community.qlik.com/t5/Official-Support-Articles/Repctl-Stop-Start-Task-using-REPCTL-command-...

I know there are other commands like "gettasklist" that you can execute, and try to combine the command above in a loop that you can try in the meantime. There is so little information on this workaround and not always works so it is not the recommended option but that's all I can think of right now as an automated solution. More here:
https://hdfiles.jitbit.com/getfile/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzM3BhdGgiOiIzMjQ0OC8yOTM5...

Let me know if you get any new information about the runTask API and if Qlik indicates solution. Thanks.