AppEvaluator RunEvaluator.ps1 not running multiple apps

[garystrader]

I found this powershell script which does everything we want to do for an initial test of a system.  No documentation, but that's fine, I can read powershell.  One of the parameters is -apps.  The script supports passing in multiple app IDs, which should then loop through each one, but it doesn't.  Using VS Code in debugging mode I found that multiple apps are getting parsed, loaded into array $apps, and then looped through and JSON created with multiple restart and run sections.  Then it gets passed in TestScheduler.exe, but then disappears into a black hole, only the first app in the list is run.

We can call this utility once for each app ID, but it would be nice if this worked.  I suspect the issue is in TestScheduler.exe.

[garystrader]

I found the documentation, which is nice, but it still doesn't work with multiple apps.

[Levi_Turner]

What's the syntax that you're using? I am not seeing issues with this:

PS C:\scalabilitytools-5.4.0\AppEvaluator> .\RunEvaluator.ps1 -apps 325c3daa-1aeb-434c-9c1d-c7ce14ee201a,e5c43d5c-9cd6-42c7-ae89-f755f24e64bd -server servername

Directory: C:\scalabilitytools-5.4.0\AppEvaluator

Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 11/1/2019 3:22 PM Temp
2019-11-01 15:22:42,826 146 1 INFO TestScheduler.Program Running test schedule at Temp\RunEvaluator.json
2019-11-01 15:22:42,858 178 1 INFO TestScheduler.Program Notification pipe token is 3822fcb2-baa2-41f5-8296-bbeab10ba016
2019-11-01 15:22:42,862 182 1 INFO TestScheduler.Program Command pipe token is e7bd0998-2986-4824-b283-a816ad5059ec
2019-11-01 15:22:42,873 193 1 INFO TestScheduler.Program Create subfolder is set to False
2019-11-01 15:22:48,563 5883 9 INFO TestScheduler.Tasks.RestartQSSTask Restart Qlik Sense Services task started
2019-11-01 15:22:48,652 5972 9 INFO TestScheduler.Tasks.RestartQSSTask Stopping service QlikSenseEngineService
2019-11-01 15:22:51,709 9029 9 INFO TestScheduler.Tasks.RestartQSSTask Service QlikSenseEngineService stopped
2019-11-01 15:22:56,721 14041 16 INFO TestScheduler.Tasks.RestartQSSTask Starting service QlikSenseEngineService
2019-11-01 15:22:57,302 14622 16 INFO TestScheduler.Tasks.RestartQSSTask Service QlikSenseEngineService started
2019-11-01 15:24:57,334 134654 18 INFO TestScheduler.Tasks.SDKExerciserTask Invoking SDKExerciserConsole: C:\scalabilitytools-5.4.0\SDKExerciser\Jun2018\SDKExerciserConsole.exe config="C:\scalabilitytools-5.4.0\AppEvaluator\Scripts\ComplexityIndexWorker.json" configname="2-Run profiler-ComplexityIndexWorker" server=servername app="325c3daa-1aeb-434c-9c1d-c7ce14ee201a" ssl=True iterations=2 usernameprefix=Exerciser concurrentusers=1 rampupdelay=1.00 executiontime=-1 instancenumber=1 appmode=Open newuserforeachiteration=True afteriterationwait=NoWait afteriterationwaittime=0 logDir="C:\scalabilitytools-5.4.0\AppEvaluator\Results"
2019-11-01 15:24:59,207 136527 22 INFO TestScheduler.Tasks.SDKExerciserTask Starting test towards Uri: wss://servername/
2019-11-01 15:24:59,267 136587 11 INFO TestScheduler.Tasks.SDKExerciserTask 15:24:59 Workers:0 Errors:0 Warnings:0 Requests:0 Sessions:0
2019-11-01 15:25:09,253 146573 24 INFO TestScheduler.Tasks.SDKExerciserTask 15:25:09 Workers:1 Errors:0 Warnings:0 Requests:25 Sessions:1
2019-11-01 15:25:19,253 156573 21 INFO TestScheduler.Tasks.SDKExerciserTask 15:25:19 Workers:1 Errors:0 Warnings:0 Requests:37 Sessions:1
2019-11-01 15:25:29,255 166575 27 INFO TestScheduler.Tasks.SDKExerciserTask 15:25:29 Workers:1 Errors:0 Warnings:0 Requests:65 Sessions:2
2019-11-01 15:25:39,256 176576 24 INFO TestScheduler.Tasks.SDKExerciserTask 15:25:39 Workers:1 Errors:0 Warnings:0 Requests:77 Sessions:2
2019-11-01 15:25:41,249 178569 24 INFO TestScheduler.Tasks.SDKExerciserTask Test Execution Finished.
2019-11-01 15:25:41,253 178573 14 INFO TestScheduler.Tasks.SDKExerciserTask WarningCount: 0
2019-11-01 15:25:41,256 178576 14 INFO TestScheduler.Tasks.SDKExerciserTask ErrorCount: 0
2019-11-01 15:25:41,260 178580 14 INFO TestScheduler.Tasks.SDKExerciserTask Finishing writing logs...
2019-11-01 15:27:41,550 298870 35 INFO TestScheduler.Tasks.RestartQSSTask Restart Qlik Sense Services task started
2019-11-01 15:27:41,575 298895 35 INFO TestScheduler.Tasks.RestartQSSTask Stopping service QlikSenseEngineService
2019-11-01 15:27:44,123 301443 35 INFO TestScheduler.Tasks.RestartQSSTask Service QlikSenseEngineService stopped
2019-11-01 15:27:49,128 306449 34 INFO TestScheduler.Tasks.RestartQSSTask Starting service QlikSenseEngineService
2019-11-01 15:27:49,603 306923 34 INFO TestScheduler.Tasks.RestartQSSTask Service QlikSenseEngineService started
2019-11-01 15:29:49,608 426928 31 INFO TestScheduler.Tasks.SDKExerciserTask Invoking SDKExerciserConsole: C:\scalabilitytools-5.4.0\SDKExerciser\Jun2018\SDKExerciserConsole.exe config="C:\scalabilitytools-5.4.0\AppEvaluator\Scripts\ComplexityIndexWorker.json" configname="4-Run profiler-ComplexityIndexWorker" server=servername app="e5c43d5c-9cd6-42c7-ae89-f755f24e64bd" ssl=True iterations=2 usernameprefix=Exerciser concurrentusers=1 rampupdelay=1.00 executiontime=-1 instancenumber=1 appmode=Open newuserforeachiteration=True afteriterationwait=NoWait afteriterationwaittime=0 logDir="C:\scalabilitytools-5.4.0\AppEvaluator\Results"
2019-11-01 15:29:51,174 428494 39 INFO TestScheduler.Tasks.SDKExerciserTask Starting test towards Uri: wss://servername/
2019-11-01 15:29:51,249 428569 13 INFO TestScheduler.Tasks.SDKExerciserTask 15:29:51 Workers:0 Errors:0 Warnings:0 Requests:0 Sessions:0
2019-11-01 15:29:59,584 436904 42 INFO TestScheduler.Tasks.SDKExerciserTask 15:29:59 1-1 No datapages found in layout for object XkJpLL of type GenericObjectLay
out
...

I am seeing two invocations of SDKExerciserConsole with the two app IDs passed to the PS1 file.

[garystrader]

Well, I see that it's working for you.  I am doing the following:

.\RunEvaluator.ps1 -apps 25c39fdd-69c9-4d12-bd9f-50bec6f2cb67,6adfc539-8ce0-4b0d-b52f-99339ec7e7e4 -server "bgl-gs-w540" -sdkExerciser "Jun2019"

I tried it both with and without double quotes around the app id list.  I also ran it without -apps argument, it prompted me and I entered the two app ids, but it still only ran one.  I debugged in VS Code, I watched it assign the app array, but it only ran through the first one as I watched it debugging.

Weird.  Not a big deal because I will just call it separately for each id in a batch file or something.

[Levi_Turner]

Version deltas perhaps? What version of the Scalability Tools are you running?

[garystrader]

Possibly.  I tried this both locally on my laptop and remote desktop into a customer server.  I tried ST 2017 (5.2.0) against a Nov 2017 server, and the latest version 5.5.1 against a Sep 2019 server.  I can't get any combination to work.  So strange.  Again, not a big deal, but I appreciate you following up.