According to my knowledge, Jmeter allows also functional testing...
This is why I believe it might be possible to use this tool to connect with Jmeter also for creating functional tests.
It would be great to find out this for sure if this is possible (and how).
It depends on what you are after when you mean functional testing. The Scalability Tool simulates certain actions a user can perform. It has been developed to be able to simulate several users at once but nothing is stopping you to simulate only one user performing actions (among those that are supported by the tool).
My perspective of functional testing for Qlik is a set of numerical checkpoints that the reporting should return... for instance:
1. for the Selection Year = 2015, Month = Jan, Customer=ABC => Object Id = CH01, Expression 1 (which is Total Sales, let's say) should give 100,000 EUR
2. For the selection Invoice Number = 100234, => Object Id = CH02 Expression 4 (which is Gross Margin %, let's say) should give 10%
3. ... or even better: a solution for matching a whole set of expected results in a single small set of instructions, something like
For the Selection Year = 2015, Month = Jan, Column1 of FILE.XLS(Sheet1,Range=... or even simpler FILE.CSV) = Object Id = CH02, Dimension 1 => Object Id = CH02, Expression 1 should match FILE.CSV, Column 2
(it can go even further: matching multiple dimensions and expressions of the Qlik object with multiple columns of the CSV validating witness report...)
4. Another area: validation the matching results that are inter-related in same or several QVW files, in various objects.
5. Beyond selection, objects and values in dimensions and expressions, at a certain moment also variable value controlling should be involved
In other words, it should validate a set of pre-expected correct calculations with the results of some objects and/or with the results (...with ... or without Jmeter).
This way we can conclude that any modification in the script hasn't lost or changed some relevant data that we already have treated in a correct manner. (4 could be used also for version comparison...)
PS: Perhaps a new place should be opened for this concept !?
The regression test capability part of the QV Scalability Tools can be used for some of your purposes.
The Regression Test Capability is described in the documentation for QV Scalability Tools. In brief you simulate a single user scenario for interacting with a certain application just like any load test with the difference that you in the drop down list for "script type" selects "RegressionScript". Then the tool saves the actual data (result sets) at protocol level and allows to compare results in-between different executions towards different versions of an application or versions of QlikView.
The regression test capability will detect if
- objects that gets updated for a certain action/selection differs
- data which an object is populated with differs
- the order of data provided to an object differs
- server-side rendered graphical objects differs (based on md5-checksum)
The regression test capability will not detect if
- web browser rendering differs
Your suggestion makes sense, but it will only validate passed/not passed, while not providing an easy way to understand what went wrong.
Will keep in mind and try whenever the time will be proper.
Tx for your input.
If you press the "Analyze in QlikView" button in the Regression Compare tool, you'll get a nice analysis of what the specific differences were. Much deeper than pass/nopass.