Skip to main content
Announcements
NEW: Seamless Public Data Sharing with Qlik's New Anonymous Access Capability: TELL ME MORE!
cancel
Showing results for 
Search instead for 
Did you mean: 
Kev_Thomas
Contributor III
Contributor III

Count where specific text is found in multiple fields

Hi there,

I have a spreadsheet with a mass of data and trying to avoid doing too much pre-processing in Excel before loading into Qlikview. The data is an extract from Jira and contains a number of fields, but the one's I'm interested in are:

Issue ID - which is the unique identifier of the Jira ticket
TPR - Target Production Release (an identifier that can group records together by a specific TPR)

Labels
Labels1
Labels2
Labels3 etc. upto 
Labels20

What I need to do is count the number of unique Issue IDs where the Labels(n) fields contain data that looks like 'RC*' (using a Wildcard search) and group this by the TPR (which is the dimension on the chart).

Not sure how to construct this in a way that will allow me to plot the number of records (count of Issue ID), by TPR.

Any clues, suggestions would be much appreciated.

Cheers

Kev

I've tried using OR statements in expressions, but can't seem to get a simple line chart going 😞

Labels (2)
31 Replies
Kev_Thomas
Contributor III
Contributor III
Author

@marcus_sommer  - thanks once again for your patience and input.

I do want to be able to do the majority of the pre-filtering on the load script, but I guess this is where I'm failing to get my ideas across/lack the knowledge to be able to accurately describe what I'm aiming for.

Essentially, I have 2 key fields:
TPR
Issue ID

And a number of Label fields from 0-65. I have used a Crosstable script to create a table with:

Issue ID as the key
Value as a list of all Label fields in one column with multiple values

 
Issue IDValue
11111RCiB19.6.1
11111Aaaaa
11111Bbbbb
22222Aaaaa
22222Bbbbb
33333RCiB19.1
44444RCiB18c.1
44444Aaaaaa
55555RCiB19.2

 

Given the table that's been created above - I want to plot the COUNT(DISTINCT([Issue ID])) where the Value matches the TPR from the main data table.

So, given that the main data table has TPR which looks like 'iB19 - 2020' - I need to understand how to select only those distinct rows on the Crosstable which are similar to the TPR.

I think I need to join on the main data and crosstable to be able to create another temp table based on:

Left(Value,6) = 'RC'&'Left(TPR,4)    >>  'RCiB19'  = 'RCiB19'

It's so frustrating because I know I'm close. but just need to get this accurate filter working to be able to power on through what I'm trying to do.

marcus_sommer

I'm not sure if I understand you right - the [Issue ID] and [TPR] are keys but you want to link/match/calculate parts from [Value] which seems only related to [Issue ID] with [TPR]? And then you want to count distinct?

Such thing worked usually only if the tables are linked properly. If it's possible in your case I couldn't say. I think I would rather trying to merge both tables by mapping per applymap() [Issue ID] and [Value] to your main data table.

- Marcus

Kev_Thomas
Contributor III
Contributor III
Author

Hehehe...I'm sorry, I'm really not managing to explain myself too well here. As far as I can tell it's a really simple thing I'm trying to do - I'm just really, really, really bad at explaining it.

If I was writing a select statement it would be something like:

SELECT DISTINCT COUNT(b.Issue_ID)
     FROM tableb
                 ,tablea
 WHERE tablea.TPR = 'iB19 - 2020'
        AND SUBSTR(tableb.Value,1,6)  = 'RC' SUBSTR(tablea.TPR,4)

So, returning every row where the value matches 'RC' + the TPR from table A....bearing in mind that I had to CrossTable the data in order to get Value instead of 65 Label fields....

marcus_sommer

It's not possible in Qlik to join tables over expressions and/or with further conditions else the join is performed on all equally named fields. This means it needs to be done a bit differently, maybe with an approach like the following:

map: mapping load 'RC' & mid(TPR, 1, 4) as Lookup, 1 as Return from TableA where TPR = 'iB19 - 2020';

crosstable: crosstable(label, value, 1) load * from TableB;

match: load *, applymap('map', value, 0) as Counter resident crosstable where value like 'RC*';

final: load [Issue ID], sum(Counter) as Counter resident match group by [Issue ID];


whereby your example looked a bit compartmentalized and if there are multiple similar ones needed it might be better just to match and flag those ones and aggregating them later within the UI.

- Marcus

Kev_Thomas
Contributor III
Contributor III
Author

Hi @marcus_sommer ,

Thanks once again for your input.

I can kind of see what you're aiming for with your example - however, my issue is that I will have multiple Target Production Releases (the iB19 - 2020, being one of them) and so was hoping not to "hard code" that as you have in your "map:" statement.

It's failing on the last statement, can't find [Issue ID].

This is my slightly modified version of what you have:

map: mapping load 'RC' & mid(TPR, 1, 4) as Lookup, 1 as Return Resident AllData;

crosstable: crosstable(Lbl, Value, 15) load * Resident AllData;

match: load *, applymap('map', Value, 0) as Counter resident crosstable where Value like 'RC*';

final: load [Issue ID], sum(Counter) as Counter resident match group by [Issue ID];  << failing at this statement

marcus_sommer

Do I understand it right that both parts come from one table? And also that the match should be performed on the record-level? If so, the whole approach might be really simpler like mentioned in my first/second answer - just keeping both parts as associated tables whereby the KEY between them would be the recno(). Means something like this:

main: load recno() as KEY, F1, F2, ... from MainTable;

crosstable: crosstable(label, value, 1) load recno() as KEY, Label-fields ... from MainTable;

dimension: load *, if(value like 'RC*', 1, 0) as Flag resident crosstable; drop tables crosstable;

and then you could use the Flag as selection or within the expression as condition. You may need multiple flags and/or you combine them in some way.

- Marcus

 

Kev_Thomas
Contributor III
Contributor III
Author

Thanks again @marcus_sommer  - we are so close with this. The suggestions you've provided has got me so much nearer my goals.

I've amended the dimensions: line to this

dimension: load *, if(Index(value,'.')=8,if(Left(value,7) = 'RC'&Left(Key,5), 1, 0),if(Left(value,6) = 'RC'&Left(Key,4), 1, 0)) as Flag resident crosstable;

which seems to be working a treat but, for some reason - some of my numbers are still out. I'm trying to determine if this is a double-counting issue - or if my original spreadsheet calculations are incorrect leading me to believe QlikView is incorrect when it's possibly showing me the right results.

we're getting there....

If you can see anything particularly flawed with the line above, please shout.

marcus_sommer

Your various matches and extracts within the if-loop may not return your expected results. Therefore I suggest to check each part separately by adding them as extra fields, like:

...
index(value,'.') as check1,
left(value,7) as check2,
...

There may some logical mistake but also that your fieldvalues are different as you think they are. They may contain some invisible chars like leading/ending spaces or even other ones. This means you need to ensure that they are like you want to query them. For it you could wrap your fields with trim() or maybe with keepchar(value, 'abc...123...') and similar funtions.

Further your if-logic may not catch all possibilities - maybe you missed here or there the else-branch and/or some condition may be never true because any earlier check were returned already a TRUE - this means the order of the checks within the if-loop needs to changed and/or some checks needs to extended to multiple checks, means something like:

if(check1 = true and check2 = true, 1, if(check1 = true, 2, 3))

and here again it might be useful to add some extra fields to see which condition returned which result.

- Marcus

Kev_Thomas
Contributor III
Contributor III
Author

Hi again @marcus_sommer ,

Trying to follow what you said, I've put this together:

rcDimension:
load *,
index(value,'.') As dotpos,
left(value,7) As vLLabel,
left(value,6) As vSLabel,
'RC'&Left(Key,5) as vLTPR,
'RC'&Left(Key,4) As vSTPR,
if(dotpos=8,if(vLLabel = vLTPR, 1, 0),if(vSLabel = vSTPR, 1, 0)) as Flag resident crosstable;

It's failing because I can't reference new columns in the IF statement by the looks of things. 😞

marcus_sommer

It wasn't meant in this way - to use the various check-parts directly within the next transforming-steps else rather as temporary standalone measures just to check logic/syntax/data-quality of the various parts.

Nevertheless this approach is quite often useful and very common and could be easily applied with a preceding load, like:

rcDimension:
load *, if(dotpos=8,if(vLLabel = vLTPR, 1, 0),if(vSLabel = vSTPR, 1, 0)) as Flag;
load *,
index(value,'.') As dotpos,
left(value,7) As vLLabel,
left(value,6) As vSLabel,
'RC'&Left(Key,5) as vLTPR,
'RC'&Left(Key,4) As vSTPR
resident crosstable;

- Marcus