Dataset comparison & clustering

shaheermecci — Thu, 08 Sep 2016 18:50:04 GMT

Hello,

Before I try a different approach with data comparison, thought I would post to the community since I will be using Qlikview to visualize the result.

I have many datasets with varying number of elements. All data is in a single table. Values within a dataset are unique.

I want to compare Values in each dataset and provide a degree of similarity. I have about 6000 such datasets with 50-200 unique Values. A hypothetical scenario below in the table:

1) How can a user choose 'Set1' to see closest dataset? in the example Set2 is closest @50%, and then Set3 @17% Match

- Should I build something in the Load script? I don't have other tools to prep the data outside

- Could using Intersect (join by Values field) to compare Default State to an Alternate State be a possible approach? I am not sure if this is possible (list dataset where Values from $ State = Values from Alt State in descending order).

2) If possible I would like to visually cluster similar datasets but that is optional

- I may need a 'similarity index' for each dataset. If anyone has done something similar I would love to know how.

Thanks!

Shaheer

Data set	Values		Data set	Values		Data set	Values
Set1	1		Set2	2		Set3	6
Set1	2		Set2	3		Set3	7
Set1	3		Set2	6		Set3	8
Set1	4		Set2	7		Set3	9
Set1	5		Set2	8		Set3	10
Set1	6					Set3	11
						Set3	12

Compare	1 & 2	2 & 1		2 & 3	3 & 2		1 & 3	3 & 1
	2	2		6	6		6	6
	3	3		7	7
	6	6		8	8
Match!	50%	60%		60%	43%		17%	14%

Re: Dataset comparison & clustering

MarcoWedel — Sun, 25 Sep 2016 21:55:09 GMT

Hi,

maybe helpful:

Community Detection

regards

Marco

topic Dataset comparison & clustering in QlikView

Dataset comparison & clustering

Re: Dataset comparison & clustering