Cluster and Apriori using associationrule minning in Python

Interesting pattern discovery is an important topic in data mining research. Many different definitions have been proposed to describe whether a pattern is interesting. Among these many definitions, unexpectedness has shown to be a highly promising measure. Mining unexpected patterns allows one to identify a failing in prior knowledge and may suggest an aspect of the data that deserves further investigation. Unexpected patterns are typically mined using belief-driven methods, but these require an established belief system. Prior studies have manually built their own partial belief systems to apply their method, but these remain laborious to create. In this study, we propose a novel approach that is able to automatically detect beliefs from data, which can in turn be used to reveal unexpected patterns. Central to this approach is a clustering-based method in which clusters represent beliefs and outliers are potential unexpected patterns. We also propose a pattern representation that captures the semantic relation between patterns rather than the lexical difference. An experimental evaluation on different datasets and a comparison to some other methods demonstrate the effectiveness of the proposed method, as well as the relevance of the discovered patterns.