Conventional clustering algorithms are restricted for use with data containing ratio or interval scale variables, so distances are used. As social studies mostly require categorical data, the literature is enriched with more complicated clustering techniques and algorithms of categorical data. The techniques are based on similarity or dissimilarity matrices, while the algorithms use density- or pattern-based approaches. This paper proposes a probabilistic nature to similarity structure. The entropy dissimilarity measure has comparable results to simple matching dissimilarity at hierarchical clustering. It overcomes dimension increase through binarization of the categorical data. It also appears that this technique could be applied to a general clustering technique.