As organizations tend to store and analyze more and more data, the need to design and implement data mining algorithms that are incremental and can deal with continuous data streams is pressing. In this talk we present a new clustering algorithm that meets that need. The algorithm clusters new points by measuring the impact these points have on the entropy of the existing clusters. We show that minimizing the entropy is related to the well-known principle of Minimum Description Length. Although the algorithm cannot guarantee finding the minimum entropy (as the problem is NP-Complete), we show via experimentation that it finds good clusterings under several clustering evaluation measures. We will show a novel method of detecting categorical outliers, using the algorithm along with statistical significant testing. We will also show how these techniques are well-suited to track the evolution of clusters in data streams, due both to the concise representations of the clustering models they produce, and incremental nature of the algorithm.