A quality of centroid-based clustering is highly dependent on initialization. In the article we propose initialization based on the probability of finding objects, which could represent individual clusters. We present results of experiments which compare the quality of clustering obtained by k-means algorithm and by selected methods for fuzzy clustering: FCM (fuzzy c-means), PCA (possibilistic clustering algorithm) and UPFC (unsupervised possibilistic fuzzy clustering) with different initializations. These experiments demonstrate an improvement in the quality of clustering when initialized by the proposed method. The concept how to estimate a ratio of added noise is also presented.