[R] Outlier Detection with k-Means

[Boris Steipe]

Three comments:
(i)    If you calculate distances like this, you are weighting all columns
       equally by absolute numbers. Depending on your application, you might
       want to normalize the columns first (and before clustering).

(ii)   Your distance calculation is not the cartesian distance. That would be:
       sqrt(rowSums(iris2[1,]^2 - centers[1,]^2)). 

(iii)  To scale to relative distances you need to define a common,
       commensurable scale. This is often done by scaling to the standard
       deviation, which gives you a probability estimate for your outliers
       if you can model the distribution as being normal. You could also
       scale to the median.

But in the end ... detecting "outliers" in the first place implies that you have an underlying model of the distribution. If you do, you should apply it. If you don't (and I don't see how such a model would be possible, given that your outliers of one group could simply be members of another) you could simply rank, and investigate (or discard) some n most distant. That said, I still don't think this would be meaningful because of the fact that k-means assigns *all* points to *some* cluster - but perhaps your specific application supports a different reasoning.

Cheers,
Boris




On 2014-05-07, at 4:34 AM, marioger wrote:

> Hi,
> 
> i am hoping you can help me with my problem. I am trying to detect outliers
> with use of the kmeans algorithm. First I perform the algorithm and choose
> those object as possible outliers which have a big distance to their cluster
> center. Instead of using the absolute distance I want to use the relative
> distance, i.e. the ration of absolute distance of the object to the cluster
> center and the average distance of all objects of the cluster to their
> cluster center. The code for outlier detection based on absolute distance is
> the following:
> 
>> # remove species from the data to cluster
>> iris2 <- iris[,1:4]
>> kmeans.result <- kmeans(iris2, centers=3)
>> # cluster centers
>> kmeans.result$centers
>> # calculate distances between objects and cluster centers
>> centers <- kmeans.result$centers[kmeans.result$cluster, ]
>> distances <- sqrt(rowSums((iris2 - centers)^2))
>> # pick top 5 largest distances
>> outliers <- order(distances, decreasing=T)[1:5]
>> # who are outliers
>> print(outliers)
> 
> But how can I use the relative instead of the absolute distance to find
> outliers?
> Thanks in advance.
> 
> Mario
> 
> 
> 
> --
> View this message in context: http://r.789695.n4.nabble.com/Outlier-Detection-with-k-Means-tp4690098.html
> Sent from the R help mailing list archive at Nabble.com.
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.