dropping factor levels in a subsetted data frame in R

Question

I have a data frame containing a factor. When I create a subset of this data frame using subset() or another indexing function, a new data frame is created. However, the factor variable retains all of its original levels -- even when they do not exist in the new data frame.

This creates headaches when doing faceted plotting or using functions that rely on factor levels.

What is the most succinct way to remove levels from a factor in my new data frame?

Here's my example:

df <- data.frame(letters=letters[1:5],
                    numbers=seq(1:5))

levels(df$letters)
## [1] "a" "b" "c" "d" "e"

subdf <- subset(df, numbers <= 3)
##   letters numbers
## 1       a       1
## 2       b       2
## 3       c       3    

## but the levels are still there!
levels(subdf$letters)
## [1] "a" "b" "c" "d" "e"

Answer 1

It is a known issue, and one possible remedy is provided by drop.levels() in the gdata package where your example becomes

> drop.levels(subdf)
  letters numbers
1       a       1
2       b       2
3       c       3
> levels(drop.levels(subdf)$letters)
[1] "a" "b" "c"

There is also the dropUnusedLevels function in the Hmisc package. However, it only works by altering the subset operator [ and is not applicable here.

As a corollary, a direct approach on a per-column basis is a simple as.factor(as.character(data)):

> levels(subdf$letters)
[1] "a" "b" "c" "d" "e"
> subdf$letters <- as.factor(as.character(subdf$letters))
> levels(subdf$letters)
[1] "a" "b" "c"

Answer 2

This is obnoxious. This is how I usually do it, to avoid loading other packages:

levels(subdf$letters)<-c("a","b","c",NA,NA)

which gets you:

> subdf$letters
[1] a b c
Levels: a b c

Note that the new levels will replace whatever occupies their index in the old levels(subdf$letters), so something like:

levels(subdf$letters)<-c(NA,"a","c",NA,"b")

won't work.

This is obviously not ideal when you have lots of levels, but for a few, it's quick and easy.

Answer 3

All you should have to do is to apply factor() to your variable again after subsetting:

> subdf$letters
[1] a b c
Levels: a b c d e
subdf$letters <- factor(subdf$letters)
> subdf$letters
[1] a b c
Levels: a b c

From the ?factor page example:

factor(ff)      # drops the levels that do not occur

Answer 4

If you don't want this behaviour, don't use factors, use character vectors instead. I think this makes more sense than patching things up afterwards. Try the following before loading your data with read.table or read.csv:

options(stringsAsFactors = FALSE)

The disadvantage is that you're restricted to alphabetical ordering. (reorder is your friend for plots)

Answer 5

Here's another way, which I believe is equivalent to the factor(..) approach:

> df <- data.frame(let=letters[1:5], num=1:5)
> subdf <- df[df$num <= 3, ]

> subdf$let <- subdf$let[ , drop=TRUE]

> levels(subdf$let)
[1] "a" "b" "c"

Answer 6

I wrote utility functions to do this. Now that I know about gdata's drop.levels, it looks pretty similar. Here they are (from here):

present_levels <- function(x) intersect(levels(x), x)

trim_levels <- function(...) UseMethod("trim_levels")

trim_levels.factor <- function(x)  factor(x, levels=present_levels(x))

trim_levels.data.frame <- function(x) {
  for (n in names(x))
    if (is.factor(x[,n]))
      x[,n] = trim_levels(x[,n])
  x
}