Changing whisker definition in geom_boxplot

Question

I'm trying to use ggplot2 / geom_boxplot to produce a boxplot where the whiskers are defined as the 5 and 95th percentile instead of 0.25 - 1.5 IQR / 0.75 + IQR and outliers from those new whiskers are plotted as usual. I can see that the geom_boxplot aesthetics include ymax / ymin, but it's not clear to me how I put values in here. It seems like:

stat_quantile(quantiles = c(0.05, 0.25, 0.5, 0.75, 0.95))

should be able to help, but I don't know how to relate the results of this stat to set the appropriate geom_boxplot() aesthetics:

geom_boxplot(aes(ymin, lower, middle, upper, ymax))

I've seen other posts where people mention essentially building a boxplot-like object manually, but I'd rather keep the whole boxplot gestalt intact, just revising the meaning of two of the variables being drawn.

Mountainous

Answer 1

geom_boxplot with stat_summary can do it:

# define the summary function
f <- function(x) {
  r <- quantile(x, probs = c(0.05, 0.25, 0.5, 0.75, 0.95))
  names(r) <- c("ymin", "lower", "middle", "upper", "ymax")
  r
}

# sample data
d <- data.frame(x=gl(2,50), y=rnorm(100))

# do it
ggplot(d, aes(x, y)) + stat_summary(fun.data = f, geom="boxplot")

# example with outliers
# define outlier as you want    
o <- function(x) {
  subset(x, x < quantile(x)[2] | quantile(x)[4] < x)
}

# do it
ggplot(d, aes(x, y)) + 
  stat_summary(fun.data=f, geom="boxplot") + 
  stat_summary(fun.y = o, geom="point")

Alcmene

kohske, that does indeed change the whiskers (thanks!), but the outliers disappear. – Mountainous

the example was updated: there are various ways to do it, but perhaps it is easiest way to plot outliers in geom_point. – Alcmene

Great! The o function should probably use the same probs = c(0.05, 0.95)[1] / [2] so the excluded points match the whiskers. Thanks again. Looks like I need to learn more about stat_summary. – Mountainous

Is it possible to put whiskers in ymin and ymax? – Bluster

in the above code, replace the line "subset(x....)" by "subset(x, x < quantile(x, 0.05) | x > quantile(x, 0.95))" – Heartstricken

Answer 2

It is now possible to specify the whiskers endpoints in ggplot2_2.1.0. Copying from the examples in ?geom_boxplot:

 # It's possible to draw a boxplot with your own computations if you
 # use stat = "identity":
 y <- rnorm(100)
 df <- data.frame(
   x = 1,
   y0 = min(y),
   y25 = quantile(y, 0.25),
   y50 = median(y),
   y75 = quantile(y, 0.75),
   y100 = max(y)
 )
 ggplot(df, aes(x)) +
   geom_boxplot(
    aes(ymin = y0, lower = y25, middle = y50, upper = y75, ymax = y100),
    stat = "identity"
  )

Waiver

Answer 3

Building on @konvas's answer, beginning in ggplot2.0.x, you can extend ggplot using the ggproto system and define your own stat.

By copying the ggplot2 stat_boxplot code and making a few edits, you can quickly define a new stat (stat_boxplot_custom) that takes the percentiles you want to use as an argument (qs) instead of the coef argument that stat_boxplot uses. The new stat is defined here:

# modified from https://github.com/tidyverse/ggplot2/blob/master/R/stat-boxplot.r
library(ggplot2)
stat_boxplot_custom <- function(mapping = NULL, data = NULL,
                     geom = "boxplot", position = "dodge",
                     ...,
                     qs = c(.05, .25, 0.5, 0.75, 0.95),
                     na.rm = FALSE,
                     show.legend = NA,
                     inherit.aes = TRUE) {
  layer(
      data = data,
      mapping = mapping,
      stat = StatBoxplotCustom,
      geom = geom,
      position = position,
      show.legend = show.legend,
      inherit.aes = inherit.aes,
      params = list(
      na.rm = na.rm,
      qs = qs,
      ...
      )
  )
}

Then, the layer function is defined. Note that b/c I copied directly from stat_boxplot, you have to access a few internal ggplot2 functions using :::. This includes a lot of stuff copied directly over from StatBoxplot, but the key area is in computing the stats directly from the qs argument: stats <- as.numeric(stats::quantile(data$y, qs)) inside of the compute_group function.

StatBoxplotCustom <- ggproto("StatBoxplotCustom", Stat,
  required_aes = c("x", "y"),
  non_missing_aes = "weight",

  setup_params = function(data, params) {
    params$width <- ggplot2:::"%||%"(
      params$width, (resolution(data$x) * 0.75)
    )

    if (is.double(data$x) && !ggplot2:::has_groups(data) && any(data$x != data$x[1L])) {
      warning(
        "Continuous x aesthetic -- did you forget aes(group=...)?",
        call. = FALSE
      )
    }

    params
  },

  compute_group = function(data, scales, width = NULL, na.rm = FALSE, qs = c(.05, .25, 0.5, 0.75, 0.95)) {

    if (!is.null(data$weight)) {
      mod <- quantreg::rq(y ~ 1, weights = weight, data = data, tau = qs)
      stats <- as.numeric(stats::coef(mod))
    } else {
    stats <- as.numeric(stats::quantile(data$y, qs))
    }
    names(stats) <- c("ymin", "lower", "middle", "upper", "ymax")
    iqr <- diff(stats[c(2, 4)])

    outliers <- (data$y < stats[1]) | (data$y > stats[5])

    if (length(unique(data$x)) > 1)
    width <- diff(range(data$x)) * 0.9

    df <- as.data.frame(as.list(stats))
    df$outliers <- list(data$y[outliers])

    if (is.null(data$weight)) {
      n <- sum(!is.na(data$y))
    } else {
      # Sum up weights for non-NA positions of y and weight
      n <- sum(data$weight[!is.na(data$y) & !is.na(data$weight)])
    }

    df$notchupper <- df$middle + 1.58 * iqr / sqrt(n)
    df$notchlower <- df$middle - 1.58 * iqr / sqrt(n)

    df$x <- if (is.factor(data$x)) data$x[1] else mean(range(data$x))
    df$width <- width
    df$relvarwidth <- sqrt(n)
    df
  }
)

There is also a gist here, containing this code.

Then, stat_boxplot_custom can be called just like stat_boxplot:

library(ggplot2)
y <- rnorm(100)
df <- data.frame(x = 1, y = y)
# whiskers extend to 5/95th percentiles by default
ggplot(df, aes(x = x, y = y)) +
  stat_boxplot_custom()
# or extend the whiskers to min/max
ggplot(df, aes(x = x, y = y)) +
  stat_boxplot_custom(qs = c(0, 0.25, 0.5, 0.75, 1))

Bodleian

This answer is excellent! The above one does not work with facet_grid. This does flawlessly. Thank you a ton!! – Aristotelian