4

I'm trying to write an extension for ggplot2 in R that lets a user add text annotations in the margins. My idea is to put the text in new rows/columns that get added to the plot's gtable. Hopefully that way space is automatically allotted in the margins and the user doesn't have to mess with the theme(plot.margin) option to manually allocate space.

Ideally my hypothetical function anno_margin could be used as below, where you can add it to a ggplot object and it returns a ggplot object so that more layers, scales, theme settings, etc., can be added:

mpg |>
  ggplot(aes(cty, hwy)) +
  geom_point() +
  anno_margin("Hello, world!", side = "t") +
  ...more layers/scales/etc...  

I can extract the plot's gtable and modify it as desired, but I have no idea how to turn the modified gtable back into a ggplot object in a way that retains all the original plot info, e.g., the aesthetic mappings. I've tried using patchwork::wrap_ggplot_grob and ggplotify::as.ggplot, which both successfully convert the gtable into a ggplot object. But since these two functions just take the gtable as input, they of course don't know about the aesthetic mappings and other plot "metadata" that I want to keep.

Here's some sample code that creates a scatter plot, gets the associated gtable, adds "Hello, world!" in a new row on top of the gtable, converts the gtable to a ggplot object, and then tries and fails to add a new geom_line layer to connect the scatter plot dots.

library(ggplot2)

# base plot
p <- mpg |>
  ggplot(aes(cty, hwy)) +
  geom_point()

# extract gtable
p_gt <- ggplotGrob(p)

# allocate a new row on top
p_gt2 <- gtable::gtable_add_rows(
  p_gt,
  heights = grid::unit(1, "lines"),
  pos = 0
)

# add text to the newly allocated top row
p_gt3 <- gtable::gtable_add_grob(
  p_gt2,
  grobs = grid::textGrob("Hello, world!"),
  t = 1,
  l = 7,
  b = 1,
  r = 7
)

# show that the text was added correctly
plot(p_gt3)

# failed attempt 1: convert to ggplot object and try to connect the dots
p_new1 <- patchwork::wrap_ggplot_grob(p_gt3)
p_new1 + geom_line()

# failed attempt 2: convert to ggplot object and try to connect the dots
p_new2 <- ggplotify::as.ggplot(p_gt3)
p_new2 + geom_line()

Is there some way to combine p with p_gt3 so that I get a ggplot object with my annotations and all the original plot metadata?

FWIW, I've asked GitHub Copilot several times to figure this out for me, but it hasn't worked.

4
  • 2
    This is quite a difficult thing to do. The least hacky way to do it is to define a new S7 child class of ggplot (eg "class_anno_ggplot"), and get your anno_margin function to convert the plot to the new class. You then need a new ggplot_build method for "class_anno_ggplot" that is just a copy of the ggplot_build method for class_ggplot, except it outputs an object of class "class_anno_ggplot_built" which again inherits from class_ggplot_built. Finally, you need a new method for ggplot_gtable which adds your text in extra rows and columns in the gtable or any other grob hacking. Commented Oct 30 at 16:07
  • @AllanCameron, your approach sounds promising but also frankly beyond my current ability. Do you have any references that implement something like this or explain the steps? There's the ggplot2 book (ggplot2-book.org) although I think maybe it hasn't been updated yet since the switch to S7. Commented Oct 30 at 17:00
  • 2
    I think this is a taller order than you anticipated. The extension system for ggplot2 is about adding new stats, geoms, themes, facets, guides, etc., not about adding new components to a plot object. ggplot2.tidyverse.org/articles/extending-ggplot2.html And I don't think there's a way to "put the toothpaste back in the tube" once you've rendered a ggplot as a gtable. Commented Oct 30 at 18:16
  • 1
    @Dagremu no, this is all brand new stuff, and my suggestion is based on the approach I would have taken when ggplot was written in S3. I don't think it's any harder in S7, but you would need a reasonable working knowledge of S7 to do it. Bearing in mind, you would also probably want the text to be able to inherit from the theme system, perhaps with its own theme elements. It's not at all impossible, but it is very involved (probably too involved for a full answer here). It would be possible to learn through the S7 docs and extending ggplot docs plus a lot of trial and error. Commented Oct 31 at 15:35

3 Answers 3

7

A ggplot object is like a blueprint for building a house. Every time you draw a ggplot object, the ggplot_build and ggplot_gtable functions act like builders, turning your blueprint into a gtable, which is what is actually drawn on the screen. The gtable is like the finished house after the building process is complete. If you later build an extension onto your house, that does not magically change the original blueprint. Similarly, modifying a gtable after it is built can't affect the ggplot object.

To get a ggplot to generate a modified final gtable, what you would need to do is include the information for your annotation in the ggplot "blueprint", and write versions of the building functions ggplot_build and ggplot_gtable that know how to incorporate the annotation in your plot.

Fortunately, ggplot_build and ggplot_gtable are generic functions, which means that as long as your annotated ggplot has its own special class, you can write methods for these functions which only apply to your new special class. Now that ggplot2 is written using the S7 object-oriented system, we can use S7 inheritance to make this work. Essentially, your anno_margin function should convert the ggplot object to a new sub-class of ggplot as well as including the annotation information, then your new build methods produce the modified gtable.

There are several caveats here. Firstly, this is probably far more complex than you were expecting. Secondly, the solution I have sketched here works in the console, but it makes use of some of ggplot2's unexported functions, and you couldn't do this inside a package unless you copied these functions from the ggplot2 code base into your own package. Thirdly, adding text grobs directly misses out on the benefits of the theme system, so at present there is no way to change the size, style and colour of your text. The easiest way to do this would be to add size, font, face and colour arguments into anno_margin which ultimately get passed to the builder methods, but ultimately it would be best to add new theme components which can be interrogated during the build process.

In view of all that, I think something like Jon Spring's answer is going to turn out to be more practical, but I thought I would show how it is possible to create a modifiable ggplot object that builds a modified gtable. If nothing else it gives a practical demonstration of the relatively new S7 system in action.


Let's start with the function that will actually modify your gtable. It takes an annotation and a side (t, r, b, l), and adds the annotation to the gtable at the correct side and at the correct angle:

library(S7)
library(ggplot2)

manipulate_gtable <- function(gtable, annotation, side) {
  
  side <- match(side, c("t", "r", "b", "l"))
  rot  <- c(0, -90, 0, 90)[side]
  sz   <- grid::unit(1, "lines")
  pos  <- c(0, -1, -1, 0)[side]
  fun  <- rep(list(gtable::gtable_add_rows, gtable::gtable_add_cols), 2)[[side]]
  
  gt <- do.call(fun, list(gtable, sz, pos))
  
  trows <- dim(gt)[1]
  tcols <- dim(gt)[2]
  t_pos <- c(1, 1, trows, 1)[side]
  b_pos <- c(1, trows, trows, trows)[side]
  l_pos <- c(1, tcols, 1, 1)[side]
  r_pos <- c(tcols, tcols, tcols, 1)[side]
  gtable::gtable_add_grob(
    gt,
    grobs = grid::textGrob(annotation, rot = rot),
    t = t_pos,
    l = l_pos,
    b = b_pos,
    r = r_pos
  )
}

To get ggplot to use this, we must first write a new ggplot subclass that is the same as a ggplot but contains slots for the annotation and the side:

class_ggplot_anno <- new_class("ggplot_anno",
  ggplot2::class_ggplot,
  constructor = function(plot, annotation, anno_side) {
    stopifnot(is.character(annotation))
    if(!anno_side %in% c("t", "r", "b", "l")) {
      stop("'anno_side must be one of 't', 'r', 'b', or 'l'")
    }
    new_object(plot, annotation = annotation, anno_side = anno_side)
  },
  properties = list(annotation = class_character, 
                    anno_side = class_character)
  )

Now we can write anno_margin in such a way that when added to a ggplot, it transforms the ggplot into our new annotated ggplot class. As far as I can tell, we need to use the old S3 generic ggplot_add here:

anno_margin <- function(annotation, side = "t") {
  structure(list(annotation = annotation, side = side), class = "anno")
}

ggplot_add.anno <- function(object, plot, ...) {
  class_ggplot_anno(plot, annotation = object$anno, anno_side = object$side)
}

We now need to propagate the annotation through the build process by declaring an annotated version of the ggplot_built class from scratch:

class_ggplot_built_anno <- new_class("ggplot_built_anno",
  parent = S7::S7_object,
  constructor = function(blank, data, layout, plot, annotation, anno_side) {
    
   new_object(blank, data = data, 
              layout = layout, plot = plot, 
              annotation = annotation, 
              anno_side = anno_side,
              built_plot = ggplot2::class_ggplot_built(data = data, 
                                                       layout = layout, 
                                                       plot = plot))
  },
  properties = list(data = class_list,
                    layout = ggplot2::class_layout,
                    plot = ggplot2::class_ggplot,
                    annotation = class_character,
                    anno_side = class_character,
                    built_plot = ggplot2::class_ggplot_built)
  )

To get our ggplot_anno built into a ggplot_built_anno, we need a new S7 method for ggplot_build. This is largely just a copy of the ggplot_build method for ggplot objects, so it contains unexported ggplot2 functions which you would need to copy into your own code base to make it work inside a package:

method(ggplot_build, class_ggplot_anno) <- function (plot, ...) {

    plot <- ggplot2:::plot_clone(plot)
    if (length(plot@layers) == 0) {
      plot <- plot + ggplot2::geom_blank()
    }
    layers <- plot@layers
    data <- rep(list(NULL), length(layers))
    scales <- plot@scales
    data <- ggplot2:::by_layer(function(l, d) l$layer_data(plot@data), 
                     layers, data, "computing layer data")
    data <- ggplot2:::by_layer(function(l, d) l$setup_layer(d, plot), layers, 
                     data, "setting up layer")
    layout <- ggplot2:::create_layout(plot@facet, plot@coordinates, plot@layout)
    data <- layout$setup(data, plot@data, plot@plot_env)
    data <- ggplot2:::by_layer(function(l, d) l$compute_aesthetics(d, plot), 
                     layers, data, "computing aesthetics")
    plot@labels <- ggplot2:::setup_plot_labels(plot, layers, data)
    data <- ggplot2:::.ignore_data(data)
    data <- lapply(data, scales$transform_df)
    scale_x <- function() scales$get_scales("x")
    scale_y <- function() scales$get_scales("y")
    layout$train_position(data, scale_x(), scale_y())
    data <- layout$map_position(data)
    data <- ggplot2:::.expose_data(data)
    data <- ggplot2:::by_layer(function(l, d) l$compute_statistic(d, layout), 
                     layers, data, "computing stat")
    data <- ggplot2:::by_layer(function(l, d) l$map_statistic(d, plot), 
                     layers, data, "mapping stat to aesthetics")
    plot@scales$add_missing(c("x", "y"), plot@plot_env)
    data <- ggplot2:::by_layer(function(l, d) l$compute_geom_1(d), layers, 
                     data, "setting up geom")
    data <- ggplot2:::by_layer(function(l, d) l$compute_position(d, layout), 
                     layers, data, "computing position")
    data <- ggplot2:::.ignore_data(data)
    layout$reset_scales()
    layout$train_position(data, scale_x(), scale_y())
    layout$setup_panel_params()
    data <- layout$map_position(data)
    layout$setup_panel_guides(plot@guides, plot@layers)
    plot@theme <- ggplot2:::plot_theme(plot)
    npscales <- scales$non_position_scales()
    if (npscales$n() > 0) {
      npscales$set_palettes(plot@theme)
      lapply(data, npscales$train_df)
      plot@guides <- plot@guides$build(npscales, plot@layers, 
                                       plot@labels, data, plot@theme)
      data <- lapply(data, npscales$map_df)
    }
    else {
      plot@guides <- plot@guides$get_custom()
    }
    data <- .expose_data(data)
    data <- ggplot2:::by_layer(function(l, d) {
      l$compute_geom_2(d, theme = plot@theme)
      }, layers, data, "setting up geom aesthetics")
    data <- ggplot2:::by_layer(function(l, d) l$finish_statistics(d), layers, 
                     data, "finishing layer stat")
    data <- layout$finish_data(data)
    plot@labels$alt <- ggplot2:::get_alt_text(plot)
    build <- class_ggplot_built_anno(S7::S7_object(),
                                data = data, layout = layout, 
                                plot = plot, annotation = plot@annotation,
                                anno_side = plot@anno_side)
    class(build) <- union(c("ggplot_built_anno", "ggplot::ggplot_built",
                            "ggplot_built"), 
                          class(build))
    build
}

Finally, we need a method of converting our ggplot_built_anno object into a gtable:

method(ggplot_gtable, class_ggplot_built_anno) <- function (plot) {
  
  built      <- ggplot2::ggplot_gtable(plot@built_plot)
  manipulate_gtable(built, plot@annotation, plot@anno_side)
}

Now we're done, so testing we get:

p <- mpg |>
  ggplot(aes(cty, hwy)) +
  geom_point() +
  anno_margin("Hello world!", side = "t") 

p

enter image description here

And we can modify it as we would modify any ggplot whilst preserving the annotation

p + geom_line() + theme_bw()

enter image description here

And it remains an actual ggplot object

ggplot2::is_ggplot(p)
#> [1] TRUE
Sign up to request clarification or add additional context in comments.

5 Comments

Thanks for adding this, there's a lot to learn from it.
This is great -- thank you! I've seen other questions on SO where a solution was to modify a ggplot's gtable. When someone then asked if the output could still be a ggplot object, the answer was something like "maybe, but it's complicated". See for example: stackoverflow.com/a/28669873/3962066. Your code is the first working example I've seen showing the answer is yes.
Your answer here solves all the hard parts of my question, so I've accepted it as the solution. But your code does have a limitation: If I try to add multiple annotations to a plot, only the final one is actually applied -- the previous ones get discarded. This can be fixed by tweaking your code in a few places. For completeness, I've posted my revision to your code as a separate answer: stackoverflow.com/a/79817085/3962066
FYI, I updated the code in my answer with a major simplification. The code is now much shorter and no longer calls any unexported ggplot2 functions. So I think that resolves two of the three caveats that you mentioned!
@Dagremu that's excellent. I suspected there would be a way to do this, but I was learning my way through S7 myself as I wrote this answer. The convert function was the missing part of the puzzle.
3

Allan Cameron wrote a great answer that solves all the hard parts of this question. His code has a limitation though: If you add multiple annotations to a plot, only the final one is actually applied -- the previous ones get discarded.

Here, I've slightly modified his code to fix this. The basic idea is to redefine Allan's class_ggplot_anno so that its only new property is a list named annotations that can grow to store multiple annotations. So then ggplot_add.anno appends a new annotation to any existing annotations. Lastly, the new ggplot_gtable method iteratively applies all the annotations to the gtable.

Aside from the above changes, this code comes from Allan's answer, and thus his explanation of the code still applies.

Update (2025-11-12): With a hint from Teun van den Brand, I've revised and majorly simplified the code. The most important change is that the custom ggplot_build method is now just two lines of code instead of 60+ and does not rely on any unexported ggplot2 functions. Basically, instead of copying and modifying the source code for method(ggplot_build, class_ggplot), I just call that method on our class_ggplot_anno object and then convert the output to our custom class_ggplot_built_anno with the new annotations property. This required changing class_ggplot_built_anno so that it inherits from class_ggplot_built rather than S7::S7_object, and the resulting definition of class_ggplot_built_anno is now much simpler than before.

One quirk is that I had to use NextMethod (from the base package) to dispatch to the parent class; I couldn't get S7::super to work here. I think that's because the ggplot2 codebase is in a transitional state where it hasn't been fully converted to S7 yet. If ggplot2 does eventually go full S7, I'm not sure if NextMethod will still work and I may need to switch to using S7::super.

library(ggplot2)

manipulate_gtable <- function(gtable, text, side) {
  side <- match(side, c("t", "r", "b", "l"))
  rot  <- c(0, -90, 0, 90)[side]
  sz   <- grid::unit(1, "lines")
  pos  <- c(0, -1, -1, 0)[side]
  fun  <- rep(list(gtable::gtable_add_rows, gtable::gtable_add_cols), 2)[[side]]

  gt <- do.call(fun, list(gtable, sz, pos))

  trows <- dim(gt)[1]
  tcols <- dim(gt)[2]
  t_pos <- c(1, 1, trows, 1)[side]
  b_pos <- c(1, trows, trows, trows)[side]
  l_pos <- c(1, tcols, 1, 1)[side]
  r_pos <- c(tcols, tcols, tcols, 1)[side]
  gtable::gtable_add_grob(
    gt,
    grobs = grid::textGrob(text, rot = rot),
    t = t_pos,
    l = l_pos,
    b = b_pos,
    r = r_pos
  )
}

class_ggplot_anno <- S7::new_class(
  "ggplot_anno",
  parent = ggplot2::class_ggplot,
  properties = list(annotations = S7::class_list),
  constructor = function(plot, annotations) {
    # The new annotations property should be a list of lists. Each inner list
    # should have two elements named "text" and "side".
    for (anno in annotations) {
      stopifnot("'text' must be a string" = is.character(anno$text) && length(anno$text) == 1)
      if(length(anno$side) != 1 || !anno$side %in% c("t", "r", "b", "l")) {
        stop("'side' must be one of 't', 'r', 'b', or 'l'")
      }
    }
    S7::new_object(plot, annotations = annotations)
  }
)

anno_margin <- function(text, side = "t") {
  structure(list(text = text, side = side), class = "anno")
}

ggplot_add.anno <- function(object, plot, ...) {
  # Append new annotation to existing annotations if there are any
  old_annotations <- tryCatch(plot@annotations, error = \(e) NULL)
  class_ggplot_anno(plot, annotations = c(old_annotations, list(object)))
}

class_ggplot_built_anno <- S7::new_class(
  "ggplot_built_anno",
  parent = ggplot2::class_ggplot_built,
  properties = list(annotations = S7::class_list)
)

S7::method(ggplot_build, class_ggplot_anno) <- function(plot, ...) {
  # Use the ggplot_build method from parent class class_ggplot, then convert the
  # result to our custom class class_ggplot_built_anno and add our new property
  build <- NextMethod(plot)
  S7::convert(build, to = class_ggplot_built_anno, annotations = plot@annotations)
}

S7::method(ggplot_gtable, class_ggplot_built_anno) <- function(plot) {
  # Use the ggplot_gtable method from parent class class_ggplot_built
  plot_table <- NextMethod(plot)

  # Iteratively apply all the annotations to the gtable
  for (anno in plot@annotations)
    plot_table <- manipulate_gtable(plot_table, anno$text, anno$side)

  plot_table
}

mpg |>
  ggplot(aes(cty, hwy)) +
  geom_point() +
  anno_margin("Hello world!", side = "t") +
  anno_margin("Goodnight moon!", side = "t") +
  geom_line() +
  theme_bw() +
  anno_margin("¡Hola mundo!", side = "r") +
  anno_margin("Bonjour le monde!", side = "l") +
  anno_margin("Hallo Welt!", side = "b")

ggplot with annotations

1 Comment

It did occur to me that this was a limitation, and I didn't fix it because my answer was more a proof of concept. Your mechanism to allow multiple annotations is very elegant - it's clear that you have a good grasp of what's going on under the hood. If you can make this into something thats appearance can be controlled by theme it would actually make a useful little package.
2

I have not made this into a function yet, but I wanted to outline a potential approach using patchwork that might be simpler to manage. To be a function, this would require adding some bookkeeping to keep track of when we need 2 vs. 3 heights/widths. (And it's not clear to me why we need 3 here for b but only 2 for d.)

wrap_elements(plot = grid::textGrob("Hello, top of the world!")) /
  p + plot_layout(height = c(1, 20)) -> a

# Add grob on bottom
# heights here needs to be 3 elements, for top / plot / bottom
a / wrap_elements(plot = grid::textGrob("Hello, bottom of the world!")) +
  plot_layout(heights = c(1, 20, 1)) -> b

# Add grob on left
wrap_elements(plot = grid::textGrob("Hello, left of the world!", rot = 90)) +
  b + plot_layout(widths = c(1, 20)) -> c

# Add another grob on top
wrap_elements(plot = grid::textGrob("Hello again, top of the world!")) /
  c + plot_layout(height = c(1, 20)) -> d

enter image description here

2 Comments

In my original question, I used grid::unit(1, "lines") to allocate 1 line of text for the annotation. That's a fixed size that I believe doesn't depend on the plot dimensions -- it should always be just the right size for one line of text. In your code you allocate roughly 5% of the plot height (or width) for the annotation. That looks good for the plot size you printed, but would presumably need to be tweaked for a different plot size. Using plot_layout is there some way to specify a fixed height (or width) for the annotation rather than a relative one?
Not that I'm aware. I know this doesn't solve your question exactly, but I'm hoping it gets you 60% of what you want with 10% of the effort of other approaches.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.