Package tests are a simple way to make sure that the statistical software you have written does what you expect, both when you run it on typical and atypical input. I also tend to use package tests when implementing new features in my software packages, as a way to check to see that the new functionality works as I expect it.
There are multiple frameworks for writing package tests, but we will focus on the framework that I find the most straightforward, which is implemented in the testthat package.
A reference for writing tests with testthat can be found at the R Packages book by Hadley Wickham.
To begin writing tests, say for a part of your software that you call
“name”, you can run the usethis function
use_test("name"). This will create a directory called
tests/testthat in the root of your R package directory, add
testthat to your Suggests: line in the
DESCRIPTION file, create a file
tests/testthat.R that will run all the tests in
tests/testthat when you run R’s package check, and create a
file tests/testthat/test-name.R. You may have multiple
groups of tests that you want to separate into different files, so you
can choose “name” however you like, e.g. test-data-input.R,
test-normalization.R, etc. However, you can also put all
your tests into a single file for the package,
e.g. test-foo.R.
The testthat.R file is very simple:
# This file is part of the standard setup for testthat.
# It is recommended that you do not modify it.
#
# Where should you do additional test configuration?
# Learn more about the roles of various files in:
# * https://r-pkgs.org/tests.html
# * https://testthat.r-lib.org/reference/test_package.html#special-files
library(testthat)
library(foo)
test_check("foo")This file stays the same way, and we will write new .R
files that go into tests/testthat which will implement the
package tests.
Suppose we run use_test("add") for our foo
package, and we want to write a test for our add function
(make sure the usethis pakage is loaded). We can do this by
opening up the file tests/testthat/test-add.R, and adding
some tests. The default file has some dummy code to show you the
style:
test_that("multiplication works", {
expect_equal(2 * 2, 4)
})But we can rewrite this for our purposes:
test_that("add works on two vectors", {
expect_equal(add(1:5,6:10), c(7,9,11,13,15))
})
test_that("simple errors for bad input", {
expect_error(add())
expect_error(add(1:5))
expect_error(add(1:5,6:10,"yes"))
})There are many possible tests that one can write, with the workhorses
probably being expect_equal and expect_true.
We can also specify a numerical tolerance (absolute or relative) for
equality, as shown in the Examples in ?expect_equal. In
order to see a list of all the expect_ functions available
in testthat, one can run the following command in R:
help(package="testthat", help_type="html")We can also check that specific messages, warnings, or errors are
output for given input to our function. These three levels of output
message the user relevant information, provide a
warning to the user about potential problems, or
stop the function from providing any output.
If we wanted the add function to warn the user about
negative values as output (just a trivial example), we could write:
add2 <- function(x,y,negative=FALSE) {
z <- x + y
if (negative) {
z <- -1 * z
}
if (any(z < 0)) {
warning("some output values are negative")
}
z
}We could then test this by saying we expect a specific warning. Note that the entire warning doesn’t need to be written out, only a regular expression that would produce a match.
library(testthat)
expect_warning(add2(1:5, -11:-15), "are negative")If we wanted to test for a message or error, we would use
expect_message or expect_error with the
message or stop function respectively.
We can check all the tests for individual files with the following
call to test_file, from within the package root:
library(devtools)
load_all()
test_file("tests/testthat/test-add.R")Or we can check all of the tests for a given package with the
following call to test_package:
test_package("foo")library(devtools)## Loading required package: usethis##
## Attaching package: 'devtools'## The following object is masked from 'package:testthat':
##
## test_filesession_info()## ─ Session info ───────────────────────────────────────────────────────────────
## setting value
## version R version 4.2.2 (2022-10-31 ucrt)
## os Windows 10 x64 (build 19045)
## system x86_64, mingw32
## ui RTerm
## language (EN)
## collate English_United States.utf8
## ctype English_United States.utf8
## tz America/New_York
## date 2023-01-23
## pandoc 2.19.2 @ C:/Program Files/RStudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)
##
## ─ Packages ───────────────────────────────────────────────────────────────────
## package * version date (UTC) lib source
## brio 1.1.3 2021-11-30 [2] CRAN (R 4.2.1)
## bslib 0.4.2 2022-12-16 [2] CRAN (R 4.2.2)
## cachem 1.0.6 2021-08-19 [2] CRAN (R 4.2.1)
## callr 3.7.3 2022-11-02 [2] CRAN (R 4.2.2)
## cli 3.5.0 2022-12-20 [2] CRAN (R 4.2.2)
## crayon 1.5.2 2022-09-29 [2] CRAN (R 4.2.1)
## desc 1.4.2 2022-09-08 [2] CRAN (R 4.2.1)
## devtools * 2.4.5 2022-10-11 [2] CRAN (R 4.2.1)
## digest 0.6.31 2022-12-11 [2] CRAN (R 4.2.2)
## ellipsis 0.3.2 2021-04-29 [2] CRAN (R 4.2.1)
## evaluate 0.19 2022-12-13 [2] CRAN (R 4.2.2)
## fastmap 1.1.0 2021-01-25 [2] CRAN (R 4.2.1)
## fs 1.5.2 2021-12-08 [2] CRAN (R 4.2.1)
## glue 1.6.2 2022-02-24 [2] CRAN (R 4.2.1)
## htmltools 0.5.4 2022-12-07 [2] CRAN (R 4.2.2)
## htmlwidgets 1.6.1 2023-01-07 [2] CRAN (R 4.2.2)
## httpuv 1.6.7 2022-12-14 [2] CRAN (R 4.2.2)
## jquerylib 0.1.4 2021-04-26 [2] CRAN (R 4.2.1)
## jsonlite 1.8.4 2022-12-06 [2] CRAN (R 4.2.2)
## knitr 1.41 2022-11-18 [2] CRAN (R 4.2.2)
## later 1.3.0 2021-08-18 [2] CRAN (R 4.2.1)
## lifecycle 1.0.3 2022-10-07 [2] CRAN (R 4.2.1)
## magrittr 2.0.3 2022-03-30 [2] CRAN (R 4.2.1)
## memoise 2.0.1 2021-11-26 [2] CRAN (R 4.2.1)
## mime 0.12 2021-09-28 [2] CRAN (R 4.2.0)
## miniUI 0.1.1.1 2018-05-18 [2] CRAN (R 4.2.1)
## pkgbuild 1.4.0 2022-11-27 [2] CRAN (R 4.2.2)
## pkgload 1.3.2 2022-11-16 [2] CRAN (R 4.2.2)
## prettyunits 1.1.1 2020-01-24 [2] CRAN (R 4.2.1)
## processx 3.8.0 2022-10-26 [2] CRAN (R 4.2.2)
## profvis 0.3.7 2020-11-02 [2] CRAN (R 4.2.1)
## promises 1.2.0.1 2021-02-11 [2] CRAN (R 4.2.1)
## ps 1.7.2 2022-10-26 [2] CRAN (R 4.2.2)
## purrr 1.0.0 2022-12-20 [2] CRAN (R 4.2.2)
## R6 2.5.1 2021-08-19 [2] CRAN (R 4.2.1)
## Rcpp 1.0.9 2022-07-08 [2] CRAN (R 4.2.1)
## remotes 2.4.2 2021-11-30 [2] CRAN (R 4.2.1)
## rlang 1.0.6 2022-09-24 [2] CRAN (R 4.2.1)
## rmarkdown 2.19 2022-12-15 [2] CRAN (R 4.2.2)
## rprojroot 2.0.3 2022-04-02 [2] CRAN (R 4.2.1)
## rstudioapi 0.14 2022-08-22 [2] CRAN (R 4.2.1)
## sass 0.4.4 2022-11-24 [2] CRAN (R 4.2.2)
## sessioninfo 1.2.2 2021-12-06 [2] CRAN (R 4.2.1)
## shiny 1.7.4 2022-12-15 [2] CRAN (R 4.2.2)
## stringi 1.7.8 2022-07-11 [2] CRAN (R 4.2.1)
## stringr 1.5.0 2022-12-02 [2] CRAN (R 4.2.2)
## testthat * 3.1.6 2022-12-09 [2] CRAN (R 4.2.2)
## urlchecker 1.0.1 2021-11-30 [2] CRAN (R 4.2.1)
## usethis * 2.1.6 2022-05-25 [2] CRAN (R 4.2.1)
## vctrs 0.5.1 2022-11-16 [2] CRAN (R 4.2.2)
## withr 2.5.0 2022-03-03 [2] CRAN (R 4.2.1)
## xfun 0.36 2022-12-21 [2] CRAN (R 4.2.2)
## xtable 1.8-4 2019-04-21 [2] CRAN (R 4.2.1)
## yaml 2.3.6 2022-10-18 [2] CRAN (R 4.2.2)
##
## [1] C:/Users/nur2/AppData/Local/R/win-library/4.2
## [2] C:/Program Files/R/R-4.2.2/library
##
## ──────────────────────────────────────────────────────────────────────────────