Skip to main content

Antarctic 2 m temperature trends from ERA5, warped with GDAL multidimensional

A toy but real workload: read a remote ERA5 Zarr array lazily through GDAL’s multidimensional API from R, warp each field to a polar grid in parallel, and look at seasonal temperature trends over Antarctic land.

gdalraster
gdal
zarr
ERA5
Antarctica
R
Author

Michael Sumner

Published

November 21, 2026

Note

This report was generated using AI under general human direction. At the time of generation, the contents have not been comprehensively reviewed by a human analyst.

This is a quick, rough-and-ready exploration of 2 m air temperature trends over Antarctica from the ERA5 reanalysis. The aim is a toy example of a real workload: read a remote 3-D Zarr array lazily with R powered by GDAL’s multidimensional API (the gdalraster package), warp each field to a polar stereographic grid, and compute per-pixel trends.

The data are the ARCO-ERA5 Zarr store on Google Cloud (hourly, 0.25 degrees, one global field per chunk). We sample one field per day at 12 UTC for 1979 to 2021, warp each to a 25 km EPSG:3031 grid, average to monthly means, and then to seasons. Trends are computed by ordinary least squares and checked with the Theil-Sen estimator, and the headline numbers are for Antarctic land (ice sheet and ice shelves) using ERA5’s own land-sea mask.

Method

Each field is one chunk of the array, so reading one hour costs one chunk fetch. For each date the pipeline is:

  1. getView("[t,...]") selects one time step lazily (nothing is read).
  2. getUnscaled()$asClassicDataset(1, 0, NULL) turns the view into a GDALRaster with x = longitude and y = latitude. This is also lazy.
  3. warp() reads only the block it needs and writes a 320 x 320 grid in EPSG:3031 (25 km, bilinear, extent of plus or minus 4000 km).

GDAL objects cannot be shared across R processes, so each of 32 PSOCK workers opens its own handle on the Zarr store and processes an interleaved share of the months. Each worker reduces its daily fields to monthly means before returning them, which keeps memory small.

The array’s time axis ends at the end of 2021, so the study period is 1979 to 2021. Time indices are hours since 1959-01-01 (this was assumed from the array’s layout, not read from its attributes).

The ice-sheet mask comes from the same store: the static /land_sea_mask array (a land fraction on the 0.25 degree grid, no time dimension) is warped to the same polar grid in the same way, and land is defined as warped fraction above 0.5. In ERA5 this treats ice shelves as land, so “land” here means ice sheet plus ice shelves plus islands inside the study disc.

Extraction code

These chunks are shown but not evaluated here: the temperature extraction took about 30 minutes for about 15,700 reads on 32 workers. The last chunk reduces that output to the small cache file this post is rendered from.

library(parallel)

worker_fun <- function(dates) {
  library(gdalraster)
  set_config_option("GDAL_DISABLE_READDIR_ON_OPEN", "EMPTY_DIR")
  era5 <- paste0('ZARR:"/vsicurl/https://storage.googleapis.com/',
                 'gcp-public-data-arco-era5/ar/',
                 '1959-2022-full_37-1h-0p25deg-chunk-1.zarr-v2"')
  ds <- new(GDALMultiDimRaster, era5)
  arr <- ds$openMDArray("/2m_temperature")
  dst <- sprintf("/vsimem/t2m_%d.tif", Sys.getpid())
  # PROJ reports "stere: Invalid latitude" for northern source points
  push_error_handler("quiet")
  on.exit({pop_error_handler(); arr$close(); ds$close()})
  vapply(dates, function(d) {
    t <- as.numeric(difftime(as.POSIXct(paste(d, "12:00:00"), tz = "UTC"),
                             as.POSIXct("1959-01-01", tz = "UTC"), units = "hours"))
    view <- arr$getView(sprintf("[%d,...]", t))
    r <- view$getUnscaled()$asClassicDataset(1, 0, NULL)
    warp(r, dst, t_srs = "EPSG:3031",
         cl_arg = c("-s_srs", "EPSG:4326", "-te", "-4000000", "-4000000",
                    "4000000", "4000000", "-tr", "25000", "25000",
                    "-r", "bilinear", "-wo", "SOURCE_EXTRA=2", "-overwrite"),
         quiet = TRUE)
    r$close(); view$close()
    out <- new(GDALRaster, dst)
    v <- out$read(1, 0, 0, 320, 320, 320, 320)
    out$close(); vsi_unlink(dst)
    v
  }, numeric(320 * 320))
}

worker_agg <- function(ym, worker_fun) {
  days <- unlist(lapply(ym, \(s) {
    d0 <- as.Date(paste0(s, "-01"))
    format(seq(d0, seq(d0, by = "month", length.out = 2)[2] - 1, by = "day"))
  }))
  X <- worker_fun(days)
  key <- substr(days, 1, 7)
  out <- vapply(ym, \(s) rowMeans(X[, key == s, drop = FALSE]), numeric(nrow(X)))
  list(ym = ym, n = as.integer(table(key)[ym]), mean = out)
}

ym <- format(seq(as.Date("1979-01-01"), as.Date("2021-12-01"), by = "month"), "%Y-%m")
cl <- makeCluster(32)
idx <- split(seq_along(ym), rep_len(seq_len(32), length(ym)))
res <- parLapply(cl, idx, \(i, f, w, y) f(y[i], w), worker_agg, worker_fun, ym)
stopCluster(cl)

monthly <- matrix(NA_real_, 320 * 320, length(ym), dimnames = list(NULL, ym))
nday <- integer(length(ym)); names(nday) <- ym
for (r in res) { monthly[, r$ym] <- r$mean; nday[r$ym] <- r$n }
saveRDS(list(ym = ym, ndays = nday, monthly_mean_K = monthly),
        "era5_t2m_3031_monthly_daily12utc.rds")

The land-sea mask is a single static field, so it is a one-off read (a few seconds):

library(gdalraster)
set_config_option("GDAL_DISABLE_READDIR_ON_OPEN", "EMPTY_DIR")
ds <- new(GDALMultiDimRaster, era5)   # `era5` is the ZARR:"/vsicurl/..." DSN above
arr <- ds$openMDArray("/land_sea_mask")  # dims: latitude, longitude
r <- arr$getUnscaled()$asClassicDataset(1, 0, NULL)
dst <- "/vsimem/lsm_3031.tif"
push_error_handler("quiet")
warp(r, dst, t_srs = "EPSG:3031",
     cl_arg = c("-s_srs", "EPSG:4326", "-te", "-4000000", "-4000000",
                "4000000", "4000000", "-tr", "25000", "25000",
                "-r", "bilinear", "-wo", "SOURCE_EXTRA=2", "-overwrite"),
     quiet = TRUE)
pop_error_handler()
r$close(); arr$close(); ds$close()
o <- new(GDALRaster, dst)
lsm <- o$read(1, 0, 0, 320, 320, 320, 320)
o$close(); vsi_unlink(dst)

The monthly means are then reduced to seasonal means, which is all the analysis needs. The days sampled in each month are used as weights, December is assigned to the following year’s DJF (so DJF 1979, which would lack December 1978, is dropped), and a season-year is kept only if all its months are present. Land pixels are stored at full seasonal resolution as integers in units of 0.01 K, a rounding that changes the land-mean trends by less than 1e-5 K/decade; the whole-disc mean of each season-year is stored at full precision. The cache is about 6 MB, against 220 MB for the monthly stack.

# `monthly`, `nday`, `ym` come from the extraction chunk; `lsm` from the mask chunk
xc <- -4e6 + 12500 + 25000 * (0:319)
grid <- expand.grid(x = xc, y = rev(xc))
inside <- sqrt(grid$x^2 + grid$y^2) <= 3.3e6
land_in <- (lsm > 0.5)[inside]

mo <- as.integer(substr(ym, 6, 7))
yr <- as.integer(substr(ym, 1, 4))
season_of <- c("DJF", "DJF", "MAM", "MAM", "MAM", "JJA",
               "JJA", "JJA", "SON", "SON", "SON", "DJF")[mo]
season_year <- ifelse(mo == 12, yr + 1, yr)

# Weighted mean over the months of each complete season-year
aggregate_seasons <- function(sel, key, n_months) {
  years <- sort(unique(key[sel]))
  Y <- lapply(years, \(y) {
    k <- which(sel & key == y)
    if (length(k) != n_months) return(NULL)
    as.vector(monthly[inside, k, drop = FALSE] %*% nday[k] / sum(nday[k]))
  })
  keep <- !vapply(Y, is.null, logical(1))
  list(years = years[keep], Y = do.call(cbind, Y[keep]))
}

seasons <- c("DJF", "MAM", "JJA", "SON")
sm <- c(
  setNames(lapply(seasons, \(s) aggregate_seasons(season_of == s, season_year, 3)), seasons),
  list(Annual = aggregate_seasons(rep(TRUE, length(ym)), yr, 12))
)

small <- list(
  meta = list(first_month = ym[1], last_month = ym[length(ym)], n_months = length(ym),
              total_days = sum(nday), missing_in_disc = sum(is.na(monthly[inside, ])),
              disc_pixels = sum(inside), land_pixels = sum(land_in), pixel_km2 = 625),
  land_fraction = lsm,
  seasons = lapply(sm, \(z) list(
    years = z$years,
    Y_cK = `storage.mode<-`(round(z$Y[land_in, , drop = FALSE] * 100), "integer"),
    disc_mean_K = colMeans(z$Y)))
)
saveRDS(small, "era5_t2m_3031_seasonal_land.rds", compress = "xz")

Load and check the cache

library(ggplot2)
small <- readRDS("era5_t2m_3031_seasonal_land.rds")
meta <- small$meta

# pixel centres of the 320 x 320 grid; rows are read top-down
xc <- -4e6 + 12500 + 25000 * (0:319)
grid <- expand.grid(x = xc, y = rev(xc))
# disc of radius 3300 km, roughly south of 60S (includes ocean and sea ice)
inside <- sqrt(grid$x^2 + grid$y^2) <= 3.3e6
# land = ice sheet, ice shelves and islands (ERA5 land fraction above 0.5)
land <- small$land_fraction > 0.5
grid <- grid[inside, ]
land_in <- land[inside]

# seasonal means (K) at land pixels, and the whole-disc mean per season-year
sm <- lapply(small$seasons, \(z) list(years = z$years,
                                       Y = z$Y_cK / 100,
                                       disc_mean = z$disc_mean_K))
stopifnot(sum(land_in) == meta$land_pixels, sum(inside) == meta$disc_pixels)
c(land_pixels = nrow(sm$DJF$Y),
  vapply(sm, \(z) length(z$years), integer(1)))
land_pixels         DJF         MAM         JJA         SON      Annual 
      21857          42          43          43          43          43 

The cache was built from 516 monthly means (1979-01 to 2021-12) of 15,706 daily fields, with 0 missing values inside the analysis disc. Of the 54,756 pixels in the disc, 21,857 are land, which is about 13.7 million km^2 at 625 km^2 per pixel (a sanity check against Antarctica’s area including ice shelves, roughly 14 million km^2).

ggplot(cbind(grid, land = land_in), aes(x, y, fill = land)) +
  geom_raster() +
  coord_equal() +
  labs(title = "ERA5 land mask on the 25 km polar grid", x = NULL, y = NULL)

Polar stereographic map of Antarctica as a filled shape on a circular ocean background, with the Peninsula reaching up and left

ERA5 land mask on the 25 km polar grid: land (ice sheet, ice shelves, islands) against ocean, within 3300 km of the pole.

Trend maps over land

The maps show Antarctic land only, with the OLS slope. Pixels with |t| < 2 are shown in grey. With about 21,857 strongly spatially correlated pixels per map, and years treated as independent, this is a descriptive filter and not a test with controlled error.

ggplot(trends, aes(x, y, fill = slope_sig)) +
  geom_raster() +
  scale_fill_gradient2(name = "K/decade", low = "blue", high = "red",
                       na.value = "grey85") +
  facet_wrap(~season) +
  coord_equal() +
  labs(title = "ERA5 2 m temperature trend over land, 1979-2021 (daily 12 UTC)",
       subtitle = "OLS slope; grey: |t| < 2", x = NULL, y = NULL)

Five polar maps of Antarctica (DJF, MAM, JJA, SON, annual) coloured blue to red by temperature trend, mostly grey with coloured patches that are largest in SON

Per-pixel OLS trend in 2 m temperature over Antarctic land, 1979-2021, by season and annual. Grey pixels have |t| < 2.
land_tr <- trends
frac_sig <- tapply(abs(land_tr$t) > 2, land_tr$season, mean)
round(100 * frac_sig, 1)
   DJF    MAM    JJA    SON Annual 
  17.2    9.2    3.1   50.6   24.5 

The share of land pixels with |t| > 2 ranges from 3% (JJA) to 51% (SON). About 5% would be expected by chance alone under independence (JJA is below that level). The spatial patches are what carry the information, and they should not be read pixel by pixel.

Theil-Sen check, per pixel

If a few anomalous years drove the OLS slopes, the Theil-Sen slopes would differ from them. The plot compares the two on every land pixel.

ggplot(land_tr, aes(slope, theil_sen)) +
  geom_point(size = 0.2, alpha = 0.2) +
  geom_abline(colour = "red") +
  facet_wrap(~season) +
  coord_equal() +
  labs(x = "OLS slope (K/decade)", y = "Theil-Sen slope (K/decade)")

Five scatterplots of Theil-Sen slope against OLS slope, with points clustered tightly along the one-to-one line

Theil-Sen against OLS slope on every land pixel, by season. The red line is 1:1.
agree <- do.call(rbind, lapply(split(land_tr, land_tr$season), \(z) {
  data.frame(correlation = cor(z$slope, z$theil_sen),
             median_abs_diff = median(abs(z$slope - z$theil_sen)),
             median_ols = median(z$slope), median_theil_sen = median(z$theil_sen))
}))
knitr::kable(agree, digits = 3,
             col.names = c("Correlation", "Median |OLS - TS|",
                           "Median OLS", "Median TS"))
Correlation Median |OLS - TS| Median OLS Median TS
DJF 0.983 0.021 0.096 0.100
MAM 0.967 0.040 0.009 0.021
JJA 0.946 0.046 -0.011 0.018
SON 0.987 0.030 0.345 0.320
Annual 0.971 0.024 0.103 0.112

Across land pixels the two estimators are highly correlated (correlation of 0.95 or more in every season), and the median absolute difference between them is at most 0.05 K/decade. So the OLS maps are not being driven by a handful of extreme years.

Continental mean series

The headline series is the unweighted mean over land pixels. For comparison, the mean over the whole disc (which includes ocean and sea ice) is also shown.

series <- do.call(rbind, lapply(names(sm), \(s) {
  rbind(
    data.frame(year = sm[[s]]$years, season = factor(s, names(sm)), domain = "Land",
               t2m = colMeans(sm[[s]]$Y) - 273.15),
    data.frame(year = sm[[s]]$years, season = factor(s, names(sm)), domain = "Whole disc",
               t2m = sm[[s]]$disc_mean - 273.15)
  )
}))

ggplot(subset(series, domain == "Land"), aes(year, t2m)) +
  geom_line() +
  geom_smooth(method = "lm", se = FALSE) +
  facet_wrap(~season, scales = "free_y") +
  labs(title = "Land mean 2 m temperature", y = "2 m temperature (degC)", x = NULL)
`geom_smooth()` using formula = 'y ~ x'

Five line plots of land-mean temperature against year with fitted straight lines, rising most clearly in SON

Land-mean 2 m temperature by season and annual, 1979-2021, with OLS trend lines.
fit <- \(z) {
  m <- lm(t2m ~ I(year / 10), z)
  c(ols = unname(coef(m)[2]), se = unname(coef(summary(m))[2, 2]),
    theil_sen = theil_sen(z$t2m, z$year))
}
tab_of <- \(d) t(sapply(split(d, d$season), fit))
land_all <- tab_of(subset(series, domain == "Land"))
land_80 <- tab_of(subset(series, domain == "Land" & year >= 1980))
disc_all <- tab_of(subset(series, domain == "Whole disc"))

tab <- data.frame(
  season = rownames(land_all),
  land_ols = land_all[, "ols"], land_se = land_all[, "se"],
  land_ts = land_all[, "theil_sen"],
  land_ols_1980 = land_80[, "ols"], land_ts_1980 = land_80[, "theil_sen"],
  disc_ols = disc_all[, "ols"], disc_ts = disc_all[, "theil_sen"],
  row.names = rownames(land_all)
)
knitr::kable(tab, digits = 3, row.names = FALSE,
             col.names = c("Season", "Land OLS", "SE", "Land Theil-Sen",
                           "Land OLS from 1980", "Land TS from 1980",
                           "Disc OLS", "Disc TS"))
Season Land OLS SE Land Theil-Sen Land OLS from 1980 Land TS from 1980 Disc OLS Disc TS
DJF 0.113 0.080 0.111 0.113 0.111 0.014 -0.001
MAM 0.023 0.112 0.092 0.023 0.110 -0.038 -0.022
JJA -0.004 0.137 0.024 -0.070 -0.022 -0.054 -0.066
SON 0.347 0.066 0.343 0.353 0.346 0.161 0.157
Annual 0.123 0.058 0.144 0.107 0.133 0.023 0.030

All trends are in K/decade. The land mean trend is largest in SON, at 0.35 (OLS, standard error 0.07, about 5.2 standard errors from zero) and 0.34 by Theil-Sen. The annual land trend is 0.12 (about 2.1 standard errors, borderline given the naive standard errors), DJF is 0.11 (1.4 standard errors), and MAM and JJA are indistinguishable from zero. The land-mean Theil-Sen and OLS slopes differ by at most 0.07 K/decade across seasons, well inside the OLS standard errors, so the series-level conclusions do not depend on the estimator.

The land mean trends are larger than the whole-disc trends in DJF, SON and the annual mean (for example SON 0.16 over the disc against 0.35 over land). This is consistent with weaker trends over the ocean and sea-ice part of the disc diluting the disc mean, although the ocean-only mean was not computed separately here. Dropping 1979 changes the JJA land OLS estimate from -0.00 to -0.07 K/decade, which is small relative to its standard error of 0.14.

Caveats

  • These are reanalysis trends. Antarctic 2 m temperature in ERA5 depends on the model and on the observations assimilated, which are sparse and changed over time (especially before 1979, which is why the period starts there).
  • Sampling is one field per day at 12 UTC, not full daily means, so the diurnal cycle is not averaged out. A fixed hour is unlikely to bias trends much but adds noise.
  • The land mask is ERA5’s own land fraction (above 0.5) warped bilinearly to 25 km. It treats ice shelves as land and has coarse (0.25 degree) coastlines, so coastal pixels mix land and ocean influence. It is not a survey-grade ice-sheet outline.
  • Pixels are weighted equally on the 25 km EPSG:3031 grid, which is only approximately equal-area (a scale error of about 1.5% at 60 degrees S).
  • The t statistics assume independent years, ignore autocorrelation and spatial correlation, and are not corrected for the number of pixels. Treat maps as descriptive. Theil-Sen is used here as a robustness check on the slope, not to provide a significance test.
  • The time index (hours since 1959-01-01) was assumed from the array layout, and the 2022 index is not available in the array, so the series stops at 2021.

Reproducibility

The extraction chunks need a GDAL build with the multidimensional API and Zarr driver, the development build of gdalraster with GDALMultiDimRaster views, and network access to the public Google Cloud bucket. The rest of the document only needs the roughly 6 MB cache file era5_t2m_3031_seasonal_land.rds next to this file.

sessionInfo()
R version 4.6.1 (2026-06-24)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 24.04.5 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so;  LAPACK version 3.12.0

locale:
 [1] LC_CTYPE=en_AU.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_AU.UTF-8        LC_COLLATE=en_AU.UTF-8    
 [5] LC_MONETARY=en_AU.UTF-8    LC_MESSAGES=en_AU.UTF-8   
 [7] LC_PAPER=en_AU.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C       

time zone: Etc/UTC
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] ggplot2_4.0.3

loaded via a namespace (and not attached):
 [1] Matrix_1.7-6       gtable_0.3.6       jsonlite_2.0.0     dplyr_1.2.1       
 [5] compiler_4.6.1     tidyselect_1.2.1   parallel_4.6.1     dichromat_2.0-1   
 [9] splines_4.6.1      scales_1.4.0       yaml_2.3.12        fastmap_1.2.0     
[13] lattice_0.23-1     R6_2.6.1           labeling_0.4.3     generics_0.1.4    
[17] knitr_1.52         htmlwidgets_1.6.4  tibble_3.3.1       pillar_1.11.1     
[21] RColorBrewer_1.1-3 rlang_1.3.0        xfun_0.60          S7_0.2.2          
[25] otel_0.2.0         cli_3.6.6          withr_3.0.3        magrittr_2.0.5    
[29] mgcv_1.9-4         digest_0.6.39      grid_4.6.1         rstudioapi_0.19.0 
[33] rappdirs_0.3.4     lifecycle_1.0.5    nlme_3.1-171       vctrs_0.7.3       
[37] evaluate_1.0.5     glue_1.8.1         farver_2.1.2       rmarkdown_2.32    
[41] tools_4.6.1        pkgconfig_2.0.3    htmltools_0.5.9