Quarto 1.10 is dominated by fixes, as much of our development effort is now going into Quarto 2, but there are some improvements we wanted to point out: more refined HTML accessibility checks, and access to Quarto’s localized strings for template authors.

You can read about these improvements and the most important fixes below. You can find all the changes in this version in the Release Notes.

Accessibility Checking Improvements

Since Quarto 1.8, setting the axe option on HTML documents runs axe-core accessibility checks on your rendered document. Quarto 1.10 makes these checks easier to rely on.

Checking works offline. Quarto now bundles its own copy of axe-core instead of loading it from a CDN in the reader’s browser. Accessibility checking works without a network connection, and viewing a rendered document no longer triggers a request to an external CDN. The axe-core version is unchanged, so results are identical.

Check against a WCAG conformance level. The new standard option scopes checks to a WCAG conformance level, including rules axe-core keeps off by default for that level. The best-practice option controls whether axe’s best-practice rules, recommendations not required by any WCAG success criterion, are also checked:

format:
  html:
    axe:
      standard: wcag21aa
      best-practice: true

A clearer report. When using output: document, violations are now listed first by impact, then WCAG conformance level. The report overlay also received its own accessibility and styling fixes: it can be scrolled with the keyboard, uses theme and brand independent colors, and no longer inherits page styling like text centering.

Learn more at HTML Accessibility.

Localized Strings in Templates

If you write custom Pandoc templates or template partials, you can now use Quarto’s localized strings through the new quarto.language template variable namespace. Quarto resolves each string for the document’s language and exposes it as a template variable:

$quarto.language.toc-title-document$
$quarto.language.crossref-fig-title$

This lets templates honor the document’s lang option without hard-coding text. For example, the orange-book Typst book extension now uses these variables in its running headers, so a book with lang: fr gets Chapitre rather than Chapter.

Learn more at Localized Strings in Templates.

Other Important Fixes

• quarto preview reliability—a long-standing bug that showed stale HTML for non-index pages is fixed, changes to a document’s format: are detected on the first render after the edit, a _brand.yml added or removed during preview is picked up, and transient .quarto_ipynb files no longer accumulate on disk.

• Shortcodes—shortcodes are now resolved inside inline and display math expressions.

• PDF font fallbacks—mainfontfallback, sansfontfallback, and monofontfallback no longer crash LuaLaTeX on TeX Live 2026, and a missing fallback font is now installed automatically.

• Typst fonts—fonts not installed locally are filtered from CSS font-family fallback lists before they reach Typst, suppressing unknown font family warnings, alongside a batch of fixes to Quarto’s CSS-to-Typst translation.

• Chrome Headless Shell—the quarto install chromium command, deprecated in 1.9, now transparently redirects to chrome-headless-shell, quarto check install warns about legacy Chromium installs, and arm64 Linux is now supported.

• Output for LLMs—fixes for websites with llms-txt enabled: headings and shortcodes inside conditional content, and clean-URL rewriting of index.html.md links.

Dependency updates:

• pandoc updated to 3.10
• typst updated to 0.15.1
• deno updated to 2.7.14
• dart-sass updated to 1.101.0
• esbuild updated to 0.28.1

Acknowledgements

We’d like to say a huge thank you to everyone who contributed to this release by opening issues and pull requests:

AlexLietard, ArthurRudolph, Artmann, C-Monaghan, ChrisJefferson, CoryMcCartan, DCEW, DrFloLinke, Guest-1013, MBe-iUS, MurzNN, ThierryO, ThomasFaria, TinasheMTapera, Voorhoeve, alderete, andrewheiss, barendgehrels, basm92, blauzo, chainsawriot, crisbour, cs-res-pub-ser-st, dustinstoltz, eculler, eneveu, github-actions[bot], gregswinehart, hwine, ianpittwood, ihrke, jdonaldson, jiangyun-fun, jidanni, jkrumbiegel, jnkatz, jph00, jtkulas, juleswg23, juliohm, kazuyanagimoto, kelli-rstudio, lsbjordao, luismmontilla, maelle, maucejo, memeplex, micedre, multimeric, nathant181, neilernst, nessan, nickvigilante, nrennie, pbosetti, reckoner, rgouveiamendes, robjhyndman, sbwiecko, seandavi, sebastiansauer, skyfroger, songwupei, stefkuypers, stragu, tiagojct, victorrssx, widlarizer, wlatendresse, xuefeng-xu, yasyf, zinc75.

The hammer and wrench emoji in the listing and social card image for this post comes from OpenMoji– the open-source emoji and icon project. License: CC BY-SA 4.0

The log summarizes findings for different areas of exploration and keeps track of next steps. To use the log, run the /eda-log slash command after you’ve started the EDA process.

Why we made this

Exploratory data analysis, the open-ended orientation to your data that often comes before anything else, can be a branching, nonlinear process. There are many questions you can ask of your data, and new areas of inquiry can open with each question you ask. Because of this, it is often hard to keep track of what you’ve looked into, where that code lives, and what you want to explore next.

Historically, the EDA process was limited by how quickly you could write code and interpret the output. Coding agents like Posit Assistant lift the first of those constraints. They can carry out EDA far faster than you can on your own, which can exacerbate the issue of keeping track of what you’ve explored.

This speed also introduces a new problem: the point of EDA is typically for you, the human, to understand your data, but coding agents can produce output faster than you can absorb it. If the agent completes an analysis but you haven’t understood the insights in the data, the exploration process hasn’t really happened.

Posit Assistant already has various features that tackle this problem, including an exploratory mode of interaction where it runs shorter turns and stops more frequently to involve the user.

The EDA log is another lightweight tool for the same goal. It keeps a running summary of what you and Posit Assistant have explored, helping your understanding keep pace with the agent’s and giving you a clearer picture of what’s already been done.

Details

Here’s what the EDA log looks like in action:

At a high level:

• When you run /eda-log, Posit Assistant will create a log for the exploration done in the conversation so far. The log then opens in the editor area in Positron.
• The underlying logs are stored as YAML files in .posit/assistant/eda-logs/, next to where plans are stored.
• Posit Assistant is instructed to loosely keep the log up to date as the conversation progresses, but you can also manually trigger an update at any time with the “Refresh” button.
• Clicking the arrow next to an area scrolls you back to the spot in the conversation where that insight originated, so you can revisit the code and context that produced it.
• Suggested next steps appear as clickable text. Clicking one sends it to Posit Assistant as your next message.
• The creation of an EDA log is always user-triggered. Posit Assistant will never create one on its own.
• The feature is currently only in Positron, but will come to RStudio soon.

Recent past newsletters

• Which models are best at spotting data quality problems?
• How to choose between AGENTS.md, skills, and MCP servers

Subscribe via RSS

Data scientists are often forced to choose between the simplicity of the classic Jupyter notebook experience and the advanced capabilities of a general-purpose IDE. With the 2026.07 release, you no longer have to choose: Positron now delivers a first-class Notebook Editor inside an IDE built for data science.

If you’re new to Positron, here’s what the Notebook Editor is like in your daily work.

Environment management

You clone your colleague’s repo and try to rerun their notebook. Which Python version do you use? What dependencies do you install and which versions? Did you previously set up an environment that you could reuse? You’re faced with all of these questions before you’ve run a single line of code.

Positron has several built-in workflows to help you manage your environments. When you open a repo, Positron discovers your installed environments, suggests the right one for the repo, and prompts you to set up a new environment if needed. The active environment is displayed front and center; click to restart or switch environments. Positron uses the same environment across your Jupyter and Quarto notebooks, scripts, and consoles. You can see inside your environments, which packages are installed, their versions, whether newer versions are available, and you can upgrade them. Environments are first-class citizens of Positron. Explore Positron’s environment management.

Interactive data exploration

When you run a cell in your notebook, your variables appear in the Variables Pane. Filter, sort, and search your data in the Data Explorer, and try out plot variations in the Visualize wizard. You can easily convert your point-and-click interactions into code by using the copy-code feature to bring them back into your notebook. If you get stuck, look up library documentation in the Help Pane. All of this works out of the box.

Streamlined version control

Notebooks store outputs and execution metadata alongside code, which is great for sharing but does not work well with version control. Positron includes settings to exclude outputs and execution metadata from the saved file, so the diff is just your code change. Explore the custom settings to clean up your git diffs.

Your notebooks stay ordinary .ipynb files and the editing experience stays classic Jupyter. When a plain-text workflow suits you better, conversion to and from .ipynb and other formats is built in.

AI assistance

Posit Assistant sees your notebook as more than just text. It works with the live session behind the notebook: your variables, your data, your plots. Ask it to fix a chart and it can inspect the dataframe, look at the plot itself, edit the cell, and run it again. It does the same things you would do, using the same panes you use. Explore notebook-aware AI assistance in Positron.

Get started

Download Positron and open a project with a Jupyter notebook. The Notebook Editor is now on by default. If you’re coming from another IDE, your existing .ipynb files will open as-is without any conversion needed.

For the full feature reference, check out our documentation.

If you run into any issues or have ideas on how we can improve Positron, reach out on GitHub.

Already using Positron? Everything in this release is in the release notes.

Thank you so much for helping us build Positron’s Jupyter notebook support from the ground up by chatting with us live, sharing your pain points, and giving us feedback on GitHub!

The problem is that many packages store the tree’s splits in different ways or offer incompatible APIs to access different characteristics. lorax helps capture this information for many different implementations.

You can install the CRAN version of lorax via

install.packages("lorax")

or get the development version using

pak::pak("tidymodels/lorax")

Let’s start by using the ranger package to create a random forest model using the food delivery data in the modeldata package. For illustration, the trees will be coerced to be more shallow than usual so that we can better plot them using the min.node.size argument:

library(tidymodels) # <- to easily get dplyr, tidyr, ggplot2, etc
library(ranger)
library(lorax)

set.seed(872)
rgr_fit <-
  ranger(
    time_to_delivery ~ .,
    data = modeldata::deliveries,
    num.trees = 1000,
    min.node.size = 5000,
    importance = "impurity"
  )

rgr_fit

Ranger result

Call:
 ranger(time_to_delivery ~ ., data = modeldata::deliveries, num.trees = 1000,      min.node.size = 5000, importance = "impurity") 

Type:                             Regression 
Number of trees:                  1000 
Sample size:                      10012 
Number of independent variables:  30 
Mtry:                             5 
Target node size:                 5000 
Variable importance mode:         impurity 
Splitrule:                        variance 
OOB prediction error (MSE):       24.26389 
R squared (OOB):                  0.4879119 

Visualizing the Trees

lorax contains methods for the as.party() function in the partykit package. This enables us to use all of the methods from that package. For example, plot.party() is an excellent visualization tool for the tree. Random forest has many trees and we can plot any of them:

rgr_fit |>
  as.party(tree = 1, data = modeldata::deliveries) |>
  plot()

rgr_fit |>
  as.party(tree = 100, data = modeldata::deliveries) |>
  plot()

rgr_fit |>
  as.party(tree = 1000, data = modeldata::deliveries) |>
  plot()

Which Predictors Were Used?

Since trees automatically conduct feature selection as the model is trained, it helps to know which ones are actually used by the model. The active_predictors() function does just that:

rgr_vars <- active_predictors(rgr_fit, tree = 1:1000)
rgr_vars

# A tibble: 1,000 × 2
   active_predictors  tree
   <list>            <int>
 1 <chr [4]>             1
 2 <chr [5]>             2
 3 <chr [3]>             3
 4 <chr [4]>             4
 5 <chr [4]>             5
 6 <chr [3]>             6
 7 <chr [3]>             7
 8 <chr [7]>             8
 9 <chr [3]>             9
10 <chr [6]>            10
# ℹ 990 more rows

# The column names are in a nested vector:
rgr_vars$active_predictors[[1]]

[1] "day"     "hour"    "item_10" "item_23"

# We can expand the list too:
rgr_vars |>
  unnest(cols = c(active_predictors))

# A tibble: 4,306 × 2
   active_predictors  tree
   <chr>             <int>
 1 day                   1
 2 hour                  1
 3 item_10               1
 4 item_23               1
 5 day                   2
 6 distance              2
 7 item_10               2
 8 item_12               2
 9 item_26               2
10 hour                  3
# ℹ 4,296 more rows

How often are predictors used?

rgr_vars |>
  unnest(cols = c(active_predictors)) |>
  count(active_predictors) |>
  arrange(active_predictors)

# A tibble: 30 × 2
   active_predictors     n
   <chr>             <int>
 1 day                 516
 2 distance            507
 3 hour                623
 4 item_01             329
 5 item_02             131
 6 item_03              57
 7 item_04              66
 8 item_05              26
 9 item_06              92
10 item_07              91
# ℹ 20 more rows

How many predictors are used in each rule (on average)? A rule is the full logical statement that defines the path to the terminal nodes.

rgr_vars |>
  mutate(num_vars = map_int(active_predictors, ~ length(.x))) |>
  summarize(mean_num_vars = mean(num_vars))

# A tibble: 1 × 1
  mean_num_vars
          <dbl>
1          4.31

Many packages can compute variable importance scores for each predictor but each has a different interface. The lorax package has an accessor function to pull these from the model called var_imp():

var_imp(rgr_fit)

# A tibble: 30 × 2
   term     estimate
   <chr>       <dbl>
 1 hour     108892. 
 2 day       16054. 
 3 distance  25689. 
 4 item_01     673. 
 5 item_02      86.3
 6 item_03      30.4
 7 item_04      43.7
 8 item_05      11.0
 9 item_06      50.1
10 item_07      55.2
# ℹ 20 more rows

Examining Rules

We can also get detailed information on the model’s rules (for each tree). Using the same ranger model fit:

rgr_rules <- extract_rules(rgr_fit, data = modeldata::deliveries)
rgr_rules

# A tibble: 5 × 3
     id rules       tree
  <int> <list>     <int>
1     3 <language>     1
2     5 <language>     1
3     7 <language>     1
4     8 <language>     1
5     9 <language>     1

# With components
rgr_rules$rules[[1]] |> class()

[1] "call"

rgr_rules$rules[[1]]

item_23 <= 0.5 & hour <= 14.6645

We can compute on these and/or use them to determine which specific data were contained in the terminal node:

modeldata::deliveries |> filter(!!rgr_rules$rules[[1]])

# A tibble: 2,415 × 31
   time_to_delivery  hour day   distance item_01 item_02 item_03 item_04 item_05
              <dbl> <dbl> <fct>    <dbl>   <int>   <int>   <int>   <int>   <int>
 1             16.1  11.9 Thu       3.15       0       0       2       0       0
 2             19.6  13.0 Sat       3.35       1       0       0       1       0
 3             17.4  11.9 Sun       2.75       0       2       1       0       0
 4             18.0  12.1 Tue       2.4        0       0       0       1       0
 5             22.1  14.4 Thu       2.69       0       0       0       0       0
 6             17.6  12.9 Sat       2.47       0       1       0       0       0
 7             17.0  12.3 Sat       3.88       0       0       0       0       0
 8             19.5  13.5 Tue       3.55       0       0       0       0       0
 9             17.6  12.9 Fri       2.88       0       0       1       1       0
10             21.6  14.3 Sat       3          0       0       0       1       0
# ℹ 2,405 more rows
# ℹ 22 more variables: item_06 <int>, item_07 <int>, item_08 <int>,
#   item_09 <int>, item_10 <int>, item_11 <int>, item_12 <int>, item_13 <int>,
#   item_14 <int>, item_15 <int>, item_16 <int>, item_17 <int>, item_18 <int>,
#   item_19 <int>, item_20 <int>, item_21 <int>, item_22 <int>, item_23 <int>,
#   item_24 <int>, item_25 <int>, item_26 <int>, item_27 <int>

A List of Methods

Here are the details of which models and methods are supported in the first CRAN version:

Note that as.party.rpart() is in the partykit package and that cforest is made out of party objects.

What’s Next?

We’ll work on adding CatBoost models to the list of supported methods. Please add an issue if there are other aspects of trees that should be quantified in these models.

Today we are announcing the first public release of ir, a small command-line tool that runs R scripts and renders Quarto documents using runtime requirements declared in the file itself.

ir is for one-file workflows that do not quite need a project but still need to be easy to share and rerun later. Simply put the package requirements, and optionally the R version, next to the code. When you render or execute the file, ir resolves the requirements, prepares cached package libraries, and launches R or Quarto with a runtime ready to use.

The interface is inspired by PEP 723 and uv run --script. A script can carry enough metadata to describe its runtime, and the runner can resolve that runtime into a cached environment on demand. ir brings that pattern to R scripts and Quarto documents, using pak, renv, and rig on the R side, and reticulate’s uv-backed helper when Python is part of the runtime.

ir focuses on two workflows:

• running or rendering self-describing scripts and documents
  (ir run, ir render)
• running or installing command-line tools distributed through R packages
  (rx, ir tool install)

Why ir?

R scripts often begin as small, local utilities: a report, a data pull, a model-training or evaluation run, a quick diagnostic, or an example shared with a colleague. Over time, the script can become important, but the setup still lives somewhere else: in a README, in a shell history, in a project library, or in the author’s current R installation.

ir makes the runtime specification part of the source file. That means a script can say, directly:

• which packages it needs
• which R should run it, when that needs to be explicit
• whether user libraries should be visible
• whether CRAN packages should be resolved as of a specific date

This should help you rerun the script reliably at a later date. Keeping the metadata in the file makes it less likely to be lost or fall out of sync with the code.

How ir fits with existing tools

ir sits alongside the R tools people already use and builds on several of them directly:

• rig installs, removes, and switches between R versions on macOS, Windows, and Linux. ir calls rig when a file requests a specific R version, or when date-only exclude-newer needs to select the latest R minor version available on that date. rig is optional for files that only declare packages.
• pak is a fast package installer with a built-in solver. ir uses pak to resolve the dependency graph from a file’s declared packages and fetch them from the appropriate repositories. pak is bootstrapped automatically on first use, so you do not need to install it separately.
• renv gives R projects isolated package libraries and lockfiles. ir uses renv’s global package cache to assemble reusable libraries without creating a renv project or lockfile.

ir builds on that stack for non-project workflows: resolving the runtime for a self-describing script or Quarto document, and running or installing command-line entry points distributed by R packages.

A self-describing R script

Here is a complete script:

#!/usr/bin/env -S ir run
#| packages:
#|   - dplyr>=1.0
#|   - tidyr
#| isolated: true
#| exclude-newer: "2024-01-15"

library(dplyr)
library(tidyr)

mtcars |> count(cyl, gear) |> pivot_wider(names_from = gear, values_from = n)

Run it with:

$ ir run script.R

Or, on macOS and Linux, make it executable and run it directly:

$ chmod +x script.R
$ ./script.R

The metadata block is YAML written in #| comments after an optional shebang. ir reads that metadata, resolves the declared packages with pak, materializes a package library with renv, and starts R with the resolved library at the front of .libPaths().

By default, user libraries remain visible as a fallback. Add isolated: true in the file, or use --isolated at the command line, to run without the user library.

Running R package tools with rx

The ir release also includes rx, a short alias for ir tool run that runs executables provided by R packages.

Package authors can expose command-line entry points through standard package subdirectories such as exec/ and bin/. Files in exec/ can be regular Rscript files, Rapp apps, or direct executable scripts. rx resolves the package, finds the requested executable, and runs it in an isolated library.

For example, this resolves the btw CLI from the btw R package on demand and runs btw --help:

$ rx btw --help

# same as:
$ ir tool run btw --help

For tools you use regularly, install them with ir tool install:

# run once
$ ir tool install btw

# now you only need
$ btw --help

Cached by design

The first run of a new dependency set does the normal work of resolving and installing packages. Later runs reuse cached resolutions and content-addressed package libraries when the same requirements are seen again.

That makes ir useful for both one-off and repeated command-line work. You can run a script, run an inline expression, render a report, or launch a package-provided executable without creating a project directory just to hold the dependency state.

ir also bootstraps its own resolver tooling on first use, so you do not need to pre-install pak or renv.

Reproducibility without a project

For R scripts that need more explicit reproducibility, exclude-newer is usually the first thing to reach for:

exclude-newer: 2024-01-15

You can also provide the same date at the command line:

$ ir run --exclude-newer 2024-01-15 script.R

This resolves packages from the Posit Package Manager snapshot for that date. When no other R selector is set, the same date also tells ir to select the latest R minor version available on that date. If you were writing with the current release of R and current CRAN packages, the date alone is usually enough.

Use r-version when the script really needs a specific installed R version or version range. This selection uses rig:

r-version: "4.3"

Together, these options let a file carry the important parts of its runtime requirements without needing a surrounding project. For reproducibility, most files should need only a list of packages and a date.

Quarto documents too

ir uses the same metadata model for Quarto documents. Put package metadata under an ir: key in the document YAML:

---
title: My report
ir:
  packages:
    - dplyr>=1.0
    - gt@1.0
  isolated: true
  exclude-newer: 2025-05-15
---

Then render with:

$ ir render report.qmd
$ ir render report.qmd --to pdf

When ir selects an R executable using --r-version, frontmatter ir.r-version, or date-only exclude-newer, it sets QUARTO_R so Quarto renders with that R. ir also seeds rmarkdown automatically for knitr-based renders unless you declare it yourself.

Python environments too

Some R scripts and Quarto documents also need Python. In an R script, declare the R packages and Python requirements in the same metadata block:

#!/usr/bin/env -S ir run
#| packages:
#|   - reticulate
#| python-packages:
#|   - pandas
#|   - matplotlib
#| python-version: "3.11"
#| exclude-newer: "2026-06-01"

library(reticulate)
pd <- import("pandas")

ir creates the environment with reticulate’s uv-backed environment helper, sets RETICULATE_PYTHON, and activates the environment for subprocesses. Declare reticulate under packages when the R script loads reticulate.

For Quarto, put Python metadata under the document’s ir: key. A knitr document that uses reticulate can mix R and Python requirements:

---
title: My report
ir:
  packages:
    - reticulate
  python-packages:
    - pandas
  exclude-newer: 2025-01-01
---

For a Quarto document that uses the Jupyter engine, the ir: metadata can contain only Python requirements:

---
title: My notebook
jupyter: python3
ir:
  python-packages:
    - matplotlib
    - pandas
  python-version: "3.11"
---

For Quarto renders, ir injects jupyter into the Python environment, passes the resolved interpreter to Quarto with QUARTO_PYTHON, and also sets RETICULATE_PYTHON so documents that use reticulate see the same interpreter. If the document uses the knitr engine and contains Python chunks, ir automatically adds reticulate to the R package manifest.

When Python metadata is present, exclude-newer is also used for Python environment resolution unless python-exclude-newer is set. Use python-exclude-newer when Python packages should use a different snapshot date from R packages.

Install ir

Install a pre-built binary on Linux or macOS:

$ curl -fsSL https://raw.githubusercontent.com/r-lib/ir/main/scripts/install.sh | sh

Install on Windows PowerShell:

> irm https://raw.githubusercontent.com/r-lib/ir/main/scripts/install.ps1 | iex

The installers download the latest GitHub release and install both ir and rx. You can also download release artifacts from the GitHub releases page or with gh release download.

You will also need R / Rscript; rig is required when selecting R by version or by date-only exclude-newer, and Quarto is required when rendering Quarto sources.

Learn more

The project is open source under the MIT license. To get started:

• Read the documentation: https://r-lib.github.io/ir/
• Browse the source: https://github.com/r-lib/ir
• Open an issue: https://github.com/r-lib/ir/issues

This is a first public release, and feedback is especially useful now. If you try ir on your own scripts or Quarto documents, we would like to hear which workflows feel natural, where the metadata model needs more room, and which command-line edges still need smoothing.

Huh. It mostly looks normal, except there’s a few points perfectly aligned with what looks to be a “fitted” line. You dig into it a bit more, and realize that the rows from one study site have their cholesterol values imputed. You set them to NA and go along your way.

Would today’s frontier LLMs catch such an oddity? We designed an LLM evaluation to help us answer this question. As it turns out, LLMs mostly miss these sorts of artifacts:

During exploratory or open-ended data analysis, Posit assistant “only runs a few bits of code at a time, then summarizes what it found and suggests next steps”. This is motivated by our stance that, for now, a data scientist should mostly keep pace with and understand what the agent is doing when analyzing data. This stance was initially informed by our observation that last year’s frontier models tended to see what they expected to see when visualizing data. While LLMs have since become much better at interpreting counterintuitive plots, bluffbench2 shows they still lag behind human data scientists in interpreting data visualizations. As such, we are still cautious on the prospect of highly autonomous data agents.

How the eval works

The eval harness is a relatively generic coding agent harness, similar to that of Claude Code or Posit Assistant. The agent has a tool to run R code in a persistent REPL and some vague prompting about data analysis:

You are an AI assistant embedded in the user’s data science IDE. You can read and modify files in the user’s workspace and execute R code in their active session, including rendering plots. Prioritize correctness and clear communication…

In each sample, the agent first carries out a few “lull” turns, making a couple plots and tables unrelated to the eval. Short user messages like “load in the csv in this folder” are decorated with “System Reminders” and other noise like that injected by popular agent harnesses.

After a few turns, the agent is asked to produce a data visualization that includes a subtle visual artifact that could feasibly result from a real data-generating process. The artifacts span a range of realistic data quality issues: stuck sensors, bad joins, points imputed onto a line, swapped columns, pseudoreplication, differing units, etc.

plot bmi vs cholesterol

Tool: Run R code

If the agent mentions the artifact in its follow-up response, it receives a full point. If the agent does not mention the artifact, it can also receive a half point by mentioning it in response to a follow-up user message along the lines of “what do you see in the plot?” If the agent never mentions the artifact, it is graded as incorrect.

Designing the eval

Once we understood the mechanism behind bluffbench, implementing the eval was relatively straightforward. bluffbench demonstrates the degree to which an LLM will ignore evidence shown in a plot in favor of its expectations. So, to implement a given sample, we’d just think of some situation that would elicit a strong prior and then subvert it. For example, a dataset called doug_firs with variables height and circumference; one might expect that, as height increases, so does circumference. So, instead, we did a transformation under the hood that made the relationship parabolic.

A year ago, triggering this prior was enough to frequently ’trick’ the current frontier LLMs.

Slipping a plotted artifact past today’s LLMs is much harder. Any human could ace bluffbench, but only an attentive data analyst would excel at bluffbench2.

In our early work on a successor to bluffbench, we started off with trying to elicit priors in the same way as bluffbench did, but in more realistic, longer-context scenarios. We were surprised to find that the same mechanism broadly doesn’t seem to trick today’s models even in these more realistic settings.¹ We then tried a ‘reverse bluffbench’, where we let the model being evaluated in on the trick, asking it to carry out the transformation itself and then look at the plotted result which was tampered with to show the original relationship. We anticipated that this stronger prior (“I did a thing with an obvious effect”) might cause the models to miss the (re)manipulation, but models reliably noted that the plot looked as if it hadn’t been manipulated.

As such, there isn’t a similar ’trick’ in bluffbench2 per se. The transcripts read like relatively normal data analysis sessions and the plotted artifacts are designed to plausibly result from real data-generating processes. Instead, the eval elicits 1) the ‘shape’ of LLMs’ vision being different than humans’ and 2) the model’s tendencies to perform progress, simulating a data analysis moving along smoothly.

Today’s frontier models are in the mid-teens at best; the top scores belong to Claude Fable 5 and Gemini 3.5 Flash at 16%. That said, we’d caution folks from interpreting the current scores on this eval as ‘LLMs don’t see plots well.’ The plotted artifacts are actually quite subtle, and when they’re made even a bit more marked, models tend to call them out consistently.

For example, the previous version of the scatterplot had a slightly more dense cluster of points:

• Previous
• Current

Opus 4.8 (medium) consistently got this sample right in the previous iteration.

Exploring the eval’s results

At least for now, there’s a loosely linear relationship between the cost to run the eval and the resulting score:

One of the most interesting learnings from examining the logs is a behavioral one. Even though we never request that LLMs introduce modeled results to plots, like fitted lines and confidence intervals with geom_smooth(method = "lm", se = TRUE), they sometimes do so anyway. For example:

In general, adding modeled results to data visualizations without first looking at data is bad practice; it makes it hard to see the data itself. In the eval, adding a modeled result like this seems to substantially lower the chances that the model will notice the plotted artifact:

More bluffbench

If you’d like to learn more about the bluffbench set of evals, take a look at these past posts:

• Introducing bluffbench: Writeup of the the original eval.
• LLMs interpret plots well, until expectations interfere: In-depth post on why models at the time didn’t perform well on bluffbench, as well as various interventions we tried to improve performance.
• LLMs are getting much better at interpreting counterintuitive plots: In spring 2026, bluffbench scores suddenly jumped as models improved.
• It’s (still) very bad to be wrong: Slides from our recent SciPy 2026 talk on building agents for correct, transparent, and reproducible data analysis in light of the bluffbench results.

If you work across Python and R or juggle multiple projects with different dependency needs, environment management is probably one of the most challenging parts of your workflow. Positron comes with several out-of-the-box ways to help simplify environment management. Here are a few tips to streamline your workflow.

1: Understand how Positron discovers your environment

Positron does not just look at your system PATH. For Python, it actively discovers venv, uv, pyenv, and conda environments. If Positron is not seeing your project’s virtual environment, you can add custom search locations via the python.interpreters.include setting, or trigger a manual rescan with Interpreter: Discover All Interpreters. Learn more about Python discovery in the Positron documentation.

For R, discovery works differently. Positron consults various sources to build the list of R interpreters. These include your PATH, R root folders based on specific operating systems, well-known executable locations, and on Windows the registry. You can customize your R discovery through a few settings including positron.r.customRootFolders and positron.r.customBinaries. Learn more about R discovery in the Positron documentation.

2. Use the Interpreter Selector

To begin your first session, click “Start Session” in the top right corner and select your preferred R or Python interpreter. Positron can run multiple R and Python interpreter sessions at once, but only one is ever the active session at a given moment. The Interpreter Selector always shows you the active session and its status (idle, busy, or shut down) and you can use it to switch between or start additional sessions.

3. Install Python with uv

If Positron does not find a usable Python on your machine, it will offer to install Python via uv to help streamline your setup. If you prefer to manage the installation yourself, you can disable uv with the python.allowUvPythonInstall setting. Check out our blog post exploring on-demand Python installation in Positron or explore configurations in our Python installation documentation.

4. Manage packages with the Packages Pane

The Packages Pane in Positron lets you manage the packages installed in your active session. Whether you use pip, uv, conda, pak, base R, or renv, you can browse installed packages and search package repositories. You can also track outdated packages and install, update, or uninstall packages without leaving Positron or writing any code.

5. View package documentation

If you need to learn more about a specific package, the Packages Pane has buttons to navigate to the source documentation or link to the package’s website. You can also pull up the Help Pane for any reference in the Console using the ? operator, including both packages and functions.

6. Start projects with New Folder from Template

The New Folder from Template flow helps you start new projects faster. Instead of running multiple setup commands you can make a few selections and Positron helps you set up an environment directory, version control, directory structure, and an interpreter instance. Learn more about the Python and R templates available in the documentation.

7. Make use of the Command Palette

We recommend taking advantage of the following commands when managing your environments:

• Interpreter: Discover All Interpreters: Find new interpreters
• Interpreter: Select Session: Select a running interpreter session
• Python: Create Environment: Create a new virtual environment

8. Let Posit Assistant help manage your environment

Posit Assistant has knowledge of your R and Python session including the language, version, names, and types of variables in your environment. You can prompt Posit Assistant to set up environments and troubleshoot issues you run into.

Have an idea for how we can improve environment management in Positron? We would love to hear from you in a discussion on GitHub.

Get it now:

# install.packages("pak")
pak::pak("httr2")

The headline changes in this release all relate to OAuth token caching. They happen behind the scenes, so you shouldn’t need to change any code, but you’ll likely need to re-authenticate once after upgrading. The rest of this post explains why, and rounds up a handful of smaller features that landed in recent patch releases. See a full list of changes in the release notes.

OAuth token caching

First, a bit of background. When you authenticate with an API using OAuth, the API typically returns a refresh token that you can reuse for a set period. That means you don’t have to log in every single time you use the API, which makes for a much nicer experience. To reuse a token across R sessions, however, httr2 has to write it to disk, and that creates a risk: anyone who can read the token could use it to access the API as you.

httr2 takes every precaution it can to keep these tokens safe. They’re stored in a user-local cache directory, which is usually excluded from backups and, because it lives outside your working directory, carries no risk of being accidentally committed to git. On top of that, the tokens are encrypted so that only httr2 can read them.

Each token’s path on disk is derived from a hash computed with rlang::hash(). We recently discovered a bug in that hashing code, and fixing it changes every hash. As a result, httr2 can no longer find your existing cached tokens, so you’ll need to re-authenticate once after upgrading.

Since that change already invalidates your cached tokens, we took the opportunity to move where they’re stored. Previously we used the rappdirs package to locate the cache directory, but since R 4.0.0 there’s been a better built-in alternative: tools::R_user_dir("httr2", which = "cache"). We’ve switched to that, which makes httr2’s cache location consistent with other packages and removes the rappdirs dependency.

Making these changes surfaced a long-standing bug. Every time httr2 loads, it’s supposed to automatically delete any cached tokens older than 30 days, so that stale tokens don’t linger indefinitely. Unfortunately, that pruning never actually worked. Now it does, so old tokens will be cleaned up once they pass 30 days, regardless of which cache directory or hashing algorithm produced them. This is good security practice: the longer a credential sits on disk, the more opportunities there are for it to leak, so removing tokens you’re no longer using shrinks that window of exposure. If you’d like to prune every cached token yourself, you can call oauth_cache_prune(max_age_days = 0).

Other features

Alongside the token caching work, this release also rolls up a few smaller features that first appeared in recent patch releases:

• Faster streaming (1.2.3): resp_stream_lines(), resp_stream_sse(), and resp_stream_aws() now decode whole chunks at a time and hold the results in a queue, instead of rescanning and recopying the buffer for every line or event. This makes memory use and run time scale linearly rather than quadratically, so large streams are dramatically faster (e.g. reading a 1 MB response of short lines is now around 200x faster and uses around 180x less memory). resp_stream_sse() is used heavily by ellmer, so this should make your LLM streaming feel a little zippier.

• OAuth server metadata discovery (1.2.3): the new oauth_server_metadata() discovers an OAuth/OIDC issuer’s endpoints from its .well-known document, and oauth_client() gains a metadata argument so you can supply all of those endpoints at once.

• OpenTelemetry tracing (1.2.2): httr2 now emits OpenTelemetry traces for all requests when tracing is enabled. This requires the otelsdk package and is part of our cross-package work to improve the observability of R packages.

• httr2_translate() (1.2.3): this new function translates an httr2 request into the equivalent curl command. It’s handy for debugging, or for creating reprexes to share with people who don’t use R.

And this community is ours. From joining my very first useR! online in 2022, through to Salzburg in 2024 - SatRdays London in between, the R Project Sprint in Warwick (UK) in 2023, the R Dev Days since, and of course posit::conf(2024) Seattle, posit::conf(2025) Atlanta, and Japan.R 2025… I may not have been part of this community for the most number of years, but I’ve certainly appreciated its warmth and openness.

And so coming back to a European useR! was exciting.

Distributing state: CRDTs for real-time collaboration

The R community knows how to distribute compute - I’ve talked extensively on this subject, mainly surrounding my creation {mirai}, an async framework that brings high-performance parallel and distributed computing to R. This lets you send tasks just as easily to a Slurm cluster as to processes on your own machine.

My talk this time was going to be about something different, but complementary: distributing state. A CRDT, or Conflict-free Replicated Data Type, is a data structure that facilitates collaboration by having a property called strong eventual consistency. Without going into the details, this allows it to always merge conflict-free, and is the technology behind many collaborative text editors. Automerge, a particular CRDT implementation, is what we’re using in Quarto 2 to make it collaborative out-of-the-box.

In my talk, I showed a Shiny app in an R session editing a document on one of our Quarto collaborative sync servers. It used the {automerge} and {autosync} packages, which we created to let R manipulate these structures and sync them over the network - on a par with reference implementations in JavaScript and Rust. Automerge was created by Ink and Switch, an independent research lab for local-first software, and we’re glad to provide a link between the two communities.

Distributing compute: mirai and mori in the wild

Apart from talking a lot about CRDTs and the three killer features of Quarto 2 (orders of magnitude faster, collaborative editing built-in with live preview, editable in the preview and source views), I also got pulled aside by people in the hallways to talk about {mirai} and {mori}.

Some of these were people working in the life sciences industry, using these packages for serious scientific research and innovation. This has always been a sector that I’ve had immense respect for - from my earliest collaboration with Will Landau (Eli Lilly & Co.). Open source software is often regarded as ‘high-leverage’ in terms of how often and widely it’s used, and when deployed in such high-impact settings, the end goals can be especially motivating and rewarding.

It’s been extremely gratifying to see that {mirai} has percolated throughout the community consciousness. Considering that I was only introducing the package at useR! 2024 in Salzburg, it was nice to be able to introduce myself to people this time simply as “the author of mirai”. By 2027, I’m hoping this will also be possible with mori!

The best part: the people

Given that only a few of my colleagues could make it to useR! this year, I was slightly apprehensive going in. This fear was dispelled at the very first reception event, where I met so many people from past R Dev Days, amongst them Tina Roszos, who simply beamed at me from across the room! R Core also turned out in force and it was nice to meet Peter Dalgaard and Robert Gentleman for the first time, as well as saying hello again to Luke Tierney, Uwe Ligges and everyone who came.

It’s always great to catch up with familiar faces - Gergely Daróczi (maintainer of R’s Weblate platform), members of the mlr group (Martin Binder, Marc Becker, Maximilian Mücke), and many others. However, some of my best conversations this year were with people I had met for the first time. I was fortunate that for some, this was their first in-person useR!, having been part of the R community for many years.

It was especially nice to be welcomed by entire groups of people, for example the NUMBATs headed by Prof. Dianne Cook, who was also the opening keynote speaker of the conference. I got to be an honorary NUMBAT for one photo:

Contributing back: R Dev Day

R Dev Day followed the conference on the Friday. Thanks again to Heather Turner and Ella Kaye for tirelessly organising these. They are simply great opportunities to contribute back to R’s source code itself. I do believe that these events have played an important role in fostering and sustaining our community.

I worked closely with Tymek (Tymoteusz Makowski) on two C-level I/O bugs. Tymek was the only person brave enough to tackle these. This proved to be a fruitful collaboration as we’d posted a patch by the end of the day (PR #19101). This patch has since been accepted into the R source, without modification! It’s proved the most productive of any R Dev Day I’ve attended thus far.

Special thanks to Mitchell O’Hara-Wild for supplying the Dev Day group photograph, which I’m using as the banner image for this blog post.

Looking ahead

There are many people I’ve not mentioned by name - that’s just to prevent this entire post from being a name-drop. I’ve enjoyed speaking to every one of you. There is so much talent out there, and I look forward to catching up again and marvelling at all you’ve been working on!

See you next at posit::conf(2026) Houston!

Welcome back to another edition of our monthly Positron updates! Each month we share highlights from our latest release and useful resources. Last release we told you that several major features were on track to leave preview in July, and the time is now here! The new Notebook editor, the Packages pane, and Posit Assistant are all now generally available.

Positron Notebook Editor

Positron’s new Notebook editor is now the default experience for Jupyter (.ipynb) files. This release brings a long list of additions, including split-pane editing, cell tag management, and executing a line or selection within a cell with Cmd/Ctrl+Shift+Enter. You can export notebooks to Quarto, Python, or R, and inline PDF rendering makes it easier to work with generated output. If you are coming from JupyterLab, you will find familiar keyboard shortcuts, and we have improved output fidelity for Mermaid diagrams, htmlwidgets, and ipywidgets.

Posit has built on and for the Jupyter ecosystem for years now, and we are excited to have recently joined the Jupyter Foundation.

Packages pane

The Packages pane has also come out of preview. It gives you a live view of the R and Python packages installed in your active session, so you can search, install, update, and remove packages, and jump to their documentation, without leaving Positron.

This release also makes the pane more capable. Clicking a package opens a detail editor with its overview, metadata, and actions, similar to the Extensions pane. Installing a package searches results as you type, and each row has a button that opens the package’s website. Update indicators appear immediately on a new session from a cached snapshot, and Update All Packages reports exactly what changed. Package operations keep your environment consistent too, resolving against a workspace requirements.txt for Python and updating the renv.lock snapshot for R.

Posit Assistant

Posit Assistant, our unified, data-science-focused approach to AI assistance, is now out of preview and generally available; Posit Assistant fully replaces the legacy Positron Assistant. We know the names are similar, and Joe Cheng recently walked through a bit of the story behind how our AI agent tools have evolved.

This release also gives you finer control over AI in Positron. The new ai.enabled setting turns off every Positron AI feature at once, and administrators can enforce it; notebook.ai.enabled does the same for notebooks specifically. The set of language model providers keeps growing, with DeepSeek joining as an experimental provider and Microsoft Foundry reaching general availability. The configuration modal now shows every provider by default with preview and experimental badges.

Data Explorer

The Data Explorer can now open Excel workbooks directly, with no code required, without first needing to load it via Python or R. Sort, filter, and profile columns, switch between worksheets, and toggle whether the first row holds column names. An Open in Excel button opens the workbook in your native spreadsheet application.

This release broadens what you can open in the Data Explorer overall. Backed by a native DuckDB engine, it now also previews compressed CSV, TSV, and Parquet files. Also, a new Open in Data Explorer code action, from the editor lightbulb or Cmd+., opens the data frame under your cursor in R, Python, and Quarto files, so you can jump straight from your code to exploring your data.

R language intelligence

Last release we introduced within-file symbol resolution for R; this release expands it across files. Go to Definition, Find References, and Rename Symbol now work across the packages and scripts in your workspace, and diagnostics and workspace symbols react to external file changes so your language intelligence stays in sync as your project evolves.

What’s coming next

• New in preview this release, Data Connections lets you browse the schemas, tables, views, and indexes of a database, open tables in the Data Explorer, and generate connection code. It currently supports DuckDB, PostgreSQL, and SQLite. Learn how to try it out and tell us what you think in the Data Connections discussion post: which databases and warehouses you need, whether the connection setup is clear, and anything confusing, missing, or broken.
• We are looking forward to posit::conf(2026) in September, where our team will have several sessions on Positron. Register now to join us in person in Houston or virtually from anywhere in the world.

If you’re new to Python, Positron has a few built-in workflows to help you skip that installation pain entirely. Most of it is powered by uv, created by Astral. We chose uv because it’s fast and has quickly become widely adopted across the Python community. Behind the scenes, uv handles the heavy lifting, downloading Python versions and building environments. Positron wraps that power in a friendly interface, so you never have to remember a command.

From zero to Python in one click

If you don’t have a suitable Python available, Positron will offer to install it for you. When you go to start a Python runtime through the Start Session button, you’ll see an option for + Install Python via uv.

When you choose it, Positron will ask to install uv for you if you don’t already have it. Then it shows you the supported Python versions (currently 3.9 through 3.14) and installs whichever one you pick. If you have a folder open, it will also offer to create a virtual environment for the project and start a Python console session using that environment. Once you have a Python available, this option disappears from the session picker.

If you’d rather not see the prompt to install Python, you can turn it off with the python.allowUvPythonInstall setting (enabled by default).

On-demand Python installation

You don’t have to wait for Positron to ask, either. Open the command palette with Cmd/Ctrl+Shift+P and run Python: Install Python via uv any time you’d like another version. This command will download uv if needed, show you the available Python versions, and optionally offer to create a virtual environment.

Since it runs on demand, it’s a good way to add another interpreter to a project you’ve already set up. And because it follows the same flow every time, it works well for teaching, since everyone ends up in the same place, the same way.

Automatic project setup support

Setting up a virtual environment isn’t limited to that first run. If Positron finds a pyproject.toml or requirements.txt in a project that doesn’t have a virtual environment yet, it’ll offer to create one with uv and install your dependencies. This comes in handy when you’ve cloned a colleague’s project and want to get up and running without extra setup.

If there’s a single requirements source, like a lone requirements.txt or pyproject.toml, Positron will prompt you to create a .venv and install everything right away. And if there are several sources, you can choose which files to include in your installation.

Positron also knows when to stay out of your way. If you already have a local virtual environment, or if you use non-uv files like environment.yml, Pipfile, or poetry.lock, you won’t get a prompt at all.

Try it yourself

Setting up Python doesn’t have to feel like a chore, whether you’re starting on a brand new machine, picking up a colleague’s project, or getting a room full of students ready to code.

Give it a try in Positron’s July 2026 release and let us know what you think.

To install the package, run:

install.packages("mcptools")

To demo these new features, I’ll deploy an R function that returns a picture to an MCP server on Posit Connect. Then, in another R session, I’ll connect to that server and ask an ellmer chat to take a look at the picture and tell me what it sees.

library(mcptools)

Images in tool results

mcptools supports “both directions” of MCP. In one direction, users can deploy R functions as MCP servers. In the other direction, users can fetch tools from third-party MCP servers as R functions. mcptools now supports both serving and fetching tools that return images.

As an example of a function that returns an image, let’s consider a tool fetch_reference_image():

library(ellmer)

tools <- list(
  fetch_reference_image = tool(
    function() {
      content_image_url(
        "https://simonpcouch.com/blog/2026-04-16-local-agents-2/featured.png"
      )
    },
    name = "fetch_reference_image",
    description = "Fetch the reference image."
  )
)

Running that function:

tools$fetch_reference_image()

Without MCP, I can ask a model to look at the image and tell me what it sees:

ch <- chat_claude("Be brief.")
ch$register_tool(tools[[1]])
ch$chat("What do you see in the reference image?")
#> I can see a beautiful Border Collie dog in the reference
#> image. The dog has:
#>
#> - A chocolate brown and white coat with distinctive
#>   coloring
#> - Amber/brown eyes
#> - Alert, perked ears
#> - A white blaze down the center of its face
#> - A pink/brown nose
#> - Long, fluffy fur typical of the breed
#>
#> The dog appears to be outdoors, positioned near what
#> looks like a wooden post or railing with wire fencing
#> visible in the background. There's greenery and trees
#> visible in the blurred background. The dog has an
#> attentive, intelligent expression that's characteristic
#> of Border Collies.

In the next two sections, I’ll deploy this function on Posit Connect and then fetch it from a fresh R session so that future ellmer chats can fetch this same image.

MCP Servers on Posit Connect

mcptools has now adopted plumber2’s _server.yml open standard. This means that, to deploy an MCP server on Posit Connect, you just need to add a _server.yml file in your project root that looks like this:

engine: mcptools
tools: tools.R

tools.R (or whatever you choose to name the file) is a file that defines ellmer tools and returns them in a list. In my case, tools.R looks exactly like the code chunk defining tools above. I can then run rsconnect::deployAPI(".", contentCategory = "mcp") to deploy the tools from tools.R as an authenticated MCP server on Posit Connect.

Fetch tools as R functions from authenticated MCP servers

Perhaps the biggest gap in mcptools before this release was that mcp_tools() did not support remote, authenticated MCP servers. Up to this point, I had written in documentation that folks ought to use npx mcp-remote, which converts remote MCP servers (like Slack, Confluence, or really any of the most well-adopted third-party MCP servers) into local ones. That meant that, even though mcptools only implemented the local half of the protocol, mcptools users could connect to remotely hosted MCP servers.

This is undesirable for a few reasons. For one, mcptools should “just work” without users having to install software from sources other than CRAN. Further, installing code via npx is particularly problematic; the node package registry has been the source of a number of particularly concerning supply chain attacks recently.

mcp_tools() now natively supports fetching tools from authenticated, third-party MCP servers. For example, a configuration that used to look like:

{
  "mcpServers": {
    "connect": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "<my_deployed_connect_listing>",
        "--header",
        "Authorization: Key ${CONNECT_API_KEY}"
      ]
    }
  }
}

Now looks like:

{
  "mcpServers": {
    "connect": {
      "url": "<my_deployed_connect_listing>",
      "headers": {
        "Authorization": "Key ${CONNECT_API_KEY}"
      }
    }
  }
}

I’ll save that latter configuration as config.json. Then, I’ll provide it to mcp_tools():

tools_fetched <- mcp_tools(config = "config.json")

class(tools_fetched[[1]])
#> [1] "ellmer::ToolDef" "function"        "S7_object"

Now I can register the fetched tools with a chat in a new R session, and it has access to the same image:

ch_new <- chat_claude()
ch_new$register_tools(tools_fetched)
ch_new$chat("What's in the reference image?")
#> The reference image features a **Border Collie** dog.
#> Here are some details:
#>
#> - **Coloring**: The dog has a striking **brown
#>   (chocolate) and white** coat, with a distinctive white
#>   stripe running down the center of its face.
#> - **Expression**: It has an alert and attentive look,
#>   with beautiful **amber/brown eyes**.
#> - **Tongue**: Its tongue is slightly visible, giving it a
#>   cute appearance.
#> - **Setting**: The dog appears to be on a **deck or
#>   porch**, with cable/wire railings visible, and a lush
#>   green, leafy background suggesting an outdoor, wooded
#>   area.
#> - **Accessories**: It appears to be wearing a small
#>   **collar tag**.
#>
#> Overall, it's a beautiful and expressive dog photo! 🐕

Taken together, the changes in this release should allow R users to do much more with MCP! For a more complete list of changes in this release, see the package changelog.

There are a variety of ways to give a coding agent new information or abilities, but it can be confusing when to use each one. Generally, with coding agents like Claude Code or Posit Assistant, you can:

• Write prompts in the chat (i.e., normal usage).
• Add information to either a project-level or user-level CLAUDE.md or AGENTS.md. The contents of a directory’s CLAUDE.md or AGENTS.md are included in the agent’s system prompt in every session in that directory. You can also create user-level versions that apply to every session.
• Write a skill or use an existing one. Skills are packaged instructions that can include both text and code. The agent loads a skill only when it’s relevant.
• Add an MCP server. An MCP (Model Context Protocol) server provides an agent with access to otherwise hard-to-find context, mostly through tools, using a standardized interface.

This list is roughly ordered from most straightforward to most complicated.¹

So when do you use one over the other? There are two axes that might matter for your decision.

The first axis is reusability: do you want the agent to perform the task or access the information just once, or many times? The more often you or others will reuse something, the more it’s worth encoding somewhere more permanent.

The second axis is reach. Prompting, CLAUDE.md/AGENTS.md files, and skills all provide guidance on how to best make use of the existing context and tools. MCP servers can provide the agent with entirely new tools (in the agent tools sense), granting the agent access to hard-to-reach information.

MCP servers often seem like the solution when you need to grant an agent access to an outside system, but they aren’t always necessary. In many cases, what seems like a task for an MCP server can actually be solved by a command-line interface (CLI) tool, or a CLI plus a skill that tells the agent how to use it. For example, GitHub has an MCP server, but the gh CLI does roughly the same thing.

The GitHub MCP server works by providing the agent with new tools, whereas the gh CLI takes advantage of the agent’s existing bash tool that lets it run arbitrary shell commands. The skill plus CLI option is therefore generally preferable from a simplicity standpoint, but also from a token standpoint: adding the GitHub MCP server would inject tens of thousands of tokens of tool definitions into every request, whereas a skill that tells the agent to use gh costs almost nothing and loads only when it’s relevant.

However, some information sources are hard to reach via the command line, because no CLI exists, the one that does isn’t fully featured, or it’s difficult for the agent to use. In some of these cases, the same sources can be accessed more effectively via MCP servers, such as design tools like Figma, knowledge repositories like Notion or Confluence, or issue trackers like Linear or Jira.

Posit news

Posit Assistant in Positron

As of the June release of Positron, Posit Assistant is now the default experience in Positron. Positron Assistant will be deprecated starting in the 2026.07 release (release date July 6).

We understand that the names are confusing! Our hope is that the transition state will be over soon and the confusion will lessen. If you want to understand why we gave different assistants very similar names, read this blog post from Posit CTO Joe Cheng: A brief and biased history of Posit data science agents.

Posit Assistant works with the same providers as Positron Assistant. If you have a working provider setup with Positron Assistant, you’ll be able to use that same setup with Posit Assistant. You can read more about available providers here.

Package updates

• raghilda v0.2, a Python package for Retrieval Augmented Generation, is now on PyPI. This release, among other things, broadens support for crawling sites.
• debrief, an R package for LLM-friendly profiling, is now on CRAN. debrief turns profvis profiling output into text-based summaries, allowing AI agents to optimize R code more effectively.

Welcome to our newsletter, posit::glimpse()!

If you’re currently reading this on our blog, consider subscribing to Product Updates - Open Source on our subscription page to receive this newsletter directly in your inbox.

Welcome to the latest edition of the posit::glimpse() newsletter, the monthly roundup of open-source news for the Posit community. I have many updates for you from across Posit. As my wonderful colleague Kristin Bott stated, “dang this is a productive bunch of humans”.

The table of contents on the right can help you navigate through all the updates. As you scroll, it will open up to show you subcategories →

Announcements

posit::conf(2026) is happening soon!

Our annual conference, posit::conf(2026), is happening September 14-16, and we would love to see you there, whether in Houston or online! Check out the speaker lineup, workshop offerings, and register for posit::conf here.

Tidy Dev Day is happening on September 17, a unique opportunity to collaboratively tackle open-source issues and work directly alongside the very developers who build and maintain the tools you use every day. As Meghan Harris stated about last year’s event, “Tidy Dev Day (TDD) gave me the PERFECT opportunity to explore this further in a low-stress, supportive environment.” Learn more about Tidy Dev Day here.

We’ve joined the Jupyter Foundation

We are proud to announce that we are deepening our commitment to the Jupyter ecosystem by becoming an official Jupyter Foundation Member!

• Learn more in the We’ve Joined the Jupyter Foundation announcement blog post.

Announcing an expanded Posit Academy

We’ve launched a new part of Posit Academy: a free, open library of product courses, hands-on labs, and live workshops available to anyone.

• Learn more in the Announcing an expanded Posit Academy blog post.

Introducing the Posit Impact Awards

Have a story to share? We just launched the Posit Impact Awards to recognize individuals and teams who used Posit to create measurable, meaningful change. Six winners will be selected, one per category, and each will receive a conference-only pass to posit::conf(2026).

• Submit your nomination before July 20th.

Key product updates and new releases

Data visualization and reporting

ggsql 0.4.1

ggsql 0.4.1 introduces spatial plotting capabilities with database-backed geometry processing, supporting WKB format data, 21 map projections for cartographic accuracy, and a built-in Natural Earth world dataset for creating choropleth maps and geographic visualizations with backends like DuckDB spatial, PostGIS, and SpatiaLite.

• Learn more in the ggsql 0.4.1: Spatial plotting and in-layer aggregation blog post.

Ask more of your dashboard with querychat and ggsql

querychat now supports ggsql-powered visualizations, enabling natural language data exploration in dashboards through SQL-only execution (no arbitrary code), with three pre-built tools for visualizing, querying, and filtering data reactively. The package works in both Python and R, integrates with Shiny dashboards, and supports Snowflake Semantic Models for business logic definitions.

• Learn more in the Ask more of your dashboard with querychat and ggsql blog post.

Great Tables 0.22.0

Great Tables v0.22.0 significantly expands Python table presentation capabilities with footnote support, group-wise and grand summary row calculations, column merging utilities for uncertainty and ranges, text transformation methods, value substitution helpers, duration and parts-per formatters, image export functionality via gtsave(), enhanced LaTeX rendering, and makes Pandas an optional dependency for Polars-only workflows. (impressive update!)

• Learn more in the Great Tables v0.22.0 blog post.

Data access

dbplyr 2.6.0

dbplyr 2.6.0 introduces ADBC support via adbi for faster Arrow-based data transfer, JDBC support, new SQL dialect separation, and query composition functions.

• Learn more in the dbplyr 2.6.0 blog post.

webR 0.6.0

webR 0.6.0 upgrades to R 4.6.0 and adds async/await support for JavaScript Promises, curl and httr2 compatibility through WebSocket traffic proxying, modern Fortran fixes for expanded package support, and updated system libraries including OpenSSL 3.5.1 and Emscripten 5.0.7. The release powers interactive R experiences in Quarto Live and Shinylive.

• Learn more in the webR 0.6.0 blog post.

Developer tools and AI

debrief 0.1.0

The debrief package converts profvis profiling output into text-based summaries designed for AI agents, enabling AI-assisted performance optimization by providing structured reports on hotspots, call trees, and memory allocations that AI systems can read and act upon.

• Learn more in the debrief 0.1.0 blog post.

pkgsite 0.1.0

pkgsite 0.1.0 converts R package .Rd documentation files into Quarto .qmd files, enabling custom documentation sites with Quarto’s freeze feature for local example rendering, unified R/Python documentation when combined with Quartodoc, and flexible template customization. The package is available on CRAN and provides an alternative to pkgdown.

• Learn more in the pkgsite 0.1.0 blog post.

watcher 0.2.0

Watcher is a lightweight R package that watches files and directories for changes and reacts in the background. It’s quietly been the engine behind Shiny’s auto-reload for the past year. With the CRAN release of 0.2.0, we’re excited to introduce it as a general-purpose filesystem watcher for R developers.

• Learn more in the watcher 0.2.0 blog post.

Development environment

Air 0.10.0

Air 0.10.0 introduces configurable assignment style enforcement allowing teams to standardize on arrow (<-), equal (=), or preserve existing styles, along with enhanced IDE integrations for Positron and RStudio, multiple installation methods via PyPI and conda-forge, pre-commit hook support, stdin integration for editors, and shell completions.

• Learn more in the Air 0.10.0 blog post.

What’s new in Positron

Positron’s June release includes a lot of highly requested features:

• Inline output for Quarto (one of Positron’s most-requested features ever!)
• Posit Assistant, the successor to Positron Assistant
• Packages pane improvements
• A more customizable interface

For more, check out the Positron June Release Highlights post and subscribe to Positron emails.

Did you know that many of the most upvoted RStudio feature requests are already implemented in Positron? Learn about ten of them in the RStudio’s Top Feature Requests … In Positron blog post!

Machine learning and modeling

brulee 1.0.0

brulee 1.0.0 significantly expands tabular deep learning capabilities in R with five new model architectures, GPU support including Apple Silicon, 32-bit precision for improved performance, and enhanced numerical stability, all integrated with the tidymodels ecosystem.

• Learn more in the brulee 1.0.0 blog post.

CatBoost support in tidymodels

CatBoost gradient boosting support is now available in tidymodels through the boost_tree() interface, providing access to CatBoost’s strong categorical feature handling with full tidymodels integration including hyperparameter tuning, cross-validation, efficient submodel optimization, and orbital package support for SQL generation and in-database predictions.

• Learn more in the CatBoost support in tidymodels blog post.

tidyclust 0.3.0

tidyclust 0.3.0 introduces three new clustering model families and achieves full integration with tidymodels by replacing tidyclust-specific functions with native tune package support.

• Learn more in the tidyclust 0.3.0 blog post.

Event roundup

We were all over the world this month, discussing how to adopt new data tools, get better at old ones, and just loving being part of the community. If you want to learn more about where we were, or see where we’ll be next, check out our event page.

Watch the recordings from some of these events:

Jun 15, 2026 | 34 min | 308 views

Neal Richardson - MCP, or not MCP | Pydata London 26

Neal Richardson - MCP, or not MCP | Pydata London 26 \Model Context Protocol is a standard for defining tools that can be made available to LLMs and AI applications. There’s a lot of noise out there about what you should use to get the best results from AI, so in this talk, I will provide some guidance on when you should use MCP, and when you should reach for some other tool. I will describe cases where MCP is the right tool for the job, and when other things, like skills or other context files, are better. I will also devote attention to questions of security and authentication, which are important for MCP, and provide concrete examples of how MCP servers can be used to unlock agentic workflows while also strengthening data governance. This talk is intended for those who are interested in using LLMs for workflows involving data. No prior experience with MCP is required. Outline: Intro: how can I get data from this API into my Claude Code session? What is MCP? When should you use it, when should you use other tools Work through an example Sharing and deploying MCP servers, alternatives and best practices Optimizing your tools for best results www.pydata.org PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R. PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases. 00:00 Welcome! 00:10 Help us add time stamps or captions to this video! See the description for details. Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVideoTimestamps

Neal Richardson

Python Tutorial Education NumFOCUS PyData Opensource Learn Software Python 3 Julia Coding Learn to Code How to Program Scientific Programming

Jun 26, 2026 | 47 min | 179 views

Agents for Correct, Transparent, and Reproducible Data Analysis - Simon Couch & Sara Altman

Agents for Correct, Transparent, and Reproducible Data Analysis - Simon Couch & Sara Altman (Posit) Abstract: How do we build competent data analysis agents? Data analysis requires a willingness to pause, question conclusions, and dig into subtleties. Frontier LLMs, however, are optimized to push tasks toward completion, not to slow down when something seems off. This tendency works well for coding agents, where success is often verifiable. But for data analysis, verification is more complicated, and autonomous work by the agent can be at odds with the spirit of the discipline. Drawing on our experience building data analysis agents, we’ll share evaluations that expose where LLM-driven analysis goes wrong and design patterns that keep analyses correct, transparent, and reproducible. Resources mentioned in the session: Presentation Slides: https://simonpcouch.github.io/gen-ai-pharma-26 Presentation GitHub repository: https://github.com/simonpcouch/gen-ai-pharma-26 bluffbench: https://github.com/simonpcouch/bluffbench Posit Assistant Terminal (TUI): https://posit-dev.github.io/assistant/docs/downloads/tui/ Posit AI Newsletter: https://opensource.posit.co/tags/ai-newsletter/ Speakers: Simon Couch builds tools that make the work of data science more joyful and effective. As an engineer on the AI Core Team at Posit, his work spans coding agents, model evaluations, inference engineering, and next-edit-suggestion systems. Drawing on his background in statistics, Simon spent several years authoring and maintaining core packages in the open-source tidymodels framework—like stacks, broom, and infer — before shifting his focus to LLMs. He blogs about his work at simonpcouch.com. Simon authors the Posit AI Newsletter along with Sara Altman. Sara Altman is a Senior Developer Advocate on the AI Core team at Posit, where she focuses on how AI can be effectively and responsibly used for data science. Previously, she helped build Posit Academy and taught data science and R at Stanford. Sara authors the Posit AI Newsletter along with Simon Couch. Presented at the 2026 R/Pharma GenAI Day

Sara Altman, Simon Couch

infer tidymodels

Showcases from the community

Brilliant Earth turned their Marketing Mix Model into a Streamlit app deployed on Posit Connect via the Snowflake Native App, so their marketing team can dig into channel performance and run scenario planning on their own. One of the campaigns in the mix: their recent Ring Pop collaboration.

• Check out the spotlight.

Leo Ohyama recently shared a fantastic Lorcana Market Data Analysis & Forecasting dashboard that showcases the power of Python, R, Positron, and Quarto working together.

Leo included detailed documentation in the GitHub repository, a great resource for anyone interested in market forecasting or multi-tool workflows. Thanks for sharing your work with the community, Leo!

• Lorcana dashboard
• Lorcana GitHub repo

We were recently joined by Thomas Lin Pedersen on the Data Science Lab, where he introduced the new ggsql package.

(Almost) immediately after the DS Lab, Dylan Poulsen wrote a blog post on exploring two years of swim data with ggsql! Dylan, we’re convinced you write blogs at the speed of ggsql.

• Exploring the New ggsql Package with Two Years of Swim Data blog post

We usually find these projects on social media. If you’re on LinkedIn, be sure to follow and tag Posit Open Source for us to share the amazing things you’re working on!

What’s next

We’re taking a short break in July before returning with more community hangouts!

• On July 21, Gina Reynolds will join the Data Science Lab (a fun, chill time with live code) to show we can extend ggplot2 by creating our own custom extensions. Register here: https://pos.it/dslab

In the meantime, check out some past Data Science Lab episodes:

May 29, 2026 | 57 min | 1.3k views

Async & Parallel R with {mirai} | Charlie Gao | Data Science Lab

The Data Science Lab is a live weekly call. Register at pos.it/dslab! Discord invites go out each week on lives calls. We’d love to have you! The Lab is an open, messy space for learning and asking questions. Think of it like pair coding with a friend or two. Learn something new, and share what you know to help others grow. On this call, Libby Heeren is joined by Charlie Gao, who walks through async and parallel programming using the mirai package for R. Charlie is the author of {mirai} and {mori}! Charlie demonstrates how mirai lets you run R code in parallel across multiple cores or even distributed across remote machines. He covers the key differences between parallel and async programming, shows how to set up worker daemons (said like “demons”), scale resources dynamically, and connect to remote machines via SSH. Whether you’re running long computations, training models, or building Shiny apps, mirai helps you make the most of your computing resources without blocking your main R session:) Hosting crew from Posit: Libby Heeren, Isabella Velasquez Charlie Gao’s GitHub: https://github.com/shikokuchuo Charlie Gao’s Bluesky: https://bsky.app/profile/shikokuchuo.net Charlie Gao’s LinkedIn: https://www.linkedin.com/in/charliegao/ Charlie Gao’s Mastodon: https://fosstodon.org/@shikokuchuo Resources mentioned in the video and chat: mirai package website: https://mirai.r-lib.org/ mirai GitHub repository: https://github.com/r-lib/mirai AskDeepSeek chatbot for mirai documentation: https://mirai.r-lib.org/ (click “ask deep wiki” button) mirai - Promises (Shiny and Plumber): https://mirai.r-lib.org/articles/v02-promises.html mirai - Serialization: https://mirai.r-lib.org/articles/v03-serialization.html mirai - OpenTelemetry: https://mirai.r-lib.org/articles/v05-opentelemetry.html mirai stop_mirai function: https://mirai.r-lib.org/reference/stop_mirai.html mirai skill for Claude Code: https://github.com/r-lib/mirai/blob/main/.claude/skills/mirai/SKILL.md Plumber2 package: https://plumber2.posit.co/ Daemon (computing) on Wikipedia: https://en.wikipedia.org/wiki/Daemon_(computing) ► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu Follow Us Here: Website: https://www.posit.co Hangout: https://pos.it/dsh The Lab: https://pos.it/dslab LinkedIn: https://www.linkedin.com/company/posit-software Bluesky: https://bsky.app/profile/posit.co Thanks for hanging out with us! Timestamps of Questions / Topics: 00:00 Introduction 03:33 “What is async programming and how is it different from parallel?” 06:57 AskDeepSeek chatbot for the Mirai package 09:40 Setting up the Positron IDE with activity bar on top 15:25 “What was the motivation or need for developing Mirai?” 22:10 Mirai map function for parallel processing 25:25 “Do the contents of Mirai inherit definitions from the global environment?” 29:00 “What’s the difference between Mirai versus promises and future?” 31:50 Demonstrating sequential vs parallel processing 36:00 “Can you use Mirai with HPC?” 37:07 Dynamically scaling workers by adding and removing daemons 38:38 Setting up daemons with URLs for network connections 42:05 Launching workers over SSH to remote machines 43:09 SSH tunneling to connect workers without open ports 51:14 “Does Mirai allow R to perform the same thing as NumPy?” 51:54 “Does Mirai ship with some kind of task viewing dashboard?” 52:35 “Can we use parallel::detectCores() to see how many daemons we can use?” 53:58 “Would parallel processing be more useful than async processing in typical data science work?” 55:00 Mirai skill for Claude Code and AI agents

Charlie Gao

mirai mori plumber plumber2 Positron Shiny

Jun 23, 2026 | 58 min | 4.9k views

Data dictionaries, parquet, & Claude | Hadley Wickham | Data Science Lab

The Data Science Lab is a live weekly call. Register at pos.it/dslab! Discord invites go out each week on lives calls. We’d love to have you! The Lab is an open, messy space for learning and asking questions. Think of it like pair coding with a friend or two. Learn something new, and share what you know to help others grow. On this call, Libby Heeren is joined by Hadley Wickham, who walks through using data dictionaries with Claude Code to clean and document datasets effectively. Hadley demonstrates a workflow using three files: a data cleaning script, a data dictionary in YAML format, and the final cleaned data as a Parquet file. He shows how Claude Code and MCP REPL can help generate and maintain data dictionaries that document what you know about your data, making it easier for both humans and AI agents to work with your datasets. Using the NYC elevators dataset as an example, he walks through data cleaning tasks like normalizing whitespace, handling missing values, fixing date formats, and investigating geocoding issues - all while keeping the data dictionary, cleaning script, and Parquet file in sync through git. Hosting crew from Posit: Libby Heeren, Isabella Velasquez Hadley Wickham’s GitHub: https://github.com/hadley Hadley Wickham’s Bluesky: https://bsky.app/profile/hadley.nz Hadley Wickham’s LinkedIn: https://www.linkedin.com/in/hadleywickham/ Resources mentioned in the video and chat: MCP REPL: https://github.com/posit-dev/mcp-repl Data Dictionary YAML Format Specification: https://github.com/hadley/data-dict.yaml Parquet files in R for Data Science: https://r4ds.hadley.nz/arrow.html#sec-parquet Elevators Dataset Used in Demo: https://github.com/EmilHvitfeldt/elevators Pointblank Package for Data Validation: https://posit-dev.github.io/pointblank/ Arrow R Book: https://arrowrbook.com/ Monaspace Font Family (with ligatures): https://monaspace.githubnext.com/ YAML Multiline Strings Reference: https://yaml-multiline.info/ UBC Course on Shiny with RAG and Parquet: https://ubc-mds.github.io/DSCI_532_vis-2_book/060-03-rag.html Tom Scott Video on Timezones: https://www.youtube.com/watch?v=-5wpm-gesOY Falsehoods Programmers Believe About Names: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ UTC Is Enough for Everyone Right: https://zachholman.com/talk/utc-is-enough-for-everyone-right Daniel Chen’s Cherry Blossom Analysis: https://chendaniely.github.io/posts/2026/2026-03-30-yvr-cherry-blossoms-marathon/ Project Drawdown Climate Impact Explorer: https://drawdown.org/explorer Green Coding Bookdown Resource: https://bookdown.org/content/d1e53ac9-28ce-472f-bc2c-f499f18264a3/ IBM Green Coding Topics: https://www.ibm.com/think/topics/green-coding Secret Elevator in Central Park Article: https://undercovernyc.home.blog/2021/02/08/a-secret-elevator-hidden-in-central-park/ Artificial Cave Beneath Central Park: https://gizmodo.com/an-artificial-cave-200-beneath-central-park-with-micha-1446538828 ► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu Follow Us Here: Website: https://www.posit.co Hangout: https://pos.it/dsh The Lab: https://pos.it/dslab LinkedIn: https://www.linkedin.com/company/posit-software Bluesky: https://bsky.app/profile/posit.co Thanks for hanging out with us! Timestamps of Questions / Topics: 00:00 Introduction 03:38 “Can you talk a little bit about what MCP is?” 05:42 Introducing the elevators dataset 07:00 Creating initial data dictionary with Claude 09:03 “Are there any cases where the CSV format is actually a better choice than Parquet?” 09:50 “What font do you use?” 12:02 Adding context from the readme to the data dictionary 14:02 “What is a good way to store data dictionaries along with datasets?” 14:38 “Is this particular data dict YAML format useful for projects with only one table of data?” 17:45 “Is Claude also going to decide when it’s a good time to make a commit?” 24:02 “Does MCP REPL work on a Windows machine and can one use other AI LLM for example ChatGPT with it?” 26:14 Converting date columns to proper date types 27:16 “Do you see a meaningful distinction between data dicts and data contracts?” 29:36 “How do you view your approach to data dictionaries and the development of pointblank?” 34:16 Eliminating placeholder values and using proper missing values 36:25 “Could you show off a diff of a Parquet file?” 38:25 Investigating geocodes and creating a map of elevators 40:20 Using a leaflet map to explore Central Park elevators 42:03 “Which model is Hadley using?” 43:55 Discussion of cost consciousness and environmental impact of LLMs 46:30 “Is there a way to quantify environmental and electrical costs?” 48:50 The mystery elevator in Central Park 54:23 “How do you know this is actually faster or more productive than just writing the code?” 55:45 The importance of deep knowledge of data in qualitative work

Hadley Wickham

bookdown bookdown.org leaflet pointblank Shiny

Feb 19, 2026 | 55 min | 1.7k views

Using R package structure for data science projects | Kylie Ainslie | Data Science Lab

The Data Science Lab is a live weekly call. Register at pos.it/dslab! Discord invites go out each week on lives calls. We’d love to have you! The Lab is an open, messy space for learning and asking questions. Think of it like pair coding with a friend or two. Learn something new, and share what you know to help others grow. On this call, Libby Heeren is joined by Kylie Ainslie who walks through how structuring data science projects as R packages provides a consistent framework that integrates documentation for you and facilitates collaboration with others by organizing things really well. Kylie says, “I stumbled on using an R package structure to organize my projects a number of years ago and it has changed how I work in such a positive way that I want to share it with others! In a world where our attention is constantly being pulled in many directions, efficiency is crucial. Structuring projects as R packages is how I work more efficiently.” Hosting crew from Posit: Libby Heeren, Isabella Velasquez Kylie’s Bluesky: @kylieainslie.bsky.social Kylie’s LinkedIn: https://www.linkedin.com/in/kylieainslie/ Kylie’s Website: https://kylieainslie.github.io/ Kylie’s GitHub: https://github.com/kylieainslie Resources from the hosts and chat: posit::conf(2026) call for talks: https://posit.co/blog/posit-conf-2026-call-for-talks/ Kylie’s posit::conf(2025) talk: https://www.youtube.com/watch?v=YzIiWg4rySA {usethis} package: https://usethis.r-lib.org/ R Packages (2e) book: https://r-pkgs.org/ Paquetes de R (R Packages in Spanish): https://davidrsch.github.io/rpkgs-es/ {box} package: https://github.com/klmr/box extdata docs in Writing R Extensions: https://cran.r-project.org/doc/manuals/R-exts.html#Data-in-packages-1 Tan Ho’s talk on NFL data: https://tanho.ca/talks/rsconf2022-github/ {rv} package: https://a2-ai.github.io/rv-docs/ Whether to Import or Depend: https://r-pkgs.org/dependencies-mindset-background.html#sec-dependencies-imports-vs-depends {pkgdown} package: https://pkgdown.r-lib.org/ Edgar Ruiz’s {pkgsite} package: https://github.com/edgararuiz/pkgsite Attendees shared examples of data packages in the chat! Here they are: https://kjhealy.github.io/nycdogs/ https://kjhealy.github.io/gssr/ https://github.com/deepshamenghani/richmondway https://github.com/kyleGrealis/nascaR.data https://github.com/ivelasq/leaidr ► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu Follow Us Here: Website: https://www.posit.co The Lab: https://pos.it/dslab Hangout: https://pos.it/dsh LinkedIn: https://www.linkedin.com/company/posit-software Bluesky: https://bsky.app/profile/posit.co Thanks for learning with us! Timestamps: 00:00 Introduction 06:17 Reviewing the disorganized project example 10:01 Creating the package structure using create_package 17:50 Organizing external data and scripts in the inst folder 22:55 Adding a README and License 29:06 “What are the advantages to packaging a project?” 33:35 Writing Roxygen2 documentation 36:06 “Do you type return at the end of your functions?” 41:55 Handling dependencies with use_package 43:53 “Can you just use require(dplyr) at the top?” 47:45 Setting up a pkgdown site 50:11 Creating vignettes 52:22 “What is the role of the usethis package?” 54:18 Loading the package with devtools::load_all

Edgar Ruiz

devtools dplyr pkgdown roxygen2 usethis

I’m a real person, and I would love to know how to make the Glimpse newsletter better! Find me on LinkedIn and Bluesky, or email me at isabella [dot] velasquez [at] posit.co.

This post walks through the major additions. The package’s fundamentals have not changed (stores, chunkers, retrievers, and the pattern for connecting to chatlas all work as before), but the surface area for building and maintaining stores in production has grown substantially.

The crawl and ingest API

The largest change in v0.2 is a new API for crawling sources and ingesting them into a store. In v0.1, building a store meant calling read_as_markdown() on individual URLs, chunking each result, and upserting them one by one. That works for a handful of pages, but it becomes unwieldy for larger collections, where you also want caching (to avoid re-fetching unchanged content) and concurrency (to finish in minutes rather than hours).

The new API introduces a clean separation between crawling and storage. On the crawl side, a crawler object produces MarkdownDocument objects from a defined scope. On the store side, store.ingest() consumes those documents lazily, applies an optional preparation step (typically chunking), and writes them to the store with configurable parallelism. The pipeline looks like this:

from raghilda.chunker import MarkdownChunker
from raghilda.crawl import CrawlScope, WebCrawler
from raghilda.embedding import EmbeddingOpenAI
from raghilda.store import DuckDBStore

crawler = WebCrawler(cache_dir=".cache/crawl", max_workers=4)

scope = CrawlScope(
    roots=["https://example.com/docs"],
    include_patterns=["https://example.com/docs/**"],
    depth=2,
)

documents = crawler.markdown_documents(scope)

store = DuckDBStore.create(
    location="raghilda.duckdb",
    embed=EmbeddingOpenAI(),
    overwrite=True,
)

summary = store.ingest(
    documents,
    prepare=MarkdownChunker(chunk_size=1000).chunk,
    max_workers=4,
)

print(summary)

IngestSummary(inserted=142, replaced=0, skipped=0)

The CrawlScope dataclass defines the traversal policy: root URLs, include/exclude patterns, depth limits, and page count caps. The crawler handles the mechanics of fetching and converting pages to Markdown, while the scope tells it where to go. This separation means you can change the backend (swap WebCrawler for CloudflareCrawler, for instance) without redefining the scope, and vice versa.

Three concrete crawlers ship with v0.2. DirectoryCrawler walks local file trees and converts supported formats to Markdown. WebCrawler fetches pages over HTTP using requests and converts them locally. CloudflareCrawler delegates both fetching and rendering to Cloudflare’s Browser Rendering API (the right tool for sites that load content through JavaScript). All three implement the same interface: origins() to discover pages, fetch_raw() and fetch_markdown() for single-page access, and markdown_documents() for the full pipeline.

Caching is built into the crawler layer. When you pass cache_dir=True (or an explicit path), each crawler stores fetched content and converted Markdown in a flat directory of files with metadata sidecars. On subsequent runs, cached entries are reused if they are still fresh. For WebCrawler and CloudflareCrawler, freshness is controlled by cache_stale_after=, a timedelta that defines how long a cached entry remains valid. This makes interrupted workflows resumable without any explicit checkpoint logic: rerun the script and the cache supplies everything that was already fetched, while only new or stale pages trigger network requests.

Concurrency operates on both sides of this boundary independently. The crawler can fetch and convert pages in parallel (controlled by max_workers= on the crawler constructor), and store.ingest() can write to the store concurrently (controlled by its own max_workers= argument). For WebCrawler, the breadth-first frontier is explored concurrently while preserving stable output order, so results come back in a consistent sequence regardless of which pages respond first.

CloudflareCrawler

The CloudflareCrawler moves the work of a crawl off the local machine. Instead of fetching pages and converting them to Markdown with local processes, it hands both jobs to Cloudflare’s Browser Rendering API, so a long crawl over a large site runs on Cloudflare’s distributed infrastructure rather than competing for local CPU and bandwidth. For collections large enough that concurrent local requests become the bottleneck, this is the primary reason to reach for it: the slow, sustained part of building a store happens remotely, and what returns is ready-to-chunk Markdown. The same arrangement resolves a problem that defeats a plain HTTP fetch, because the API renders each page in a real browser, executing JavaScript and waiting for the DOM to settle before extracting content. Sites built with React, Vue, or Angular, which an ordinary request reduces to an empty shell, are therefore handled without extra configuration or a locally installed headless browser.

The usage looks almost identical to WebCrawler, because both share the same crawl interface. The key difference is that the constructor takes Cloudflare credentials instead of an HTTP session, and Cloudflare’s infrastructure handles the rendering remotely (so there is no need to install Playwright, Selenium, or any other local headless browser):

import os
from raghilda.crawl import CloudflareCrawler, CrawlScope

crawler = CloudflareCrawler(
    account_id=os.environ["CLOUDFLARE_ACCOUNT_ID"],
    api_token=os.environ["CLOUDFLARE_API_TOKEN"],
    cache_dir=True,
    render=True,
    max_workers=4,
)

scope = CrawlScope(
    roots=["https://my-spa-docs.example.com/"],
    depth=2,
    include_patterns=["https://my-spa-docs.example.com/**"],
    limit=500,
)

documents = crawler.markdown_documents(scope)

Iterating the result performs the crawl lazily, yielding one MarkdownDocument per page. Each document exposes the page’s origin alongside its rendered Markdown content, so a quick pass confirms that the JavaScript-rendered pages came back with real text rather than the empty shells a plain HTTP fetch would have produced:

for doc in documents:
    print(doc.origin, f"({len(doc.content):,} chars)")

https://my-spa-docs.example.com/ (3,214 chars)
https://my-spa-docs.example.com/guide/install (5,902 chars)
https://my-spa-docs.example.com/guide/config (8,477 chars)
https://my-spa-docs.example.com/api/reference (12,043 chars)

The render=True default tells Cloudflare to execute JavaScript before extracting content. For server-rendered sites where JavaScript execution is unnecessary, setting render=False reduces crawl time and API usage. The source= parameter controls how pages are discovered: "all" (the default) combines multiple discovery methods, "sitemap" reads from the site’s sitemap.xml, "crawl" follows links from the rendered DOM, and "urls" processes only the explicitly provided roots.

For stores that need regular updates, the modified_since= parameter restricts the crawl to pages modified after a given Unix timestamp, keeping refresh jobs lightweight. Combined with the crawl cache and the store’s own deduplication (identical documents are not re-embedded), an incremental update script can run daily without redundant work:

import time
from datetime import timedelta

one_week_ago = int(time.time()) - (7 * 24 * 60 * 60)

crawler = CloudflareCrawler(
    account_id=os.environ["CLOUDFLARE_ACCOUNT_ID"],
    api_token=os.environ["CLOUDFLARE_API_TOKEN"],
    cache_dir=True,
    cache_stale_after=timedelta(days=1),
    modified_since=one_week_ago,
)

The tradeoff is cost: CloudflareCrawler requires a Cloudflare account with Browser Rendering access. For static HTML sites where a plain HTTP fetch returns the full content, WebCrawler remains the simpler and free option. Both crawlers share the same interface, so switching between them requires only a constructor change.

PostgreSQL store

raghilda v0.1 shipped with three store backends: DuckDB (local, zero-config), ChromaDB, and OpenAI Vector Stores. v0.2 adds PostgreSQLStore, backed by psycopg2 and pgvector. This is the natural choice for production deployments where the store needs to be shared across services, or where you already have PostgreSQL infrastructure.

from raghilda.store import PostgreSQLStore
from raghilda.embedding import EmbeddingOpenAI

store = PostgreSQLStore.create(
    connection="postgresql://user:pass@localhost:5432/mydb",
    embed=EmbeddingOpenAI(),
    name="docs_store",
    overwrite=True,
)

The store supports full-text search via PostgreSQL’s native tsvector/tsquery with a pre-computed column and GIN index, vector similarity search via pgvector with HNSW indexes (supporting cosine, L2, and inner product distance metrics), and combined retrieval that merges both result sets with deoverlap support.

Retrieval uses the same interface as every other backend: a single retrieve() call returns a ranked list of chunks, each carrying its similarity score under metrics and its heading-hierarchy context. Running a query against a store populated with the raghilda documentation returns the most relevant chunks first:

results = store.retrieve("How are vector indexes configured?", top_k=2)

for r in results:
    print(f"Score: {r.metrics[0].value:.4f}")
    print(r.context)
    print(r.text)
    print("---")

Score: 0.5142
# PostgreSQL store
Vector similarity search runs through pgvector with HNSW indexes,
supporting cosine, L2, and inner product distance metrics. Call
build_index() after ingestion to create them.
---
Score: 0.4417
# PostgreSQL store > Combined retrieval
Combined retrieval merges the vector and full-text result sets and
applies deoverlap to drop redundant overlapping chunks, returning a
single ranked list from one retrieve() call.
---

Attributes work as expected: scalar types map to columns, struct types map to JSONB, and attribute filters can query into JSONB fields using the ->> operator. The build_index() method creates HNSW indexes after ingestion, and the vss_index= parameter on create() controls the default index type.

Connection strings are accepted directly in create() and connect(), so you can point the store at any PostgreSQL instance with pgvector installed. If the pgvector extension is missing, the store raises an informative error rather than failing cryptically on the first vector operation.

NVIDIA NIM embeddings

The embedding layer gains a new provider: EmbeddingNVIDIA, which connects to NVIDIA’s OpenAI-compatible embedding API. The default model is nvidia/llama-nemotron-embed-1b-v2, a compact embedding model suitable for retrieval workloads. The provider reads its API key from the NVIDIA_API_KEY environment variable.

from raghilda.embedding import EmbeddingNVIDIA

embedding = EmbeddingNVIDIA()

One notable feature of NVIDIA’s embedding API is differentiated input types: queries and documents are embedded with different prefixes ("query" and "passage"), which can improve retrieval quality for asymmetric search where the query is short and the documents are long. raghilda handles this distinction automatically when the store calls the embedding provider during ingestion and retrieval.

The provider includes built-in rate limit handling with exponential backoff. NVIDIA’s 429 responses carry no Retry-After header or rate limit metadata, so backoff is the only viable strategy. For users who want to keep embedding computation entirely local, NVIDIA NIM can also be self-hosted, in which case you point the provider at your local endpoint with a base_url= override.

Improved DuckDB error messages

A smaller but practical improvement: DuckDBStore now raises clear, actionable errors when BM25 retrieval is attempted before the index has been built, or after writes have made the index stale. Previously, this produced a cryptic CatalogException from DuckDB about a missing match_bm25 function. The new message tells you exactly what to do:

RuntimeError: DuckDBStore retrieval requires a current BM25 index.
Call `store.build_index("bm25")` after inserting or updating documents
and before calling `retrieve_bm25()` or `retrieve()`.

The store now tracks BM25 freshness internally: the index is marked stale after any upsert() call and marked current after build_index("bm25"). This tracking happens off the retrieval hot path, so there is no per-query overhead. HNSW indexes are unaffected because DuckDB maintains them across writes automatically.

Why even use raghilda?

raghilda is a retrieval library, not an orchestration framework. Larger projects like LangChain and LlamaIndex offer composable retrieval components too, but they also ship agent runtimes, chain abstractions, prompt management, and memory systems. If all you need is the retrieval pipeline (crawl, chunk, embed, store, retrieve), raghilda gives you that without the surrounding framework. The API surface is small: plain dataclasses, iterators, and direct function calls. There are fewer layers of indirection between your code and the underlying operations, which makes the pipeline easier to debug and reason about.

raghilda v0.2 makes that focused scope practical at scale. The crawl API adds caching and concurrency while keeping each step a separate, inspectable call. The storage layer lets you start with a local DuckDB file and move to PostgreSQL or OpenAI Vector Stores later without changing retrieval code. And every backend provides hybrid retrieval (semantic search, BM25, and attribute filtering combined in a single retrieve() call) out of the box, without assembling separate retriever classes or configuring a pipeline graph.

Getting started

raghilda v0.2 is available now on PyPI (pip install raghilda). The raghilda documentation site covers all of the features described here in more detail. The Getting Started guide walks through building a store from scratch, and the Crawling and Ingestion guide covers the new crawl API in depth. A dedicated CloudflareCrawler guide explains browser rendering, page discovery, caching, and incremental updates. The GitHub repository has the source, issue tracker, and full changelog. If you run into problems or have feature requests, open an issue there.

watcher is a lightweight R package that watches files and directories for changes and reacts in the background. It shipped quietly last year as the engine behind Shiny’s auto-reload, and until now that is mostly where it lived. watcher 0.2.0 is on CRAN, and we’re taking this opportunity to introduce it as a general-purpose filesystem watcher for R developers to use.

Install it from CRAN:

install.packages("watcher")

The engine behind Shiny auto-reload

The inner loop of building a Shiny app is edit, save, switch to the browser, reload. Auto-reload removes the friction of the last two steps: you save, and the app reloads itself.

The easiest way to turn it on is Shiny’s Developer Mode, which flips on a handful of developer-friendly options for the session – auto-reload among them, alongside unminified JavaScript and full stack traces. Call devmode() once and run your app as usual:

shiny::devmode()
shiny::runApp("app.R")

Now every time you save a file in the app directory, the running app reloads to match:

Editing app.R on the left; the app on the right reloads on each save. No manual refresh or restart.

Under the hood, Developer Mode sets shiny.autoreload = TRUE (you can set that option directly if you’d rather not switch on the rest of Developer Mode), which hands your app directory to watcher and starts it watching in the background. The instant you save a file, watcher’s callback fires and Shiny pushes a reload to the browser.

Previously Shiny did this by polling: every few hundred milliseconds it re-listed the directory and compared modification times. That works, but the cost scales with the size of your project and the reload only ever happens on the next tick. watcher instead subscribes to the operating system’s own filesystem-change notifications, so the reload fires on the save itself, and an idle app does no work at all.

For version 0.2.0, we’ve simplified how the package installs from source. This means we can have it power auto-reload by default: the next release of Shiny will require watcher outright, so it’s installed alongside Shiny with nothing to install by hand. (Shiny 1.14.0 uses watcher when it’s present and falls back to polling otherwise – so for now it’s worth installing watcher yourself).

watcher, the package

This same machinery is available directly, outside of Shiny. watcher() returns an R6 object that you can start and stop:

library(watcher)

dir.create("data")

w <- watcher(
  path = "data",
  callback = \(paths) cat("changed:", paths, "\n"),
  latency = 0.5
)
w$start()

From now on, any change under data/ calls your function back with the paths that changed:

file.create("data/report.csv")
#> changed: /home/you/project/data/report.csv

Three arguments to watcher() cover most needs:

• path – a file, a directory (watched recursively), or a vector of paths. Defaults to the working directory.
• callback – a function taking one argument: a character vector of the paths that changed. The default, NULL, simply writes the changed paths to stdout.
• latency – seconds to debounce events before reporting them. Defaults to 1.

w$stop()

The returned object also has $stop(), $is_running() and $get_path() alongside $start().

watcher runs on a background thread, but your callback runs on R’s main thread, scheduled through later. It fires when R is idle at the top level, whenever you call later::run_now(), or automatically inside an event loop such as Shiny’s. This is what lets it slot easily into Shiny or plumber without blocking the session.

Put it to work

watcher is deliberately small and general. Anything you want to happen when a file changes, you can wire to it:

• rebuild a report or re-render a document when its source or data changes
• reprocess a directory as new files land in it
• re-run tests or reload package code while you develop
• reload configuration without restarting a long-running service

The shape is always the same – give watcher() a path and a function:

library(watcher)

rebuild <- function(paths) {
  message(format(Sys.time()), ": ", length(paths), " file(s) changed")
  # ... your render / test / reload step here ...
}

w <- watcher("data", rebuild, latency = 1)
w$start()

The callback receives the paths that changed, so you can act on exactly what triggered it. File created, updated, removed and renamed events are monitored, for individual files or whole directory trees.

From here:

• watcher package site – reference and examples.
• r-lib/watcher – source, issues, and feedback.
• For Shiny, enable Developer Mode with shiny::devmode() (or set options(shiny.autoreload = TRUE) directly).

Acknowledgments

watcher builds on libfswatch by Enrico M. Crisostomo and Alan Dipert. This is a mature C++ filesystem-monitoring library, which uses the native, event-driven notification API on each platform.

Thanks go to Garrick Aden-Buie for wiring watcher into Shiny’s auto-reload, and to everyone who has contributed issues, fixes, and feedback.

Questions and ideas are very welcome on GitHub: r-lib/watcher.

If you use Positron, the Air Extension has probably already updated you to the new release! Otherwise, follow one of Air’s editor guides to install the latest version for your editor.

To see every bug fix, check out our CLI changelog and extension changelog.

Assignment style 😎

The headlining feature of 0.10.0 is a new assignment-style option, one of our top requests! This allows you to enforce <- or = for assignment throughout your codebase. Choose one of "arrow" for <-, "equal" for =, or "preserve" to leave existing code as is. For example, with assignment-style = "arrow":

# This
a = 1
fn = function(x, y) {
  z <- x + y
  z
}

# Is standardized to
a <- 1
fn <- function(x, y) {
  z <- x + y
  z
}

This option defaults to enforcing <- everywhere, which is what the style guide already recommended. Note that this is a breaking change for Air, as the previous behavior didn’t enforce any assignment style. To revert to the old behavior, use "preserve".

This feature is surprisingly sophisticated! In R, you can always convert from = to <- without changing the meaning of the code, but the reverse isn’t true. Consider the following:

quote(x <- 1)

This quotes the expression x <- 1 and returns it. Blindly changing to = results in:

quote(x = 1)

That’s very different! This sets an argument named x to 1. Trying to run this will result in an error since quote() doesn’t have an x argument. Air knows every case where <- is allowed but = isn’t, so you can rest assured that this option won’t impact the meaning of your code in any way.

Positron terminal support

Positron’s integration with Air has gotten even better! When the Air extension activates, it now prepends the bundled Air executable to your integrated terminal’s PATH. This means:

• You can now run Air commands from Positron’s terminal, even without installing the CLI separately.
• Your terminal version of Air should now always match the version used by the editor itself.

This is particularly useful with agents like Claude Code, where you’ll often want to tell your agent to run Air on any R files it touches (see usethis::use_tidy_agents() in the development version of usethis for a way to automatically add this to an AGENTS.md).

If you’ve already installed the Air CLI separately, note that the bundled version will take precedence in Positron’s terminals by default. To force both the editor and the integrated terminal to use your externally installed version of Air, set air.executableStrategy: "environment". To opt-out of terminal support, set air.addExecutableToTerminalPath: false.

RStudio improvements

Thanks to Kevin Ushey, RStudio’s support for Air is improving rapidly!

Note that you’ll need RStudio 2026.06.0 for this, which isn’t out yet, but should be by the end of next week.

RStudio now has “native” support for Air, rather than just being configurable as an External Formatter. If you opt-in to Air, RStudio will now automatically download the latest version of the air binary the first time you save a file, making the whole experience much smoother.

You can turn this on via Tools -> Global Options... -> Code -> Formatting. I recommend checking both Use Air for code formatting and Reformat documents on save.

Lions, tigers, and bears uv, pixi, and mise, oh my!

Air now lives on PyPi as air-formatter (sadly air was already taken), which means that you can now install Air’s CLI via uv:

# Install globally
uv tool install air-formatter
air format path/to/file.R

# Run one-off command
uvx --from air-formatter air format path/to/file.R

It might sound a little absurd to put an R code formatting tool on Python’s PyPI, but in a world where an increasingly large number of R users are also uv users, it’s pretty darn convenient to be able to invoke it this way!

Additionally, Air is on conda-forge thanks to @salim-b, which means you can also install it with pixi and mise:

# Add to a project
pixi add air
mise use conda:air

# Install globally
pixi global install air
mise use --global conda:air

# Run one-off command
pixi exec air format path/to/my/script.R
mise exec conda:air -- air format path/to/my/script.R

pre-commit support

Air now has pre-commit and prek support via posit-dev/air-pre-commit. To run Air on changed R files before every commit, add the following to your .pre-commit-config.yaml:

repos:
- repo: https://github.com/posit-dev/air-pre-commit
  # Air version
  rev: 0.10.0
  hooks:
    # Run the formatter
    - id: air-format

stdin support

Air’s CLI now has stdin support! You can activate this with the new --stdin-file-path option like so:

cat R/across.R | air format --stdin-file-path R/across.R

air will receive the contents of cat R/across.R on stdin, format it, and then emit the formatted result back out on stdout.

You typically won’t call this directly, but it’s very useful for editors and IDEs without full language server support. For example, with Zed you can configure Air as an external formatter with the following setup:

"formatter": {
  "external": {
    "command": "air",
    "arguments": ["format", "--stdin-file-path", "{buffer_path}"]
  }
}

This allows Zed to invoke Air on every file save.

Note that Zed does have full language server support, and it’s recommended to use our Air extension for Zed instead, but it makes for a nice example anyways!

Invoking Air via stdin can also be useful in some embedded contexts, such as inside arf, an R console.

Shell completions

Thanks to @salim-b, Air can now generate shell completions for itself via a new air generate-shell-completion <shell> command, where <shell> can be one of zsh, bash, powershell, fish, or elvish. Read the documentation for how to set this up for your specific shell. With shell completions set up, you can Tab your way through air commands at the command line:

Installing CatBoost

The one wrinkle is installation. The CatBoost R package is not on CRAN, so you can’t reach for install.packages("catboost") directly.

Grab the URL for your platform from the CatBoost R installation guide and install it with the remotes package. For example, on an Apple Silicon or Intel Mac:

install.packages("remotes")
remotes::install_url(
  "https://github.com/catboost/catboost/releases/download/v1.2.10/catboost-R-darwin-universal2-1.2.10.tgz",
  INSTALL_opts = c("--no-multiarch", "--no-test-load", "--no-staged-install")
)

Swap in the release version, operating system, and architecture that match your setup. The guide lists the full URL pattern and the binaries available for each release.

Once CatBoost itself is installed, the tidymodels packages is just the usual packages:

# install.packages("pak")
pak::pak(c("tidymodels", "bonsai"))

You’ll need bonsai 0.4.1 (or later) and parsnip 1.4.0 (or later), which is where the CatBoost engine landed and got polished.

Fitting a CatBoost model

CatBoost is supported as an engine for []boost_tree()](https://parsnip.tidymodels.org/reference/details_boost_tree_catboost.html). Loading bonsai registers the engine, and from there it behaves like any other parsnip model spec:

library(tidymodels)
library(bonsai)

cat_spec <-
  boost_tree(trees = 500, learn_rate = 0.05) |>
  set_engine("catboost") |>
  set_mode("regression")

cat_fit <- fit(cat_spec, mpg ~ ., data = mtcars)
cat_fit

parsnip model object

CatBoost model (500 trees)
Loss function: RMSE
Fit to 10 feature(s)

Tuning

The CatBoost engine supports the standard boost_tree() tuning parameters, and the recent releases made tuning both faster and more correct.

A typical tuning setup looks like this:

cat_spec <-
  boost_tree(trees = tune(), learn_rate = tune()) |>
  set_engine("catboost") |>
  set_mode("regression")

cat_wf <- workflow(mpg ~ ., cat_spec)

set.seed(123)
folds <- vfold_cv(mtcars, v = 5)

tune_res <- tune_grid(cat_wf, resamples = folds, grid = 20)

show_best(tune_res, metric = "rmse")

# A tibble: 5 × 8
  trees learn_rate .metric .estimator  mean     n std_err .config         
  <int>      <dbl> <chr>   <chr>      <dbl> <int>   <dbl> <chr>           
1  1473     0.0379 rmse    standard    2.76     5   0.419 pre0_mod15_post0
2   527     0.0513 rmse    standard    2.81     5   0.443 pre0_mod06_post0
3  1053     0.0941 rmse    standard    2.83     5   0.447 pre0_mod11_post0
4   211     0.127  rmse    standard    2.84     5   0.400 pre0_mod03_post0
5   632     0.234  rmse    standard    2.86     5   0.480 pre0_mod07_post0

Thanks to the submodel trick tuning trees doesn’t require refitting the model from scratch at every candidate value.

orbital support

CatBoost models also work with orbital, allowing you to turn your fitted catboost model into SQL and run predictions directly inside a database.

library(orbital)

cat_spec <-
  boost_tree(trees = 50, learn_rate = 0.05) |>
  set_engine("catboost") |>
  set_mode("regression")

cat_wf <- workflow(mpg ~ ., cat_spec)
cat_fit <- fit(cat_wf, data = mtcars)

orbital_obj <- orbital(cat_fit)
orbital_obj

── orbital Object ──────────────────────────────────────────────────────────────
• .pred = dplyr::case_when(cyl <= 5 ~ dplyr::case_when(wt <= 2.17 ~ dplyr ...
────────────────────────────────────────────────────────────────────────────────
1 equations in total.

From here you can predict in-database with predict() against a database connection, or generate the SQL directly with orbital_sql(). See the orbital documentation for the full set of supported backends.

Wrapping up

CatBoost is a great addition to the gradient boosting options available in tidymodels, especially if you work with categorical features or want a strong out-of-the-box model.

For the full details, see the bonsai changelog and the parsnip 1.4.0 release notes.

The v0.22.0 release is the largest step in that direction so far. It introduces footnotes, group-wise summary rows, a family of column-merging methods, a suite of text transformations, several value-substitution helpers, two new formatting methods, and a modern image-export pipeline through gtsave(). The LaTeX output gained the ability to render stubs and row groups, and Pandas is no longer a required dependency. There is a great deal to cover, so this post walks through each addition in turn, with a small working example for every one.

Footnotes with tab_footnote()

Footnotes are one of the oldest conventions in tabular presentation, and they solve a real problem: sometimes a value, a label, or a heading needs a short explanation that would clutter the table if placed inline. The new tab_footnote() method attaches a footnote to any location in the table and manages the marks for you, numbering them sequentially in the order they appear and collecting the notes themselves in the table’s footer.

A location is specified with one of the loc.* helpers, the same ones used elsewhere in the library for styling. You can attach a note to cells in the stub, to a column label, to the subtitle, or to body cells. Because the footnote text accepts md() and html(), you can format it with Markdown or raw HTML just as you would any other piece of table content.

import polars as pl
from great_tables import GT, loc, md
from great_tables.data import towny

towny_mini = (
    pl.from_pandas(towny)
    .filter(pl.col("csd_type") == "city")
    .select(["name", "density_2021", "population_2021"])
    .top_k(10, by="population_2021")
    .sort("population_2021", descending=True)
)

(
    GT(towny_mini, rowname_col="name")
    .tab_header(
        title=md("The 10 Largest Municipalities in `towny`"),
        subtitle="Population values taken from the 2021 census.",
    )
    .fmt_integer()
    .cols_label(density_2021="Density", population_2021="Population")
    .tab_footnote(
        footnote="Part of the Greater Toronto Area.",
        locations=loc.stub(rows=[
            "Toronto", "Mississauga", "Brampton", "Markham", "Vaughan"
        ]),
    )
    .tab_footnote(
        footnote=md("Density is in terms of persons per {{km^2}}."),
        locations=loc.column_labels(columns="density_2021"),
    )
    .tab_footnote(
        footnote="Census results made public on February 9, 2022.",
        locations=loc.subtitle(),
    )
    .opt_footnote_marks(marks="letters")
)

The marks themselves are configurable through opt_footnote_marks(). The default is a standard set of typographic symbols, but you can switch to numbers or letters, as we did above with marks="letters". The placement= argument on tab_footnote() controls whether a mark sits to the left or right of the cell content, and the default "auto" chooses a side based on the cell’s alignment.

Group-wise summaries with summary_rows()

When a table is divided into row groups, readers frequently want a per-group summary: a total, a mean, a minimum and maximum. The summary_rows() method computes these and inserts them as labeled rows within each group, either at the bottom (the default) or at the top.

The aggregations are described with the fns= argument, a dictionary whose keys become the row labels and whose values are the expressions to evaluate. The expressions can be Polars expressions, which is the most concise option when your data is a Polars DataFrame, or plain Python callables that receive a DataFrame subset. A formatting function from the vals.* family can be passed through fmt= so that the summary values match the formatting of the rest of the table.

import polars as pl
from great_tables import GT, vals
from great_tables.data import gtcars

gtcars_mini = (
    pl.from_pandas(gtcars)
    .select(["mfr", "model", "hp", "trq"])
    .head(12)
)

(
    GT(gtcars_mini, rowname_col="model", groupname_col="mfr")
    .summary_rows(
        fns={
            "Min": pl.col("hp", "trq").min(),
            "Max": pl.col("hp", "trq").max(),
        },
        fmt=vals.fmt_integer,
    )
)

By default the summary applies to every group, but the groups= argument narrows it to a named subset when you only need summaries in certain places. The release also includes grand_summary_rows(), a companion method that produces a single summary across the entire table rather than one per group. Both kinds of summary can be targeted for styling through loc.summary() and loc.grand_summary(), so you can shade them or set them apart from the regular body rows.

Merging columns together

Tables often hold several columns that, conceptually, describe a single quantity. A value and its uncertainty, the lower and upper ends of a range, or a count paired with its percentage all read better as one column than as two. The release adds a family of merge methods for exactly these situations, along with a generic method for everything else.

The most specialized of these is cols_merge_uncert(), which combines a measured value with its uncertainty and renders the pair with a plus-or-minus separator. You provide the value column and the uncertainty column, and the second column is hidden automatically once it has been folded into the first.

from great_tables import GT
from great_tables.data import exibble
import polars as pl

exibble_mini = (
    pl.from_pandas(exibble)
    .select("num", "currency")
    .slice(0, 7)
)

(
    GT(exibble_mini)
    .fmt_number(columns="num", decimals=3, use_seps=False)
    .cols_merge_uncert(col_val="currency", col_uncert="num")
    .cols_label(currency="value + uncert.")
)

The cols_merge_range() method works the same way for a pair of columns that mark the beginning and end of a range, joining them with an en dash by default (the separator is adjustable through sep=). The cols_merge_n_pct() method pairs a count with a percentage, rendering values in the familiar 10 (16.70%) form and suppressing the percentage when the count is zero.

from great_tables import GT
import polars as pl

df = pl.DataFrame({
    "category": ["A", "B", "C"],
    "n": [10, 20, 30],
    "pct": [0.167, 0.333, 0.500],
})

(
    GT(df)
    .fmt_percent(columns="pct")
    .cols_merge_n_pct(col_n="n", col_pct="pct")
    .cols_label(n="Count (%)")
)

For anything that does not fit those three patterns there is the generic cols_merge(), which takes a list of columns and a pattern= template. The template uses zero-based indices in braces to refer to the columns, so a pattern of "{0} to {1}" interleaves the first and second columns with the literal text between them. The first column named becomes the visible, merged column, and the rest are hidden by default. This is the general mechanism on which the specialized methods are built, and it is the right tool when your desired arrangement is unusual.

A related convenience is cols_reorder(), which rearranges every column in a single call. Previously, a full reordering meant a sequence of cols_move_*() invocations; now you can list the columns in the order you want and have the table laid out accordingly. The method expects every column to appear exactly once, raising an error if any are omitted or duplicated, which guards against the silent loss of a column.

from great_tables import GT
from great_tables.data import exibble

exibble_mini = exibble[["num", "char", "fctr", "date", "time"]]

(
    GT(exibble_mini)
    .cols_reorder(["fctr", "date", "time", "char", "num"])
)

A suite of text transformations

Formatting methods handle numbers, dates, and currencies, but cell content sometimes needs a transformation that no formatter anticipates. The release introduces four text_*() methods that operate on the rendered text of cells, each addressing a different shape of problem.

The most general is text_transform(), which applies an arbitrary function to the text of the targeted cells. The function receives the cell’s current string and returns a new one, which makes it suitable for any transformation you can express in Python. Because it runs after formatting, you can format a value first and then decorate the result.

from great_tables import GT, loc, exibble

(
    GT(exibble[["num", "char"]].head(4))
    .fmt_number(columns="num", decimals=2)
    .text_transform(
        locations=[loc.body(columns="num"), loc.body(columns="char")],
        fn=lambda x: f"~ {x}",
    )
)

When the transformation is a regular-expression substitution, text_replace() is more direct. It takes a pattern= and a replacement=, and it supports capture groups, so you can wrap or rearrange matched text. The example below finds parenthetical text and emphasizes it with HTML tags.

import pandas as pd
from great_tables import GT, loc

df = pd.DataFrame({
    "item": ["Column A (details)", "Column B (info)"],
    "value": [1, 2],
})

(
    GT(df)
    .text_replace(
        pattern=r"\((.+?)\)",
        replacement=r"(<em>\1</em>)",
        locations=loc.body(columns="item"),
    )
)

The remaining two methods cover conditional replacement. text_case_match() is a switch-like construct: each case is a tuple pairing one or more values to match against a replacement string, with an optional default= for everything unmatched. text_case_when() generalizes this to predicates, where each case pairs a function that returns a boolean with the replacement to use when it is true. The case ordering matters, since the first matching predicate wins, which makes it a natural fit for binning a numeric column into labels.

import pandas as pd
from great_tables import GT, loc

df = pd.DataFrame({"score": [95, 72, 88, 61, 100]})

(
    GT(df)
    .fmt_number(columns="score", decimals=0)
    .text_case_when(
        (lambda x: int(x) >= 90, "A"),
        (lambda x: int(x) >= 80, "B"),
        (lambda x: int(x) >= 70, "C"),
        default="F",
        locations=loc.body(columns="score"),
    )
)

Substituting specific values

Closely related to text transformation is the act of replacing particular values for the sake of readability. A column of measurements might contain values too small to be meaningful, or zeros that would be better shown as a dash, or missing entries that should read as something other than a blank. The release adds a family of sub_*() methods for these cases: sub_missing() for missing values, sub_zero() for zeros, sub_small_vals() and sub_large_vals() for values beyond a threshold, and the general sub_values() for replacing any specified value.

The small-value substitution is representative. It replaces values whose magnitude falls below a threshold= with a chosen pattern, which is useful when very small numbers carry no real information and only distract. The sign= argument restricts the substitution to positive or negative values, so you can treat the two tails of a distribution differently.

from great_tables import GT
import polars as pl

neg_vals_df = pl.DataFrame({
    "i": range(1, 6),
    "numbers": [-0.0001, -0.005, -0.05, -1.0, -100.0],
})

(
    GT(neg_vals_df)
    .fmt_number(columns="numbers")
    .sub_small_vals(sign="-", threshold=0.01, small_pattern="~0")
)

These methods operate on the underlying values rather than on rendered text, so they compose cleanly with the formatting methods. You decide what counts as missing, zero, small, or large, and the table presents those cases consistently wherever they occur.

Two new formatters: durations and parts-per

The formatting family gained two members. The first, fmt_duration(), renders durations in any of several styles. Numeric inputs are interpreted according to an input_units= setting (seconds, minutes, hours, days, or weeks), while Polars Duration columns are detected automatically. The duration_style= argument selects between a narrow style such as 5d 3h, a wide style such as 5 days, 3 hours, a colon-separated style such as 02:15:30, and ISO 8601. The example below renders race times as zero-padded HH:MM:SS.

import pandas as pd
from great_tables import GT

df = pd.DataFrame({
    "event": ["Marathon", "Half Marathon", "10K", "Mile"],
    "winning_time_s": [7377, 3542, 1620, 233],
})

(
    GT(df)
    .fmt_duration(
        columns="winning_time_s",
        input_units="seconds",
        duration_style="colon-sep",
        output_units=["hours", "minutes", "seconds"],
    )
)

The second formatter, fmt_partsper(), handles parts-per quantities: per-mille, parts per million, parts per billion, and finer scales still. The to_units= argument names the target quantity, the values are scaled to match unless you opt out with scale_values=False, and the symbol is rendered appropriately for both HTML and LaTeX output. The example formats gas concentrations as parts per billion by volume.

import polars as pl
from great_tables import GT

concentrations = pl.DataFrame({
    "gas": ["CO", "NO2", "O3"],
    "conc": [1.5, 35.0, 120.0],
})

(
    GT(concentrations)
    .fmt_partsper(
        columns="conc",
        to_units="ppb",
        scale_values=False,
        symbol="ppbV",
    )
)

Saving tables as images with gtsave()

A display table is often destined for a slide deck, a report, or a README, and in those settings you need an image rather than live HTML. The new gtsave() method produces one by rendering the table in a headless instance of Chrome and capturing it. It writes PNG, JPEG, WebP, and PDF, choosing the format from the file extension you supply.

from great_tables import GT
from great_tables.data import gtcars
import polars as pl

gtcars_mini = (
    pl.from_pandas(gtcars)
    .select(["mfr", "model", "msrp"])
    .head(5)
)

(
    GT(gtcars_mini)
    .tab_header(title="Some Cars from gtcars")
    .fmt_currency(columns="msrp")
    .gtsave("my_table.png")
)

Several arguments control the capture. The zoom= factor governs the resolution of raster output, with higher values producing sharper images, while expand= adds padding around the table and vwidth=/vheight= set the viewport. The gtsave() method replaces the older save(), which is now deprecated; existing code will continue to work for the time being, but new work should use gtsave().

Better LaTeX output

Great Tables can render to LaTeX as well as HTML, and that path received substantial attention in this release. LaTeX output now supports the stub and row groups, including spanning column headers and the row-group-as-column layout, which means that tables relying on these structural features are no longer limited to HTML. In addition, Markdown and HTML content placed in cells, headers, or footnotes is now converted to its LaTeX equivalent during rendering, so styled text survives the trip into a LaTeX document rather than appearing as literal markup. For anyone producing tables destined for a paper or a typeset report, the LaTeX output is now much closer in capability to the HTML output.

Polars without Pandas

Until now, Great Tables required Pandas even if all of your work was in Polars. As of this release, Pandas is an optional dependency, and the library is fully functional with Polars alone. For Polars-first projects and for lightweight environments where every dependency counts, this removes a sizable transitive install that was not actually needed. Pandas users are unaffected: a DataFrame from either library works as input exactly as before, and the choice of backend remains yours.

Getting started

Great Tables v0.22.0 is available now on PyPI, so a pip install great-tables (or an upgrade of an existing install) brings everything described here. The documentation site, which moved to Great Docs as its generator in this release, covers each method in detail with runnable examples, and the User Guide has been updated to reflect the new features. The GitHub repository holds the source, the full changelog, and the issue tracker. This release also welcomed several first-time contributors, and if you would like to join them, or simply have a feature to request or a bug to report, the issue tracker is the place to start.

Since its inception, Project Jupyter has grown into one of the world’s most widely used open source ecosystems for interactive computing, powering breakthroughs in research, education, and industry. The Jupyter Foundation, hosted by the Linux Foundation, was established to support the long-term sustainability of this work by bringing together organizations committed to investing in the project’s future.

By joining as a Foundation Member, we are becoming active stewards of its future. Our membership helps fund the core infrastructure, the release engineering, and the community events that keep Jupyter at the cutting edge of scientific computing. This move reinforces our promise to the open source community: we will not only build on these tools, but we will also build up the people and processes that create them.

Open source is a shared resource, and it requires shared responsibility. We encourage our partners and peers in the industry to join us in supporting the foundations that make our work possible.

To learn more about our commitment to open research and technical innovation, visit our About Page.