Skip to content

feat(pkg-r): support sqlite engine in PinSource#251

Open
gadenbuie wants to merge 2 commits into
mainfrom
fix/pin-source-sqlite
Open

feat(pkg-r): support sqlite engine in PinSource#251
gadenbuie wants to merge 2 commits into
mainfrom
fix/pin-source-sqlite

Conversation

@gadenbuie

Copy link
Copy Markdown
Contributor

Summary

PinSource previously always used DuckDB, even for pins that deserialize to a plain data frame. Since duckdb and RSQLite are both optional (Suggests) in the R package, a user who installed only RSQLite could use DataFrameSource but not PinSource. This PR aligns PinSource with DataFrameSource's engine handling.

Key changes:

  • Extracted DataFrameSource's inline connection setup into a shared internal helper, new_dataframe_connection(df, table_name, engine), which creates an in-memory connection, registers the data frame, and applies the DuckDB security lockdown. DataFrameSource$initialize() now calls it.
  • PinSource$new() gains an engine argument (default getOption("querychat.DataFrameSource.engine", NULL), resolved via the same get_default_dataframe_engine() as DataFrameSource). Data-frame pins now share DataFrameSource's code path, so the SQL flavor reported by the inherited get_db_type() is resolved consistently.
  • The efficient direct DuckDB file read is kept for parquet/CSV/JSON pins under the duckdb engine. Requesting engine = "sqlite" for those pin types falls back to pin_read() with a warning that DuckDB reads them more efficiently.
  • duckdb is no longer unconditionally required by PinSource — it is only needed when actually used.

While here, this PR also backfills a changelog entry for the original PinSource feature (#246), which was merged without one, in both NEWS.md and the Python CHANGELOG.md. The Python side is otherwise intentionally unchanged: duckdb is a core dependency there and DataFrameSource/PinSource are DuckDB-only by design, so no equivalent gap exists.

Verification

board <- pins::board_temp()
pins::pin_write(board, mtcars, "mtcars", type = "rds")

# Force the SQLite engine for a data-frame pin
ps <- querychat::PinSource$new(board, "mtcars", engine = "sqlite")
ps$get_db_type()
#> [1] "SQLite"
ps$execute_query("SELECT * FROM mtcars WHERE mpg > 30")

# A file-type pin under sqlite warns, then reads via pin_read()
pins::pin_write(board, mtcars, "cars", type = "csv")
ps2 <- querychat::PinSource$new(board, "cars", engine = "sqlite")
#> Warning: Reading csv pin "cars" into SQLite.
#> i The duckdb engine reads csv pins more efficiently. ...
ps2$cleanup()
ps$cleanup()

New tests cover the SQLite data-frame path and the file-type fallback warning in tests/testthat/test-PinSource.R.

PinSource gains an engine argument mirroring DataFrameSource. Data-frame
pins (and any pin loaded under the sqlite engine) now share
DataFrameSource's connection path via the new new_dataframe_connection()
helper, so the SQL flavor is resolved consistently. The efficient direct
DuckDB file read is kept for parquet/csv/json pins under the duckdb
engine; requesting sqlite for those pins falls back to pin_read() with a
warning.
The PinSource feature (#246) was added without a NEWS/changelog entry.
@gadenbuie gadenbuie requested a review from cpsievert June 18, 2026 15:17
@gadenbuie

Copy link
Copy Markdown
Contributor Author

Python E2E failures are unrelated, this PR only touches R code

@gadenbuie gadenbuie marked this pull request as ready for review June 18, 2026 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants