Repository navigation

feat: add lazy Arrow and database viewing - #1804

Closed

Fred-Wu wants to merge 82 commits into

REditorSupport:mainfrom

Fred-Wu:feature/lazy-data-view-final

Closed

feat: add lazy Arrow and database viewing#1804
Fred-Wu wants to merge 82 commits into
REditorSupport:mainfrom
Fred-Wu:feature/lazy-data-view-final

Commits on Sep 28, 2026
- feat: support lazy Arrow datasets in data viewer
  Fred Wu
  committed
  5ea6524
- refactor: separate lazy Arrow data viewer operations
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  f608629
- fix: preserve Arrow int64 column type
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  48dd22e
- fix: serialize Arrow list columns in data viewer
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  4978222
- fix: disable Arrow nested column operations
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  2a38711
- fix: serialize Arrow struct columns in data viewer
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  d0d94e0
- fix: normalize Arrow list columns before page binding
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  92313a8
- fix: reset Arrow cache when filters change
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  1044520
- perf: reduce Arrow data viewer cache materialization
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  b4e23fd
- fix: isolate Arrow query cache from source rows
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  69cecbb
- perf: fetch Arrow query pages by fragment
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  75021be
- fix: preserve Arrow query page types
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  aef7a90
- fix: format Arrow filtered pages by schema
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  1068bc5
- fix: preserve Arrow types before page handling
  
  Fred-Wu
  authored and
  Fred Wu
  committed
  be73678
- perf: imporve fetch pages by fragment
  Fred Wu
  committed
  a594b03
Commits on Sep 29, 2026
- feat: add lazy DBI data viewer
  Fred-Wu
  committed
  5729b28
- perf: optimize Arrow data viewer page fetching
  Fred Wu
  committed
  c3bd279
- Pef: optimize DBI data viewer page fetching
  Fred Wu
  committed
  f7db425
- refactor: align lazy data viewer paging
  Fred-Wu
  committed
  75261a9
- Add arrow, DBI and dbplyr to suggested packages
  Fred Wu
  committed
  962bfcb
- Set a configurable session request timeout
  Fred Wu
  committed
  0bc1895
- fix: wrap plain DBI table sources
  Fred-Wu
  committed
  69ab6b7
- fix: preserve DBI table source structure
  Fred-Wu
  committed
  0d46983
- fix: preserve DBI dates and cancel viewer loads
  Fred-Wu
  committed
  a0ad35c
Commits on Sep 30, 2026
- fix: preserve DBI date types and cancel closed viewer fetches
  Fred Wu
  committed
  b3c196e
- revert: remove DBI date projection workaround
  Fred-Wu
  committed
  875f861
- feat: add sess.quiet option and fix duplicate prompt at startup (#1795)
  
  grcatlin
  authored and
  Fred Wu
  committed
  6252778
- Merge branch 'REditorSupport:main' into feature/arrow-lazy-dataview
  Fred-Wu
  authored
  c8eea0e
- refactor: use DBI driver column types
  Fred-Wu
  committed
  1e0963a
- fix: preserve DBI query ordering
  Fred-Wu
  committed
  19459cf
- perf: optimize Arrow query and Parquet paging
  Fred-Wu
  committed
  83ed407
- refactor: align lazy data viewer behavior
  Fred-Wu
  committed
  75deff1
- feat: add lazy Arrow and database viewing
  Fred Wu
  committed
  35397ad
- Fix DBI test error and Arrow reader assertions in tests
  Fred Wu
  committed
  ce3642e
- Fix indentations that reulted in lint errors
  Fred Wu
  committed
  e70dff6
Commits on Oct 3, 2026
- feat: support lazy Arrow datasets in data viewer
  
  Fred Wu
  authored and
  Fred-Wu
  committed
  24313a8
- refactor: separate lazy Arrow data viewer operations
  Fred-Wu
  committed
  17ddf08
- fix: preserve Arrow int64 column type
  Fred-Wu
  committed
  0f7d05b
- fix: serialize Arrow list columns in data viewer
  Fred-Wu
  committed
  86482f2
- fix: disable Arrow nested column operations
  Fred-Wu
  committed
  22512f6
- fix: serialize Arrow struct columns in data viewer
  Fred-Wu
  committed
  6cc6871
- fix: normalize Arrow list columns before page binding
  Fred-Wu
  committed
  ce1deed
- fix: reset Arrow cache when filters change
  Fred-Wu
  committed
  411b40c
- perf: reduce Arrow data viewer cache materialization
  Fred-Wu
  committed
  e55087d
- fix: isolate Arrow query cache from source rows
  Fred-Wu
  committed
  8b5acca
- perf: fetch Arrow query pages by fragment
  Fred-Wu
  committed
  d8fba9d
- fix: preserve Arrow query page types
  Fred-Wu
  committed
  8f77556
- fix: format Arrow filtered pages by schema
  Fred-Wu
  committed
  a09195d
- fix: preserve Arrow types before page handling
  Fred-Wu
  committed
  92e9e3a
- perf: imporve fetch pages by fragment
  
  Fred Wu
  authored and
  Fred-Wu
  committed
  4c2e5a6
- feat: add lazy DBI data viewer
  Fred-Wu
  committed
  36a46e8
- perf: optimize Arrow data viewer page fetching
  
  Fred Wu
  authored and
  Fred-Wu
  committed
  6739174
- Pef: optimize DBI data viewer page fetching
  
  Fred Wu
  authored and
  Fred-Wu
  committed
  576f601
- refactor: align lazy data viewer paging
  Fred-Wu
  committed
  2a5a037
- Add arrow, DBI and dbplyr to suggested packages
  
  Fred Wu
  authored and
  Fred-Wu
  committed
  05a2148
- Set a configurable session request timeout
  
  Fred Wu
  authored and
  Fred-Wu
  committed
  ecf6259
- fix: wrap plain DBI table sources
  Fred-Wu
  committed
  e981453
- fix: preserve DBI table source structure
  Fred-Wu
  committed
  35c3e4a
- fix: preserve DBI dates and cancel viewer loads
  Fred-Wu
  committed
  b2943e3
- fix: preserve DBI date types and cancel closed viewer fetches
  
  Fred Wu
  authored and
  Fred-Wu
  committed
  e592771
- revert: remove DBI date projection workaround
  Fred-Wu
  committed
  5d83bd0
- refactor: use DBI driver column types
  Fred-Wu
  committed
  cd83c9c
- fix: preserve DBI query ordering
  Fred-Wu
  committed
  c22d88a
- perf: optimize Arrow query and Parquet paging
  Fred-Wu
  committed
  56f63d9
- refactor: align lazy data viewer behavior
  Fred-Wu
  committed
  629e52d
- fix: preserve lazy query results in data viewer
  Fred-Wu
  committed
  606ebe1
- refactor: delegate DBI SQL generation to dbplyr
  Fred-Wu
  committed
  b1a4fe3
- perf: cache Arrow lazy query results
  Fred-Wu
  committed
  1eab71d
- perf: cache DBI lazy query results
  Fred-Wu
  committed
  d58e857
- perf: cache DBI lazy query results
  Fred-Wu
  committed
  4e4e6b5
Commits on Oct 8, 2026
- Merge latest master into feature/arrow-lazy-dataview
  Fred-Wu
  committed
  e3871d6
- Merge remote feature/arrow-lazy-dataview, preserving current viewer implementation
  Fred-Wu
  committed
  9ddcd64
- fix: use dbplyr to look up primary key
  Fred-Wu
  committed
  88807c3
Commits on Oct 14, 2026
- Merge main into feature/arrow-lazy-dataview
  Fred Wu
  committed
  ecae666
- Merge remote feature/arrow-lazy-dataview
  Fred Wu
  committed
  ee7344d
- Merge current lazy data viewer work and latest main
  Fred Wu
  committed
  5883ea5
- fix: correctly detect bare database tables and repair DBI paging tests
  Fred Wu
  committed
  0d7ad03
- fix: keep random access inside arrow before conversion into R
  Fred Wu
  committed
  0d6b48d
- fix: preserve lazy query ordering and recover safely from fetch failures
  Fred Wu
  committed
  8545d0e
Commits on Oct 15, 2026
- feat: enable filtering and sorting for lazy Data Viewer queries
  Fred Wu
  committed
  12e5959
- fix: handle lazy Data Viewer failures and unknown row counts
  Fred Wu
  committed
  dbdb194
- fix(test): remove Arrow batch-order assumptions from Data Viewer tests
  Fred Wu
  committed
  4f9df9c

Commits

Commits on Sep 28, 2026

Commits on Sep 29, 2026

Commits on Sep 30, 2026

Commits on Oct 3, 2026

Commits on Oct 8, 2026

Commits on Oct 14, 2026

Commits on Oct 15, 2026