picks #270

gogonzo · 2025-09-11T12:48:24Z

Closes insightsengineering/NEST-roadmap#36

Related issues in the milestone:

https://github.com/insightsengineering/teal.transform/milestone/1

See related PRs:

Installation

remotes::install_github("insightsengineering/teal.modules.general@redesign_extraction@main")

Table of content (not interactive)

Motivation
- Merging relational data
  - Handling ambiguous variables
- Merging interactively
1. Configure possible selection

Motivation

Merging relational data

Consider following tables orders, order_items, products, customers connected with join keys
following sql convention {child}.{$parent}_id = {$parent+"s"}.id, for example
orders.customer_id = customers.id. teal.data setup would look like this:

Sample data setup with relational tables and join keys

library(teal.data)
data <- within(teal_data(), {
  customers <- tibble::tribble(
    ~id, ~name, ~age, ~country,
    1, "Alice Johnson", 30, "USA",
    2, "Bob Smith", 25, "Canada",
    3, "Charlie Brown", 35, "UK",
    4, "David Wilson", 28, "Australia",
    5, "Emma Davis", 32, "USA",
    6, "Frank Miller", 27, "Canada",
    7, "Grace Taylor", 29, "UK",
    8, "Henry Clark", 33, "Australia",
    9, "Isabella Martinez", 26, "USA",
    10, "Jack Thompson", 31, "Canada"
  )

  orders <- tibble::tribble(
    ~id, ~customer_id, ~order_date, ~total_amount,
    101, 1, as.Date("2024-01-15"), 250.00,
    102, 1, as.Date("2024-02-01"), 150.00,
    103, 2, as.Date("2024-02-10"), 125.00,
    104, 3, as.Date("2024-02-15"), 200.00,
    105, 4, as.Date("2024-02-20"), 175.00,
    106, 5, as.Date("2024-03-01"), 300.00,
    107, 6, as.Date("2024-03-05"), 50.00,
    108, 7, as.Date("2024-03-10"), 225.00,
    109, 8, as.Date("2024-03-12"), 100.00,
    110, 9, as.Date("2024-03-15"), 275.00,
    111, 10, as.Date("2024-03-18"), 125.00,
    112, 2, as.Date("2024-03-20"), 150.00
  )

  order_items <- tibble::tribble(
    ~id, ~order_id, ~product_id, ~quantity, ~unit_price, ~total_price,
    201, 101, 1, 2, 100.00, 200.00,
    202, 101, 2, 1, 50.00, 50.00,
    203, 102, 2, 3, 50.00, 150.00,
    204, 103, 2, 1, 50.00, 50.00,
    205, 103, 3, 1, 75.00, 75.00,
    206, 104, 1, 2, 100.00, 200.00,
    207, 105, 3, 2, 75.00, 150.00,
    208, 105, 2, 1, 50.00, 50.00,
    209, 106, 1, 3, 100.00, 300.00,
    210, 107, 2, 1, 50.00, 50.00,
    211, 108, 1, 1, 100.00, 100.00,
    212, 108, 3, 2, 75.00, 150.00,
    213, 109, 2, 2, 50.00, 100.00,
    214, 110, 1, 2, 100.00, 200.00,
    215, 110, 3, 1, 75.00, 75.00,
    216, 111, 2, 2, 50.00, 100.00,
    217, 111, 1, 1, 100.00, 100.00,
    218, 112, 3, 2, 75.00, 150.00
  )

  order_files <- tibble::tribble(
    ~id, ~order_id, ~file_name, ~file_type,
    301, 101, "invoice_101.pdf", "invoice",
    302, 102, "receipt_102.pdf", "receipt",
    303, 103, "invoice_103.pdf", "invoice",
    304, 104, "receipt_104.pdf", "receipt",
    305, 105, "invoice_105.pdf", "invoice",
    306, 106, "receipt_106.pdf", "receipt",
    307, 107, "invoice_107.pdf", "invoice",
    308, 108, "receipt_108.pdf", "receipt",
    309, 109, "invoice_109.pdf", "invoice",
    310, 110, "receipt_110.pdf", "receipt",
    311, 111, "invoice_111.pdf", "invoice",
    312, 112, "receipt_112.pdf", "receipt"
  )

  products <- tibble::tribble(
    ~id, ~name, ~price, ~category, ~stock_quantity,
    401, "Laptop Pro", 100.00, "Electronics", 15,
    402, "Wireless Mouse", 50.00, "Electronics", 50,
    403, "Office Chair", 75.00, "Furniture", 8
  )

  product_components <- tibble::tribble(
    ~id, ~product_id, ~component_name, ~component_type, ~quantity_required, ~cost,
    501, 401, "CPU", "Processor", 1, 25.00,
    502, 401, "RAM", "Memory", 2, 15.00,
    503, 401, "SSD", "Storage", 1, 20.00,
    504, 401, "Screen", "Display", 1, 30.00,
    505, 402, "Optical Sensor", "Sensor", 1, 8.00,
    506, 402, "Wireless Module", "Connectivity", 1, 12.00,
    507, 402, "Battery", "Power", 1, 5.00,
    508, 403, "Steel Frame", "Structure", 1, 35.00,
    509, 403, "Cushion", "Comfort", 1, 20.00,
    510, 403, "Wheels", "Mobility", 5, 3.00
  )
})

join_keys(data) <- join_keys(
  join_key("customers", keys = "id"),
  join_key("orders", keys = c("id")),
  join_key("products", keys = c("id")),
  join_key("product_components", keys = c("id")),
  # foreign keys
  join_key("customers", "orders", keys = c(id = "customer_id")),
  join_key("products", "order_items", keys = c(id = "product_id")),
  join_key("products", "product_components", keys = c(id = "product_id")),
  join_key("orders", "order_items", keys = c(id = "order_id"))
)

print(join_keys(data))

Imagine now a scenario of a ggplot where one wants to select x, y, color, facet_rows,
facet_cols from any variable in any dataset.

Example ggplot with dynamic variable selection

ggplot(
  data = ?,
  aes(
    x = !!sym(input$x), # orders.order_date
    y = !!sym(input$y), # order_items.total_price
    color = !!sym(input$color) # products.category
  )
) +
  geom_line() +
  facet_grid(
    vars(!!sym(input$facet_rows)) # customers.country
  )

In order to create above visualization, datasets need to be merged as ggplot::aes is related to single
data object. Problem is solvable as teal.data has enough information to determine correct
merge call based and selected variables and join_keys (describing relationships between datasets).

Using dplyr only we need to perform following merge operation given that following variables have
been selected:

x: orders.order_date
y: order_items.order_items.total_price
color: products.category
facet_rows: customers.country

Merge operation using dplyr joins

data_w_merged <- within(data, {
  anl <- dplyr::select(orders, id, customer_id, order_date) %>%
    dplyr::left_join(dplyr::select(order_items, order_id, product_id, total_price), by = c(id = "order_id")) %>%
    dplyr::left_join(dplyr::select(products, id, category), by = c(product_id = "id")) %>%
    dplyr::left_join(dplyr::select(customers, id, country), by = c(customer_id = "id"))
})

Now anl can produce desired visualization

Creating visualization with merged dataset

# Create the visualization with merged data - sum moved to ggplot
data_w_plot <- within(data_w_merged, {
  library(ggplot2)

  # Create ggplot with sum calculation inside
  plot <- ggplot(
    data = anl,
    aes(
      x = order_date,
      y = total_price,
      color = category
    )
  ) +
    geom_line() +
    facet_grid(
      rows = vars(country),
      labeller = label_both
    )
  
  print(plot)
})

get_outputs(data_w_plot)[[1]]

Handling ambiguous variables

When merging datasets containing duplicated variable names dplyr::*_join(suffix = c(".x", ".y")) automatically
adds a suffix to the columns, so that names are alway unique.

Merging interactively

Developing system which can interactively handle merge is a challenging task not only for a reasons described above
but also due to additional layers which need to control described operation. These layers include:

Providing an API for app-developer to enable and set specific merge configuration.
Handling reactive relationship between all inputs (dataset -> variable -> value) and avoid unnecessary triggers.
Providing robust, easy to use merge-modules which can handle weirdest app-developer needs (1) and provide meaningful information to the app-user about consequences of the data/variable selections.

1. Configure possible selection

picks and choices/selected

We came with an idea of picks which allows app-developer to specify datasets, variables and values to be
selected by app-user during an app run. Each of them is based on the idea of choices/selected where app-developer
provides choices and what is selected by default. App-user though changes selected interactively.

graph TB
    subgraph AppDeveloper["👨‍💻 App Developer (Configuration Time)"]
        Dev[App Developer]
    end

    subgraph Variables["variables()"]
        VAR_Choices["choices"]
        VAR_Selected["selected"]
    end

    subgraph AppUser["👤 App User (Runtime)"]
        User[App User]
        UI["selectInput<br/>(UI Component)"]
    end

    Dev -->|"sets choices"| VAR_Choices
    Dev -->|"sets default selected"| VAR_Selected
    VAR_Choices -->|"displayed in"| UI
    VAR_Selected -->|"initial value in"| UI
    UI -->|"presents choices"| User
    User -->|"changes selection"| VAR_Selected

    classDef devStyle fill:#e1f5ff,stroke:#0066cc,stroke-width:2px
    classDef userStyle fill:#fff4e1,stroke:#cc6600,stroke-width:2px
    classDef choicesStyle fill:#d4edda,stroke:#28a745,stroke-width:2px
    classDef selectedStyle fill:#fff3cd,stroke:#ffc107,stroke-width:2px

    class Dev devStyle
    class User,UI userStyle
    class VAR_Choices choicesStyle
    class VAR_Selected selectedStyle

New design bases on an idea that a module can consume its arguments referring to any variable in any dataset. Consider following example, where:

a module uses x, y and facet arguments to create an interactive inputs,
user can select a variable from any dataset for x, y, facet
visualization will be build on a merged dataset containing these three variables.

# pseudocode
tm_example <- function(x, y, facet) {
  ui = function(id, x, y, facet) ...., # inputs will be here
  server = function(id, x, y, facet) {
    moduleServer(id, function(input, output, session) {
      output$plot <- renderPlot({
        merged_dataset |>
          ggplot(
            aes(
              x = <selected x var>,
              y = <selected y var>
            )          
          ) + geom_point() + facet_wrap(vars(<selected facet var>))
      })
    })
  }
}

To provide choices and default selection for x, y and facet we propose following api:

Proposed API using picks() for variable selection

# pseudocode
tm_example(
  x = picks(
    datasets(<choices>, <selected>),
    variables(<choices>, <selected>)
  ),
  y = picks(
    datasets(<choices>, <selected>),
    variables(<choices>, <selected>)
  ),
  facet = picks(
    datasets(<choices>, <selected>),
    variables(<choices>, <selected>)
  )
)

Where each function creates an object which holds the information consumed by the framework. choices and selected can be either:

tidyselect selection_helpers (?tidyselect::language)
Integer denoting index of column(s)
explicit character denoting the name of the objects/columns/level

Relationship between `picks` elements

Each picks element is evaluated in a sequence starting from datasets. selected in one of them determines possible choices of the next one. For example:

If datasets is selected to be iris, then following variables's choices will be variables of iris. selected can't be something else than a choices and so on.

graph TB
    subgraph "picks()"
        subgraph "datasets()"
            DS_Choices["choices<br/>(available datasets)"]
            DS_Selected["selected<br/>(chosen dataset)"]
        end

        subgraph "variables()"
            VAR_Choices["choices<br/>(available variables)"]
            VAR_Selected["selected<br/>(chosen variable)"]
        end
    end

    DS_Choices -->|"user selects from"| DS_Selected
    DS_Selected -->|"determines"| VAR_Choices
    VAR_Choices -->|"user selects from"| VAR_Selected

    DS_Selected -.->|"e.g., if 'iris' selected"| VAR_Choices
    VAR_Choices -.->|"then choices become<br/>iris columns"| VAR_Selected

    classDef choicesStyle fill:#d4edda,stroke:#28a745,stroke-width:2px
    classDef selectedStyle fill:#fff3cd,stroke:#ffc107,stroke-width:2px
    classDef flowStyle fill:#e8f4f8,stroke:#0066cc,stroke-width:1px,stroke-dasharray: 5 5

    class DS_Choices,VAR_Choices choicesStyle
    class DS_Selected,VAR_Selected selectedStyle

    style DS_Selected stroke-width:3px
    style VAR_Choices stroke-width:3px

Example settings

Please read carefully the code and see the description to understand how picks work.

Strict variables picks

picks below will create an input in the module where single variable can be selected from c("Sepal.Length", "Sepal.Width"). multiple = FALSE disallow user to select more than one choice.

Example: Strict variable picks with single selection

picks(
  datasets(choices = "iris", selected = "iris"),
  variables(choices = c("Sepal.Length", "Sepal.Width"), selected = "Sepal.Length", multiple = FALSE)
)

Dynamic variables choices

Following picks will create an input in the module where user will be able to select any variable from iris (any = everything()) and by default 1-st will be selected. Be careful, setting explicit selected when choices throws a warning as it is not certain for example that "Species" %in% everything().

Example: Dynamic variable choices with tidyselect

picks(
  datasets(choices = "iris", selected = "iris"),
  variables(choices = tidyselect::everything(), selected = 1, multiple = FALSE)
)

Dynamic variables from multiple datasets

Consider a situation when one wants to select a variable from either iris or mtcars. Instead of forcing app-developer to enumerate all possible choices for iris and mtcars. Following picks will create two related inputs for datasets and for variables. Input for variables will automatically update when dataset selection changes.

Example: Multiple datasets with dynamic variables

picks(
  datasets(choices = c("iris", "mtcars"), selected = "iris"),
  variables(choices = tidyselect::everything(), selected = 1, multiple = FALSE)
)

Dynamic everything

In extreme scenario also lists of datasets could be unknown. Or to avoid writing too much text, one can specify following picks.

Example: Fully dynamic dataset and variable selection

picks(
  datasets(choices = tidyselect::where(is.data.frame), selected = 1),
  variables(choices = tidyselect::everything(), selected = 1, multiple = FALSE)
)

Implementation in `teal_module`

teal_module will accept x, y and facet and hand-over them to both ui and server.

Module definition with picks arguments

tm_example <- function(x, y, facet) {
  module(
    ui = ui_example,
    server = srv_example,
    ui_args = list(x = x, y = y, facet = facet),
    server_args = list(x = x, y = y, facet = facet)
  )
}

On the ui part it is necessary to call picks_ui for each picks object.

UI implementation with picks_ui

ui_example <- function(id, x, y, facet) {
  ns <- NS(id)
  div(
    picks_ui(id = ns("x"), picks =  x),
    picks_ui(id = ns("y"), picks =  y),
    picks_ui(id = ns("facet"), picks =  facet),
    plotOutput(ns("plot"))
  )
}

In the server, picks are utilized in picks_srv which can be called per each pick, or for all at once (as in the example below). picks_srv is used only to resolve dynamic choices/selected and to handle interaction between inputs. selectors contain a list of selected datasets/variables for each pick. In this example selectors structure looks like this:

x: (reactiveVal)
  datasets: 
    choices: ...
    selected: ...
  variables: 
    choices: ...
    selected: ...
y: ...
facet: ...

To create a merged-dataset using information from app-user selection one needs to call merge_srv. picks_srv doesn't do anything else than controlling a selection for a number of reasons:

One might want to use different set of variables to perform merge. For example, some might be controlled with picks_ui/srv and have an UI element and some might be fixed and added optionally to the merge_srv(selectors).
Before merge is performed, one might want to validate if the selection is correct from a module perspective.

merge_srv returns a list with two reactives:

data: teal_data object with merged dataset and
merge_vars: named list of variables. List is named after selector name, for example merge_vars()$facet

Server implementation with merge_srv

srv_example <- function(id, data, x, y, facet) {
  moduleServer(id, function(input, output, session) {
    selectors <- picks_srv(data = data, picks =  list(x = x, y = y, facet = facet))

    merged <- merge_srv("merge", data = data, selectors = selectors)

    plot_q <- reactive({
      within(merged$data(),
        {
          merged %>%
            ggplot(aes(x = x, y = y)) +
            geom_point() +
            facet_wrap(vars(facet))
        },
        x = str2lang(merged$variables()$x),
        y = str2lang(merged$variables()$y),
        facet = str2lang(merged$variables()$facet)
      )
    })

    output$plot <- renderPlot({
      req(plot_q())
      rev(get_outputs(plot_q()))[[1]]
    })
  })
}

App example

Complete working app example with relational data and dynamic merging

devtools::load_all("teal.data")
devtools::load_all("teal.transform")
devtools::load_all("teal")
devtools::load_all("teal.modules.general")
library(dplyr)

data <- within(teal.data::teal_data(), {
  customers <- tibble::tribble(
    ~id, ~name, ~age, ~country,
    1, "Alice Johnson", 30, "USA",
    2, "Bob Smith", 25, "Canada",
    3, "Charlie Brown", 35, "UK",
    4, "David Wilson", 28, "Australia",
    5, "Emma Davis", 32, "USA",
    6, "Frank Miller", 27, "Canada",
    7, "Grace Taylor", 29, "UK",
    8, "Henry Clark", 33, "Australia",
    9, "Isabella Martinez", 26, "USA",
    10, "Jack Thompson", 31, "Canada"
  )

  orders <- tibble::tribble(
    ~id, ~customer_id, ~order_date, ~total_amount,
    101, 1, as.Date("2024-01-15"), 250.00,
    102, 1, as.Date("2024-02-01"), 150.00,
    103, 2, as.Date("2024-02-10"), 125.00,
    104, 3, as.Date("2024-02-15"), 200.00,
    105, 4, as.Date("2024-02-20"), 175.00,
    106, 5, as.Date("2024-03-01"), 300.00,
    107, 6, as.Date("2024-03-05"), 50.00,
    108, 7, as.Date("2024-03-10"), 225.00,
    109, 8, as.Date("2024-03-12"), 100.00,
    110, 9, as.Date("2024-03-15"), 275.00,
    111, 10, as.Date("2024-03-18"), 125.00,
    112, 2, as.Date("2024-03-20"), 150.00
  )

  order_items <- tibble::tribble(
    ~id, ~order_id, ~product_id, ~quantity, ~unit_price, ~total_price,
    201, 101, 401, 2, 100.00, 200.00,
    202, 101, 402, 1, 50.00, 50.00,
    203, 102, 402, 3, 50.00, 150.00,
    204, 103, 402, 1, 50.00, 50.00,
    205, 103, 403, 1, 75.00, 75.00,
    206, 104, 401, 2, 100.00, 200.00,
    207, 105, 403, 2, 75.00, 150.00,
    208, 105, 402, 1, 50.00, 50.00,
    209, 106, 401, 3, 100.00, 300.00,
    210, 107, 402, 1, 50.00, 50.00,
    211, 108, 401, 1, 100.00, 100.00,
    212, 108, 403, 2, 75.00, 150.00,
    213, 109, 402, 2, 50.00, 100.00,
    214, 110, 401, 2, 100.00, 200.00,
    215, 110, 403, 1, 75.00, 75.00,
    216, 111, 402, 2, 50.00, 100.00,
    217, 111, 401, 1, 100.00, 100.00,
    218, 112, 403, 2, 75.00, 150.00
  )

  order_files <- tibble::tribble(
    ~id, ~order_id, ~file_name, ~file_type,
    301, 101, "invoice_101.pdf", "invoice",
    302, 102, "receipt_102.pdf", "receipt",
    303, 103, "invoice_103.pdf", "invoice",
    304, 104, "receipt_104.pdf", "receipt",
    305, 105, "invoice_105.pdf", "invoice",
    306, 106, "receipt_106.pdf", "receipt",
    307, 107, "invoice_107.pdf", "invoice",
    308, 108, "receipt_108.pdf", "receipt",
    309, 109, "invoice_109.pdf", "invoice",
    310, 110, "receipt_110.pdf", "receipt",
    311, 111, "invoice_111.pdf", "invoice",
    312, 112, "receipt_112.pdf", "receipt"
  )

  products <- tibble::tribble(
    ~id, ~name, ~price, ~category, ~stock_quantity,
    401, "Laptop Pro", 100.00, "Electronics", 15,
    402, "Wireless Mouse", 50.00, "Electronics", 50,
    403, "Office Chair", 75.00, "Furniture", 8
  )

  product_components <- tibble::tribble(
    ~id, ~product_id, ~component_name, ~component_type, ~quantity_required, ~cost,
    501, 401, "CPU", "Processor", 1, 25.00,
    502, 401, "RAM", "Memory", 2, 15.00,
    503, 401, "SSD", "Storage", 1, 20.00,
    504, 401, "Screen", "Display", 1, 30.00,
    505, 402, "Optical Sensor", "Sensor", 1, 8.00,
    506, 402, "Wireless Module", "Connectivity", 1, 12.00,
    507, 402, "Battery", "Power", 1, 5.00,
    508, 403, "Steel Frame", "Structure", 1, 35.00,
    509, 403, "Cushion", "Comfort", 1, 20.00,
    510, 403, "Wheels", "Mobility", 5, 3.00
  )

  iris <- iris
  mtcars <- mtcars
  iris$id <- seq_len(nrow(iris))
  mtcars$id <- seq_len(nrow(mtcars))
  ADSL <- rADSL
  ADTTE <- rADTTE
  ADRS <- rADRS
  ADAE <- rADAE
  ADLB <- rADLB
  ADTR <- rADTR
})

join_keys(data) <- c(
  teal.data::default_cdisc_join_keys[c("ADSL", "ADTTE", "ADRS", "ADAE", "ADQS", "ADTR", "ADLB")],
  teal.data::join_keys(
    join_key("iris", keys = "id"),
    join_key("mtcars", keys = "id"),
    teal.data::join_key("customers", keys = "id"),
    teal.data::join_key("orders", keys = c("id")),
    teal.data::join_key("products", keys = c("id")),
    teal.data::join_key("product_components", keys = c("id")),
    # foreign keys
    teal.data::join_key("customers", "orders", keys = c(id = "customer_id")),
    teal.data::join_key("products", "order_items", keys = c(id = "product_id")),
    teal.data::join_key("products", "product_components", keys = c(id = "product_id")),
    teal.data::join_key("orders", "order_items", keys = c(id = "order_id")),
    # add missing keys
    teal.data::join_key("ADTR", "ADTR", keys = c("STUDYID", "USUBJID", "PARAMCD", "AVISIT")),
    teal.data::join_key("ADSL", "ADTR", keys = c("STUDYID", "USUBJID"))
  )
)

tm_example <- function(x, y, facet) {
  module(
    ui = ui_example,
    server = srv_example,
    ui_args = list(x = x, y = y, facet = facet),
    server_args = list(x = x, y = y, facet = facet)
  )
}

ui_example <- function(id, x, y, facet) {
  ns <- NS(id)
  div(
    picks_ui(id = ns("x"), picks =  x),
    picks_ui(id = ns("y"), picks =  y),
    picks_ui(id = ns("facet"), picks =  facet),
    plotOutput(ns("plot"))
  )
}

srv_example <- function(id, data, x, y, facet) {
  moduleServer(id, function(input, output, session) {
    selectors <- picks_srv(data = data, picks =  list(x = x, y = y, facet = facet))

    merged <- merge_srv("merge", data = data, selectors = selectors)

    plot_q <- reactive({
      within(merged$data(),
        {
          anl %>%
            ggplot(aes(x = x, y = y)) +
            geom_point() +
            facet_wrap(vars(facet))
        },
        x = str2lang(merged$variables()$x),
        y = str2lang(merged$variables()$y),
        facet = str2lang(merged$variables()$facet)
      )
    })

    output$plot <- renderPlot({
      req(plot_q())
      rev(get_outputs(plot_q()))[[1]]
    })
  })
}

app <- init(
  data = data,
  modules = modules(
     tm_example(
      x = picks(
        datasets("orders"),
        variables(selected = "order_date")
      ),
      y = picks(
        datasets("order_items"),
        variables(selected = "total_price")
      ),
      facet = picks(
        datasets("customers"),
        variables(selected = "country")

      )
    ),
    modules(
      label = "Display e2e configuration",
      tm_merge(
        label = "adam",
        inputs = list(
          a = picks(
            datasets("ADTTE"),
            variables(multiple = TRUE)
          ),
          b = picks(
            datasets(choices = tidyselect::where(is.data.frame), selected = "ADSL"),
            variables(is_categorical(min.len = 2, max.len = 20), selected = 1, multiple = TRUE)
          ),
          c = picks(
            datasets(tidyselect::everything(), "ADTTE"),
            variables(choices = c(AGE:ARM, PARAMCD), selected = AGE, multiple = TRUE)
          ),
          d = picks(
            datasets(choices = "ADRS", selected = "ADRS"),
            variables(choices = "PARAM", selected = "PARAM"),
            values(selected = tidyselect::everything(), multiple = TRUE)
          ),
          e = picks(
            datasets(selected = "ADSL"),
            variables(
              choices = variable_choices("whatever", subset = function(data) {
                idx <- vapply(data, is.factor, logical(1))
                names(data)[idx]
              })
            )
          )
        )
      ),
      tm_merge(
        label = "non adam",
        inputs = list(
          a = picks(
            datasets(
              choices = tidyselect::where(is.data.frame) & !tidyselect::starts_with("AD"),
              selected = "orders"
            ),
            variables(
              selected = "order_date",
              multiple = TRUE
            )
          ),
          b = picks(
            datasets(selected = "products"),
            variables(selected = "price", multiple = TRUE)
          ),
          c = picks(
            datasets(selected = "order_items"),
            variables(multiple = TRUE)
          )
        )
      )
    )
  )
)

shinyApp(app$ui, app$server, enableBookmarking = "server")

Select specific variable(s) from a specific dataset

data_extract_spec(
  data = "iris"
  select = select_spec(
    choices = c("Sepal.Length", "Species"), 
    selected = "Species"
  )
)

# to
picks(
  datasets("iris", "iris"),
  variables(
    choices = c("Sepal.Length", "Species"), 
    selected = "Species"
  )
)

Select specific variable(s) from a selected dataset

list(
  data_extract_spec(
    data = "iris"
    select = select_spec(
      choices = c("Sepal.Length", "Species"), 
      selected = "Species"
    )
  ),
  data_extract_spec(
    data = "mtcars"
    select = select_spec(
      choices = c("mpg", "cyl"), 
      selected = "mpg"
    )
  )
)

# to
picks(
  datasets(c("iris", "mtcars"), "iris"),
  variables(
    choices = c("Sepal.Length", "Species", "mpg", "cyl"), 
    selected = c("Species", "mpg")
  )
)

Select unknown variable(s) from a selected dataset

list(
  data_extract_spec(
    data = "iris"
    select = select_spec(
      choices = variable_choices("iris"), 
      selected = first_choice()
    )
  ),
  data_extract_spec(
    data = "mtcars"
    select = select_spec(
      choices = variable_choices("mtcars"), 
      selected = first_choice()
    )
  )
)

# to
picks(
  datasets(c("iris", "mtcars"), "iris"),
  variables(
    choices = tidyselect::everything(), 
    selected = 1L
  )
)

filtering by any variable

picks provides no equivalent to filter_spec feature. To achieve this, please create a teal_transform_module with
a filtering mechanism.

list(
  data_extract_spec(
    data = "iris"
    select = select_spec(
      choices = c("Sepal.Length", "Species"), 
      selected = first_choice()
    ),
    filter = filter_spec(
      vars = "Species",
      choices = c("setosa", "versicolor", "virginica"),
      selected = "setosa"
    ) 
  )
)

# to picks and transformators
picks(
  datasets("iris", "iris"),
  variables(
    choices = c("Sepal.Length", "Species"),
    selected = 1L
  )
)

# Apply filtering through teal_transform_module
transformators = teal_transform_module(
  ui = function(id) {
    ns <- NS(id)
    selectInput(
      ns("species"), 
      label = "Select species", 
      choices = c("setosa", "versicolor", "virginica"),
      selected = "setosa"
    )
  },
  server = function(id, data) {
    moduleServer(id, function(input, output, session) {
      reactive({
        req(input$species)
        within(
          data(), {
            iris <- iris %>% dplyr::filter(Species %in% !!filter_values)
          }, 
          filter_values = input$species)
      })
    })
  }
)

Conversion from to data-extract-spec

picks will completelly replace data_extract_spec (des) and will be the only tool to select-and-merge
in teal framework. So far des will be supported as soft deprecated. help("as.picks")
contains the information how to convert des into picks but in a limited scope.

data_extract_spec (or a list of des) containing only select_spec are convertible 1:1 to the picks
filter_spec is not convertible to picks as it variables used in filter can be different than variables selected in select_spec, thus hierarchical form of picks can't handle this case.
filter_spec can be converted to teal_transform_module and used in transformators argument instead and we recommend to do so. teal.transform::teal_transform_filter provides a simplified way to create such transformator.

Issues with `data_extract_spec`

API of data_extract_spec is bad encapsulation, hard to extend, confusing and easy to break.

Bad encapsulation

In filter_spec one can specify choices = value_choices(dataname, subset = function(data)), this is vulnerable
for multiple failures:

value_choices("dataname") can be completelly different than data_extract_spec(dataname), guess what happens? ;]
subset = function(data) is a function of a dataset, it means that even if vars = "Species" has been provideed one still have to do levels(data$Species) instead of levels(column)

As you can see, there are few places where scope of the classes is not well separated which leads to:

repeating same information on multiple levels of the hierarchy data_extract_spec(dataname) and value_choices(dataname), vars = "Species", levels(data$Species)
Repeating the same information also requires to be cautious to have both places correctly specified, otherwise error will occur

data_extract_spec(
  dataname = "iris",
  select = select_spec(
    choices = variable_choices(subset = function(data) names(data))
    selected = "Sepal.Length"
  ),
  filter = filter_spec(
    vars = "Species",
    choices = value_choices("iris, subset = function(data) levels(data$Species)),
    selected = first_choice()
  )
)

Conclusion:

value_choices(data) shouldn't have an argument data as it is "given" by the data_extract_spec. Same applies to variable_choices(data).
value_choices(subset = function(data)) should be a function of column which is set in filter_spec(vars)

Hard to extend

Recently one user asked to make data_extract_spec(datanames) delayed, so that it will adjust automatically to the existing datasets when resolved. Problem is API allow to have des for multiple datasets only when one knows their names. It is just done by making a list-of-des. Following code will produce dropdown with "iris" and "mtcars" and app-user will be able to switch between datasets (switching between des)

# pseudocode
list(
  data_extract_spec(dataname = "iris", ...),
  data_extract_spec(dataname = "mtcars", ...)
)

Proposition was that dataname = dataset_choices(), similar to the filter_spec(vars = variable_choices()). Let's consider how would it look like:

data_extract_spec(
  dataname = dataset_choices(choices = all_datasets(), selected = first_choice()),
  select = select_spec(
    choices = variable_choices(dataname = ??, choices = function(x) names(data))
  )
  filter = filter_spec(
    vars = variable_choices(dataname = ??, choices = function(x) names(data)), # how to obtain delayed `dataname`
    choices = value_choices(dataname = ??, choices = function(x) data$?? |> levels()) # how to obtain delayed `vars`
  )
)

To achive this, package would have to be seriously refactored, to be able to do following:

data_extract_spec(
  dataname = dataset_choices(choices = all_datasets(), selected = first_choice()),
  select = select_spec(
    choices = variable_choices(choices = function(x) names(data))
  ),
  filter = filter_spec(
    vars = variable_choices(choices = function(data) names(data)),
    choices = value_choices(choices = function(column) column |> levels())
  )
)

Let's just use above example and change function names:

picks(
  datanames(choices = all_datasets(), selected = first_choice()),
  variables(
    choices = variable_choices(choices = function(x) names(data))
  ),
  values(
    vars = variable_choices(choices = function(data) names(data)),
    choices = value_choices(choices = function(column) column |> levels())
  )
)

- multiple variables concatenates values

- tests

testing coverage

github-actions · 2025-10-30T15:01:38Z

Code Coverage Summary

Filename                          Stmts    Miss  Cover    Missing
------------------------------  -------  ------  -------  ------------------------------------------------------------------------------
R/0-as_picks.R                      127      77  39.37%   58, 115-198, 216-218
R/0-badge_dropdown.R                 26      26  0.00%    13-38
R/0-call_utils.R                    125      24  80.80%   23-28, 65, 67, 69, 132-138, 256-264
R/0-module_merge.R                  202       1  99.50%   301
R/0-module_picks.R                  254      32  87.40%   40-72, 105-106, 115, 200-201, 259, 403, 441, 453, 461
R/0-picks.R                         146       0  100.00%
R/0-print.R                          38       2  94.74%   50, 58
R/0-resolver.R                      123       0  100.00%
R/0-tidyselect-helpers.R             17      17  0.00%    18-35
R/0-tm_merge.R                       49      49  0.00%    42-96
R/check_selector.R                   33       0  100.00%
R/choices_labeled.R                 157      27  82.80%   67, 73, 78, 85, 101, 218-222, 226-231, 354-355, 357, 363, 390-397
R/choices_selected.R                 81      11  86.42%   211-237, 274
R/column_functions.R                  3       3  0.00%    15-18
R/data_extract_datanames.R           30       8  73.33%   15-19, 80-82
R/data_extract_filter_module.R      105      47  55.24%   95-108, 110-111, 113-130, 146-165
R/data_extract_module.R             305      67  78.03%   139, 144, 161, 164-169, 171, 190-193, 223-269, 507, 512, 688, 699-700, 778-783
R/data_extract_read_module.R        137       7  94.89%   34, 39-41, 43, 138, 155
R/data_extract_select_module.R       32      18  43.75%   29-46
R/data_extract_single_module.R       60       2  96.67%   30, 43
R/data_extract_spec.R                32       0  100.00%
R/delayed_choices.R                  34       1  97.06%   86
R/filter_spec.R                     186       1  99.46%   279
R/format_data_extract.R              16       1  93.75%   48
R/get_dplyr_call.R                  297       0  100.00%
R/get_merge_call.R                  276      29  89.49%   31-37, 48, 214-223, 387, 402-414
R/input_checks.R                     11       2  81.82%   15-16
R/merge_data_utils.R                  2       0  100.00%
R/merge_datasets.R                  134       6  95.52%   122, 248-252
R/merge_expression_module.R          60      11  81.67%   160-165, 183, 359-364
R/Queue.R                            23      15  34.78%   53-55, 78-104
R/resolve_delayed.R                  16       4  75.00%   76-79
R/resolve.R                         115      45  60.87%   46, 180-286
R/select_spec.R                      64       8  87.50%   98, 178-185
R/utils.R                            37      24  35.14%   31-44, 172-185
R/zzz.R                               3       3  0.00%    2-4
TOTAL                              3356     568  83.08%

Diff against main

Filename                    Stmts    Miss  Cover
------------------------  -------  ------  --------
R/0-as_picks.R               +127     +77  +39.37%
R/0-badge_dropdown.R          +26     +26  +100.00%
R/0-call_utils.R             +125     +24  +80.80%
R/0-module_merge.R           +202      +1  +99.50%
R/0-module_picks.R           +254     +32  +87.40%
R/0-picks.R                  +146       0  +100.00%
R/0-print.R                   +38      +2  +94.74%
R/0-resolver.R               +123       0  +100.00%
R/0-tidyselect-helpers.R      +17     +17  +100.00%
R/0-tm_merge.R                +49     +49  +100.00%
R/delayed_choices.R             0      -5  +14.71%
R/get_merge_call.R             -2       0  -0.08%
R/Queue.R                       0     +15  -65.22%
TOTAL                       +1105    +238  +1.94%

Results for commit: 621603d

Minimum allowed coverage is 80%

♻️ This comment has been updated with latest results

github-actions · 2025-10-30T15:01:57Z

Unit Tests Summary

1 files 28 suites 17s ⏱️
356 tests 353 ✅ 3 💤 0 ❌
942 runs 939 ✅ 3 💤 0 ❌

Results for commit 621603d.

♻️ This comment has been updated with latest results.

github-actions · 2025-10-30T15:02:01Z

Unit Test Performance Difference

Test Suite	$Status$	Time on `main`	$±Time$	$±Tests$	$±Skipped$	$±Failures$	$±Errors$
0-as_picks	👶		$+0.07$	$+5$	$0$	$0$	$0$
0-module_merge	👶		$+3.63$	$+44$	$0$	$0$	$0$
0-module_picks	👶		$+4.59$	$+76$	$+3$	$0$	$0$
0-picks	👶		$+2.45$	$+126$	$0$	$0$	$0$
0-print	👶		$+0.09$	$+14$	$0$	$0$	$0$
Queue	💀	$0.10$	$-0.10$	$-21$	$0$	$0$	$0$

Additional test case details

Test Suite	$Status$	Time on `main`	$±Time$	Test Case
0-as_picks	👶		$+0.01$	delayed_select_spec_is_convertible_to_variables
0-as_picks	👶		$+0.02$	eager_select_spec_is_convertible_to_variables
0-as_picks	👶		$+0.01$	select_spec_with_multiple_selected_convertible_to_variables
0-as_picks	👶		$+0.02$	select_spec_with_selected_NULL_is_convertible_to_variables
0-as_picks	👶		$+0.01$	throws_warning_with_teal_tranform_filter_instruction_for_eager_filter_spec
0-module_merge	👶		$+0.03$	merge_srv_accepts_data_argument_accepts_shiny_reactive_teal_data
0-module_merge	👶		$+0.04$	merge_srv_accepts_data_argument_doesn_t_accept_non_reactive_teal_data
0-module_merge	👶		$+0.03$	merge_srv_accepts_selectors_argument_accepts_empty_list_of_selectors
0-module_merge	👶		$+0.24$	merge_srv_accepts_selectors_argument_accepts_named_list_of_shiny_reactive_picks
0-module_merge	👶		$+0.05$	merge_srv_accepts_selectors_argument_doesn_t_accept_non_reactive_list_elements
0-module_merge	👶		$+0.04$	merge_srv_accepts_selectors_argument_doesn_t_accept_unnamed_list_of_selectors
0-module_merge	👶		$+0.55$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_anl_can_merge_deep_join_tree_by_pair_keys_and_finds_correct_merge_order
0-module_merge	👶		$+0.10$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_anl_contains_colnames_with_original_names_when_duplicated_for_the_same_dataset
0-module_merge	👶		$+0.23$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_anl_contains_selected_colnames_with_original_names_if_selected_from_a_multiple_datasets_and_not_duplicated
0-module_merge	👶		$+0.07$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_anl_contains_selected_colnames_with_original_names_if_variables_are_selected_from_a_single_dataset
0-module_merge	👶		$+0.22$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_anl_contains_selected_colnames_with_suffixes_names_if_duplicated_across_datasets
0-module_merge	👶		$+0.12$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_anl_is_filtered_by_POSIXct_variable_when_values_is_selected
0-module_merge	👶		$+0.11$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_anl_is_filtered_by_date_variable_when_values_is_selected
0-module_merge	👶		$+0.10$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_anl_is_filtered_by_factor_variable_when_values_is_selected
0-module_merge	👶		$+0.08$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_anl_is_filtered_by_logical_variable_when_values_is_selected
0-module_merge	👶		$+0.10$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_anl_is_filtered_by_numeric_variable_when_values_is_selected
0-module_merge	👶		$+0.07$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_data_returns_reactive_containing_teal_data_with_object_output_name_
0-module_merge	👶		$+0.19$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_data_returns_teal_data_with_merged_anl_using_join_fun
0-module_merge	👶		$+0.12$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_fails_when_selected_from_multiple_datasets_and_no_join_keys_between_selected_datasets
0-module_merge	👶		$+0.05$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_fails_when_selected_from_multiple_datasets_and_no_join_keys_primary_keys
0-module_merge	👶		$+0.19$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_fails_when_unresolved_picks_are_passed_to_the_module
0-module_merge	👶		$+0.40$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_join_keys_are_updated_to_contains_anl_anl_components
0-module_merge	👶		$+0.04$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_returns_list_with_two_reactives_variables_and_data
0-module_merge	👶		$+0.40$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_selected_join_keys_across_multiple_datasets_refers_to_the_same_column_in_anl_c_O.O_
0-module_merge	👶		$+0.05$	merge_srv_returns_list_with_data_teal_data_with_anl_and_variables_selected_anl_variables_variables_returns_reactive_list_named_after_selectors
0-module_picks	👶		$+0.04$	each_list_element_is_reactiveVal
0-module_picks	👶		$+0.04$	each_list_element_is_reactiveVal_containing_picks
0-module_picks	👶		$+0.04$	list_is_named_as_pick_argument
0-module_picks	👶		$+0.02$	picks_srv_accepts_data_as_reactive_environment
0-module_picks	👶		$+0.02$	picks_srv_accepts_data_as_reactive_named_list
0-module_picks	👶		$+0.02$	picks_srv_accepts_data_doesn_t_accept_non_reactive_list_environment_teal_data
0-module_picks	👶		$+0.03$	picks_srv_accepts_data_doesn_t_accept_reactive_non_named_list_or_non_environment
0-module_picks	👶		$+0.01$	picks_srv_accepts_picks_accepts_empty_list
0-module_picks	👶		$+0.04$	picks_srv_accepts_picks_as_list_of_picks_objects
0-module_picks	👶		$+0.07$	picks_srv_accepts_picks_as_single_picks_object
0-module_picks	👶		$+0.01$	picks_srv_accepts_picks_doesn_t_accept_NULL_picks
0-module_picks	👶		$+0.01$	picks_srv_accepts_picks_doesn_t_accept_list_of_non_picks
0-module_picks	👶		$+0.02$	picks_srv_accepts_picks_doesn_t_accept_list_of_picks_with_duplicated_names
0-module_picks	👶		$+0.01$	picks_srv_accepts_picks_doesn_t_accept_unnamed_list_of_picks
0-module_picks	👶		$+0.03$	picks_srv_resolves_datasets_datasets_predicate_are_resolved_on_init
0-module_picks	👶		$+0.10$	picks_srv_resolves_datasets_datasets_tidyselect_are_resolved_on_init
0-module_picks	👶		$+0.03$	picks_srv_resolves_datasets_provided_non_delayed_datasets_are_adjusted_to_possible_datanames
0-module_picks	👶		$+0.12$	picks_srv_resolves_picks_interactively_Setting_numeric_variable_resolves_values_to_be_a_slider_input_with_variable_range
0-module_picks	👶		$+0.03$	picks_srv_resolves_picks_interactively_adding_a_column_to_data_adds_new_choice_to_variables_choices
0-module_picks	👶		$+0.01$	picks_srv_resolves_picks_interactively_adding_a_dataset_to_data_adds_new_choice_to_dataset_choices
0-module_picks	👶		$+0.65$	picks_srv_resolves_picks_interactively_change_of_dataset_input_resolves_variables
0-module_picks	👶		$+0.07$	picks_srv_resolves_picks_interactively_changing_picks_resolved_doesn_t_change_picker_input
0-module_picks	👶		$+0.12$	picks_srv_resolves_picks_interactively_current_datasets_choices_selected_are_produced_in_picker_inputs
0-module_picks	👶		$+0.06$	picks_srv_resolves_picks_interactively_custom_choices_label_set_in_data_is_displayed_in_a_picker_input
0-module_picks	👶		$+0.06$	picks_srv_resolves_picks_interactively_custom_choices_label_set_in_picks_has_priority_over_data_label_is_displayed_in_a_picker_input
0-module_picks	👶		$+0.07$	picks_srv_resolves_picks_interactively_custom_choices_label_set_in_picks_is_displayed_in_a_picker_input
0-module_picks	👶		$+0.01$	picks_srv_resolves_picks_interactively_picker_input_choices_produces_class_specific_icons_for_datasets
0-module_picks	👶		$+0.14$	picks_srv_resolves_picks_interactively_picker_input_choices_produces_class_specific_icons_for_variable
0-module_picks	👶		$+0.28$	picks_srv_resolves_picks_interactively_removing_a_selected_dataset_from_data_removes_choice_from_dataset_choices_and_from_selection_with_warning
0-module_picks	👶		$+0.31$	picks_srv_resolves_picks_interactively_removing_a_selected_variable_from_data_removes_choice_from_dataset_choices_and_from_selection
0-module_picks	👶		$+0.22$	picks_srv_resolves_picks_interactively_switching_dataset_input_changes_variables_input
0-module_picks	👶		$+0.23$	picks_srv_resolves_picks_interactively_switching_variables_input_changes_values_input
0-module_picks	👶		$+0.24$	picks_srv_resolves_picks_interactively_variables_ordered_TRUE_returns_input_following_a_selection_order_instead_of_choices_order
0-module_picks	👶		$+0.05$	picks_srv_resolves_picks_named_non_delayed_picks_preserve_names
0-module_picks	👶		$+0.05$	picks_srv_resolves_picks_non_delayed_picks_are_returned_unchanged
0-module_picks	👶		$+0.18$	picks_srv_resolves_picks_non_delayed_picks_with_values_are_returned_unchanged_if_within_a_possible_choices
0-module_picks	👶		$+0.06$	picks_srv_resolves_picks_pick_elements_are_nullified_if_previous_selected_NULL
0-module_picks	👶		$+0.10$	picks_srv_resolves_picks_pick_elements_are_resolved_sequentially
0-module_picks	👶		$+0.04$	picks_srv_resolves_picks_picks_converted_from_des_with_variable_choices_are_resolved
0-module_picks	👶		$+0.04$	picks_srv_resolves_picks_picks_converted_from_variable_choices_fun_are_resolved
0-module_picks	👶		$+0.05$	picks_srv_resolves_picks_picks_with_multiple_FALSE_defaults_to_single_value_even_if_multiple_values_provided
0-module_picks	👶		$+0.05$	picks_srv_resolves_values_values_character_are_adjusted_to_possible_levels
0-module_picks	👶		$+0.06$	picks_srv_resolves_values_values_character_are_set_to_data_range_when_column_is_numeric
0-module_picks	👶		$+0.06$	picks_srv_resolves_values_values_numeric_are_adjusted_to_possible_range
0-module_picks	👶		$+0.06$	picks_srv_resolves_values_values_numeric_are_emptied_with_warning_when_column_is_not_numeric
0-module_picks	👶		$+0.06$	picks_srv_resolves_values_values_on_multiple_columns_are_resolved_to_be_concatenated_choices
0-module_picks	👶		$+0.06$	picks_srv_resolves_values_values_predicate_are_emptied_with_warning_when_data_returns_infinite
0-module_picks	👶		$+0.08$	picks_srv_resolves_values_values_predicate_are_resolved_on_init
0-module_picks	👶		$+0.08$	picks_srv_resolves_values_values_predicate_are_set_to_data_range_when_predicate_doesn_t_match_anything
0-module_picks	👶		$+0.06$	picks_srv_resolves_values_values_predicate_are_set_to_delayed_range_when_data_range_returns_infinite
0-module_picks	👶		$+0.08$	picks_srv_resolves_values_values_range_are_preserved_when_related_data_lacks_finite_values
0-module_picks	👶		$+0.05$	picks_srv_resolves_variables_variables_are_nullified_with_warning_when_selected_dataset_has_no_columns
0-module_picks	👶		$+0.06$	picks_srv_resolves_variables_variables_eager_are_adjusted_to_possible_column_names
0-module_picks	👶		$+0.04$	picks_srv_resolves_variables_variables_predicate_are_resolved_on_init
0-module_picks	👶		$+0.09$	picks_srv_resolves_variables_variables_tidyselect_are_resolved_on_init
0-picks	👶		$+0.00$	datasets_attributes_allows_explicit_fixed_FALSE_override
0-picks	👶		$+0.00$	datasets_attributes_allows_explicit_fixed_TRUE_override
0-picks	👶		$+0.00$	datasets_attributes_passes_additional_arguments_via_...
0-picks	👶		$+0.05$	datasets_attributes_sets_fixed_to_FALSE_for_delayed_choices_tidyselect_
0-picks	👶		$+0.00$	datasets_attributes_sets_fixed_to_FALSE_for_multiple_choices
0-picks	👶		$+0.00$	datasets_attributes_sets_fixed_to_TRUE_for_single_non_delayed_choice
0-picks	👶		$+0.00$	datasets_attributes_sets_multiple_attribute_to_FALSE_always_single_selection_
0-picks	👶		$+0.13$	datasets_basic_asserts_datasets_choices_argument_accepts_character_integer_predicate_function_and_tidyselect
0-picks	👶		$+0.02$	datasets_basic_asserts_datasets_choices_can_t_be_empty
0-picks	👶		$+0.23$	datasets_basic_asserts_datasets_selected_argument_character_1_integer_1_predicate_and_tidyselect_or_empty
0-picks	👶		$+0.01$	datasets_basic_asserts_datasets_selected_must_be_a_subset_of_choices
0-picks	👶		$+0.05$	datasets_basic_asserts_datasets_selected_warns_if_choices_are_delayed_and_selected_eager
0-picks	👶		$+0.01$	datasets_integration_with_tidyselect_helpers_accepts_tidyselect_all_of_
0-picks	👶		$+0.05$	datasets_integration_with_tidyselect_helpers_accepts_tidyselect_any_of_
0-picks	👶		$+0.04$	datasets_integration_with_tidyselect_helpers_stores_tidyselect_contains_as_a_quosure
0-picks	👶		$+0.05$	datasets_integration_with_tidyselect_helpers_stores_tidyselect_ends_with_as_a_quosure
0-picks	👶		$+0.05$	datasets_integration_with_tidyselect_helpers_stores_tidyselect_matches_as_a_quosure
0-picks	👶		$+0.05$	datasets_integration_with_tidyselect_helpers_stores_tidyselect_starts_with_as_a_quosure
0-picks	👶		$+0.00$	datasets_returns_datasets_defaults_selected_to_1_st_when_not_specified
0-picks	👶		$+0.01$	datasets_returns_datasets_returns_a_list_with_choices_and_selected_elements
0-picks	👶		$+0.00$	datasets_returns_datasets_returns_an_object_of_class_datasets_and_type_
0-picks	👶		$+0.00$	datasets_returns_datasets_sets_fixed_to_TRUE_when_single_choice
0-picks	👶		$+0.00$	datasets_returns_datasets_stores_custom_selected_value
0-picks	👶		$+0.01$	datasets_returns_datasets_stores_integer_selected_value_as_quosure
0-picks	👶		$+0.00$	datasets_returns_datasets_stores_static_character_vector_in_choices
0-picks	👶		$+0.01$	datasets_returns_quosures_for_delayed_evaluation_does_not_store_static_character_vector_as_a_quosure_in_choices
0-picks	👶		$+0.05$	datasets_returns_quosures_for_delayed_evaluation_stores_combined_tidyselect_expressions_as_a_quosure_in_choices
0-picks	👶		$+0.00$	datasets_returns_quosures_for_delayed_evaluation_stores_numeric_range_1_5_as_a_quosure_in_choices
0-picks	👶		$+0.00$	datasets_returns_quosures_for_delayed_evaluation_stores_symbol_range_a_b_as_a_quosure_in_choices
0-picks	👶		$+0.07$	datasets_returns_quosures_for_delayed_evaluation_stores_tidyselect_ends_with_as_a_quosure_in_choices
0-picks	👶		$+0.05$	datasets_returns_quosures_for_delayed_evaluation_stores_tidyselect_everything_as_a_quosure_in_choices
0-picks	👶		$+0.04$	datasets_returns_quosures_for_delayed_evaluation_stores_tidyselect_starts_with_as_a_quosure_in_choices
0-picks	👶		$+0.00$	datasets_returns_quosures_for_delayed_evaluation_stores_tidyselect_where_as_a_predicate_function_in_choices
0-picks	👶		$+0.00$	datasets_validation_and_warnings_does_not_warn_when_both_choices_and_selected_are_static
0-picks	👶		$+0.05$	datasets_validation_and_warnings_does_not_warn_when_selected_is_numeric_and_choices_are_delayed
0-picks	👶		$+0.05$	datasets_validation_and_warnings_warns_when_selected_is_explicit_and_choices_are_delayed
0-picks	👶		$+0.07$	picks_assertions_fails_when_first_element_is_not_datasets
0-picks	👶		$+0.01$	picks_assertions_fails_when_input_is_not_of_class_type_
0-picks	👶		$+0.05$	picks_assertions_fails_when_mixing_valid_and_invalid_types
0-picks	👶		$+0.10$	picks_assertions_fails_when_values_doesn_t_immediately_follow_variables
0-picks	👶		$+0.06$	picks_assertions_fails_when_values_exists_without_variables
0-picks	👶		$+0.01$	picks_assertions_fails_with_empty_input
0-picks	👶		$+0.05$	picks_assertions_succeeds_when_first_element_is_datasets
0-picks	👶		$+0.10$	picks_assertions_succeeds_when_values_immediately_follows_variables
0-picks	👶		$+0.10$	picks_assertions_succeeds_with_only_datasets_and_variables_no_values_
0-picks	👶		$+0.01$	picks_assertions_warns_when_element_with_dynamic_choices_is_followed_by_element_with_eager_choices
0-picks	👶		$+0.10$	picks_basic_structure_creates_a_picks_object_with_datasets_and_variables
0-picks	👶		$+0.10$	picks_basic_structure_creates_a_picks_object_with_datasets_variables_and_values
0-picks	👶		$+0.05$	picks_basic_structure_creates_a_picks_object_with_single_datasets_element
0-picks	👶		$+0.09$	picks_basic_structure_ignores_trailing_empty_arguments
0-picks	👶		$+0.05$	picks_basic_structure_returns_an_object_of_class_picks_and_list_
0-picks	👶		$+0.12$	values_assertions_values_choices_accepts_predicate_functions_character_numeric_2_date_2_and_posixct_2_._No_tidyselect
0-picks	👶		$+0.00$	values_assertions_values_succeeds_by_default
0-picks	👶		$+0.00$	values_attributes_fixed_FALSE_when_choices_is_a_predicate
0-picks	👶		$+0.00$	values_attributes_fixed_FALSE_when_choices_length_1
0-picks	👶		$+0.01$	values_attributes_fixed_TRUE_when_single_choice_is_provided
0-picks	👶		$+0.01$	values_attributes_multiple_set_to_TRUE_by_default
0-picks	👶		$+0.00$	variables_allow_clear_attribute_sets_allow_clear_to_FALSE_for_multiple_numeric_selected_tidyselect_
0-picks	👶		$+0.01$	variables_allow_clear_attribute_sets_allow_clear_to_FALSE_for_single_non_NULL_selected
0-picks	👶		$+0.00$	variables_allow_clear_attribute_sets_allow_clear_to_FALSE_for_single_numeric_selected
0-picks	👶		$+0.06$	variables_allow_clear_attribute_sets_allow_clear_to_FALSE_for_tidyselect_selected
0-picks	👶		$+0.01$	variables_allow_clear_attribute_sets_allow_clear_to_TRUE_for_multiple_selected
0-picks	👶		$+0.01$	variables_allow_clear_attribute_sets_allow_clear_to_TRUE_when_selected_is_NULL
0-picks	👶		$+0.02$	variables_attribute_interactions_all_three_attributes_can_be_set_independently
0-picks	👶		$+0.01$	variables_attribute_interactions_allow_clear_depends_on_multiple_when_selected_is_character
0-picks	👶		$+0.01$	variables_attribute_interactions_multiple_FALSE_and_ordered_FALSE_work_together
0-picks	👶		$+0.01$	variables_attribute_interactions_multiple_TRUE_and_ordered_TRUE_work_together
0-picks	👶		$+0.00$	variables_multiple_attribute_auto_detects_multiple_FALSE_from_single_selected
0-picks	👶		$+0.01$	variables_multiple_attribute_auto_detects_multiple_TRUE_from_selected_vector_length
0-picks	👶		$+0.06$	variables_multiple_attribute_defaults_to_NULL_for_tidyselect_selected
0-picks	👶		$+0.00$	variables_multiple_attribute_explicit_multiple_overrides_auto_detection
0-picks	👶		$+0.01$	variables_multiple_attribute_sets_multiple_to_FALSE_for_single_selected_value
0-picks	👶		$+0.00$	variables_multiple_attribute_sets_multiple_to_FALSE_when_explicitly_specified
0-picks	👶		$+0.01$	variables_multiple_attribute_sets_multiple_to_TRUE_for_multiple_selected_values
0-picks	👶		$+0.00$	variables_multiple_attribute_sets_multiple_to_TRUE_when_explicitly_specified
0-picks	👶		$+0.00$	variables_ordered_attribute_defaults_to_FALSE
0-picks	👶		$+0.01$	variables_ordered_attribute_sets_ordered_to_FALSE_when_explicitly_specified
0-picks	👶		$+0.01$	variables_ordered_attribute_sets_ordered_to_TRUE_when_specified
0-print	👶		$+0.00$	format.picks_for_picks_collection_formats_picks_with_datasets_variables_and_values_by_showing_them_all_explicitly
0-print	👶		$+0.01$	format.type_for_datasets_formats_datasets_with_character_choices_by_printing_them_explicitly
0-print	👶		$+0.05$	format.type_for_datasets_formats_datasets_with_tidyselect_choices_by_printing_matched_call_s_argument
0-print	👶		$+0.00$	format.type_for_values_formats_values_with_character_choices_by_printing_them_explicitly_with_their_attributes
0-print	👶		$+0.00$	print.type_for_variables_formats_variables_with_ordered_attribute_correctly
0-print	👶		$+0.00$	print.type_for_variables_prints_variables_with_character_choices_by_printing_them_explicitly
0-print	👶		$+0.01$	print_methods_output_correctly_print.picks_outputs_picks_to_console_with_its_class_and_elements
0-print	👶		$+0.01$	print_methods_output_correctly_print.type_outputs_datasets_to_console_with_class_name_choices_and_selected
0-print	👶		$+0.00$	print_methods_output_correctly_print_returns_invisibly
Queue	💀	$0.04$	$-0.04$	Queue_can_be_initialized
Queue	💀	$0.01$	$-0.01$	empty_method_removes_all_elements_from_queue
Queue	💀	$0.00$	$-0.00$	get_method_returns_elements_of_queue
Queue	💀	$0.00$	$-0.00$	pop_method_removes_first_element_from_queue
Queue	💀	$0.00$	$-0.00$	print_method_displays_proper_format
Queue	💀	$0.01$	$-0.01$	push_method_adds_elements_to_queue
Queue	💀	$0.01$	$-0.01$	push_method_can_add_multiple_elements
Queue	💀	$0.00$	$-0.00$	remove_method_can_remove_several_elements
Queue	💀	$0.01$	$-0.01$	remove_method_removes_specified_element_from_queue
Queue	💀	$0.01$	$-0.01$	size_method_returns_number_of_elements_in_queue

Results for commit 74816b8

♻️ This comment has been updated with latest results.

osenan

Great job!

The PR is very big. I need to take a look again to the tests.
I really like the testing is very complete. I also would need to check the issue and its documentation to verify that the issue is solved.

However I provided comments, they are basically related with code duplication. I also need that for package tests there is no need to always refer to testthat::my_function as we are allowed to load library(testthat) as part of the testthat.R file.

Please take a look at my comments while I study the issue and its documentation to provide more review.

osenan · 2025-11-12T13:54:37Z

DESCRIPTION

    shinyvalidate (>= 0.1.3),
+    shinyWidgets,
    stats,
+    teal,


there is no minimal version for teal and teal.data?

osenan · 2025-11-12T14:06:59Z

R/0-as_picks.R

+#' )
+#'
+#' @export
+as.picks <- function(x) { # nolint


is all lintrs that we need ot exclude or can we exclude specific lintr while keeping others?

osenan · 2025-11-12T14:10:17Z

R/0-as_picks.R

+  }
+}
+
+.as.picks.filter <- function(x, dataname) { # nolint


same as above, if the problem is from 1-3 lintr checks we can ignore specific lintrs while keeping lintr checks for the rest of the function. Otherwise it is ok to exclude all lintrs

osenan · 2025-11-12T14:11:39Z

R/0-as_picks.R

+        as.call(
+          c(
+            quote(paste),
+            lapply(variables, as.name),


do variables need to be always of the same type of vector (list, numeric, character) if is numeric or character, maybe it makes sense to make it more strict using some purrr::map_chr or similar to detect errors

osenan · 2025-11-12T14:14:43Z

R/0-as_picks.R

+.select_spec_to_variables <- function(x) {
+  if (length(x)) {
+    variables(
+      choices = if (inherits(x$choices, "delayed_data")) {


because we are assigning "delayed_data" and "des-delayed" data many times, why not assigning those strings to variables like:

desdelayed_class <- "des-delayed" delayed_class <- "delayed"data"

To reuse the class string and only require to change the name in one place if necessary?

osenan · 2025-11-12T18:40:33Z

inst/badge-dropdown/style.css

+.badge-dropdown .btn::after {
+  margin-left: 0.25rem;
+  vertical-align: 0.1em;
+}


please add newline at EOF

osenan · 2025-11-12T18:45:28Z

tests/testthat/test-0-as_picks.R

@@ -0,0 +1,51 @@
+testthat::describe("as.picks turns select_spec to variables", {


why don't loading library(testthat) at testthat.R file here? This is repeating testthat::it many times. This are not package functions, are tests and is ok to load testthat lib.

osenan · 2025-11-12T18:47:11Z

tests/testthat/test-0-as_picks.R

+        data_extract_spec(
+          dataname = "iris",
+          filter = filter_spec(vars = "Species", choices = levels(iris$Species), selected = levels(iris$Species)),
+        )


because is our warning we could put the message to be checked as well to distinguish from a generic warning

osenan · 2025-11-12T18:50:39Z

tests/testthat/test-0-module_merge.R

@@ -0,0 +1,987 @@
+testthat::describe("merge_srv accepts selectors argument", {
+  it("accepts named list of shiny::reactive picks", {
+    data <- teal.data::teal_data()


Ythe data, adsl and adae are repeated across all the test. You could declare them in describe and reduce code duplication.
Then in each it call the teal.data operations and the selectors

osenan · 2025-11-12T18:53:19Z

tests/testthat/test-0-module_merge.R

+
+testthat::describe("merge_srv returns list with data (teal_data with anl) and variables (selected anl variables)", {
+  it("returns list with two reactives: variables and data", {
+    shiny::reactiveConsole(TRUE)


From line 163 to 165 is repeated in all it. We can reduce code lines by declaring in describe

llrs-roche added 30 commits March 3, 2025 09:09

Add some initial definitions

e0e1571

Add some infrastructure

707224e

Resolution tested up to variables

9f9aacb

Add function for when a new selection is chosen

0d89dce

Raising more errors and silencing some tests

58182d2

Improve update_spec

c21591f

Simplify tests

8023b1c

Add print method

37225be

Uncomment some tests

e1ec420

Make print work

a8bb226

Move delayed to attributes

f4bf89a

Add missing method!

a1678fa

Simplify type creation

06c0eb9

Simplify resolver

208fb53

Delayed takes into consideration the selection too

cb94d6d

Fix mistake and remove delayed class

158eda9

Resolve better by names

d545331

Fix issues

99cadc2

Simplify c.type

b26695a

Use Group Generic Functions

6600d17

Improve tests

df47b48

Make it work for simple cases of values

e90025c

Allow OR combinations

a92f051

Make it easier to work with types

41800f3

Avoid conflicting names

3a39de2

Simplify according to new methods

6920d6e

Updating docs

d728739

Fix some checks

6a85027

Simplify and reorganize delayed

9993468

Basic operations work more flexible

3624441

gogonzo added 5 commits September 23, 2025 07:07

minor errors

e4e2b36

WIP

8b33a34

POC1

db9d038

don't trigger when changing selection

905c787

value(selected = everything) by default

5bde18e

llrs-roche mentioned this pull request Sep 25, 2025

Design data extract and data merge insightsengineering/NEST-roadmap#36

Open

gogonzo added 6 commits September 26, 2025 13:15

implement keep_order

67493a7

allow-empty

764333d

- adding icons to pickerInput-choicesOpt

1307329

- multiple variables concatenates values

WIP cleanup

e6c39df

- tests

WIP cleanup

3302ffb

testing coverage

WIP

c3f8e42

This was referenced Oct 30, 2025

picks insightsengineering/teal#1642

Open

picks insightsengineering/teal.modules.general#942

Open

gogonzo changed the title ~~Redesign teal_module(s) interactive arguments~~ picks Oct 30, 2025

gogonzo marked this pull request as ready for review October 30, 2025 14:49

insightsengineering unlocked this conversation Oct 30, 2025

github-actions bot and others added 2 commits October 30, 2025 14:51

[skip style] [skip vbump] Restyle files

a556485

r cmd check

394cbea

gogonzo mentioned this pull request Oct 30, 2025

picks insightsengineering/teal.gallery#247

Open

remove refactor notes

566a18c

pkgdown

621603d

gogonzo mentioned this pull request Oct 31, 2025

picks (short) insightsengineering/teal.gallery#248

Open

osenan assigned osenan and unassigned osenan Nov 12, 2025

osenan reviewed Nov 12, 2025

View reviewed changes

		@@ -0,0 +1,51 @@
		testthat::describe("as.picks turns select_spec to variables", {

Uh oh!

picks #270

Are you sure you want to change the base?

picks #270

Uh oh!

Conversation

gogonzo commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Table of content (not interactive)

Motivation

Merging relational data

Handling ambiguous variables

Merging interactively

1. Configure possible selection

picks and choices/selected

Relationship between picks elements

Example settings

Strict variables picks

Dynamic variables choices

Dynamic variables from multiple datasets

Dynamic everything

Implementation in teal_module

App example

Select specific variable(s) from a specific dataset

Select specific variable(s) from a selected dataset

Select unknown variable(s) from a selected dataset

filtering by any variable

Conversion from to data-extract-spec

Issues with data_extract_spec

Bad encapsulation

Hard to extend

Uh oh!

github-actions bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Summary

Diff against main

Uh oh!

github-actions bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit Tests Summary

Uh oh!

github-actions bot commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Unit Test Performance Difference

Uh oh!

osenan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gogonzo commented Sep 11, 2025 •

edited

Loading

Relationship between `picks` elements

Implementation in `teal_module`

Issues with `data_extract_spec`

github-actions bot commented Oct 30, 2025 •

edited

Loading

github-actions bot commented Oct 30, 2025 •

edited

Loading

github-actions bot commented Oct 30, 2025 •

edited

Loading