@@ -467,30 +467,106 @@ Response:
467467
468468Map diverse API documentation terms to consistent `dlt` parameters. Identify the API ' s term first, then find the corresponding `dlt` key.
469469
470- | Concept | Common API Terms | dlt Parameter Key(s) | Notes |
471- | -------------------- - | ---------------------------------------------------- - | ---------------------- | -------------------------------------------- |
472- | ** Client** | | | |
473- | Base URL | Base URL , API Endpoint, Root URL , Service URL | `client.base_url` | Include version path (e.g., `/ v1/ ` ) |
474- | API Key (value) | API Key, Access Token, Secret, Token, Key | `client.auth.api_key` | (Handled via Secret Handling patterns) |
475- | API Key (param name)| `api_key` , `token` , `key` , `access_token` | `client.auth.name` | Query param name or Header name |
476- | API Key (location) | Query parameter, Header | `client.auth.location` | `query` or `header` |
477- | Bearer Token | Bearer Token, JWT | `client.auth.token` | (Handled via Secret Handling patterns) |
478- | ** Pagination** | | | ** Define per- resource if strategies differ!** |
479- | Next Cursor (source)| `next_cursor` , `next_page` , `nextToken` , `marker` | `paginator.cursor_path` | JSON path in response |
480- | Next Cursor (param) | `cursor` , `page_token` , `after` , `next ` , `marker` | `paginator.cursor_param` | Query param name to send cursor |
481- | Offset (param) | `offset` , `skip` , `start` , `startIndex` | `paginator.offset_param` | Query param name |
482- | Page Number (param) | `page` , `page_number` , `pageNum` | `paginator.page_param` | Query param name |
483- | Page Size (param) | `limit` , `per_page` , `page_size` , `count` , `maxItems` | `paginator.limit_param` | Query param name |
484- | Total Items (source)| `total` , `total_count` , `total_results` , `count` | `paginator.total_path` | Optional JSON path in response |
485- | Link Header Relation| `next ` , `last` | `paginator.next_url_path` | `rel` value in `Link` header |
486- | ** Incremental** | | | ** Define per- resource if strategies differ!** |
487- | Timestamp (param) | `since` , `updated_since` , `modified_since` , `from ` | `incremental.start_param` | Query param name |
488- | Timestamp (source) | `updated_at` , `modified` , `last_updated` , `ts` | `incremental.cursor_path` | JSON path in response item |
489- | ID / Sequence (param) | `since_id` , `min_id` , `after_id` , `sequence` | `incremental.start_param` | Query param name |
490- | ID / Sequence (source)| `id ` , `event_id` , `sequence_id` , `_id` | `incremental.cursor_path` | JSON path in response item |
491- | Initial Value | N/ A | `incremental.initial_value` | Start value for first run |
492- | ** Data** | | | |
493- | Data Array Path | `data` , `results` , `items` , `records` , `entries` | `endpoint.data_selector` | JSON path to the list of items |
470+ ```yaml
471+ client:
472+ base_url:
473+ common_api_terms: [" Base URL" , " API Endpoint" , " Root URL" , " Service URL" ]
474+ dlt_parameter: " client.base_url"
475+ notes: " Include version path (e.g., /v1/)"
476+
477+ auth:
478+ api_key_value:
479+ common_api_terms: [" API Key" , " Access Token" , " Secret" , " Token" , " Key" ]
480+ dlt_parameter: " client.auth.api_key"
481+ notes: " Handled via Secret Handling patterns"
482+
483+ api_key_param_name:
484+ common_api_terms: [" api_key" , " token" , " key" , " access_token" ]
485+ dlt_parameter: " client.auth.name"
486+ notes: " Query param name or Header name"
487+
488+ api_key_location:
489+ common_api_terms: [" Query parameter" , " Header" ]
490+ dlt_parameter: " client.auth.location"
491+ notes: " query or header"
492+
493+ bearer_token:
494+ common_api_terms: [" Bearer Token" , " JWT" ]
495+ dlt_parameter: " client.auth.token"
496+ notes: " Handled via Secret Handling patterns"
497+
498+ pagination:
499+ note: " Define per-resource if strategies differ!"
500+
501+ next_cursor_source:
502+ common_api_terms: [" next_cursor" , " next_page" , " nextToken" , " marker" ]
503+ dlt_parameter: " paginator.cursor_path"
504+ notes: " JSON path in response"
505+
506+ next_cursor_param:
507+ common_api_terms: [" cursor" , " page_token" , " after" , " next" , " marker" ]
508+ dlt_parameter: " paginator.cursor_param"
509+ notes: " Query param name to send cursor"
510+
511+ offset_param:
512+ common_api_terms: [" offset" , " skip" , " start" , " startIndex" ]
513+ dlt_parameter: " paginator.offset_param"
514+ notes: " Query param name"
515+
516+ page_number_param:
517+ common_api_terms: [" page" , " page_number" , " pageNum" ]
518+ dlt_parameter: " paginator.page_param"
519+ notes: " Query param name"
520+
521+ page_size_param:
522+ common_api_terms: [" limit" , " per_page" , " page_size" , " count" , " maxItems" ]
523+ dlt_parameter: " paginator.limit_param"
524+ notes: " Query param name"
525+
526+ total_items_source:
527+ common_api_terms: [" total" , " total_count" , " total_results" , " count" ]
528+ dlt_parameter: " paginator.total_path"
529+ notes: " Optional JSON path in response"
530+
531+ link_header_relation:
532+ common_api_terms: [" next" , " last" ]
533+ dlt_parameter: " paginator.next_url_path"
534+ notes: " rel value in Link header"
535+
536+ incremental:
537+ note: " Define per-resource if strategies differ!"
538+
539+ timestamp_param:
540+ common_api_terms: [" since" , " updated_since" , " modified_since" , " from" ]
541+ dlt_parameter: " incremental.start_param"
542+ notes: " Query param name"
543+
544+ timestamp_source:
545+ common_api_terms: [" updated_at" , " modified" , " last_updated" , " ts" ]
546+ dlt_parameter: " incremental.cursor_path"
547+ notes: " JSON path in response item"
548+
549+ id_sequence_param:
550+ common_api_terms: [" since_id" , " min_id" , " after_id" , " sequence" ]
551+ dlt_parameter: " incremental.start_param"
552+ notes: " Query param name"
553+
554+ id_sequence_source:
555+ common_api_terms: [" id" , " event_id" , " sequence_id" , " _id" ]
556+ dlt_parameter: " incremental.cursor_path"
557+ notes: " JSON path in response item"
558+
559+ initial_value:
560+ common_api_terms: [" N/A" ]
561+ dlt_parameter: " incremental.initial_value"
562+ notes: " Start value for first run"
563+
564+ data:
565+ data_array_path:
566+ common_api_terms: [" data" , " results" , " items" , " records" , " entries" ]
567+ dlt_parameter: " endpoint.data_selector"
568+ notes: " JSON path to the list of items"
569+ ```
494570
495571# # 7. Verification Checklist
496572
@@ -708,9 +784,3 @@ The presence of a field in the JSON that looks like a cryptic token (often base6
708784API docs using terminology like “cursor”, “continuation token”, “next token”, or showing request examples with parameters such as after, nextToken, cursor, etc.
709785Choose JSONResponseCursorPaginator if the API ’s pagination is driven by such tokens in the response body. You will configure cursor_path to point at the JSON field containing the token (e.g. " cursors.next" as default, or " next_cursor" , etc.), and cursor_param to the name of the query parameter the API expects (commonly " cursor" or " after" ). The paginator will then automatically extract the token and append it as ? cursor=< token> (or your specified param name) on subsequent calls
710786
711- # # Example markdown
712-
713- # Example markdown
714-
715- This is where you can write your first rule.
716-
0 commit comments