Skip to content

About

ESP32 DHT11 telemetry firmware in ESP-IDF (C): six components, core-pinned FreeRTOS tasks, RAM queue buffering, and HTTP JSON upload.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

5 Commits

Folders and files

Repository files navigation

GetDHTData

GetDHTData is an ESP-IDF C firmware application (declared as project Paperdata in CMakeLists.txt) that samples temperature and humidity from a DHT11 single-wire sensor on an ESP32. A FreeRTOS task pipeline reads the sensor periodically, serializes measurements into JSON with cJSON, buffers payloads in a RAM pointer array while Wi-Fi is disconnected, and uploads them via HTTP POST using esp_http_client. For an I2C SHT31 variant of this firmware, see the sister repository GetSHT31Data.

Architecture and component layout

+---------------+     worker_dht_json_handler     +---------------+
|  dht_sensor   |     (Queue, len 5, struct)      |feature_extract|
| read_dht_data | ------------------------------> | dht_json_task |
| (Core 1, P7)  |                                 | (Core 1, P8)  |
+---------------+                                 +---------------+
                                                          |
                                  worker_json_net_handler |
                                  (Queue, len 5, char *)  v
+-------------------+                                 +---------------------+
|     network.c     |   wifi_event_handler (BITS)     |       worker        |
| event_handler &   | ------------------------------> |json_network_retrieve|
| life_guard (C1,P9)|   BIT0: SUCCESS / BIT1: FAIL    |    (Core 0, P7)     |
+-------------------+                                 +---------------------+
                                                        |                 |
                                 WIFI_CONNECT_FAIL      |                 | WIFI_CONNECT_SUCCESS
                                 (buffer up to 127)     v                 v (drain buffer first,
                                                +---------------+         |  then live queue)
                                                |  json_buffer  |         |
                                                |  [128] (RAM)  | --------+
                                                +---------------+         |
                                                                          v
                                                                +-------------------+
                                                                |    http_build     |
                                                                | http_upload_build |
                                                                | (max 3 attempts)  |
                                                                +-------------------+

components/feature_extract (feature.c) performs cJSON serialization and RAM queue buffering rather than signal feature extraction.

Directory and component layout

Directory Files Responsibility
main main.c, Kconfig, CMakeLists.txt Initializes NVS flash (erasing and retrying on error), calls network_init(), initializes the HTTP client via http_init(), and starts the pipeline via worker_session().
components/common common.h, CMakeLists.txt Header-only shared constants (NORMAL_STACKSIZE, NETWORK_STACKSIZE, JSON_BUFFER_SIZE), dht_int_data_t, QueueGroupHandle_t, and workflow error codes (WORKFLOW_ERR_BASE + 1..3).
components/dht_sensor dht11.c, dht11.h, Kconfig, CMakeLists.txt, idf_component.yml Polls DHT11 via esp-idf-lib/dht in read_dht_data(), converts float readings to tenths (int16_t), and sends dht_int_data_t structs to worker_dht_json_handler.
components/feature_extract feature.c, feature.h, CMakeLists.txt, idf_component.yml Serializes dht_int_data_t to unformatted JSON strings using espressif/cjson in dht_json_task() and manages the RAM fallback array json_buffer via network_failed_buffer() and json_buffer_retrieve().
components/http_build http_build.c, http_build.h, Kconfig, CMakeLists.txt Configures esp_http_client for JSON HTTP POST in http_init() and runs bounded retry uploads in http_upload_build().
components/network network.c, network.h, Kconfig, CMakeLists.txt Configures Wi-Fi STA mode in network_init(), handles Wi-Fi/IP events updating wifi_event_handler, and runs the life_guard reconnect task.
components/worker worker.c, worker.h, CMakeLists.txt Creates the two FreeRTOS queues and three pipeline tasks in worker_session(), and runs json_network_retrieve() for live upload, offline buffering, and buffer drain.

FreeRTOS tasks

Function Task name string Stack size Priority Pinned core ID Queues / event groups
read_dht_data "READ SENSOR DATA" NORMAL_STACKSIZE (2048) 7 1 Sends dht_int_data_t to worker_dht_json_handler (len 5)
dht_json_task "JSON TO NETWORK" NORMAL_STACKSIZE (2048) 8 1 Receives from worker_dht_json_handler (len 5); sends char * to worker_json_net_handler (len 5)
json_network_retrieve "WIFI DATA TRANSFER" NETWORK_STACKSIZE (8192) 7 0 Waits on wifi_event_handler (WIFI_CONNECT_SUCCESS | WIFI_CONNECT_FAIL); receives char * from worker_json_net_handler (len 5)
life_guard "WIFI LIFE GUARD" NORMAL_STACKSIZE (2048) 9 1 Waits on wifi_event_handler (WIFI_CONNECT_FAIL)

Sampling, buffering, and HTTP retry behavior

  • Sampling interval and timing API (components/dht_sensor/dht11.c):

    • On the first execution (!initate_sensor), read_dht_data() reads once via dht_read_float_data(DHT_TYPE_DHT11, CONFIG_DHT_GPIO, &humidity, &tempreture) and drops that initial reading, sets initate_sensor = true, and records lastwaketime = xTaskGetTickCount().
    • Each loop iteration sleeps with xTaskDelayUntil(&lastwaketime, pdMS_TO_TICKS(DHT_WAKETIME)) (DHT_WAKETIME = 1500 ms) for fixed-period scheduling rather than relative vTaskDelay.
    • On ESP_OK, tempreture and humidity are multiplied by 10 and cast to int16_t, status is set to true, and data_success_int() pushes dht_int_data_t to worker_dht_json_handler with xQueueSend(..., pdMS_TO_TICKS(DHT_WAITTIME)) (DHT_WAITTIME = 500 ms, returning ESP_ERR_QUEUE_FULL if full).
    • On sensor read error, data_failed() sets status = false and sends dht_int_data_t with the same 500 ms queue timeout.
  • Queue capacities, Wi-Fi state, and buffering (components/worker/worker.c, components/feature_extract/feature.c, components/network/network.c):

    • worker_session() allocates worker_dht_json_handler (5 items of dht_int_data_t) and worker_json_net_handler (5 items of char *).
    • dht_json_task blocks with portMAX_DELAY receiving from worker_dht_json_handler, builds an unformatted JSON string with cJSON_PrintUnformatted(), and blocks with portMAX_DELAY sending the char * pointer to worker_json_net_handler.
    • network_init() starts Wi-Fi STA (WIFI_AUTH_WPA2_PSK, PMF capable, PMF not required). On WIFI_EVENT_STA_DISCONNECTED, event_handler() clears WIFI_CONNECT_SUCCESS (BIT0) and retries esp_wifi_connect() immediately while current_retry < MAX_RETRY (MAX_RETRY = 3; the vTaskDelay backoff using NETWORK_WAKETIME = 2000 ms is commented out). After 3 reconnect retries (once current_retry reaches 3, on the next disconnect event), WIFI_CONNECT_FAIL (BIT1) is set; life_guard wakes on BIT1, waits LIFE_GUARD_WAITTIME (3000 ms) via vTaskDelay, and calls esp_wifi_connect().
    • IP_EVENT_STA_GOT_IP resets current_retry = 0 and sets WIFI_CONNECT_SUCCESS (BIT0). IP_EVENT_STA_LOST_IP clears WIFI_CONNECT_SUCCESS (BIT0) and calls esp_wifi_connect().
    • json_network_retrieve waits via xEventGroupWaitBits(wifi_event_handler, (WIFI_CONNECT_SUCCESS | WIFI_CONNECT_FAIL), pdFALSE, pdFALSE, portMAX_DELAY):
      • When WIFI_CONNECT_FAIL is active (and WIFI_CONNECT_SUCCESS is clear): it receives each char * from worker_json_net_handler, sets is_failed_before = true, and stores the pointer in static char * json_buffer[JSON_BUFFER_SIZE] (JSON_BUFFER_SIZE = 128 in common.h) via network_failed_buffer(). Because network_failed_buffer() rejects items once (*amount_pointer) >= JSON_BUFFER_SIZE - 1, capacity is 127 buffered JSON payloads (0..126). Once full, newer incoming payloads are freed immediately with cJSON_free() and dropped (ESP_ERR_QUEUE_FULL).
      • When WIFI_CONNECT_SUCCESS is active: if is_failed_before && queue_stored_amount != 0, it drains json_buffer in FIFO order starting at real_data_position (0). Each successful http_upload_build() frees the entry via cJSON_free(), sets the slot to NULL, and advances real_data_position. On HTTP failure, the drain loop breaks and keeps remaining items at real_data_position for a subsequent pass. Once real_data_position == queue_stored_amount, it resets queue_stored_amount = 0, real_data_position = 0, and is_failed_before = false.
      • When !is_failed_before && queue_stored_amount == 0: it receives a live pointer from worker_json_net_handler, calls http_upload_build(json_receive), and unconditionally frees json_receive with cJSON_free().
  • HTTP client settings and retries (components/http_build/http_build.c):

    • http_init() configures esp_http_client_config_t with:
      • url: CONFIG_HTTP_SERVER_URL
      • method: HTTP_METHOD_POST
      • timeout_ms: 4000
      • event_handler: http_event_handler (logs HTTP_EVENT_ERROR, HTTP_EVENT_ON_DATA, HTTP_EVENT_DISCONNECTED)
      • buffer_size: 512
      • buffer_size_tx: 512
      • keep_alive_enable: true
    • Sets header Content-Type: application/json via esp_http_client_set_header().
    • http_upload_build() calls esp_http_client_set_post_field(http_handler, json_body, strlen(json_body)) and executes esp_http_client_perform() up to max_retry = 3 total attempts with no delay between attempts.
    • On ESP_OK from esp_http_client_perform(), it checks esp_http_client_get_status_code(): 200 <= status_code < 300 returns ESP_OK; any non-2xx status logs a warning ([WARN]Server returned non-2xx status code: %d) and continues until 3 total attempts complete. Any transport error (check != ESP_OK) also runs up to 3 total attempts before returning ESP_FAIL.

Configuration (Kconfig)

Symbol Kconfig file / menu Type Default value Description
CONFIG_ESP_DEBUG_LOGV main/Kconfig (ESP_MAIN_SETTING) bool y "SHOW THE INFORMATION OF DEBUG" (defined in Kconfig; currently unused in .c files)
CONFIG_ESP_DEVICE_ID main/Kconfig (ESP_MAIN_SETTING) string "ESP32_1" "DEVICE ID" sent as "device_id" in the JSON payload
CONFIG_DHT_GPIO components/dht_sensor/Kconfig (DHT_GPIO) int None (must be set) "specify DHT11 pin" connected to the DHT11 data line
CONFIG_HTTP_SERVER_URL components/http_build/Kconfig (HTTP Setting) string None (must be set) "SERVER_URL" passed to esp_http_client_config_t.url
CONFIG_NETWORK_SSID components/network/Kconfig (NETWORK_CFG) string None (must be set) "SSID" for Wi-Fi station connection
CONFIG_NETWORK_PASSWORD components/network/Kconfig (NETWORK_CFG) string None (must be set) "PASSWORD" for Wi-Fi station connection

HTTP JSON payload

dht_body_build() in components/feature_extract/feature.c builds an unformatted JSON object using cJSON_PrintUnformatted() and sends it as an HTTP POST body with Content-Type: application/json:

{"device_id":"ESP32_1","temperature":240,"humidity":550,"status":true}
  • device_id: string from CONFIG_ESP_DEVICE_ID (default "ESP32_1").
  • temperature: int16_t integer scaled by 10 from the float reading in dht11.c (e.g., 240 for 24.0 deg C).
  • humidity: int16_t integer scaled by 10 from the float reading in dht11.c (e.g., 550 for 55.0 %RH).
  • status: JSON boolean (true on successful dht_read_float_data() read, false on read failure).

This repository contains only the ESP32 firmware; the companion HTTP JSON ingestion server is maintained in a separate repository.

Build and flash

CMakeLists.txt sets cmake_minimum_required(VERSION 3.22) and project(Paperdata). dependencies.lock (lockfile 3.0.0) records target: esp32, ESP-IDF 6.1.0, esp-idf-lib/dht 1.2.0 ("^1.2.0" in components/dht_sensor/idf_component.yml), esp-idf-lib/esp_idf_lib_helpers 1.4.0, and espressif/cjson 1.7.19~2 ("^1.7.19" in components/feature_extract/idf_component.yml, requiring idf >= 5.0).

idf.py set-target esp32
idf.py menuconfig
idf.py build
idf.py flash monitor

Set CONFIG_DHT_GPIO, CONFIG_HTTP_SERVER_URL, CONFIG_NETWORK_SSID, CONFIG_NETWORK_PASSWORD, and optionally CONFIG_ESP_DEVICE_ID in idf.py menuconfig before building. Pass -p <PORT> to idf.py flash monitor if the serial port is not detected automatically.

Limitations

  • Volatile RAM-only offline buffer: json_buffer is a static RAM array of 128 pointers (127 usable slots before dropping new entries). All buffered payloads are lost on reboot or power loss.
  • Live HTTP failures while Wi-Fi is connected are discarded: In components/worker/worker.c, json_network_retrieve() buffers payloads into json_buffer only in the WIFI_CONNECT_FAIL branch. When WIFI_CONNECT_SUCCESS is active and json_buffer is empty, http_upload_build(json_receive) is followed unconditionally by cJSON_free(json_receive), so readings that fail HTTP POST across all 3 attempts (max_retry = 3) are lost.
  • Plaintext HTTP and missing TLS setup: components/http_build/http_build.c sets .url = CONFIG_HTTP_SERVER_URL without configuring cert_pem, crt_bundle_attach, or authentication headers. Using http:// sends payloads unencrypted, and https:// URLs lack explicit server certificate verification configuration in esp_http_client_config_t.
  • Wi-Fi reconnect and life_guard defects in components/network/network.c:
    • network_init() passes life_guard_task_handler (NULL) by value instead of &life_guard_task_handler to xTaskCreatePinnedToCore(), leaving life_guard_task_handler permanently NULL. Consequently, vTaskSuspend(life_guard_task_handler) in WIFI_EVENT_STA_START is skipped, and eTaskGetState(life_guard_task_handler) on MAX_RETRY is called with NULL.
    • IP_EVENT_STA_GOT_IP sets WIFI_CONNECT_SUCCESS (BIT0) without clearing WIFI_CONNECT_FAIL (BIT1). Because life_guard waits on WIFI_CONNECT_FAIL with xClearOnExit = false, BIT1 stays set after reconnection and life_guard continues waking every 3000 ms to call esp_wifi_connect().
    • The exponential delay in WIFI_EVENT_STA_DISCONNECTED (//vTaskDelay(pdMS_TO_TICKS(NETWORK_WAKETIME*current_retry));) is commented out, so the initial 3 reconnect attempts run consecutively with no pause.
  • DHT11 sensor characteristics and failure-path struct state:
    • The DHT11 single-wire sensor has lower measurement resolution and slower response than I2C digital sensors (such as SHT31); dht11.c polls every 1500 ms, discards the first boot sample, and truncates (float * 10) to int16_t.
    • When dht_read_float_data() fails, data_failed() sets status = false and sends dht_int_datas without resetting tempreture and humidity. If the first post-init read fails, those struct fields hold uninitialized stack contents; on subsequent failures, they retain the previous successful sample.
  • Dual-core affinity, missing Kconfig defaults, and naming mismatches:
    • Tasks pin explicitly to Core 0 and Core 1 in worker.c and network.c, so single-core targets cannot run the unmodified code.
    • CONFIG_DHT_GPIO, CONFIG_HTTP_SERVER_URL, CONFIG_NETWORK_SSID, and CONFIG_NETWORK_PASSWORD have no Kconfig defaults.
    • CONFIG_ESP_DEBUG_LOGV is declared in main/Kconfig without any C source references, CMakeLists.txt declares project(Paperdata), and components/feature_extract performs JSON serialization and buffering rather than feature extraction.

About

ESP32 DHT11 telemetry firmware in ESP-IDF (C): six components, core-pinned FreeRTOS tasks, RAM queue buffering, and HTTP JSON upload.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages