Skip to content

Commit d1000b8

Browse files
feat(AM335X): Add webserver demo user guide
Add the documentation for the new Webserver Demo added showcasing the audio analytics and performance stats. Signed-off-by: Telukula Jeevan Kumar Sahu <j-sahu@ti.com>
1 parent 7737bf1 commit d1000b8

File tree

9 files changed

+180
-8
lines changed

9 files changed

+180
-8
lines changed

configs/AM335X/AM335X_linux_toc.txt

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,7 @@ linux/Foundational_Components/Virtualization/Docker
100100
linux/Foundational_Components_Machine_Learning
101101
linux/Foundational_Components/Machine_Learning/armnn
102102
linux/Foundational_Components/Machine_Learning/tflite
103+
linux/Foundational_Components/Machine_Learning/nnstreamer
103104

104105
linux/Foundational_Components/Graphics/index
105106
linux/Foundational_Components/Graphics/Common/Display
@@ -113,6 +114,9 @@ linux/Foundational_Components/Graphics/SGX/Overview
113114
linux/Foundational_Components/Graphics/SGX/SGX_Debug_Info
114115
linux/Foundational_Components/Graphics/SGX/Build_Guide
115116

117+
linux/Demo_User_Guides/index_Demos
118+
linux/Demo_User_Guides/Webserver_Demo_User_Guide
119+
116120
linux/Examples_and_Demos
117121
linux/Examples_and_Demos_Application_Demos
118122
linux/Examples_and_Demos/Application_Demos/QT_Thermostat_HMI_Demo

source/devices/AM335X/linux/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ Processor SDK Linux Software Developer's Guide
1616
/linux/Overview
1717
Release_Specific
1818
/linux/Foundational_Components
19+
/linux/Demo_User_Guides/index_Demos
1920
/linux/Industrial_Protocols
2021
/linux/Examples_and_Demos
2122
/linux/How_to_Guides
260 KB
Loading
216 KB
Loading
330 KB
Loading
Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
.. _Webserver-Demo-User-Guide-label:
2+
3+
############################
4+
Web server demo - User Guide
5+
############################
6+
7+
*********
8+
Overview
9+
*********
10+
11+
This document describes the Out of Box Web Server Demo Application, which the Linux
12+
SDK delivers with the |__SDK_FULL_NAME__| |__PART_FAMILY_NAME__|, applicable to boards
13+
such as the `BEAGL-BONE-GRN-ECO <https://www.ti.com/tool/BEAGL-BONE-GRN-ECO>`__ and
14+
`TMDXEVM3358 <https://www.ti.com/tool/TMDXEVM3358>`__. The main purpose of this demo is to
15+
showcase a glimpse of the ARM analytics AI stack integrated into the filesystem,
16+
providing an out-of-box experience even users do not connect a display to the board.
17+
18+
The demo showcases a web server running on an AM335x based board, providing a web interface with three key sections:
19+
20+
* **Audio Classification Demo**: Users can connect a USB audio capture device to run real-time audio classification, displaying classification statistics.
21+
* **Live CPU Performance Metrics**: Provides a real-time CPU usage indicator, historical usage data for the last 5 minutes, and detailed CPU information.
22+
* **Documentation Links**: Offers convenient access to relevant documentation.
23+
24+
This document provides all necessary equipment requirements and instructions.
25+
26+
**********************
27+
Hardware Prerequisites
28+
**********************
29+
30+
- TI AM335x based board (e.g., `BEAGL-BONE-GRN-ECO <https://www.ti.com/tool/BEAGL-BONE-GRN-ECO>`__ and `TMDXEVM3358 <https://www.ti.com/tool/TMDXEVM3358>`__)
31+
(Note: For the Beaglebone Green Eco, a display is available via an `HDMI cape <https://wiki.seeedstudio.com/BeagleBone_Green_HDMI_Cape>`__)
32+
- PC (Windows or Linux)
33+
- Ethernet cables
34+
- Ethernet switch or ethernet router with DHCP service
35+
- SD card (minimum 16GB)
36+
- Audio Capture Device
37+
38+
***********
39+
Get Started
40+
***********
41+
42+
1. Flash an SD card with the :file:`tisdk-default-image`. User can download the :file:`tisdk-default-image` wic
43+
image from |__SDK_DOWNLOAD_URL__|. Please follow the instructions from here to :ref:`Flash an SD card <processor-sdk-linux-create-sd-card>`.
44+
45+
2. Insert the SD card into the AM335x based board and set it to boot via SD card Boot mode.
46+
47+
3. Connect an ethernet cable from your ethernet switch or router to the
48+
AM335x based board.
49+
50+
4. Connect your PC to the same ethernet switch or router.
51+
52+
5. Connect the UART to the PC's USB port.
53+
54+
6. Open a terminal program (like TeraTerm or minicom) and connect to the
55+
serial port. Set the port to 115200bps, 8 bit, no parity, 1 stop bit, no flow control.
56+
57+
7. Power on the AM335x based board.
58+
59+
8. After the Linux boot completes, login as "root". Use the :command:`ifconfig` command
60+
to find out the IP address of the device.
61+
62+
9. On the host PC, open a Internet Browser and enter in the following: ``http://<IP_ADDRESS_OF_AM335x_Board>:3000``
63+
64+
10. The following web page will be displayed, the home page shows as below.
65+
66+
.. image:: /images/Webserver_home_page.PNG
67+
:alt: Webserver Demo Page
68+
:width: 75%
69+
70+
*************************
71+
Audio Classification Demo
72+
*************************
73+
74+
The web interface provides an audio classification demo that uses the integrated ARM analytics AI stack.
75+
76+
.. image:: /images/Webserver_audio_classification.PNG
77+
:alt: Audio Classification Demo
78+
:width: 75%
79+
80+
To use the demo:
81+
82+
1. Connect a USB audio device (e.g., a USB headset) to the AM335x based board.
83+
2. Click the "Refresh Devices" button to detect the connected audio capture device.
84+
3. Select the desired audio capture device from the list of available devices.
85+
4. Click the "Start Classification" button to begin real-time audio classification.
86+
5. The "Live Classification" section will display the current classification, total classifications, unique classes, session time, and updates per minute.
87+
6. A "Classification History" log is also available, showing a timestamped record of classifications.
88+
7. Click the "Stop Classification" button to end the demo.
89+
90+
**Technical Details**
91+
92+
The audio classification demo leverages :ref:`NNStreamer <nnstreamer-label>`, a GStreamer-based neural network framework, to create and run the processing pipeline.
93+
94+
* **Model:** The demo uses the YAMNet sound classification model (:file:`yamnet_audio_classification.tflite`).
95+
* **Deep Learning Runtime:** The model is executed using the :ref:`TensorFlow Lite <tflite-label>` runtime.
96+
97+
**GStreamer Pipeline**
98+
99+
The following GStreamer pipeline is used to process the audio from the microphone, run the inference, and decode the results:
100+
101+
.. code-block:: console
102+
103+
gst-launch-1.0 alsasrc device=<device> ! \
104+
audioconvert ! \
105+
audio/x-raw,format=S16LE,channels=1,rate=16000,layout=interleaved ! \
106+
tensor_converter frames-per-tensor=3900 ! \
107+
tensor_aggregator frames-in=3900 frames-out=15600 frames-flush=3900 frames-dim=1 ! \
108+
tensor_transform mode=arithmetic option=typecast:float32,add:0.5,div:32767.5 ! \
109+
tensor_transform mode=transpose option=1:0:2:3 ! \
110+
queue leaky=2 max-size-buffers=10 ! \
111+
tensor_filter framework=tensorflow2-lite model=/usr/share/oob-demo-assets/models/yamnet_audio_classification.tflite custom=Delegate:XNNPACK,NumThreads:2 ! \
112+
tensor_decoder mode=image_labeling option1=/usr/share/oob-demo-assets/labels/yamnet_label_list.txt ! \
113+
filesink buffer-mode=2 location=<fifo_path>
114+
115+
****************************
116+
Live CPU Performance Metrics
117+
****************************
118+
119+
The web interface also provides a live view of the CPU performance metrics.
120+
121+
.. image:: /images/Webserver_CPU_Performance.PNG
122+
:alt: CPU Performance Metrics
123+
:width: 75%
124+
125+
This section includes:
126+
127+
* **CPU Usage**: A real-time gauge showing the current CPU utilization.
128+
* **CPU History**: A graph displaying the CPU usage history over the last 5 minutes.
129+
* **Statistics**: Average and maximum CPU usage calculated for the data from CPU History data.
130+
131+
*********************
132+
Software Architecture
133+
*********************
134+
135+
The demo is comprised of three main components: a web interface, a backend web server, and a set of Linux applications.
136+
137+
* **Web Interface (GUI):** A dynamic HTML page with JavaScript that runs in the user's browser. It provides the user interface for the audio classification demo and CPU performance metrics, making asynchronous requests to the web server for data updates.
138+
139+
* **Web Server:** A lightweight HTTP server (e.g., Node.js) running on the AM335x's Arm A8 core. It serves the static web page and provides a simple REST API for the frontend to interact with the underlying Linux applications.
140+
141+
* **Linux Applications:**
142+
* A C application that reads real-time system information (like CPU stats from :file:`/proc/stat`) and makes it available to the web server.
143+
- The audio classification demo, which is a GStreamer pipeline that leverages the NNStreamer framework. This pipeline reads from a USB audio device, processes the audio with the YAMNet model using the TensorFlow Lite runtime, and sends the classification results to the web server.
144+
145+
*******************
146+
Directory Structure
147+
*******************
148+
149+
Yocto recipe for building this demo can be found at `github:webserver-oob_git.bb <https://github.com/TexasInstruments/meta-tisdk/blob/scarthgap/meta-ti-foundational/recipes-demos/webserver-oob/webserver-oob_git.bb>`__.
150+
151+
The source code for the demo can be found at `github/TexasInstruments/webserver-oob-demo <https://github.com/TexasInstruments/webserver-oob-demo>`__ .
152+
For instructions on building the utilities see the `README <https://github.com/TexasInstruments/webserver-oob-demo/blob/main/README.md>`__ file.
153+
154+
+------+----------------------------------+---------------------------------------------+
155+
| | **Directory Name** | **Description** |
156+
+======+==================================+=============================================+
157+
| 1 | :file:`webserver_app/app` | GUI code (HTML, CSS, JavaScript) |
158+
+------+----------------------------------+---------------------------------------------+
159+
| 2 | :file:`webserver_app/linux_app` | Linux application code (C code) |
160+
+------+----------------------------------+---------------------------------------------+
161+
| 3 | :file:`webserver_app/webserver` | Web server code (Node.js) |
162+
+------+----------------------------------+---------------------------------------------+

source/linux/Demo_User_Guides/index_Demos.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ The SDK supports the following Out-Of-Box demo-applications
1414
Seva_Store
1515
Chromium_Browser
1616
Benchmark_Demo_User_Guide
17+
Webserver_Demo_User_Guide
1718
Display_Cluster_User_Guide
1819
TI_LVGL_Demo_User_Guide
1920
AM62D_Dsp_Offload_User_Guide.rst

source/linux/Foundational_Components/Machine_Learning/nnstreamer.rst

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
.. _nnstreamer-label:
2+
13
NNStreamer
24
==========
35

@@ -37,7 +39,7 @@ Run the included unit tests:
3739
To list available elements:
3840

3941
.. code-block:: console
40-
42+
4143
root@am62xx-evm:~# gst-inspect-1.0 | grep nnstreamer
4244
nnstreamer: tensor_aggregator: TensorAggregator
4345
nnstreamer: tensor_converter: TensorConverter

source/linux/Foundational_Components/Machine_Learning/tflite.rst

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
.. _tflite-label:
2+
13
########################
24
TensorFlow Lite (LiteRT)
35
########################
@@ -176,22 +178,22 @@ The following performance numbers are captured with :command:`benchmark_model` o
176178
Based on the above data, using the XNNPACK delegate significantly improves inference times across all SoCs, though it generally increases initialization time and overall memory footprint.
177179

178180
.. note::
179-
181+
180182
The performance numbers mentioned above were recorded after stopping the out-of-box (OOB) demos included in the TI SDK.
181-
183+
182184
********************
183185
Example Applications
184186
********************
185-
186-
|__SDK_FULL_NAME__| has integrated opensource components like NNStreamer which can be used for neural network inferencing using the sample tflite models under :file:`/usr/share/oob-demo-assets/models/`
187+
188+
|__SDK_FULL_NAME__| has integrated opensource components like NNStreamer which can be used for neural network inferencing using the sample tflite models under :file:`/usr/share/oob-demo-assets/models/`
187189
Checkout the Object Detection usecase under :ref:`TI Apps Launcher - User Guide <TI-Apps-Launcher-User-Guide-label>`
188190

189191
Alternatively, if a display is connected, you can run the Object Detection pipeline using this command,
190192

191193
.. ifconfig:: CONFIG_part_variant in ('AM62X', 'AM62LX')
192194

193195
.. code-block:: console
194-
196+
195197
gst-launch-1.0 multifilesrc location=/usr/share/oob-demo-assets/videos/oob-gui-video-objects.h264 loop=true ! \
196198
h264parse ! avdec_h264 ! \
197199
tee name=tee_split0 \
@@ -220,7 +222,7 @@ Alternatively, if a display is connected, you can run the Object Detection pipel
220222
.. ifconfig:: CONFIG_part_variant in ('AM62PX')
221223

222224
.. code-block:: console
223-
225+
224226
gst-launch-1.0 multifilesrc location=/usr/share/oob-demo-assets/videos/oob-gui-video-objects.h264 loop=true caps=video/x-h264,width=1280,height=720,framerate=1/1 ! \
225227
h264parse ! v4l2h264dec capture-io-mode=4 ! \
226228
tee name=tee_split0 \
@@ -247,5 +249,5 @@ Alternatively, if a display is connected, you can run the Object Detection pipel
247249
The above GStreamer pipeline reads an H.264 video file, decodes it, and processes it for object detection using a TensorFlow Lite model, displaying bounding boxes around detected objects. The processed video is then composited and rendered on the screen using the ``kmssink`` element.
248250

249251
.. attention::
250-
252+
251253
The Example Applications section is not applicable for AM64x

0 commit comments

Comments
 (0)