Skip to content

Commit 28d304f

Browse files
frallaxfrallax
committed
[WIP] TSP description, vision and guidelines
My interpretation of the TSP. A tentative description of what it is, what it aims to accomplish and the use cases it aims to solve. Some initial guidelines to follow when extending the TSP to accomodate new use cases.
1 parent fa3a7d9 commit 28d304f

File tree

2 files changed

+215
-5
lines changed

2 files changed

+215
-5
lines changed

‎README.md‎

Lines changed: 95 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,105 @@
11
# trace-server-protocol
22

3-
Specification of the Trace Server Protocol
4-
5-
This protocol is built to decouple the backend and frontend of trace analysers, allowing traces to reside and be analysed on the backend, and visual models to be exchanged with a variety of clients.
6-
7-
The protocol is meant to be RESTful, over HTTP.
3+
Specification of the Trace Server Protocol (TSP).
84

95
The specification is currently written in **OpenAPI 3.0** and can be pretty-visualized in the [github pages][tspGhPages].
106

117
**👋 Want to help?** Read our [contributor guide][contributing].
128

9+
## About, why and goals of the TSP:
10+
11+
Similarly to the philosophy behind the Language Server Protocol (LSP),
12+
the TSP is an open, HTTP-based protocol for use between *analytics or
13+
interactive visualization web applications* and *servers that provide
14+
trace analysis specific features* such as:
15+
16+
- generation of data that can interpeted as charts from trace analysis
17+
- selection, filtering and correlation of elements in charts using data
18+
contained in the traces
19+
- statistics calculation from data contained in the traces
20+
21+
The goal of the protocol is to allow trace analysis support to be
22+
implemented and distributed independently of any given analytics or
23+
interactive visualization web application.
24+
25+
## What the TSP is:
26+
27+
The TSP is a RESTful API on top of the HTTP protocol that enables to:
28+
29+
- analyze time series data (e.g. traces and logs);
30+
- return the result of the analysis in form of data that represents
31+
*charts* or *charts components*;
32+
- *navigate (or explore)* through the returned data;
33+
34+
Summarizing:
35+
36+
> the TSP is an API to explore data and navigate charts that has been
37+
> generated from the interpretation of time series data.
38+
39+
With the term *chart* we mean different type of visualizations such as
40+
XY charts, timegraph charts, tables, pie charts, etc.
41+
42+
With the term *chart components* we mean different elements that
43+
can be used to build or enrich charts such as arrows, annotations,
44+
bookmarks, etc.
45+
46+
With the terms *exploration and navigation* we mean interactions with
47+
the charts, such as:
48+
49+
- **filter** specific components of the charts. For example:
50+
- remove from the chart all components that do not match a condition;
51+
- return components of the chart that do match a condition;
52+
- **select** a subset of all data points of a chart. For example:
53+
- given a Timegraph chart, return only the data between a specific
54+
time interval
55+
- **select** specific components of the charts. For example:
56+
- when analyzing the trace of a SW execution, select only some
57+
components of the chart that represent specific SW functions;
58+
- when analyzing the trace of a SW execution, select only some
59+
components of the chart executed on specific CPUs;
60+
- given a Timegraph chart, automatically select areas of the chart
61+
where the components match a specific condition (**zoom-on-crash-area**)
62+
- **correlate** components between charts. For example:
63+
- given 2 charts generated by the analysis of the same time series data,
64+
it enables to correlate the chart components that represent the same
65+
information. For example, given a table containing the max length
66+
of an interval in a time chart, it enables to identify the corresponding
67+
interval in the time chart (**go-to-min/max**)
68+
- **correlate** chart components and their information. For example:
69+
- given a interval in a time chart, extrapolate all information related
70+
to that interval (e.g. **source location**)
71+
- **order** specific parts of the charts. For example:
72+
- return the **N longest** intervals of a timegraph chart;
73+
- **customize** charts and charts components. For example:
74+
- given a table containing statistics from analysis of all input data,
75+
generate a new table containing statistics from analysis of a
76+
subset of the input data;
77+
- given a time chart where intervals represent thread execution over
78+
time, and they are connected using arrows, generate a new time chart
79+
showing the longest path of connected intervals (e.g. **critical path**)
80+
81+
## Guidelines for the TSP implementation
82+
83+
The main resources (endpoints) of the API should be:
84+
85+
- **data sources** (e.g. time series data such as traces and logs). These
86+
are analyzed and used to create the data that can be retrieved;
87+
- **charts** (e.g. tables, XY charts), used to retrieve data representing
88+
a chart;
89+
- **chart components** (e.g. annotations, styles), used to retrieve data
90+
representing specific information for a chart;
91+
92+
The *exploration and navigation* should be achieved by:
93+
94+
- parameters in the HTTP message body data (usually when POST-ing), or
95+
as query strings (usually when GET-ing). These parameters should be:
96+
- **data source related**. For example, considering that data sources
97+
have a strong relation to time, a classic parameter can be "time"
98+
related (e.g. *requested_timerange*);
99+
- **chart related**. For example, when "talking" to a resource representing
100+
an XY chart, a parameter can be related to "XY dimensions" (e.g.
101+
*requested_Y_elements*);
102+
13103
## Current version
14104

15105
The current version of the specification is currently implemented and supported in the [Trace Compass trace-server][tcServer] (reference implementation) and what is currently supported by the [tsp-typescript-client][tspClient].

‎TSP-examples.md‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
# How to follow TSP guidelines
2+
3+
This file/document will be removed. It is meant to give some examples
4+
of how to follow the TSP guidelines when extending the TSP.
5+
6+
NOTE:
7+
8+
- the examples below are just pseudocode to try to give a concrete
9+
feeling on how to implement the exploration/navigation features and
10+
follow the TSP guidelines. They are not meant to be seen as concrete
11+
proposals to extend/modify the existing TSP;
12+
- the GET / POST methods are chosen just to be more RESTful (i.e. try
13+
to not use a POST when we do not create new resources)
14+
- the parameters in the requests are always written as body params,
15+
just to make it easier to write. But they do not have to be body params.
16+
17+
## Timegraph states (a.k.a. intervals)
18+
19+
```
20+
GET tsp/api/experiments/{expUUID}/outputs/timeGraph/{outputId}/states
21+
{"parameters":
22+
{"requested_timerange":
23+
{"start": 111111111,
24+
"end": 222222222,
25+
"nbTimes": 1920}, // to follow "TSP guidelines", would it be better "precision"? or "samples"?
26+
"requested_items": [1,2] // to follow "TSP guidelines", would it be better "requested_row_ids"?
27+
}
28+
}
29+
```
30+
31+
## Filter rows of a timegraph
32+
33+
```
34+
GET tsp/api/experiments/{expUUID}/outputs/timeGraph/{outputId}/states
35+
{"parameters":
36+
{"requested_timerange":
37+
{"start": 111111111,
38+
"end": 222222222,
39+
"nbTimes": 1920}, // to follow "TSP guidelines", would it be better "precision"? or "samples"?
40+
"requested_row_labels":
41+
[CPU0,CPU2] // this is the label of the rows. How does the cli know about name of rows?
42+
// The cli "asks" to the server about the labels of rows in the chart (e.g.
43+
// from the timegraph/tree endpoint)
44+
}
45+
}
46+
```
47+
48+
## Filter states (a.k.a. intervals) of a timegraph chart
49+
50+
```
51+
GET tsp/api/experiments/{expUUID}/outputs/timeGraph/{outputId}/states
52+
{"parameters":
53+
{"requested_timerange":
54+
{"start": 111111111,
55+
"end": 222222222,
56+
"nbTimes": 1982},
57+
"requested_intervals":
58+
[ThreadA*,FunctionB*,BankTransactionC*]
59+
}
60+
}
61+
```
62+
63+
## Filter states (a.k.a. intervals) of the chart with fullsearch
64+
65+
```
66+
GET tsp/api/experiments/{expUUID}/outputs/timeGraph/{outputId}/states
67+
{"parameters":
68+
{"requested_timerange":
69+
{"start": 111111111,
70+
"end": 222222222,
71+
"nbTimes": inf}, // or "max", however give the idea that we are trying to get all possible samples
72+
"requested_intervals":
73+
[ThreadA*,FunctionB*]
74+
}
75+
}
76+
```
77+
78+
## Correlate components between charts (e.g. go-to-max)
79+
80+
An example of how to jump from a value in a table to the interval/state
81+
that it represents in another chart.
82+
83+
First, ask some info to the table:
84+
85+
```
86+
cli (ask): GET tsp/api/experiments/{expUUID}/outputs/<chart-type>/<chart-id>/tree
87+
{"parameters":{"requested_times":[0,100000000]}} // Side question: why is "requested_times" needed?
88+
srv (ret): {"headers":[{Min},{Max},...], "entries" :[{1 sec},{2 min},...]}
89+
```
90+
91+
In order to implement the "go-to-max" functionality, the client asks to the server
92+
to return the "time series data" (in this case an interval) that was used to calculate
93+
the value "2 min" at the table index 1,1:
94+
95+
```
96+
cli (ask): GET tsp/api/experiments/{expUUID}/outputs/<chart-type>/<chart-id>/tree
97+
{"parameters":{"table_row": 1, "table_col":1]}}
98+
srv (ret): {"data":[{"start": 1234, "end": 2345, "label": "ThreadA"}]}
99+
```
100+
101+
The client uses than that info to "go to max" i.e. zoom in, filter or whaterver.
102+
In theory it could also be possible to enable the server to "auto-align" other charts,
103+
not sure if it is a good idea.
104+
105+
## Customize (e.g. select a subset of the input data to use to create a new chart)
106+
107+
Cli asks to the "time series input data" (e.g. the event table) to return some info on the input data:
108+
109+
```
110+
cli (ask): GET tsp/api/experiments/{expUUID}/outputs/table/<chart-id>/columns
111+
srv (ret): {"model":[{"name": "timestamp", "type":"number", ...}, {"name": "Device", "type":"string", ...}, ...]}
112+
```
113+
114+
Cli asks to create a new statistics table using a subset of the input data.
115+
The subset is selected using some of the previously returned params.
116+
117+
```
118+
cli (ask): POST tsp/api/experiments/{expUUID}/outputs/<chart-type>
119+
{"parameters":{"outputId/chart-id":"my.custom.chart", "include":[{"Device": "CPU0"},...]}}
120+
```

0 commit comments

Comments
 (0)