88
99# Discussion: Compositional Loggers
1010
11- LoggingExtras is designs around allowing you to build arbitrarily complicated
11+ LoggingExtras is designs around allowing you to build arbitrarily complicated
1212systems for "log plumbing". That is to say basically routing logged information to different places.
1313It is built around the idea of simple parts which are composed together,
1414to allow for powerful and flexible definition of your logging system.
@@ -30,16 +30,15 @@ The loggers defined in this package are all pure.
3030The Filters, only filter, the Sinks only sink, the transformers only Transform.
3131
3232We can contrast this to the the ` ConsoleLogger ` (the standard logger in the REPL).
33- The ` ConsoleLogger ` is an in-pure sink.
33+ The ` ConsoleLogger ` is an impure sink.
3434As well as displaying logs to the user (as a Sink);
3535it uses the log content, in the form of the ` max_log ` kwarg to decide if a log should be displayed (Active Filtering);
36- and it has a min_enabled_level setting, that controls if it will accept a message at all
36+ and it has a min_enabled_level setting, that controls if it will accept a message at all
3737(Early Filtering, in particular see ` MinLevelLogger ` ).
3838If it was to be defined in a compositional way,
39- we would write;
39+ we would write something along the lines of:
4040```
41-
42- ConsoleLogger(stream, min_level) =
41+ ConsoleLogger(stream, min_level) =
4342 MinLevelLogger(
4443 ActiveFilteredLogger(max_log_filter,
4544 PureConsoleLogger(stream)
@@ -76,27 +75,21 @@ logger = global_logger()
7675```
7776
7877# Loggers introduced by this package:
79-
80-
81- This package introduces 5 new loggers.
82- The ` DemuxLogger ` , the ` FileLogger ` , and 3 types of filtered logger.
78+ This package introduces 6 new loggers.
79+ The ` DemuxLogger ` , the ` TransformerLogger ` , 3 types of filtered logger, and the ` FileLogger ` .
8380All of them just wrap existing loggers.
8481 - The ` DemuxLogger ` sends the logs to multiple different loggers.
8582 - The ` TransformerLogger ` applies a function to modify log messages before passing them on.
86- - The ` FileLogger ` is a simple logger sink that writes to file.
8783 - The 3 filter loggers are used to control if a message is written or not
8884 - The ` MinLevelLogger ` only allowes messages to pass that are above a given level of severity
8985 - The ` EarlyFilteredLogger ` lets you write filter rules based on the ` level ` , ` module ` , ` group ` and ` id ` of the log message
9086 - The ` ActiveFilteredLogger ` lets you filter based on the full content
91-
87+ - The ` FileLogger ` is a simple logger sink that writes to file.
9288
9389By combining ` DemuxLogger ` with filter loggers you can arbitrarily route log messages, wherever you want.
9490
95- The ` FileLogger ` is just a convience wrapper around the base julia ` SimpleLogger ` ,
96- to make it easier to pass in a filename, rather than a stream.
97-
9891
99- ## ` DemuxLogger ` and ` FileLogger `
92+ ## ` DemuxLogger `
10093
10194The ` DemuxLogger ` sends the log messages to multiple places.
10295It takes a list of loggers.
@@ -107,13 +100,18 @@ It is up to those loggers to determine if they will accept it.
107100Which they do using their methods for ` shouldlog ` and ` min_enabled_level ` .
108101Or you can do, by wrapping them in a filtered logger as discussed below.
109102
103+ ## ` FileLogger `
110104The ` FileLogger ` does logging to file.
105+ It is just a convience wrapper around the base julia ` SimpleLogger ` ,
106+ to make it easier to pass in a filename, rather than a stream.
111107It is really simple.
112- It takes a filename,
108+ - It takes a filename,
113109 - a kwarg to check if should ` always_flush ` (default: ` true ` ).
114110 - a kwarg to ` append ` rather than overwrite (default ` false ` . i.e. overwrite by default)
111+ The resulting file format is similar to that which is shown in the REPL.
112+ (Not identical, but similar)
115113
116- ### Demo
114+ ### Demo: ` DemuxLogger ` and ` FileLogger `
117115We are going to log info and above to one file,
118116and warnings and above to another.
119117
@@ -123,7 +121,7 @@ julia> using Logging; using LoggingExtras;
123121julia> demux_logger = DemuxLogger(
124122 MinLevelLogger(FileLogger("info.log"), Logging.Info),
125123 MinLevelLogger(FileLogger("warn.log"), Logging.Warn),
126- include_current_global=false
124+ include_current_global=false
127125);
128126
129127
@@ -164,7 +162,7 @@ We want to filter to only log strings staring with `"Yo Dawg!"`.
164162julia> function yodawg_filter(log_args)
165163 startswith(log_args.message, "Yo Dawg!")
166164end
167- yodawg_filter (generic function with 1 method)
165+ yodawg_filter (generic function with 1 method)
168166
169167julia> filtered_logger = ActiveFilteredLogger(yodawg_filter, global_logger());
170168
@@ -186,7 +184,7 @@ but it runs earlier in the logging pipeline.
186184In particular it runs before the message is computed.
187185It can be useful to filter things early if creating the log message is expensive.
188186E.g. if it includes summary statistics of the error.
189- The filter function for early filter logging only has access to the
187+ The filter function for early filter logging only has access to the
190188` level ` , ` _module ` , ` id ` and ` group ` fields of the log message.
191189The most notable use of it is to filter based on modules,
192190see the HTTP example below.
@@ -197,35 +195,35 @@ Another example is using them to stop messages every being repeated within a giv
197195using Dates, Logging, LoggingExtras
198196
199197julia> function make_throttled_logger(period)
200- history = Dict{Symbol, DateTime}()
201- # We are going to use a closure
202- EarlyFilteredLogger(global_logger()) do log
203- if !haskey(history, log.id) || (period < now() - history[log.id])
204- # then we will log it, and update record of when we did
205- history[log.id] = now()
206- return true
207- else
208- return false
209- end
210- end
211- end
198+ history = Dict{Symbol, DateTime}()
199+ # We are going to use a closure
200+ EarlyFilteredLogger(global_logger()) do log
201+ if !haskey(history, log.id) || (period < now() - history[log.id])
202+ # then we will log it, and update record of when we did
203+ history[log.id] = now()
204+ return true
205+ else
206+ return false
207+ end
208+ end
209+ end
212210make_throttled_logger (generic function with 1 method)
213211
214212julia> throttled_logger = make_throttled_logger(Second(3));
215213
216214julia> with_logger(throttled_logger) do
217- for ii in 1:10
218- sleep(1)
219- @info "It happen " ii
220- end
221- end
222- ┌ Info: It happen
215+ for ii in 1:10
216+ sleep(1)
217+ @info "It happened " ii
218+ end
219+ end
220+ ┌ Info: It happened
223221└ ii = 1
224- ┌ Info: It happen
222+ ┌ Info: It happened
225223└ ii = 4
226- ┌ Info: It happen
224+ ┌ Info: It happened
227225└ ii = 7
228- ┌ Info: It happen
226+ ┌ Info: It happened
229227└ ii = 10
230228```
231229
@@ -249,19 +247,19 @@ A simple example of its use is truncating messages.
249247julia> using Logging, LoggingExtras
250248
251249julia> truncating_logger = TransformerLogger(global_logger()) do log
252- if length(log.message) > 128
253- short_message = log.message[1:min(end, 125)] * "..."
254- return merge(log, (;message=short_message))
255- else
256- return log
257- end
258- end;
250+ if length(log.message) > 128
251+ short_message = log.message[1:min(end, 125)] * "..."
252+ return merge(log, (;message=short_message))
253+ else
254+ return log
255+ end
256+ end;
259257
260258julia> with_logger(truncating_logger) do
261- @info "the truncating logger only truncates long messages"
262- @info "Like this one that is this is a long and rambling message, it just keeps going and going and going, and it seems like it will never end."
263- @info "Not like this one, that is is short"
264- end
259+ @info "the truncating logger only truncates long messages"
260+ @info "Like this one that is this is a long and rambling message, it just keeps going and going and going, and it seems like it will never end."
261+ @info "Not like this one, that is is short"
262+ end
265263[ Info: the truncating logger only truncates long messages
266264[ Info: Like this one that is this is a long and rambling message, it just keeps going and going and going, and it seems like it wil...
267265[ Info: Not like this one, that is is short
@@ -279,7 +277,7 @@ using LoggingExtras
279277using Logging
280278
281279function sensible_message_filter(log)
282- length(log.message) < 1028
280+ length(log.message) < 1028
283281end
284282
285283global_logger(ActiveFilteredLogger(sensible_message_filter, global_logger()))
@@ -294,7 +292,7 @@ using Logging
294292using HTTP
295293
296294function not_HTTP_message_filter(log)
297- log._module != HTTP
295+ log._module != HTTP
298296end
299297
300298global_logger(EarlyFilteredLogger(not_HTTP_message_filter, global_logger()))
@@ -308,8 +306,8 @@ using Logging
308306using HTTP
309307
310308transformer_logger(global_logger()) do log
311- if log._module == HTTP && log.level=Logging.Debug
312- # Merge can be used to construct a new NamedTuple
309+ if log._module == HTTP && log.level=Logging.Debug
310+ # Merge can be used to construct a new NamedTuple
313311 # which effectively is the overwriting of fields of a NamedTuple
314312 return merge(log, (; level=Logging.Info))
315313 else
319317
320318global_logger(transformer_logger)
321319```
322-
323-
0 commit comments