Timelion is the clawing, gnashing, zebra killing, pluggable time series interface for everything. If your datastore can produce a time series, then you have all of the awesome power of Timelion at your disposal. Timelion lets you compare, combine, and combobulate datasets across multiple datasources with one easy-to-master expression syntax. This tutorial focuses on Elasticsearch, but you'll quickly discover that what you learn here applies to any datasource Timelion supports.
Ready to get started? Click Next. Want to skip the tutorial and view the docs? Jump to the function reference.
If you're using Logstash, you don't need to configure anything to
start exploring your log data with Timelion. To search other
indices, go to Management / Kibana / Advanced Settings
and configure the timelion:es.default_index
and timelion:es.timefield
settings to match your
indices.
You'll also see some other Timelion settings. For now, you don't need to worry about them. Later, you'll see that you can set most of them on the fly if you need to.
We validated your default index and your timefield and everything looks ok. We found data from {{es.stats.min}} to {{es.stats.max}}. You're probably all set. If this doesn't look right, see First time configuration for information about configuring the Elasticsearch datasource.
You should already see one chart, but you might need to make a couple adjustments before you see any interesting data:
The interval selector at the right of the input bar lets you
control the sampling frequency. It's currently set to
{{state.interval}}
.
You're all set!
Set it to auto
to let Timelion choose an
appropriate interval.
If Timelion thinks your combination of time range and interval
will produce too many data points, it throws an error. You can
adjust that limit by configuring timelion:max_buckets
in Management/Kibana/Advanced Settings.
Use the timepicker in the Kibana toolbar to select the time period that contains the data you want to visualize. Make sure you select a time period that includes all or part of the time range shown above.
Now, you should see a line chart that displays a count of your data points over time.
Now that we've validated that you have a working Elasticsearch
datasource, you can start submitting queries. For starters,
enter .es(*)
in the input bar and hit enter.
This says hey Elasticsearch, find everything in my default
index. If you want to find a subset, you could enter something
like .es(html)
to count events that match html,
or .es('user:bob AND bytes:>100')
to find events
that contain bob in the user
field and have a
bytes
field that is greater than 100. Note that this query
is enclosed in single quotes—that's because it contains
spaces. You can enter any
Lucene query string
as the first argument to the .es()
function.
Timelion has a number of shortcuts that make it easy to do common
things. One is that for simple arguments that don't contain spaces or
special characters, you don't need to use quotes. Many functions also
have defaults. For example, .es()
and .es(*)
do the same thing. Arguments also have names, so you don't have to
specify them in a specific order. For example, you can enter
.es(index='logstash-*', q='*')
to tell the
Elasticsearch datasource use * as the q (query) for the
logstash-* index.
Counting events is all well and good, but the Elasticsearch datasource
also supports any
Elasticsearch metric aggregation
that returns a single value. Some of the most useful are
min
, max
, avg
, sum
,
and cardinality
. Let's say you want a unique count of the
src_ip
field. Simply use the cardinality
metric: .es(*, metric='cardinality:src_ip')
. To get the
average of the bytes
field, you can use the
avg
metric: .es(metric='avg:bytes')
.
Every expression starts with a datasource function. From there, you can append new functions to the datasource to transform and augment it.
By the way, from here on out you probably know more about your data than we do. Feel free to replace the sample queries with something more meaningful!
We're going to experiment, so click Add in the Kibana toolbar to add another chart or three. Then, select a chart, copy one of the following expressions, paste it into the input bar, and hit enter. Rinse, repeat to try out the other expressions.
.es(*), .es(US) |
Double the fun. Two expressions on the same chart. |
.es(*).color(#f66), .es(US).bars(1) |
Custom styling. Colorizes the first series red and uses 1 pixel wide bars for the second series. |
.es(*).color(#f66).lines(fill=3),
.es(US).bars(1).points(radius=3, weight=1)
|
Named arguments. Forget trying to remember what order you need to specify arguments in, use named arguments to make the expressions easier to read and write. |
(.es(*), .es(GB)).points() |
Grouped expressions. You can also chain groups of expressions to functions. Here, both series are shown as points instead of lines. |
Timelion provides additional view transformation functions you can use to customize the appearance of your charts. For the complete list, see the Function reference.
Now that you've mastered the basics, it's time to unleash the power of Timelion. Let's figure out what percentage some subset of our data represents of the whole, over time. For example, what percentage of our web traffic comes from the US?
First, we need to find all events that contain US:
.es('US')
.
Next, we want to calculate the ratio of US events to the whole. To
divide 'US'
by everything, we can use the
divide
function: .es('US').divide(.es())
.
Not bad, but this gives us a number between 0 and 1. To convert it
to a percentage, simply multiply by 100:
.es('US').divide(.es()).multiply(100)
.
Now we know what percentage of our traffic comes from the US, and
can see how it has changed over time!
Timelion has a number of built-in arithmetic functions, such as
sum
, subtract
, multiply
, and
divide
. Many of these can take a series or a number.
There are also other useful data transformation functions, such as
movingaverage
, abs
, and
derivative
.
Now that you're familiar with the syntax, refer to the Function reference to see how to use all of the available Timelion functions. You can view the reference at any time by clicking Docs in the Kibana toolbar. To get back to this tutorial, click the Tutorial link at the top of the reference.
.{{function.name}}() | {{function.help}} | ||||||
This function does not accept any arguments.
Well that's simple, isn't it?
|