Timelion. Timeline. Get it? Ok, enough with the puns. Timelion is the, clawing, gnashing, zebra killing, pluggable timeseries interface for everything. If your datastore can produce a timeseries, then you have all of the awesome power of Timelion at your disposal. Timelion lets you compare, combine and combobulate (not actually a word) datasets across multiple data sources, even entirely different technologies, all with the same easy-to-master expression syntax. While the beginning of this tutorial will focus on Elasticsearch, once you're rolling you'll discover you can use nearly everything you learn here with any datasource timelion supports.
Why start with elasticsearch? Well, you're using timelion, so we know you have Kibana, so you definitely have Elasticsearch. So the answer is: Because its easy. Timelion want everything to be easy. Ok, let's do this thing. If you're already familar with Timelion's syntax, Jump to the function reference, otherwise click the Next button in the lower right corner.
If you're using logstash you're already done. Otherwise,
pop over to Kibana's "Advanced Settigns" and update timelion:es.timefield
and timelion:es.default_index
parameters to match your environment.
You'll see some other timelion parameters in there too, we won't be messing with them for now, but you can probably guess what they do. And frankly, most can be specified on-the-fly with the timelion expression syntax. More on that in a bit.
Or at least, things look ok. I validated your default index and your timefield and everything looks ok. Given your settings I found data between {{es.stats.min}} and {{es.stats.max}}. You're probably all set. If this doesn't look right, Click here for instructions on configuring the elasticsearch data source.
{{state.interval}}
.
Looks good!
Set it to auto
.
If timelion thinks your combination of time range and interval will produce too many data points it will throw an error. You can configure that limit in Advanced Settings
We're going to start off talking about the Elasticsearch datasource, because we've already validated that one works for you. Enter .es(*)
in the expression input, if its not there already. Hit enter.
This said "hey elasticsearch, find everything in my default index". If you wanted to find a subset you might do something like .es(html)
to count events matching html, or .es('user:bob AND bytes:>100')
to find events with bob in the user field, and a bytes field that is greater than 100. Note that we surrounded our query in single quotes this time, because it has spaces. You can enter any lucene query string as the first argument to the .es()
function.
.es()
and .es(*)
do the same thing. Arguments also have names, so you don't have to remember their position, you can pass .es(index='logstash-*', q='*')
to tell the elasticsearch data source "use * as the q (query) for the logstash-* index"
Counting events is all well and good, but the elasticsearch data source also supports any Elasticsearch metric that returns a single value. Min, max, avg, sum and cardinality are some of the most useful. Let's say you want a unique count of the src_ip
field. You could do say, .es(*, metric='cardinality:src_ip')
. To get the average of the bytes field you would run: .es(metric='avg:bytes')
.
Every timelion expression starts with a datasource function. From there, the sky is the limit and new functions can be appended, or "chained", to the data source to transform and augment it. From here we're going to assume you know something about your data. Feel free to replace the elasticsearch query with something more meaningful to you.
Up until now we've dealt with just the one chart. We're going to experiment, so add a few more. Click the Menu icon in the top right to expand the menu. Then click the Add Chart button.
.es(*) |
One expression |
.es(*), .es(US) |
Two expressions. Two expressions on the same chart! |
.es(*).color(#f66), .es(US).bars(1) |
Red expression. Let's colorize the first series red instead. Also, instead of lines for 2nd series, we'll have some bars, with a 1 pixel width. |
.es(*).color(#f66).lines(fill=3), .es(US).bars(1).points(radius=3, weight=1) |
Wooo expressions. In the last example we used un-named arguments to color() and bars , which rely on the arguments position in a comma separated list. We can use named arugments to make expressions easier to read and arguments easier to remember. |
(.es(*), .es(GB)).points() |
Also grouped expressions. Groups of expressions can be chained to functions as well. Both series will be shown as points instead of lines. |
We can make our charts pretty all day, but its time for businessing. As an example exercise, we're going to figure out what percentage some subset of our data represents of the whole, over time. For example, what percentage of my web traffic comes from the US? Let's start with finding all events that contain US: .es('US')
. Now, to find that ratio to the whole, we'd need to divide 'US'
by everything, try this: .es('US').divide(.es())
. Ah, not bad, but of course this provides us with a number between 0 and 1, let's correct that to a percentage: .es('US').divide(.es()).multiply(100)
. There, now we've divided all US traffic by all worldwide traffic, then multiplied the result by 100 to get a percentage.
Timelion has a number of built in arithmetic functions, such as sum
, subtract
, multiply
and divide
, many of which can take a series or a number. There are also other data transformation functions including movingaverage
, abs
and derivative
. In addition there are other view transformation functions than the ones we learned on the previous page. See the function reference for the complete list of transforming, and drawing functions.
Now that you know the syntax, jump over to the Function Reference for detailed info on all of Timelions available functions.
.{{function.name}}() | {{function.help}} | ||||||
This function does not accept any arguments. Well that's simple, isn't it?
|