Welcome to Timelion!

Timelion is the clawing, gnashing, zebra killing, pluggable time series interface for everything. If your datastore can produce a time series, then you have all of the awesome power of Timelion at your disposal. Timelion lets you compare, combine, and combobulate datasets across multiple datasources with one easy-to-master expression syntax. This tutorial focuses on Elasticsearch, but you'll quickly discover that what you learn here applies to any datasource Timelion supports.

Ready to get started? Click Next. Want to skip the tutorial and view the docs? Jump to the function reference.

First time configuration

If you're using Logstash, you don't need to configure anything to start exploring your log data with Timelion. To search other indices, go to Management / Kibana / Advanced Settings and configure the timelion:es.default_index and timelion:es.timefield settings to match your indices.

You'll also see some other Timelion settings. For now, you don't need to worry about them. Later, you'll see that you can set most of them on the fly if you need to.

Could not validate Elasticsearch settings: {{es.invalidReason}}. Check your Advanced Settings and try again. ({{es.invalidCount}})

Good news, Elasticsearch is configured correctly!

We validated your default index and your timefield and everything looks ok. We found data from {{es.stats.min}} to {{es.stats.max}}. You're probably all set. If this doesn't look right, see First time configuration for information about configuring the Elasticsearch datasource.

You should already see one chart, but you might need to make a couple adjustments before you see any interesting data:

  • Intervals

    The interval selector at the right of the input bar lets you control the sampling frequency. It's currently set to {{state.interval}}. You're all set! Set it to auto to let Timelion choose an appropriate interval. If Timelion thinks your combination of time range and interval will produce too many data points, it throws an error. You can adjust that limit by configuring timelion:max_buckets in Management/Kibana/Advanced Settings.

  • Time range

    Use the timepicker in the Kibana toolbar to select the time period that contains the data you want to visualize. Make sure you select a time period that includes all or part of the time range shown above.

Now, you should see a line chart that displays a count of your data points over time.

Querying the Elasticsearch datasource

Now that we've validated that you have a working Elasticsearch datasource, you can start submitting queries. For starters, enter .es(*) in the input bar and hit enter.

This says hey Elasticsearch, find everything in my default index. If you want to find a subset, you could enter something like .es(html) to count events that match html, or .es('user:bob AND bytes:>100') to find events that contain bob in the user field and have a bytes field that is greater than 100. Note that this query is enclosed in single quotes—that's because it contains spaces. You can enter any Lucene query string as the first argument to the .es() function.

Passing arguments

Timelion has a number of shortcuts that make it easy to do common things. One is that for simple arguments that don't contain spaces or special characters, you don't need to use quotes. Many functions also have defaults. For example, .es() and .es(*) do the same thing. Arguments also have names, so you don't have to specify them in a specific order. For example, you can enter .es(index='logstash-*', q='*') to tell the Elasticsearch datasource use * as the q (query) for the logstash-* index.

Beyond count

Counting events is all well and good, but the Elasticsearch datasource also supports any Elasticsearch metric aggregation that returns a single value. Some of the most useful are min, max, avg, sum, and cardinality. Let's say you want a unique count of the src_ip field. Simply use the cardinality metric: .es(*, metric='cardinality:src_ip'). To get the average of the bytes field, you can use the avg metric: .es(metric='avg:bytes').

Expressing yourself with expressions

Every expression starts with a datasource function. From there, you can append new functions to the datasource to transform and augment it.

By the way, from here on out you probably know more about your data than we do. Feel free to replace the sample queries with something more meaningful!

We're going to experiment, so click Add in the Kibana toolbar to add another chart or three. Then, select a chart, copy one of the following expressions, paste it into the input bar, and hit enter. Rinse, repeat to try out the other expressions.

.es(*), .es(US) Double the fun. Two expressions on the same chart.
.es(*).color(#f66), .es(US).bars(1) Custom styling. Colorizes the first series red and uses 1 pixel wide bars for the second series.
.es(*).color(#f66).lines(fill=3), .es(US).bars(1).points(radius=3, weight=1) Named arguments. Forget trying to remember what order you need to specify arguments in, use named arguments to make the expressions easier to read and write.
(.es(*), .es(GB)).points() Grouped expressions. You can also chain groups of expressions to functions. Here, both series are shown as points instead of lines.

Timelion provides additional view transformation functions you can use to customize the appearance of your charts. For the complete list, see the Function reference.

Transforming your data: the real fun begins!

Now that you've mastered the basics, it's time to unleash the power of Timelion. Let's figure out what percentage some subset of our data represents of the whole, over time. For example, what percentage of our web traffic comes from the US?

First, we need to find all events that contain US: .es('US').

Next, we want to calculate the ratio of US events to the whole. To divide 'US' by everything, we can use the divide function: .es('US').divide(.es()).

Not bad, but this gives us a number between 0 and 1. To convert it to a percentage, simply multiply by 100: .es('US').divide(.es()).multiply(100).

Now we know what percentage of our traffic comes from the US, and can see how it has changed over time! Timelion has a number of built-in arithmetic functions, such as sum, subtract, multiply, and divide. Many of these can take a series or a number. There are also other useful data transformation functions, such as movingaverage, abs, and derivative.

Now that you're familiar with the syntax, refer to the Function reference to see how to use all of the available Timelion functions. You can view the reference at any time by clicking Docs in the Kibana toolbar. To get back to this tutorial, click the Tutorial link at the top of the reference.

Function reference
Click any function for more information. Just getting started? Check out the tutorial.
.{{function.name}}() {{function.help}}
Argument Name Accepted Types Information
{{arg.name}} {{arg.types.join(', ')}} {{arg.help}}
This function does not accept any arguments. Well that's simple, isn't it?