Step 3.1: Framework time: FILTER

Using TIMESPLIT we compute our first time series operation but WarpScript offers other richer tools to manipulate them. Five frameworks are available and they all have their specific usage and important place in Time-Series analytics:

Let’s start with the FILTER framework.

The FILTER framework

The first framework we are going to discover in this tutorial is FILTER. It provides the tooling to select specific sub-set of Time-series in a list thereof.

Let’s resume what we have on the stack: first a kepler-11 time series are loaded using the FETCH function. Then the data are split according to a quiesce period using TIMESPLIT. We are now going to select only a few records to work on using FILTER.

Input parameters

The filter function takes a list containing 3 elements as input parameter: the time series as list or singleton to filter, then a list of labels to compute an equivalence class, and finally a filter function.

Do not focus on the equivalence class: during this step, and keep an empty list. We will see later how it could be used.

// Filter framwork
[
    $gts                                // Series list or Singleton
    []                                  // Labels to compute equivalence class
    filter.function                     // A selector function
]
FILTER

To have a working filter, replace the function keyword by an existing filter function: filter.bylabels or filter.byclass for instance. Those specific filter function requires a parameter on top of the stack. A more concret example is detailed below, based on the NASA lightcurve data.

Hello Exo World step

In our data-flow, the goal is now to select only 2 to 3 records of those series to see if it’s possible to visualize specific drops corresponding to an exoplanet in those series. As a result we will then be able to build a generic method to detect exoplanet. You will have to use a variable as seen in the first step to place the time series list in first position of the element list of the FILTER.

The filter function we will use in our case corresponds to filter.bylabels. This function expects a selector map on top of the stack that have to be specify. As an example you can try to use the selector labels map specifies in the follozing spec in your filter function.

Resume

To resume, to filter a data set, we have to use the FILTER framework. It expects on the stack a list of parameter: the time series as list or as singleton to filter, then a list of labels to compute an equivalence class, and finally a filter function. The filter function as we saw with the filter.bylabels can also need some parameter(s) on top of stack.

The result of this step corresponds to a reduced list. In our case only 4 records (2 to 5) were selected using a regular expression. A list containing 4 time series is now let on top of the WarpScript stack.

To be continued

As an exoplanet hunter, you may be asking yourself why you should reduced the amount of data you will be working with. Be patient, and remember that taking your time during the first steps will making you win a lot of time in the longer run! Stay strong young hunter, more will come in the next step as a framework very useful for time series downsampling will be briefly introduced.

Previous
Solution