Getting Started

Types of Aggregation​

Use the following extensions to achieve the required aggregation on a metric

Average.avg or no extensionNot allowed for text fields
Percentile.95P, .99P etcNot allowed for text fields
Max.maxNot allowed for text fields
Min.minNot allowed for text fields
Count.count
Terminal.termRetrieves the last value of the metric from DBDocument type has to be specified in the query
Current.currRetrieves the last value of the metric in the last 8 mins. If no value is available, the result is displayed as NADocument type has to be specified in the query

Global vs Custom timeline​

• A component by default uses the timeline selected in the global component

• However, a user can evaluate a component for a specific custom using the following steps

• Edit the component and go to the advanced section

• Select “Custom Time”

• Go to the JSON Editor and edit the time fields as shown below. In this example, we evaluate the component for last 1 hour. User can specify 30m for 30 minutes, 1d for 1day etc.

Constructing Queries​

SnappyFlow provides SQL like query language to create components through a JSON editor provided. Features in JSON editor are selectively exposed based on options selected in advanced section

• Every document that is collected in SnappyFlow will have these tags: _plugin, _documentType, _tag_projectName, _tag_appName

• Whenever you create a new component, it will always have _tag_appName == $value by default, which is automatically substituted with current application name. This ensures that data is being queried for the current application only • Examples Example Use CaseQuery Collect a metric for a specific instanceselect CPUUtil where _tag_Name == instance1 Collect metrics for all instances in an applicationselect CPUUtil where _tag_Name == * Collect metric based on a dropdown conditionselect CPUUtil where _tag_Name == DropDownXX.$value
Note: DropDownXX is the name of the dropdown. DropDown.\$value is the value selected in the dropdown
Collect a terminal value of a metricSelect clusterStatus.term where _plugin == elasticsearch and _documentType == clusterStats
Note: Whenever you query a terminal value, a _documentType has to be specified
Create a logic expressionSelect path.count where code >= 400 and code < 600
Equal: ==
Not Equal: !==
Greater or Less: >, >=, <, <=
Variable belongs to a list: X ==(A,B,C..)
String operationsquery_string == message:"Index to be deleted"
Query_string: Directive or operator for search by keyword
Message: Field name in which the pattern has to be searched
Search Text: Specify exact text in quotes To see all supported options, refer to Log Overview Search
Building queries for nested data
Note: When using a nested field, enclose metric within square brackets.
Enable Add Nested Fields option in the advanced section. This will add the section nestedFields in JSON editor.
Specify the nested fields used in the query under nestedFields section.
• Combined query: SnappyFlow allows upto 2 queries for SummaryBox and Tables and can be enabled from the advanced section. This is useful when querying terminal values from two different documentTypes

• Render: This feature is useful when using combined query for tables and summary charts

It has 2 uses:

• change the order of rendering metrics when combined query is used
• collect a number of metrics, transform the metrics and render only a subset of the collected metrics. Enable the feature from advanced section

Histogram Intervals​

Histogram interval is relevant for a Line chart and Bar chart histogram. It is a sub interval in which a metric is evaluated or aggregated. Every histogram interval will correspond to a point in the chart.

By default, histogram interval is calculated based on time range represented in the graph. If user deselects ‘Adaptive Interval’ option from the advanced section, the histogram interval is fixed at 60s.

Time Range represented in the chartLine Chart IntervalBar Chart Interval
<= 1 minute1s1s
<= 5 minutes2s4s
<= 10 minutes4s10s
<= 20 minutes10s10s
<= 30 minutes30s30s
<= 1 hr30s60s
<= 3 hrs30s5m
<= 6 hrs60s5m
<= 8 hrs5m5m
<= 12 hrs5m10m
<= 1 day5m30m
<= 7 days30m3h
<= 30 days2h12h
<= 60 days4h1d
<= 90 days8h1d
<= 120 days8h1d
<= 240 days16h1d
> 240 days1d1d

Metric Properties​

User can enable the metric properties from advanced section for:

• Formatting the data rendered in the component
• Applying text/box color properties
metricsFilterUsed to display specified portions of the string in the component or to shorten the displayed stringConsider a table component displaying pod details with _tag_podName as the table key. If the pod names have the form apmmanager-apm-sfapm-apm, we can shorten the displayed name using metricsFilterApplicable for Table, Tabbed table, Line chart, Bar chart components
For Example: This will display apmmanager-apm-sfapm-apm-5cd8946d64-2smb9 as apm-5cd8946d64-2smb9
decimalBy default, data is shown upto 1 decimal placeUser can override this setting by using this optionApplicable for Table, Tabbed table, Summary box, Line chart and Bar chart histogram components
If we have a table component with a header CPU (%), data under this column will be rounded to 3 decimal places.
Note: To change the number of decimals for all metrics in the component, use default: <num_of_decimals>
formatFormat time field from epoch milliseconds/epoch seconds format to DateTime formatApplicable for Table, Tabbed table, Bar charts, Key based line chart componentsAvailable options are DateTime, DateTime(ms) and DateTime(us)
In the above example, the time displayed under Last Seen column of table is formatted to DateTime format
colorDisplay table cell with colors based on conditionsRefer the section “Display table cell with colors based on conditions” in the Table Component section belowApplicable for Table, Tabbed table and Excel Table components
rateUsed to display the urate/unit rate of the metric in Table and Summary Box componentsUsage: <var_name>: urateApplicable for Summary Box and Table components
urate is calculated as value of the metric/(time range used in query in seconds)
nullValuesNull Values are displayed as NA by defaultUse this option to override the default settingThe null values will be displayed as -- instead of NAApplicable for Table component
boxColorDisplay box color for Summary box based on conditionThe condition should be specified in the same way as the table cell color. Refer Table Component section below.Applicable for Summary Box
textColorDisplay text color for Summary box based on conditionThe condition should be specified in the same way as the table cell color.Refer Table Component section below.Applicable for Summary Box

Table Component​

Aggregation Table​

• All tables by default have aggregated option selected

• In this option, metrics are computed for a bucket that corresponds to a unique value of the key

Example:

If we need documents such as (Group: A, Name: x, Age: 10, Weight: 20) , (Group B, Name: y, Age: 12, Weight: 22), (Group A, Name: xx, Age: 25, Weight: 100), (Group B, Name: yy, Age: 15, Weight: 70), a table for Select Age.avg, Weight.avg where Group == * provides an output as shown below.

GroupAgeWeight
A17.560
B13.546

*Group is the table key* in the above example

Non Aggregation Table:​

• User can choose this option by disabling “Aggregation” in advanced option or by leaving the “Table Key” field as empty

• This option is used to represent values from a sequence of JSON in a tabular format, without performing any aggregations

Example: For the same example above, a query “Select Name, Group, Age, Weight” would provide the result as shown below that merely represents the data in a tabular form. The ordering by default will be in descending order of time

GroupNameAgeWeight
Ax1020
By1222
Cxx25100
Dyy1570

Change Sort Field​

• Data for Aggregation tables by default is sorted in ascending order of table key

• In order to change the sorting field, enable “Change Sort Field” in advanced section

• Go to JSON editor and enter the raw metric (not a transformed field) that should be used for sorting

• Please note that a metric that has a .term/.count aggregation cannot be specified as a sort field

• By choosing “Change Sort Order” in advanced section, user can change the option to descending or ascending for the sort key. Please see in the example below

Filter​

• In many cases, a query may return a large number of results and we would want to filter the results from the database based on a certain values of a column

• Click on the filter icon

• Define filters for specific columns. We can specify one or more filters at a time. In the example below we are asking to filter all API path values in databases that contain the key “snappyflow” and number of 4xx errors is GT 5

• Please note the following limitations and rules to follow

• If combined query is used, filters can be applied only to table key
• Filter cannot be applied to a variable that is a transformation of more than one metric

Display table cell with colors based on conditions​

• Example- we are computing the total number of 4XX and 5XX errors in the query below and we would like color the cell containing #Errors in red if the number of errors are GT 0

• Go to the advanced section and enable metric properties. This will bring up a section called metric properties under which add the condition as shown below for color

• This will cause the color to rendered for the cell if #Errors is GT 0

• Hyperlink allows you to navigate from a table to another pane. The value of the hyperlink are propagated to a dropdown of the pane

• Go to Edit Component’s advanced section and enable Hyperlink checkbox, which can be found under General category

• A hyperlink section is enabled in the JSON editor. You will need to define the redirection target which comprises of Group (Group is a collection of panes. If the target pane does not belong a group, leave this field empty), Pane (dashboard pane name), Component (dropdown in the pane where the value of the hyperlink has to be propagated)

• Hyperlinks are enabled for the table. Click on the value “200” will redirect to the pane “Transaction Analysis” which belongs to group “Nginx Access” and render this pane with DropDown11 value = 200

The Hyperlink to Tracing feature allows you to navigate from table to tracing page.

1. Go to Edit Component’s advanced section and enable Hyperlink to Tracing checkbox, which can be found under the General category.

1. In the JSON Editor tab, provide the configuration object as mentioned below in the traceHyperlink list.
{ field:<tracing_field_name>, page:<tracing_page_name>, column:<table_column_name>}

Example

traceHyperlink:[    {        field: traceId,        page: realTime,        column: Trace ID    }]            

Field

The field refers to the tracing field in the tracing feature. It can be any of the following tracings fields: service, transactionType, method, resourceApi, result, status, origin, originIP, traceId, transactionId, nodeName, podName, forwardedIP.

Page

The page refers to the pages of the tracing feature. It can be any of the following pages: aggregate, realTime, history.

Column

The column refers to the column of the table to which the hyperlink will be enabled.

Generic Filter Component​

The Generic Filter component is used to filter data displayed in the components in a pane. Generic Filter component gives more flexible filtering capabilities compared to Dropdown.

info

In any pane, only one of Generic Filter or Dropdown components can be added.

An example of Generic Filter in use in an Nginx dashboard where browser used is Chrome and OS is Windows.

Generic filter can be added by selecting Generic Filter option from add component list

note

Only one generic filter component can be added to a pane

Using Generic Filter​

The results in a pane can be filtered by adding condition(s) in the Generic Filter component. Multiple conditions can be added to further fine tune the results. Once all the condition(s) are added, click the search icon to filter the results.

A condition contains 3 components:

• Field: Field name on which search has to be performed
• Operator: The search operator
• Value: The value to be searched for the provided field name

For Example, if one of the condition is code == 200, all the components in the pane will only show results where the value of code field is 200.

Here,

code is the field name

== is the operator

200 is the value

Use the search box to select from a list of recommended fields. Alternatively, type in the field name and press enter.

Based on the type of the selected field, operator recommendations will be provided.

• If the field type is number, then the operators will be >, >=, <, <=, !=, ==
• If the field type is text, then the operators will be ~(contains), ~-(does not contain), ==(equal to).
note

If the selected field has different types of values, all the opertors will be listed in operator recommendation.