Shape data to work with the Geo package
Functions in the Geo package require the following data schema:
- an s2_cell_id tag containing the S2 Cell ID as a token
- a
lat
field field containing the latitude in decimal degrees (WGS 84) - a
lon
field field containing the longitude in decimal degrees (WGS 84)
Shape geo-temporal data
If your data already contains latitude and longitude fields, use the
geo.shapeData()
function
to rename the fields to match the requirements of the Geo package, pivot the data
into row-wise sets, and generate S2 cell ID tokens for each point.
import "experimental/geo"
from(bucket: "example-bucket")
|> range(start: -1h)
|> filter(fn: (r) => r._measurement == "example-measurement")
|> geo.shapeData(
latField: "latitude",
lonField: "longitude",
level: 10
)
Generate S2 cell ID tokens
The Geo package uses the S2 Geometry Library to represent geographic coordinates on a three-dimensional sphere. The sphere is divided into cells, each with a unique 64-bit identifier (S2 cell ID). Grid and S2 cell ID accuracy are defined by a level.
To filter more quickly, use higher S2 Cell ID levels, but know that that higher levels increase series cardinality.
The Geo package requires S2 cell IDs as tokens. To generate add S2 cell IDs tokens to your data, use one of the following options:
- Generate S2 cell ID tokens with Telegraf
- Generate S2 cell ID tokens language-specific libraries
- Generate S2 cell ID tokens with Flux
Generate S2 cell ID tokens with Telegraf
Enable the Telegraf S2 Geo (s2geo
) processor
to generate S2 cell ID tokens at a specified cell_level
using lat
and lon
field values.
Add the processors.s2geo
configuration to your Telegraf configuration file (telegraf.conf
):
[[processors.s2geo]]
## The name of the lat and lon fields containing WGS-84 latitude and
## longitude in decimal degrees.
lat_field = "lat"
lon_field = "lon"
## New tag to create
tag_key = "s2_cell_id"
## Cell level (see https://s2geometry.io/resources/s2cell_statistics.html)
cell_level = 9
Telegraf stores the S2 cell ID token in the s2_cell_id
tag.
Generate S2 cell ID tokens language-specific libraries
Many programming languages offer S2 Libraries with methods for generating S2 cell ID tokens.
Use latitude and longitude with the s2.CellID.ToToken
endpoint of the S2 Geometry
Library to generate s2_cell_id
tags. For example:
- Go: s2.CellID.ToToken()
- Python: s2sphere.CellId.to_token()
- JavaScript: s2.cellid.toToken()
Generate S2 cell ID tokens with Flux
Use the geo.s2CellIDToken()
function
with existing longitude (lon
) and latitude (lat
) field values to generate and add the S2 cell ID token.
First, use the geo.toRows()
function
to pivot lat and lon fields into row-wise sets:
import "experimental/geo"
from(bucket: "example-bucket")
|> range(start: -1h)
|> filter(fn: (r) => r._measurement == "example-measurement")
|> geo.toRows()
|> map(fn: (r) => ({ r with
s2_cell_id: geo.s2CellIDToken(point: {lon: r.lon, lat: r.lat}, level: 10)
}))
The geo.shapeData()
function
generates S2 cell ID tokens as well.