Time Extractors

Time extractors are used to extract a timestamp from a given log message. The extracted timestamp is then used for sorting and retention purposes.

There are 3 types of time extractors:

  • FieldTimeExtractor - extracts the timestamp from a JSON field in the log message using the provided format string.
  • RegexTimeExtractor - extracts the timestamp from the log message using a regular expression, and then parses it using the provided format string.
  • RegexFieldTimeExtractor - extracts the timestamp from a JSON field in the log message using a regular expression, and then parses it using the provided format string.

If an extractor fails to extract a timestamp from a log message, then it will be assigned the current time as its timestamp.

Field Time Extractor

The FieldTimeExtractor is used to extract a timestamp from a JSON field in the log message using the provided format string. If no Time Extractor is specified for an input, the default is to use this type with the following configuration:

  • field: timestamp
  • format: %s

This means that by default, the extractor will look for a JSON field called timestamp, and attempt to parse it as a Unix timestamp with seconds precision. If no such field is found, or if parsing fails, then the current time will be used as the timestamp.

Regex Time Extractor

The RegexTimeExtractor is used to extract a timestamp from the log message using a regular expression, and then parse it using the provided format string. This type of extractor can be useful for parsing timestamps that are not in JSON format, or for which the field name is not known in advance.

It has two parameeters to configure:

  • regex: A regular expression pattern used to match and extract the timestamp from the log message. The timestamp must be captured by a named group called timestamp.
  • format: A format string used to parse the extracted timestamp.

Regex Field Time Extractor

The RegexFieldTimeExtractor is used to extract a timestamp from a JSON field in the log message using a regular expression, and then parse it using the provided format string. This type of extractor can be useful for parsing timestamps that have additional characters or formatting around them, such as quotes or brackets.

It has three parameters to configure:

  • field: The name of the JSON field containing the timestamp.
  • regex: A regular expression pattern used to match and extract the timestamp from the field value. The timestamp must be captured by a named group called timestamp.
  • format: A format string used to parse the extracted timestamp.