Time Extractors

Time extractors are used to extract a timestamp from a given log message. The extracted timestamp is then used for sorting and retention purposes.

There are 3 types of time extractors. In JSON configuration they are selected with a type field:

• Field - extracts the timestamp from a JSON field in the log message using the provided format string.
• Simple - extracts the timestamp from a raw log message using a regular expression, and then parses it using the provided format string.
• RegexField - extracts the timestamp from a JSON field using a regular expression, and then parses it using the provided format string.

If an extractor fails to extract a timestamp from a log message, then it will be assigned the current time as its timestamp.

Field

The Field extractor is used to extract a timestamp from a JSON field in the log message using the provided format string. If no Time Extractor is specified for an input, the default is to use this type with the following configuration:

• field: timestamp
• format: %+

This means that by default, the extractor will look for a JSON field called timestamp, and attempt to parse it as an RFC 3339 / ISO 8601 timestamp. If no such field is found, or if parsing fails, then the current time will be used as the timestamp.

{
  "type": "Field",
  "field": "timestamp",
  "format": "%+"
}

Simple

The Simple extractor is used to extract a timestamp from a raw log message using a regular expression, and then parse it using the provided format string. This type of extractor can be useful for parsing timestamps that are not in JSON format, or for which the field name is not known in advance.

It has two parameters to configure:

• regex: A regular expression pattern used to match and extract the timestamp from the log message. The timestamp must be captured by a named group called timestamp.
• format: A format string used to parse the extracted timestamp.

{
  "type": "Simple",
  "regex": "^(?<timestamp>[^\\s]+)",
  "format": "%+"
}

RegexField

The RegexField extractor is used to extract a timestamp from a JSON field in the log message using a regular expression, and then parse it using the provided format string. This type of extractor can be useful for parsing timestamps that have additional characters or formatting around them, such as quotes or brackets.

It has three parameters to configure:

• field: The name of the JSON field containing the timestamp.
• regex: A regular expression pattern used to match and extract the timestamp from the field value. The timestamp must be captured by a named group called timestamp.
• format: A format string used to parse the extracted timestamp.

{
  "type": "RegexField",
  "field": "message",
  "regex": "^\\[(?<timestamp>[^\\]]+)\\]",
  "format": "%+"
}