# Log Transformation & Enrichment

### Advanced Usage

The previous example doesn't show how to parse the correct timestamp or how to ship multi-line logs. These are important issues that can easily be addressed.

To get started, let's assume our logs contain a timestamp (UTC), severity and message.  If your logs have different structure you'll need to make small adjustments to the configs below.

`2015-10-03 12:01:58,345 ERROR Processing request failed.`

Multi-line messages require a slight changed input section.

```
input {
  file {
    path => "/var/log/test.log"
    start_position => "beginning"
    codec => multiline {
      pattern => "^%{TIMESTAMP_ISO8601}"
      negate => true
      what => "previous"
    }
  }
}
The pattern looks for log lines starting with a timestamp and, until a new match is found, all lines are considered part of the event. This is done by setting the negate parameter to true.
Parsing logs means adding a new section to the config file:
filter {
  grok {
    match => [ "message", "(?m)%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:severity} %{GREEDYDATA:message}" ]
    overwrite => [ "message" ]
  }
  date {
    match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss,SSS" ]
  }
}
```

\
The [grok ](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html)filter splits the log content into 3 variable. The (?m) in the beginning of the regexp is used for multi-line matching. Otherwise, only the first line would be read. The patterns used in the [regexp](https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/grok-patterns) used in the regexp are provided with LogStash and should be used when possible to simplify regexps. By default, the timestamp of the log line is considered the moment when the log line is read from the file. the [date ](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html)filter changes the timestamp to the one matched earlier by the grok filter.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.cloudfabrix.io/cfxloganalytics/log-transformation-and-enrichment.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.