# Collecting Logs from Linux

#### Prerequisites:

* Filebeat 6.3 (or higher) installed on your system. Please see the[ installation instructions](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation.html)
* Allow outgoing traffic to destination port 5044 for Logstash and port 9200 for elastic search.
* Either port `5044` or `9200` should be open not both.

### Installation on Linux / Mac

**Step 1**: Download the certificate

```
wget https://raw.githubusercontent.com/cfxDLAio/public-certificates/master/COMODORSADomainValidationSecureServerCA.crt
```

**Step 2**: Copy the certificate to desired location

```
sudo mkdir -p /etc/pki/tls/certs
sudo cp COMODORSADomainValidationSecureServerCA.crt /etc/pki/tls/certs/
```

**Step 3**: Configure Filebeat

On Linux file systems, the configuration should be placed user

```
/etc/filebeat/filebeat.yml
```

Either you can use configuration wizard to create the configuration file or you can manually edit the configuration file. Always make sure that you have taken backup of original configuration file. 

The file contains two main sections:

* **`prospectors`**: define the files that needs to be monitored
* **`outputs`**: responsible for sending the data in JSON format

**Note 1**: 

There is a difference when configuring a prospector that harvests JSON files vs a prospector that harvests simple text line logs. Please follow the following guidelines.

When harvesting JSON files, please add **`cfxDLAio_codec`** equals to **`json`** to the fields object. 

When harvesting text files, please add **`cfxDLAio_codec`**  equals to **`plain`** to the fields object.<br>

**Note 2**:

The **`fields_under_root`** property should always set to be true.

**Note 3**:

The **`type`** property should be defined, it is used to identify the type of log data and provide you with a richer logging experience. For complete list of known types, please click [here](https://support.logz.io/hc/en-us/articles/210205985-Which-log-types-are-preconfigured-on-the-Logz-io-platform-). If the type you are looking for is not listed, please select the type that is closer or best describes your data.

The sample configuration file will look like the following:

```
filebeat.inputs:
- type: log
  paths: 
  - /path/to/log/file.log
  fields:
    cfxDLAio_codec: plain
    token: GsfEyZvknXZFtrNfqdmrmiJXvYIKJbID
    type: MY_LOG_TYPE
  fields_under_root: true
  encoding: utf-8
  ignore_older: 3h
  
- type: log
  paths: 
  - /path/to/json/file.json
  fields:
    cfxDLAio_codec: json
    token: GsfEyZvknXZFtrNfqdmrmiJXvYIKJbID
    type: MY_JSON_LOG_TYPE
  fields_under_root: true
  encoding: utf-8
  ignore_older: 3h
  
#For version 6.x and lower uncomment the line below and remove the line after it 
#filebeat.registry_file: /var/lib/filebeat/registry 
 
filebeat.registry.path: /var/lib/filebeat
 
#The following processors are to ensure compatibility with version 7
processors:
- rename:
    fields:
     - from: "agent"
       to: "beat_agent"
    ignore_missing: true
- rename:
    fields:
     - from: "log.file.path"
       to: "source"
    ignore_missing: true
    
############################# Output ##########################################
output:
  logstash:
    hosts: ["listener-ca.cfxdla.io:5044"]
    ssl:
      certificate_authorities: ['/etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt']
```

You can add as many file entries as you like. All file entries should contain the **`cfxDLA`** token.