Lumu Virtual Appliance Metadata Collection with Logstash

Some enterprises use the Elastic stack (ELK) to collect, index, and analyze logs from multiple devices. If yours is one of those organizations, deploy a Lumu Virtual Appliance and create collectors that will receive data directly from existing Logstash devices to seamlessly integrate your network infrastructure with Lumu while layering Continuous Compromise Assessment.

This article provides you with an example of using the typical ELK stack to collect, index, and analyze logs and how to leverage Logstash devices to forward events to a Lumu Virtual Appliance for Continuous Compromise Assessment.

Typical ELK stack deployment with Lumu VA

Logstash configuration

Configure Logstash’s filter and output to forward raw messages from different network devices to the Lumu Virtual Appliance, avoiding the transformation of data that will be pushed into the ELK Stack’s components.

The following is an example of how to add a clone block inside your filter configuration to preserve the raw event, avoid conflicts between original data parsing and the transformation process, and send raw data to the Lumu VA. The clone block must be the first one inside the filter block.

filter {
	clone {
		clones => [“lumu”]
    }
    …
}

This example will preserve the original event and create a clone with a field named type set to “ raw ”. The next step will be adding an additional output configuration to forward the cloned event to the Lumu Virtual Appliance.

output {
	if “lumu” in [tags] {
		udp|tcp {
			host => “<Lumu VA IP/hostname>”
			port => “<Lumu VA collector port>”
			codec => line { format => “%{message}” }
		}
	}
}

This example assumes events do not have a tag field set to “lumu”. Please set these conditions based on Logstash’s current configuration for collecting events. You must preserve other output configurations to maintain the operation of your ELK deployment. We encourage you to implement additional filtering to avoid lumu-tagged events from being sent to Elastic. Here is an example

if [type] == "fortigate" and "lumu" not in [tags] {
  elasticsearch {
    ...
  }
}

The line codec combined with the format tag, will preserve the original format of the event received by Logstash (saved in the message field).

Create and Configure Virtual Appliance Collectors

To create and configure Virtual Appliance collectors, please consult our documentation for details. When configuring the collector’s details in the Virtual Appliance, do not use Syslog encapsulation to avoid event processing malfunctions. Logstash’s forwarding feature will add header information that could cause irregular Syslog formatting.

In some cases, you will need to tune the collector configuration manually.