Lumu Virtual Appliance Metadatada Collection with Logstash

Lumu Virtual Appliance Metadatada Collection with Logstash

Some enterprises use the Elastic stack (ELK) to collect, index, and analyze logs from multiple devices. If yours is one of those organizations, deploy a Lumu Virtual Appliance and create collectors that will receive data directly from existing Logstash devices to seamlessly integrate your network infrastructure with Lumu while layering Continuous Compromise Assessment.

This article provides you with an example of using the typical ELK stack to collect, index, and analyze logs and how to leverage Logstash devices to forward events to a Lumu Virtual Appliance for Continuous Compromise Assessment.
Typical ELK stack deployment with Lumu VATypical ELK stack deployment with Lumu VA

Logstash configuration

Configure Logstash’s filter and output to forward raw messages from different network devices to the Lumu Virtual Appliance, avoiding the transformation of data that will be pushed into the ELK Stack’s components.
The following is an example of how to add a clone block inside your filter configuration to preserve the raw event, avoid conflicts between original data parsing and the transformation process, and send raw data to the Lumu VA. The clone block must be the first one inside the filter block.
  1. filter { clone { clones => [“raw”] } … }
This example will preserve the original event and create a clone with a field named type set to “ raw ”. The next step will be adding an additional output configuration to forward the cloned event to the Lumu Virtual Appliance.
  1. output { if “raw” in [type] { udp|tcp { host => “<Lumu VA IP/hostname>” port => “<Lumu VA collector port>” codec => line { format => “%{message}” } } } }
This example assumes events do not have a type field set to “ raw ”. Please set these conditions based on Logstash’s current configuration for collecting events.
The line codec combined with the format tag, will preserve the original format of the event received by Logstash (saved in the message field).

Create and Configure Virtual Appliance Collectors

To create and configure Virtual Appliance collectors, please consult our documentation  for details. When configuring the collector’s details in the Virtual Appliance, do not use Syslog encapsulation to avoid event processing malfunctions. Logstash’s forwarding feature will add header information that could cause irregular Syslog formatting. 
In some cases, you will need to tune the collector configuration manually.

        • Related Articles

        • Custom Collector API Integration With Packetbeat and Logstash

          Some enterprises may already be using Logstash as part of their pipelines to centralize their logs. In such cases, you have the option to deploy custom collectors to seamlessly integrate your network infrastructure with Lumu while layering Continuous ...
        • Virtual Appliance Collectors

          The Lumu Virtual Appliance (VA) offers the option to create VA Collectors, a seamless way to collect the network metadata of your entire enterprise and forward it to the Lumu cloud with the lowest impact on the network operation. In this quick guide, ...
        • Introduction to Lumu Virtual Appliances

          The Lumu Virtual Appliance (VA) is a pre-configured lightweight virtual machine solution that collects the network metadata of your entire enterprise and forwards it to the Lumu cloud with the lowest impact on the network operation. This document ...
        • Configure Netflow/IPFIX Collector on Lumu Virtual Appliance

          The Lumu Virtual Appliance (VA) can work alongside your current security infrastructure to provide an even better Continuous Compromise Assessment. The Lumu VA offers the option to create Collectors, a seamless way to integrate with metadata ...
        • Set Lumu as primary network DNS Resolver on Virtual Appliances

          The Lumu Virtual Appliance (VA) offers a seamless way to integrate the network metadata of your entire enterprise into the Lumu platform. You can easily deploy and set up a Lumu Virtual appliance to collect DNS network metadata through the Virtual ...