Illumio Custom Data Collection Integration

Illumio Custom Data Collection Integration

Learn how to enhance the detection & response capabilities of your organization by integrating Illumio with Lumu’s data collection capabilities to pull, transform and inject the activity network logs recorded by Illumio into Lumu.

Requirements

  • An active Illumio Segmentation subscription.
    • An Illumio administrator user.
  • Lumu Custom Collector API configuration for Firewall Logs.
    • A Lumu custom collector ID and client key are required to set up the collection process. Information on how to create a custom collector in your Lumu portal can be found in Manage Custom Collectors.
  • Script host.
    • A scripting host is required to deploy the integration. This host must have Internet visibility over Lumu Defender API endpoints and Illumio public services. According to the deployment model you select, you will need a host with:
      • Python 3.13+
        or
      • A Docker-enabled host.
  • Script package.
    • Contact the Lumu support team to request the package we created to deploy the required files.

Contacted hosts

Ensure your script host can communicate with the following host. They are required for the operation of this integration.

  • console.illum.io
  • api.lumu.io
  • docker.io
  • ghcr.io
  • *.ubuntu.com
  • *.launchpad.net
  • canonical.com
  • debian.org
  • *.debian.org
  • debian-security.org
  • pypi.python.org
  • pypi.org
  • pythonhosted.org
  • files.pythonhosted.org

Integration overview

Lumu custom data collection integration with Illumio uses its API to run traffic queries, collect their results, process them as Lumu events, and send them to Lumu Cloud.

Preliminary Setup - Illumio

To set up the integration, you must prepare your Illumio instance to communicate with the Lumu integration. To do this, you need the following:

  • Create a dedicated integration user
  • Create an API Key.

The following sections will guide you on how to perform these tasks

Create a dedicated integration user

We encourage you to create a dedicated Illumio integration user. It allows you to segregate and trace activities beyond the regular administration tasks. To create your new integration user, log in to your Illumio console and follow these steps:

1. Go to Access (1) in the left navigation menu, and click on Users (2).


2. In the Users window, click on Add (1).

3. Fill in the New User information following these guidelines:

    1. Type in the First and Last Name.
    2. Type in a valid Email.
    3. Assign the Viewer role to the user (1).
    4. When finished, click on Add (2).

4. You will receive an account activation email. Activate your new integration user by following the instructions given in the Illumio activation email. Keep the credentials at hand, they will be used during the Set up the configuration files step.

Create an API Token

You must create an API key. This credential allows the integration to use the Illumio API to manage the Illumio IP list by updating it with Lumu IPs. To create it, log in to your Illumio portal with your integration user and follow these steps:

1. Open the profile menu (1) in the top-right corner of the screen. Then click on My API Keys (2) from the list.

2. In the API Keys window, copy the listed API Endpoint (1) and Organization ID (2) values and keep them at hand, they will be used during the Set up the configuration files step. Finally, click on Add (3).

3. Fill in the Create API Key form. Give the key a distinctive Name. Optionally, give it a Description. When finished, click on Create (1).

4. Copy the API key information. Alternatively, click on Download Credentials (1) to download the API credentials into a text file (.txt). Save them for later, they will be used during the Set up the configuration files step.

Preliminary setup - Lumu portal

The integration setup process needs you to collect this information from Lumu portal:

  • Lumu Collector Key
  • Lumu Collector ID
  • Company UUID

Log in to your Lumu portal and run the following procedures to collect this data.

Collect your Lumu Collector Key

To collect the Lumu Collector key, please refer to the Collector key document.

Collect your Lumu Collector ID

To collect the Lumu Custom Collector key, please refer to the Collector ID document.

Collect your Lumu company UUID

To collect your Lumu company UUID, log in to your Lumu portal. Once you are in the main window, copy the string below your company name.


Preliminary Setup - Choose your integration environment

There are 2 environment options to deploy the script. Select the one that best fits your current infrastructure.

  • Run it as a Python script (Unix-based systems and Windows)
  • Run it as a Docker container.
    • By using the Makefile model (Unix-based systems)(Recommended).
    • By using Docker commands (Unix-based systems and Docker Desktop for Windows).

Whichever alternative you select, you must unpack the integration package shared by our Support team.

Unpack the deployment package provided by Lumu in your preferred path/folder. Keep in mind this location, as it will be required for further configurations. From now on, we will refer to this folder as <app_lumu_root>.

Prepare your integration environment

Notes Before starting, ensure your integration environment can communicate with the all hosts listed in the Contacted hosts section.

You can deploy your integration using the following alternatives:

  • Run your integration using Python
  • Run your integration using Docker

Follow the instructions based on the selected deployment method.

Prepare Python on your environment

Notes If Docker is your chosen deployment method, you may skip this step.

If Python is your chosen deployment method, you will need to create a virtual environment for each integration to avoid conflicts between them and your operating system tools. Make sure you follow the steps in our Preparing Environment for Custom Integrations article.

Prepare Docker on your environment

Notes If you chose Python as your deployment method, you may skip this step.

If Docker is your chosen deployment method, you must follow the Docker installation documentation that corresponds to your OS. Ensure you follow the Post-installation steps for Linux before deploying the integration.

Notes For Windows users, follow the Install Docker Desktop for Windows documentation to install the Docker Engine.

Set up the configuration files

You need to add and edit the integrations.yml configuration file to set up the integration.

Notes You will find the integrations_template.yml sample file inside the integrations package. Use it to build your configuration file.
Notes All the parameters in red should be replaced with the real data necessary for your integration deployment. For example, the parameter “COMPANY-UUID” should end up as something similar to “aa11bb22bb33-123a-456b-789c-11aa22bb33cc”. Follow these indications for all similar parameters.
lumu: uuid: "COMPANY-UUID" collector_key: "COLLECTOR-KEY"
  collector_id: "COLLECTOR-ID"
app: name: "UNIQUE-NAME" api: api_endpoint: "API-ENDPOINT" # https://us-scp41.illum.io/api/v2/ username: "USERNAME" secret: "SECRET" organization_id: "ORGANIZATION-ID"

Replace the highlighted placeholders as follows:

  • COMPANY-UUID with the Company UUID collected in the Collect your Lumu company UUID section.
  • COLLECTOR-KEY with the Collector KEY collected in the Collect your Lumu Collector Key section.
  • COLLECTOR-ID with the Collector ID collected in the Collect your Lumu Collector ID section.
  • UNIQUE-NAME with a distinctive name for the integration. We recommend using the customer's or Illumio tenant’s name.
  • API-ENDPOINT with the API Endpoint collected in the Create an API Token section. Delete the <resource> part at the end of the string if present. The value should look as follows:

api_endpoint: "https://us-scp41.illum.io/api/v2/"

  • USERNAME with the username of the user created in the API token collected in the Create an API Token section.
  • SECRET with the secret of the user created in the API token collected in the Create an API Token section.
  • ORGANIZATION-ID with the organization ID collected in the Create an API Token section.
Alert You must fill in the configuration data carefully. If there are any mistakes or missing data, you’ll receive errors during the deployment of the integration.

To streamline the deployment process, Lumu introduced the Makefile model integration that allows you to easily deploy integrations as a Docker container. To deploy the integration, locate yourself in the <app_lumu_root> folder, and run the following command:

make docker-run-build
Notes Monitor the console output for any unexpected errors. If there are any errors present, fix them and run the command again. Check the Troubleshooting section for further reference

Deploy as a Python script

Notes In some Python installations, the executable name could vary from python to python3. If any Python command shows an error, change the python string in the presented command by python3

We encourage you to create a Python environment to deploy the integration as a Python script. You will find specific instructions in the Create a Virtual Environment document. Install the required dependencies by running the following commands:

For Windows environments:

ENV_FOLDER\Scripts\activate.bat
python -m pip install -r requirements

For Unix-based environments:

source ENV_FOLDER\bin\activate
python -m pip install -r requirements

Replace the ENV_FOLDER placeholder with the name of your virtual environment folder.

Script details

To use the script, you must locate yourself on the path selected for deployment (<app_lumu_root>). Use the following command to show all options available for the package:

python run.py --help

Usage: run.py [OPTIONS]

╭─ Options ─────────────────────────────────────────────────────────────────────────────────╮
│ --verbose -v                             Enable verbose mode.                             │
│ --logging-type -l [screen|file]          Logging output type: 'screen' or 'file'          │
│                                          [default: screen]                                │
│ --config TEXT                            Path to the configuration file.                  │
│                                          [default: integrations.yml]                      │
│ --help                                   Show this message and exit.                      │
╰───────────────────────────────────────────────────────────────────────────────────────────╯
Options Description
-h, --help Show this message and exit.
--config TEXT Path to the configuration file. [default: integrations.yml]
--logging-type -l [screen|file] Logging output type: 'screen' or 'file' [default: screen]
--verbose, -v Enable verbose mode.

Usage Examples

Task: poll and inject Illumio Traffic logs into Lumu

Run the following command to poll all the Illumio logs and push them into the Lumu custom data collector.

python run.py

Deploy as a Docker container (Optional)

If you have a Docker environment, you can select this option to run the integration as a Docker process. To deploy and run your integration as a Docker container, locate yourself in the <app_lumu_root> folder, and follow these instructions:

1. To build the container, run the following command. Change all the flags based on the reference given in the script section above.

docker build --tag img-illumio-collection --file Dockerfile .
Notes Do not forget the dot "." at the end of the line

2. To run the container, run the following command:

docker run -v ./integrations.yml:/app/integrations.yml -v ./dead_letters:/app/dead_letters -d --restart unless-stopped --log-driver json-file --log-opt max-size=30m --log-opt max-file=3 --name lumu-illumio-collection img-illumio-collection

Expected results

After you configure the integration, you will see the processed events in the custom collector created in Lumu portal. Lumu integrations will process events from the previous 10 minutes since the integration activation time.

Troubleshooting

The commands defined in this section will allow you to troubleshoot the operation of your integration. Keep in mind that you must locate yourself in the <app_lumu_root> folder before running any of them.

Deployment as a Docker container via Makefile

The following are the troubleshooting commands for this deployment option:

  • Checking integration logs
    Run the following command to check your integration logs.
make docker-logs


  • Checking integration errors
    Run the following command to check errors in your integration.
make docker-errors
  • Check the status of the integration
    Run the following command to check the status of the integration.
make docker-ps

  • Stopping the integration
    Run the following command if you need to stop the integration.
make docker-stop
  • Starting the integration
    Run the following command to start the integration.
make docker-start
  • Fixing issues with sudo for Docker
    If you cannot run Docker commands with your current user, run the following command.
make docker-fix-sudo
  • Reinstalling integration from scratch
    Run the following command to reinstall the integration from scratch.
docker-reset-force
  • Collecting and packaging logs for Lumu support
    Run the following command to collect and package the integration logs to share them with the Lumu support team. This command will create the support.tar package file that contains relevant information for the Lumu support team.
make docker-support

Deployment via Python script

To identify failures in the script, please use the -v flag. This will allow you to identify failures in the script execution.

Deployment as a Docker container

For troubleshooting purposes, you can run the following commands:

  • Logging in to the container using an interactive shell
docker exec -it lumu-illumio-collection bash
  • Collecting integration logs
docker logs -f lumu-illumio-collection

        • Related Articles

        • Netskope Log Streaming Custom Data Collection Integration

          In this article, you will find out how to configure your Netskope Log Streaming subscription and its Lumu Custom Data Collection integration to pull, transform, and inject the Web Transactions by Netskope Log Streaming into Lumu to enhance the ...
        • DNSFilter Custom Data Collection Integration

          In this article, you will find out how to configure your DNSFilter subscription and its Lumu Custom Data Collection integration to pull, transform, and inject the query logs recorded by DNSFilter into Lumu to enhance the detection & response ...
        • Akamai SIA Custom Data Collection Integration

          In this article, you will find out how to configure your Akamai Secure Internet Access Enterprise (SIA) subscription and the Lumu Custom Data Collection integration to pull, transform, and inject the DNS query and Proxy logs recorded by Akamai into ...
        • Illumio Custom Response Integration

          Learn how to leverage the Lumu Defender API and Illumio API to mitigate security risks. Response integration between Illumio and Lumu Requirements An active Illumio Segmentation subscription. You need an Illumio administrator user to set up the ...
        • Zero Networks Custom Data Collection Integration

          Learn how to enhance the detection & response capabilities of your organization by integrating Zero Networks with Lumu’s data collection capabilities to pull, transform and inject the activity network logs recorded by Zero Networks into Lumu. ...