The Custom Collector API is a seamless way to integrate your network infrastructure with Lumu while layering Continuous Compromise Assessment. It allows sending network metadata captured from third-party platforms/services/appliances to Lumu, and it can also be used as an alternative for obtaining greater visibility in cases where the enterprise network restricts the use of Virtual Appliances.
This article provides you with Custom Collector API endpoints specifications, requirements, sample JSON, and request parameters.
The authentication of the Lumu Customer Collector API is handled by a unique key referred to in this document as {lumu-client-key}
. Each custom collector also has a unique ID that is referred to as {collector-id}
to perform diagnostic/support tasks smoothly.
The Custom Collector API key (1) and collectors ID (2) are found in the Lumu Portal and are self-managed by company administrators.
The Lumu Custom Collectors is a REST API restricted to HTTPS and is available to make requests at: https://api.lumu.io
application/json
header.
All the post requests to Custom Collector API are identified as traffic of the company by the required query parameters {lumu_client_key}
and {collector_id}
, as in the following example:
https://api.lumu.io/collectors/64c1a892-9ea3-4c7b-b8af-e1f5b2ff6b0c/dns/packets?key=64c1a892-9ea3-4c7b-b8af-e1f5b2ff6b0c.
This method post arrays of DNS packets captured using third-party solutions such as Logstash, Graylog, and other DNS packet capture tools.
POST /collectors/{collector-id}/dns/packets?key={lumu-client-key}
[ { "id": 69, "timestamp": "2021-01-05T14:37:02.228Z", "client_ip": "192.168.0.103", "client_name": "User9800", "op_code": "QUERY", "response_code": "NOERROR", "question": { "type": "A", "name": "
www.example.com", "class": "IN" }, "flags": { "authoritative": false, "recursion_available": true, "truncated_response": false, "checking_disabled": false, "recursion_desired": true, "authentic_data": false }, "answers": [ { "name": "www.example.com", "type": "A", "class": "IN", "ttl": 2549, "data": "69.172.201.153" } ] } ]
Body parameters (details)
Name |
Data Type |
Description |
---|---|---|
id |
integer
|
A 16-bit numeric identifier assigned by the software that generates the query, such as Packetbeat, Logstash, Graylog, etc.
|
timestamp |
date-time
|
The UTC date and time when query request was performed in ISO format, e.g. "
2021-01-07T14:37:02.228Z"
|
client_ip |
string
|
The dotted-quad formatted or colon-separated IPv4/IPv6 address of the client that initiated the request, e.g. "
192.168.1.16" or "2001:db8:85a3::8a2e:370:7334"
|
client_name |
string
(optional)
|
The asset identification as you want it to be shown at the Lumu Portal, e.g. the user or the device hostname from which the query originated.
|
op_code |
string
|
The DNS operation code that specifies the type of query in the message, e.g. "QUERY"
|
response_code |
string
|
The keyword of the DNS response code, e.g. "
NOERROR"
|
question.type |
string
|
The type of records being queried, e.g. "A" or "
AAAA"
|
question.name |
string
|
|
question.class |
string
|
The class of records being queried, e.g. "
IN"
|
flags.authoritative |
boolean
|
A flag specifying that the responding server is an authority for the domain name used in the question, e.g.
false
.
|
flags.recursion_available |
boolean
|
A boolean flag specifying whether recursive query support is available in the name server, e.g.
true
|
flags.truncated_response |
boolean
|
A flag specifying that only the first 512 bytes of the reply were returned., e.g.
false
|
flags.checking_disabled |
boolean
|
A flag specifying if the client disabled the server signature validation of the query, e.g.
true
|
flags.recursion_desired |
boolean
|
A flag specifying that the client directs the server to pursue a query recursively. Recursive query support is optional, e.g.
false
|
flags.authentic_data |
boolean
|
A flag specifying that the recursive server considers the response authentic, e.g. true
|
answers |
array
(optional)
|
Objects with each answer section returned by the server
|
answers.name |
array
|
The domain name to which this resource record pertains
|
answers.type |
string |
The type of data contained in this resource record, e.g. "
A"
or "
AAAA"
|
answers.class |
string
|
The class of records being queried, e.g. "
IN"
|
answers.ttl |
integer
|
The time interval in seconds that this resource record may be cached before it should be discarded. This field is a long numeric type where zero values mean that the data should not be cached, e.g.
2549
|
answers.data |
string
|
The data describing the resource. The meaning of this data depends on the type and class of the resource record, e.g. "
69.172.201.153"
|
Use this method to post simple DNS queries extracted from DNS servers’ logs. It is a simplified version of the DNS Packets method. While this form of posting DNS data is suitable for Lumu’s Illumination Process, it lacks information that could be useful for understanding more complex adversarial activity.
This method receives an array of DNS queries, which are identified as traffic of the company by the {lumu-client-key}
in the Lumu Portal. The path parameter {collector-id}
is a unique identifier for each collector to perform diagnostic/support tasks easily.
POST /collectors/{collector-id}/dns/queries?key={lumu-client-key}
[ { "timestamp": "2021-01-06T14:37:02.228Z", "name": "
www.example.com", "client_ip": "192.168.0.103", "client_name": "MACHINE-0987", "type": "A" } ]
Body parameters (details)
Name
|
Data Type
|
Description
|
timestamp |
date-time
|
The UTC date and time when query request was performed in ISO format, e.g. "
2021-01-07T14:37:02.228Z"
|
name |
array
|
The domain name being queried. Non-printable characters are represented as in C; i.e., as \DDD, where DDD is the octal ASCII character code. Backslashes and quotes are escaped. Tabs, carriage returns, and line feeds are converted to \t, \r, and \n respectively, e.g. "www.example.com
"
|
client_ip |
string
|
The dotted-quad formatted or colon-separated IPv4/IPv6 address of the client that initiated the request, e.g. "
192.168.1.16"
or "
2001:db8:85a3::8a2e:370:7334"
|
client_name |
string
(optional)
|
The name of the device or the user from which the query was originated
|
type |
string
(optional)
|
The type of data contained in this resource record, e.g. "
A"
or "
AAAA"
|
Use this method to post proxy entries composed of information extracted from the client proxy log. This method receives an array of Proxy entries, which are identified as traffic of the company by the {lumu-client-key}
.
POST /collectors/{collector-id}/proxy/entries?key={lumu-client-key}
[ { "timestamp": "2021-01-07T14:37:02.228Z", "client_ip": "192.168.0.103", "client_name": "optional DeviceName", "user_name": "optional UserName", "remote_ip": "201.98.87.1", "time_taken": 6532, "request": { "uri": { "host": "birazoku.com", "port": 443, "scheme": "https", "path": "/index.html", "query": "name=myname&wlkd=ksi" }, "method": "post", "length": 1034, "body_length": 887, "referrer": "referrer.com", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X1" }, "response": { "code": 200, "phrase": "Success", "length": 3049, "body_length": 2978 } } ]
Body parameters (details)
Name
|
Data Type
|
Description
|
timestamp |
date-time
|
The UTC date and time when query request was performed in ISO format, e.g. "2021-01-07T14:37:02.228Z"
|
client_ip |
string
|
A string containing the dotted quad formatted or colon-separated IPv4/IPv6 address of the client that initiated the request, e.g. "192.168.1.16" or "2001:db8:85a3::8a2e:370:7334"
|
client_name |
string
(optional)
|
The name of the device or the user from which the query was originated
|
user_name |
string
(optional)
|
The authenticated user in the proxy session
|
remote_ip |
string
(optional)
|
The dotted quad-formatted or colon-separated IPv4/IPv6 address of the server which response the request, e.g. "192.168.1.16" or "2001:db8:85a3::8a2e:370:7334"
|
time_taken |
integer
(optional)
|
The time used to complete the request in milliseconds, e.g. 2549
|
request.uri.host |
string
|
The domain or IP against which the request was made, e.g. "example.com"
|
request.uri.port |
integer
|
An integer containing the TCP port used to make the request, e.g. 443
|
request.uri.scheme |
string
|
The scheme used to perform the request, e.g. "http" or "https"
|
request.uri.path |
string
(optional)
|
The specified resource that was being requested, default is “/”
|
request.uri.query |
string
(optional)
|
The parameters used in the request, e.g. "name=myname&wlkd=ksi"
|
request.method |
string
|
The HTTP method used to run the request, e.g. "post", "get," "put"
|
request.length |
integer
(optional)
|
The size of the request in bytes, e.g. 1034
|
request.body_length |
integer
(optional)
|
The size of the body of the request in bytes, e.g. 900
|
request.referer |
string
(optional)
|
The address of the webpage which is linked to the resource being requested, e.g. "referrer.com"
|
request.user_agent |
string
(optional)
|
The user agent of the browser that made the request, e.g. "Mozilla/5.0 (Macintosh; Intel Mac OS X1"
|
response.code |
integer
|
The HTTP code response for the request, e.g. 200
|
response.phrase |
string
|
The HTTP string response for the request, e.g. "Success"
|
response.length |
integer
(optional)
|
The size of the response in bytes, e.g. 1200
|
response.body_length |
integer
(optional)
|
The size of the body of the response in bytes, e.g. 800
|
Use this method to post firewall entries composed of information extracted from the client firewall log. This method receives an array of Firewall entries, which are identified as traffic of the company by the {lumu-client-key}
at the Lumu Portal.
POST /collectors/{collector-id}/firewall/entries?key={lumu-client-key}
[ { "timestamp": "2021-01-06T14:37:02.228Z", "time_taken": 1630, "user_name": "optional UserName", "action": "accept", "protocol": "http", "additional": { "path": "/mhiQtgAAAAE", "query": "key=value&t=846" }, "source": { "ip": "10.0.0.46", "port": 54786, "name": "FW_SGM_DC_01" }, "destination": { "ip": "52.91.3.178", "port": 443, "name": "google.com" }, "sent": { "bytes": 53489, "packets": 183 }, "received": { "bytes": 67385, "packets": 163 } } ]
Body parameters (details)
Name
|
Data Type
|
Description
|
timestamp |
date-time
|
The UTC date and time when query request was performed in ISO format, e.g. "2021-01-07T14:37:02.228Z"
|
time_taken |
integer
(optional)
|
The time used to complete the request in milliseconds, e.g. 2549
|
user_name |
string
(optional)
|
The authenticated user in the firewall session
|
action |
string
|
The action performed by the firewall with the request, being accepted or blocked are common expected scenarios, e.g. "Accept"
|
protocol |
string
|
The protocol used by the API request, whether is layer seven or four, e.g. "http", "https", "ftp", "udp"
|
additional.path |
string
(optional)
|
the specified resource the request was requesting, default is “/”
|
additional.query |
string
(optional)
|
The parameters used in the request, e.g. "name=myname&wlkd=ksi"
|
source.ip |
string
|
The dotted-quad formatted or colon-separated IPv4/IPv6 address of the client that initiated the request, e.g. "192.168.1.16 or 2001:db8:85a3::8a2e:370:7334"
|
source.port |
integer
|
An integer containing the TCP port used to make the request, e.g. 54786
|
source.name |
string
(optional)
|
The name of the device or the user where the query originated. For outgoing connections, this will be shown in the Lumu Portal, e.g. "FW_SGM_DC_01"
|
destination.ip |
string
|
The dotted-quad formatted or colon-separated IPv4/IPv6 address of the server which response the request, e.g. "192.168.1.16" or "2001:db8:85a3::8a2e:370:7334"
|
destination.port |
integer
|
An integer containing the TCP port used by the destination to receive the request, e.g. 443
|
destination.name |
string
(optional)
|
The domain or hostname against which the request was made, e.g. "google.com"
|
sent.bytes |
integer
(optional)
|
The amount of bytes sent from source to destination during the request, e.g. 51034
|
sent.packets |
integer
(optional)
|
The amount of packets sent from source to destination during the request., e.g. 134
|
received.bytes |
integer
(optional)
|
The amount of bytes sent from destination to source during the request, e.g. 51034
|
received.packets |
integer
(optional)
|
The amount of packets sent from destination to source during the request, e.g. 130
|
The Lumu Custom Collector API uses standard HTTP features and response codes to indicate the success or failure of API requests
HTTP Code
|
Description
|
200/202
|
OK The request was successfully received, understood, and accepted
|
400/422
|
Bad Request/Unprocessable Entity . Client-side error status, verify malformed request syntax, invalid request message parameters, etc.
|
500
|
Internal server error . Our application server encountered a problem. Try again later and if the problem persists, please let us know.
|
Lumu provides an example of using third-party tools as metadata collectors, check our documentation on Custom Collector API Integration with Packetbeat and Logstash.