#94 -IBM Spectrum Protect Plus Audit Log Workflow #95
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| # Workflow basic information: | ||
|
|
||
| * Author: Daniel Wendler | ||
| * Maintainer: dwendler, dwendler(at)de(dot)ibm(dot)com | ||
| * Workflow Version Number: 1.0 | ||
| * [Endpoint Documentation of Spectrum Protect Plus REST API](https://www.ibm.com/docs/en/SSNQFQ_10.1.10/pdf/restapi.pdf) | ||
| * [SPP landing page](https://www.ibm.com/docs/en/spp/10.1.10) | ||
| * supported endpoints: Audit Logs via api/endeavour/log/audit | ||
|
|
||
|
|
||
| # Workflow parameters | ||
|
|
||
| * #host# (required): IP_Address or Hostname, no protocol prefix, e.g. "mySppHost.myCompany.com" will be assebled to https://mySppHost.myCompany.com:443 | ||
| * #port# (not required, default is 443): 443 | ||
|
There was a problem hiding this comment. Here the port claims to be I've added some suggestions on handling this near the |
||
| * #username# (required): e.g. monitorUser, this user needs the correct RBAC within SPP to query the SPP audit logs | ||
| * #password# (required): password of above user | ||
| * #pageSize# (not required, leave default): number of audit log entries to retrieve with a single REST API get operation, the workflow will use pagination until no new events exist in the audit log queue. default = 100 is suggested by the API for this endpoint | ||
|
|
||
|
|
||
| # tested REST API of IBM Spectrum Protect Plus versions: | ||
| This Workflow has been tested with SPP version 10.1.9 and 10.1.10. | ||
| The SPP REST API in versions 10.1.8 and earlier do not provide the required | ||
| information and functionalities required by this workflow. | ||
|
|
||
| # QRadar Log Source Configuration | ||
|
|
||
| If you want to ingest data from an endpoint using Universal Rest API Protocol, configure a log source on the QRadar® Console using the Workflow field so that the defined endpoint can communicate with QRadar by using the Universal Rest API protocol. | ||
|
|
||
| 1. Log in to QRadar. | ||
| 2. Click the _Admin_ tab. | ||
| 3. To open the app, click the _Log Sources_ app icon and _launch_ the app to select _Log Sources - Manage Log Sources_ | ||
| 4. Click _New Log Source_ > _Single Log Source_. | ||
| 5. On the _Select a Log Source Type_ page, Select the Log Source Type _Universal DSM_ and click _Select Protocol Type_ > _Universal Cloud REST API_. | ||
| 6. *Important:* disable the function _Coalescing Events_, otherwise, similar Audit Logs may be interpreted as a single event. | ||
| 7. On the Select a Protocol Type page, select a protocol and click _Configure Log Source Parameters_. | ||
| 8. On the Configure the Log Source parameters page, configure the log source parameters and click _Configure Protocol | ||
| Parameters_. | ||
| 9. On the Configure the Protocol Parameters page, configure the protocol-specific parameters (Workflow and Workflow | ||
| Parameter Values). | ||
| 10. In the Test protocol parameters window, click _Start Test_. | ||
| 11. To fix any errors, click _Configure Protocol Parameters_. Configure the parameters and click Test Protocol Parameters. | ||
| 12. Click _Finish_ | ||
|
|
||
| # sample API response of an audit log entry | ||
|
|
||
| ``` | ||
| { | ||
| "accessTime": 1648475990938, | ||
| "serverTime": 1648475990938, | ||
| "user": "monitorUser", | ||
| "groups": "", | ||
| "operation": "Login", | ||
| "description": "Login failed for user monitorUser.", | ||
| "requesterIp": "AAA.BBB.CCC.100", | ||
| "sppserver": "AAA.BBB.CCC.120" | ||
| } | ||
| ``` | ||
|
|
||
|
|
||
| # conversion from epoch time to date and vice versa | ||
|
|
||
| **Note:** SPP utilizes epoch timestamp in milliseconds -> multiply / devide with 1000 may be required | ||
|
|
||
| ``` | ||
| date +%s # converts local time to epoch time in seconds (not MS) | ||
| date -d @1648466798 # convert epoch timestamp in seconds (not MS) to local date | ||
| ``` | ||
|
|
||
|
|
||
| # sample test tool execution and debug logs - sanitized | ||
|
|
||
| > time /opt/qradar/bin/test-workflow.sh -u -w /tmp/spp/spp-Workflow.xml -wp /tmp/spp/spp-Workflow-Parameter-Values.xml | ||
| ``` | ||
| SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] | ||
| 2022-03-28 16:30:59 [INFO ][LogAction] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> debug: true | ||
| 2022-03-28 16:30:59 [INFO ][LogAction] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> bookmark (start): 1648461176749 | ||
| 2022-03-28 16:30:59 [INFO ][LogAction] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> counter (start): 0 | ||
| 2022-03-28 16:30:59 [INFO ][LogAction] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> URL: https://AAA.BBB.CCC.100:443/api/endeavour/log/audit | ||
| 2022-03-28 16:30:59 [INFO ][UniversalCloudRESTAPITest] Received 1 events from AAA.BBB.CCC.100 | ||
| 2022-03-28 16:30:59 [INFO ][UniversalCloudRESTAPITest] {"accessTime":1648461176749,"serverTime":1648461176749,"user":"restapiuser","groups":"","operation":"Login","description":"Login failed for user restapiuser (request originated from IP address: AAA.BBB.CCC.120).","requesterIp":"AAA.BBB.CCC.120","sppserver":"AAA.BBB.CCC.100"} | ||
|
|
||
| ... | ||
|
|
||
| 2022-03-28 16:31:01 [INFO ][LogAction] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> bookmark (queryPage): 1648475895751 | ||
| 2022-03-28 16:31:01 [INFO ][LogAction] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> URL: https://AAA.BBB.CCC.100/api/endeavour/log/audit?sort=%5B%7B%22property%22%3A+%22accessTime%22%2C+%22direction%22%3A+%22ASC%22%7D%5D&filter=%5B%7B%22property%22%3A+%22accessTime%22%2C+%22op%22%3A+%22%3E%3D%22%2C+%22value%22%3A+%221648461176749%22%7D%5D&pageSize=100&pageStartIndex=100 | ||
|
|
||
| ... | ||
|
|
||
| 2022-03-28 16:31:01 [INFO ][UniversalCloudRESTAPITest] Received 1 events from AAA.BBB.CCC.100 | ||
| 2022-03-28 16:31:01 [INFO ][UniversalCloudRESTAPITest] {"accessTime":1648477857858,"serverTime":1648477857858,"user":"monitorUser","groups":"","operation":"Login","description":"Login successful for user monitorUser.","requesterIp":"QRadarInstance1","sppserver":"AAA.BBB.CCC.100"} | ||
| 2022-03-28 16:31:01 [INFO ][LogAction] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> bookmark (queryPage): 1648477857858 | ||
| 2022-03-28 16:31:01 [INFO ][LogAction] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> bookmark (final): 1648477857858 | ||
| 2022-03-28 16:31:01 [INFO ][LogAction] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> collected events: 114 | ||
| ``` | ||
|
|
||
|
|
||
| # sample QRadar log output | ||
|
|
||
| >tail -f /var/log/qradar.log | grep SPP | ||
|
|
||
| ``` | ||
| Mar 28 16:13:19 ::ffff:QRadarInstance1 [ecs-ec-ingress.ecs-ec-ingress] [Thread-7638764] com.q1labs.semsources.sources.universalcloudrestapi.v1.workflow.action.LogAction: [INFO] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> debug: true | ||
| Mar 28 16:13:19 ::ffff:QRadarInstance1 [ecs-ec-ingress.ecs-ec-ingress] [Thread-7638764] com.q1labs.semsources.sources.universalcloudrestapi.v1.workflow.action.LogAction: [INFO] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> bookmark (start): 1648461176749 | ||
| Mar 28 16:13:19 ::ffff:QRadarInstance1 [ecs-ec-ingress.ecs-ec-ingress] [Thread-7638764] com.q1labs.semsources.sources.universalcloudrestapi.v1.workflow.action.LogAction: [INFO] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> counter (start): 0 | ||
| Mar 28 16:13:19 ::ffff:QRadarInstance1 [ecs-ec-ingress.ecs-ec-ingress] [Thread-7638764] com.q1labs.semsources.sources.universalcloudrestapi.v1.workflow.action.LogAction: [INFO] [NOT:0000006000][QRadarInstance1/- -] [-/- -]SPP IQUCRA> URL: https://AAA.BBB.CCC.100:443/api/endeavour/log/audit | ||
| ``` | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| <?xml version="1.0" encoding="UTF-8" ?> | ||
| <WorkflowParameterValues xmlns="http://qradar.ibm.com/UniversalCloudRESTAPI/WorkflowParameterValues/V1"> | ||
| <Value name="host" value="IP_or_HOSTNAME"/> | ||
| <Value name="port" value="443"/> | ||
| <Value name="username" value="monitorUser" /> | ||
| <Value name="userpwd" value="Passw0rd" /> | ||
| <Value name="pageSize" value="100" /> | ||
|
There was a problem hiding this comment.
|
||
| </WorkflowParameterValues> | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| <?xml version="1.0" encoding="UTF-8" ?> | ||
| <!-- IBM Spectrum Protect Plus Audit logs --> | ||
| <Workflow name="SPPauditLogCollector" version="1.0" minimumRecurrence="180" xmlns="http://qradar.ibm.com/UniversalCloudRESTAPI/Workflow/V1"> | ||
|
|
||
| <Parameters> | ||
| <Parameter name="host" label="host" required="true" /> | ||
| <Parameter name="port" label="port" required="true" /> | ||
|
There was a problem hiding this comment. As mentioned above the readme says this parameter is optional but marked required here. Additional comments on line 15 |
||
| <Parameter name="username" label="username" required="true" /> | ||
| <Parameter name="userpwd" label="userpwd" required="true" secret="true" /> | ||
| <Parameter name="pageSize" label="pageSize" required="false" /> | ||
| </Parameters> | ||
|
|
||
| <Actions> | ||
| <Set path="/debug" value="false" /> | ||
| <Set path="/sppaddress" value="https://${/host}:${/port}" /> | ||
|
There was a problem hiding this comment. Have you considered adding logic when constructing |
||
|
|
||
|
|
||
| <!-- get Access Token aka SPP sessionid --> | ||
| <CallEndpoint url="${/sppaddress}/api/endeavour/session" method="POST" savePath="/get_access_token"> | ||
| <BasicAuthentication username="${/username}" password="${/userpwd}" /> | ||
| <RequestHeader name="Accept" value="application/json" /> | ||
| <RequestHeader name="Content-type" value="application/json" /> | ||
| </CallEndpoint> | ||
|
|
||
| <!-- Handle Errors --> | ||
| <If condition="/get_access_token/status_code != 200"> | ||
| <Abort reason="${/get_access_token/body}" /> | ||
| </If> | ||
|
|
||
| <!-- Extract the Access Token and set the initial query URL in nextpage --> | ||
| <Set path="/access_token" value="${/get_access_token/body/sessionid}" /> | ||
| <Set path="/next_page" value="${/sppaddress}/api/endeavour/log/audit" /> | ||
| <Set path="/counter" value="0" /> | ||
|
|
||
| <!-- the first API call differs from subsequent calls by its parameters --> | ||
| <Set path="/firstApiCall" value="true" /> | ||
|
|
||
| <!-- | ||
| initialize the bookmark, will only set the below value if bookmark is not already initialized | ||
| The bookmark value is epoch time in milliseconds. If no value is set, the start time is calculated | ||
| one week back. | ||
| Example: <set path="/bookmark" value="1647868275392" /> | ||
| converts to "Mon, Mar 28, 2022 1:26:38 PM" | ||
| date +%s # converts local time to epoch time in seconds (not MS) | ||
| date -d @1648466798 #c onvert timestamp in seconds (not MS) to local date | ||
| --> | ||
|
|
||
| <If condition="/debug = true"> | ||
| <Set path="/bookmark" value="1648461176749" /> | ||
| <Log type="INFO" message="SPP IQUCRA> debug: true" /> | ||
| </If> | ||
| <Else> | ||
| <Initialize path="/bookmark" value="${time() - 7 * 24 * 60 * 60 * 1000}" /> | ||
| </Else> | ||
|
|
||
| <Log type="INFO" message="SPP IQUCRA> bookmark (start): ${/bookmark}" /> | ||
| <Log type="INFO" message="SPP IQUCRA> counter (start): ${/counter}" /> | ||
|
There was a problem hiding this comment. This one might be more of a candidate for |
||
|
|
||
|
|
||
| <!-- Get List of logs with API token --> | ||
| <DoWhile condition="/next_page != null"> | ||
| <Log type="INFO" message="SPP IQUCRA> URL: ${/next_page}" /> | ||
|
There was a problem hiding this comment. This one might be more of a candidate for |
||
|
|
||
| <!-- | ||
| Due to a feature in the Spectrum Protect Plus REST API code we should add the parameters | ||
| pageSize, filter and search >>>> ONLY <<<< for the FIRST page query, all subsequent | ||
| page queries contain already the search, filter and pageSize parameters | ||
| --> | ||
| <If condition="/firstApiCall = true"> | ||
| <CallEndpoint url="${/next_page}" method="GET" savePath="/response"> | ||
| <QueryParameter name="pageSize" value="${/pageSize}" omitIfEmpty="true" /> | ||
|
There was a problem hiding this comment. pageSize is defined on line 10 as |
||
| <QueryParameter name="sort" value='[{"property": "accessTime", "direction": "ASC"}]' omitIfEmpty="true" /> | ||
| <QueryParameter name="filter" value='[{"property": "accessTime", "op": ">=", "value": "${/bookmark}"}]' omitIfEmpty="true" /> | ||
| <RequestHeader name="Accept" value="application/json" /> | ||
| <RequestHeader name="Content-type" value="application/json" /> | ||
| <RequestHeader name="x-endeavour-sessionid" value="${/access_token}" /> | ||
| </CallEndpoint> | ||
| <Set path="/firstApiCall" value="false" /> | ||
| </If> | ||
| <Else> | ||
| <CallEndpoint url="${/next_page}" method="GET" savePath="/response"> | ||
| <RequestHeader name="Accept" value="application/json" /> | ||
| <RequestHeader name="Content-type" value="application/json" /> | ||
| <RequestHeader name="x-endeavour-sessionid" value="${/access_token}" /> | ||
| </CallEndpoint> | ||
| </Else> | ||
|
|
||
| <!-- Handle Errors --> | ||
| <If condition="/response/status_code != 200"> | ||
| <Abort reason="API endpoint query not successfull, error: ${/response/body}" /> | ||
| </If> | ||
|
|
||
|
|
||
| <!-- here itemList is the array of Logs --> | ||
| <Set path="/itemList" value="${/response/body/logs}" /> | ||
|
|
||
| <ForEach item="/item" items="/itemList"> | ||
| <!-- remote HATEOAS information from responses and leave audit log info only --> | ||
| <Delete path="/item/links" /> | ||
| <Delete path="/item/up" /> | ||
|
|
||
| <!-- add the SPP server IP or Hostname as JSON attribute into response for easy parsing with DSM --> | ||
| <Set path="/item/sppserver" value="${/host}" /> | ||
| <Set path="/itemMessage" value="${/item}" /> | ||
| <PostEvent path="/itemMessage" source="${/host}"/> | ||
| <Set path="/counter" value="${/counter + 1}" /> | ||
| </ForEach> | ||
|
|
||
| <If condition="exists /response/body/links/nextPage/href"> | ||
| <Set path="/next_page" value="${/response/body/links/nextPage/href}" /> | ||
| </If> | ||
| <Else> | ||
| <Delete path="/next_page" /> | ||
| </Else> | ||
|
|
||
| <!-- get timestamp of last item in the itemList (in this loop) --> | ||
| <Set path="/lastItem" value="${count(/itemList) -(1)}" /> | ||
| <Set path="/bookmark" value="${/itemList[/lastItem]/accessTime}" /> | ||
| <Log type="INFO" message="SPP IQUCRA> bookmark (queryPage): ${/bookmark}" /> | ||
|
There was a problem hiding this comment. This one might be more of a candidate for |
||
|
|
||
| </DoWhile> | ||
|
|
||
| <Log type="INFO" message="SPP IQUCRA> bookmark (final): ${/bookmark}" /> | ||
| <Log type="INFO" message="SPP IQUCRA> collected events: ${/counter}" /> | ||
|
|
||
| </Actions> | ||
|
|
||
| <Tests> | ||
| <TCPConnectionTest host="${/host}" /> | ||
|
There was a problem hiding this comment. It would be a good idea to include the DNS and the SSL Handshake tests here as well. |
||
| </Tests> | ||
| </Workflow> | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.