Elasticsearch self-hosted provider. #4080
-
|
Hello, Is it possible to connect Keep to Elasticsearch self-hoster service? |
Beta Was this translation helpful? Give feedback.
Replies: 14 comments 40 replies
-
|
Привіт @OlegVPA-UA! 🤖 Yes, it is possible to connect Keep to a self-hosted Elasticsearch service using either an API key or a username/password. Ensure that your provider configuration includes the correct To continue talking to Dosu, mention @dosu. Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other |
Beta Was this translation helpful? Give feedback.
-
|
It's accessible, I have testes connectivity from host when Keep deployed. The curl command return me successful request. |
Beta Was this translation helpful? Give feedback.
-
|
hey @OlegVPA-UA - thanks for opening this! as far as I know elasticsearch self hosted is supported. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, I see just this in logs: 2025-03-17T17:49:34: Action elastic-step evaluated to run! Reason: no condition, hence true. 2025-03-17T17:49:34: Checking throttling for action elastic-step 2025-03-17T17:49:34: Action elastic-step is not throttled 2025-03-17T17:49:34: Running elastic-step StepType.STEP, current retry: 0 2025-03-17T17:49:34: HEAD https://10.20.16.180:9200/ [status:N/A duration:0.007s] 2025-03-17T17:49:34: Node <Urllib3HttpNode(https://10.20.16.180:9200)> has failed for 1 times in a row, putting on 1 second timeout 2025-03-17T17:49:34: Retrying request after failure (attempt 0 of 3) 2025-03-17T17:49:34: HEAD https://10.20.16.180:9200/ [status:N/A duration:0.008s] 2025-03-17T17:49:34: Node <Urllib3HttpNode(https://10.20.16.180:9200)> has failed for 2 times in a row, putting on 2 second timeout 2025-03-17T17:49:34: Retrying request after failure (attempt 1 of 3) 2025-03-17T17:49:34: HEAD https://10.20.16.180:9200/ [status:N/A duration:0.007s] 2025-03-17T17:49:34: Node <Urllib3HttpNode(https://10.20.16.180:9200)> has failed for 3 times in a row, putting on 4 second timeout 2025-03-17T17:49:34: Retrying request after failure (attempt 2 of 3) 2025-03-17T17:49:34: HEAD https://10.20.16.180:9200/ [status:N/A duration:0.007s] 2025-03-17T17:49:34: Node <Urllib3HttpNode(https://10.20.16.180:9200)> has failed for 4 times in a row, putting on 8 second timeout 2025-03-17T17:49:34: Failed to run step elastic-step with error Could not connect to ElasticSearch 2025-03-17T17:49:34: Error running workflow 86d30706-6ead-4319-863e-6fca162d6d8f 2025-03-17T17:49:34: Failed to run workflow 86d30706-6ead-4319-863e-6fca162d6d8f...` |
Beta Was this translation helpful? Give feedback.
-
|
Also, if I try to use just username/password I got an error: Connection Problem |
Beta Was this translation helpful? Give feedback.
-
That's my bad. @shahargl is pushing a fix for that now |
Beta Was this translation helpful? Give feedback.
-
|
I think I found root cause ( {"asctime": "2025-03-17 17:58:38,817", "message": "Retrying request after failure (attempt 0 of 3)", "levelname": "WARNING", "name": "elastic_transport.transport", "filename": "_transport.py", "otelTraceID": "40510aa4c86731eeb0f65ed3ec935a99", "otelSpanID": "74908091a6a66485", "otelTraceSampled": true, "otelServiceName": "keep-api", "threadName": "WorkflowScheduler_1", "process": 22, "module": "_transport", "exc_info": "Traceback (most recent call last):\n File "/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 466, in _make_request\n self._validate_conn(conn)\n File "/venv/lib/python3.11/site-packages/elastic_transport/_node/_urllib3_chain_certs.py", line 96, in _validate_conn\n super(HTTPSConnectionPool, self)._validate_conn(conn)\n File "/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1095, in _validate_conn\n conn.connect()\n File "/venv/lib/python3.11/site-packages/elastic_transport/_node/_urllib3_chain_certs.py", line 45, in connect\n super().connect()\n File "/venv/lib/python3.11/site-packages/urllib3/connection.py", line 730, in connect\n sock_and_verified = _ssl_wrap_socket_and_match_hostname(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/urllib3/connection.py", line 909, in ssl_wrap_socket_and_match_hostname\n ssl_sock = ssl_wrap_socket(\n ^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/urllib3/util/ssl.py", line 469, in ssl_wrap_socket\n ssl_sock = ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/urllib3/util/ssl.py", line 513, in _ssl_wrap_socket_impl\n return ssl_context.wrap_socket(sock, server_hostname=server_hostname)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/ssl.py", line 517, in wrap_socket\n return self.sslsocket_class._create(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/ssl.py", line 1104, in _create\n self.do_handshake()\n File "/usr/local/lib/python3.11/ssl.py", line 1382, in do_handshake\n self._sslobj.do_handshake()\nssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1006)\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/venv/lib/python3.11/site-packages/elastic_transport/_node/_http_urllib3.py", line 167, in perform_request\n response = self.pool.urlopen(\n ^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 843, in urlopen\n retries = retries.increment(\n ^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/urllib3/util/retry.py", line 449, in increment\n raise reraise(type(error), error, _stacktrace)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/urllib3/util/util.py", line 39, in reraise\n raise value\n File "/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 789, in urlopen\n response = self._make_request(\n ^^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 490, in _make_request\n raise new_e\nurllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1006)\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/venv/lib/python3.11/site-packages/elastic_transport/_transport.py", line 342, in perform_request\n resp = node.perform_request(\n ^^^^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/elastic_transport/_node/_http_urllib3.py", line 202, in perform_request\n raise err from e\nelastic_transport.TlsError: TLS error caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1006))", "extra": {}, "workflow_id": "86d30706-6ead-4319-863e-6fca162d6d8f", "workflow_execution_id": "cb680339-0764-4501-8a2d-f9a4ce9f152d", "tenant_id": "keep", "context": {"step_id": "elastic-step"}} |
Beta Was this translation helpful? Give feedback.
-
|
@OlegVPA-UA let me add it. will release a version in few minutes. |
Beta Was this translation helpful? Give feedback.
-
|
@OlegVPA-UA relased on |
Beta Was this translation helpful? Give feedback.
-
|
It's works... A lot of thanks!!!! But have another issue with elasticsearch ( it should be: I can't save this ( |
Beta Was this translation helpful? Give feedback.
-
|
I see this error: {"asctime": "2025-03-17 20:08:33,559", "message": "Failed to run workflow 86d30706-6ead-4319-863e-6fca162d6d8f...", "levelname": "ERROR", "name": "keep.workflowmanager.workflowscheduler", "filename": "workflowscheduler.py", "otelTraceID": "0", "otelSpanID": "0", "otelTraceSampled": false, "otelServiceName": "keep-api", "threadName": "WorkflowScheduler_1", "process": 19, "module": "workflowscheduler", "exc_info": "Traceback (most recent call last):\n File "/venv/lib/python3.11/site-packages/keep/step/step.py", line 341, in _run_single\n step_output = self.provider.query(\n ^^^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/keep/providers/base/base_provider.py", line 327, in query\n results = self._query(**kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/keep/providers/elastic_provider/elastic_provider.py", line 193, in _query\n return self._run_eql_query(query, index)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/keep/providers/elastic_provider/elastic_provider.py", line 221, in _run_eql_query\n query = json.loads(query)\n ^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/json/init.py", line 346, in loads\n return _default_decoder.decode(s)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode\n obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.11/json/decoder.py", line 353, in raw_decode\n obj, end = self.scan_once(s, idx)\n ^^^^^^^^^^^^^^^^^^^^^^\njson.decoder.JSONDecodeError: Expecting ',' delimiter: line 7 column 1 (char 67)\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/venv/lib/python3.11/site-packages/keep/step/step.py", line 355, in _run_single\n raise StepError(e)\nkeep.step.step.StepError: Expecting ',' delimiter: line 7 column 1 (char 67)\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/venv/lib/python3.11/site-packages/keep/step/step.py", line 74, in run\n did_action_run = self._run_single()\n ^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/keep/step/step.py", line 374, in _run_single\n raise StepError(e)\nkeep.step.step.StepError: Expecting ',' delimiter: line 7 column 1 (char 67)\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/venv/lib/python3.11/site-packages/keep/workflowmanager/workflowscheduler.py", line 221, in _run_workflow\n errors, _ = self.workflow_manager._run_workflow(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/keep/workflowmanager/workflowscheduler.py", line 54, in wrapper\n result = func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/keep/workflowmanager/workflowmanager.py", line 446, in _run_workflow\n errors = workflow.run(workflow_execution_id)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/keep/workflowmanager/workflow.py", line 163, in run\n self.run_steps()\n File "/venv/lib/python3.11/site-packages/keep/workflowmanager/workflow.py", line 70, in run_steps\n step_ran = step.run()\n ^^^^^^^^^^\n File "/venv/lib/python3.11/site-packages/keep/step/step.py", line 86, in run\n raise ActionError(e)\nkeep.exceptions.action_error.ActionError: Expecting ',' delimiter: line 7 column 1 (char 67)", "workflow_id": "86d30706-6ead-4319-863e-6fca162d6d8f", "workflow_execution_id": "f2fcfdde-cb48-4a88-8e54-5962960eb9ca", "tenant_id": "keep", "extra": {}, "context": {"step_id": "elastic-step"}} |
Beta Was this translation helpful? Give feedback.
-
|
Let me check! |
Beta Was this translation helpful? Give feedback.
-
|
@OlegVPA-UA I see some problem with saving the workflow from the UI. I'll dig into it. Meanwhile, you can edit the workflow on your favorite IDE and upload it from a file |
Beta Was this translation helpful? Give feedback.
-
|
Yes... I already did this... Also, I see than keep try to get with _sql from ElasticSearch. So, my request doesn't works:
|
Beta Was this translation helpful? Give feedback.
-
|
@OlegVPA-UA looks smth with enoding. Will be fixed tomorrow. |
Beta Was this translation helpful? Give feedback.
-
|
Opened this for tracking - #4087 Its super late here so I'll wrap it up tomorrow :) |
Beta Was this translation helpful? Give feedback.
-
|
Hello, |
Beta Was this translation helpful? Give feedback.
-
@OlegVPA-UA can you paste the workflow yaml you are using so I can further check? |
Beta Was this translation helpful? Give feedback.
-
|
Sure.
|
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
A lot of thanks!!!! Everything now works ) |
Beta Was this translation helpful? Give feedback.
-
|
Ok. Now I need to parse everything what I got from elasticsearch and create alert on every message from elastic. Maybe you have some examples for that, how to do it in Keep? I understood need to do this using actions. |
Beta Was this translation helpful? Give feedback.
-
|
I have tested... Thanks it works. add this 2 attribute |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
A lot of thank! Next question, can I use this attributes in action for send messages to telegram/slack or I need to create new workflow for this? |
Beta Was this translation helpful? Give feedback.
-
|
Also, extraction is not works if message is:
|
Beta Was this translation helpful? Give feedback.
-
|
Never mind... I found solution after investigate python2 regex ) |
Beta Was this translation helpful? Give feedback.
@OlegVPA-UA relased on
0.39.8. images will be ready in ~5 minutes.