Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: sync failure webhooks gain errorType and errorOrigin #55809

Merged
merged 1 commit into from
Mar 18, 2025
Merged

docs: sync failure webhooks gain errorType and errorOrigin #55809

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 34 additions & 22 deletions docs/cloud/managing-airbyte-cloud/manage-airbyte-cloud-notifications.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,16 @@ This page provides guidance on how to manage notifications for Airbyte, allowing

## Notification Event Types

| Type of Notification | Description |
| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Failed Syncs** | A sync from any of your connections fails. Note that if sync runs frequently or if there are many syncs in the workspace these types of events can be noisy |
| **Successful Syncs** | A sync from any of your connections succeeds. Note that if sync runs frequently or if there are many syncs in the workspace these types of events can be noisy |
| **Automated Connection Updates** | A connection is updated automatically (ex. a source schema is automatically updated) |
| **Connection Updates Requiring Action** | A connection update requires you to take action (ex. a breaking schema change is detected) |
| **Warning - Repeated Failures** | A connection will be disabled soon due to repeated failures. It has failed 20 times consecutively and there were only failed jobs in the past 4 days |
| Type of Notification | Description |
| ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Failed Syncs** | A sync from any of your connections fails. Note that if sync runs frequently or if there are many syncs in the workspace these types of events can be noisy |
| **Successful Syncs** | A sync from any of your connections succeeds. Note that if sync runs frequently or if there are many syncs in the workspace these types of events can be noisy |
| **Automated Connection Updates** | A connection is updated automatically (ex. a source schema is automatically updated) |
| **Connection Updates Requiring Action** | A connection update requires you to take action (ex. a breaking schema change is detected) |
| **Warning - Repeated Failures** | A connection will be disabled soon due to repeated failures. It has failed 20 times consecutively and there were only failed jobs in the past 4 days |
| **Sync Disabled - Repeated Failures** | A connection was automatically disabled due to repeated failures. It will be disabled when it has failed 30 times consecutively and has been failing for 7 days in a row |
| **Warning - Upgrade Required** (Cloud only) | A new connector version is available and requires manual upgrade |
| **Sync Disabled - Upgrade Required** (Cloud only) | One or more connections were automatically disabled due to a connector upgrade deadline passing |
| **Warning - Upgrade Required** (Cloud only) | A new connector version is available and requires manual upgrade |
| **Sync Disabled - Upgrade Required** (Cloud only) | One or more connections were automatically disabled due to a connector upgrade deadline passing |

### Enabling schema update notifications

Expand All @@ -44,10 +44,12 @@ All email notifications except for Successful Syncs are enabled by default.
To change the recipient, edit and save the **notification email recipient**. If you would like to send email notifications to more than one recipient, you can enter an email distribution list (ie Google Group) as the recipient.

## Configure Webhook Notification Settings

Airbyte can send notifications to any generic webhook service. This is helpful when using a downstream service to trigger transformations or other tasks in your data stack.

### Example Webhook Notification Payload
Open each section to see an example of the payload returned for the notification type.

Open each section to see an example of the payload returned for the notification type.

:::info
Airbyte passes both the `data` payload along with text blocks that are intended for Slack usage.
Expand Down Expand Up @@ -87,6 +89,8 @@ Airbyte passes both the `data` payload along with text blocks that are intended
"recordsEmitted":89,
"recordsCommitted":45,
"errorMessage":"Something failed",
"errorType": "config_error",
"errorOrigin": "source",
"bytesEmittedFormatted": "1000 B",
"bytesCommittedFormatted":"90 B",
"success":false,
Expand All @@ -96,6 +100,8 @@ Airbyte passes both the `data` payload along with text blocks that are intended
}
```

Note that `errorType` refers to the type of error that occurred, and may indicate the need for a followup action. For example `config_error` indicates a problem with the source or destination configuration (look to `errorOrigin`), while `transient_error` indicates a temporary issue that may resolve itself.

Copy link
Contributor

@ian-at-airbyte ian-at-airbyte Mar 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comment: it looks like these are the only two possible values, but the way the example is written, it feels like other error types might be possible. As a reader I'd feel more confident if we were firm about the possibilities here. "Two error types are possible: config_error and transient_error..." and so forth.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree! I'd much rather link out to another page where we enumerate all the possible failure types... do we have that? I can't find it. There's:

  • config_error
  • system_error
  • transient_error
  • ... and others!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that I can find. ChatGPT wasn't much help which tells me this has probably never been documented publicly. Maybe let's merge this for now and I'll see if Devin can put together a list quickly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the list of failure reasons in the code for reference - https://github.com/airbytehq/airbyte-platform-internal/blob/master/oss/airbyte-config/config-models/src/main/resources/types/FailureReason.yaml#L22-L32

</details>
<details>
<summary>Successful Sync</summary>
Expand Down Expand Up @@ -144,40 +150,46 @@ Airbyte passes both the `data` payload along with text blocks that are intended
<details>
<summary>Automated Connection Updates</summary>

Webhook does not contain payload and only works for Slack notifications
Webhook does not contain payload and only works for Slack notifications

</details>

<details>
<summary>Connection Updates Requiring Action</summary>

Webhook does not contain payload and only works for Slack notifications

Webhook does not contain payload and only works for Slack notifications

</details>

<details>
<summary>Warning - Repeated Failures</summary>

Webhook does not contain payload and only works for Slack notifications

Webhook does not contain payload and only works for Slack notifications

</details>

<details>
<summary>Sync Disabled - Repeated Failures</summary>

Webhook does not contain payload and only works for Slack notifications

Webhook does not contain payload and only works for Slack notifications

</details>
<details>
<summary>Warning - Upgrade Required</summary>

Webhook does not contain payload and only works for Slack notifications

Webhook does not contain payload and only works for Slack notifications

</details>
<details>
<summary>Sync Disabled - Upgrade Required</summary>

Webhook does not contain payload and only works for Slack notifications

Webhook does not contain payload and only works for Slack notifications

</details>

### Configuring Slack Notifications

The webhook notification also integrates easily with Slack.
The webhook notification also integrates easily with Slack.

If you're more of a visual learner, head over to [this video](https://www.youtube.com/watch?v=NjYm8F-KiFc&ab_channel=Airbyte) to learn how to set up a Slack app to receive notifications. You can also refer to the Slack documentation on how to [create an incoming webhook for Slack](https://api.slack.com/messaging/webhooks).

Expand Down
Loading