feat(ourlogs): Adjust 'log' protocol per sdk feedback #4592
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
k-fish
force-pushed
the
feat/ourlogs/adjust-log-item-type
branch
3 times, most recently
from
4dfab55 to
94acbee
Compare
Now that we've mostly finalized the logs protocol with sdks (see develop doc for more info), we want to update Relay to allow the 'log' item type to be sent in this format. Also in this PR is some shimming between the updated protocol / event schema and the kafka consumers. This shimming is temporary until the generic EAP items consumers are ready, at which point we'll have to transform the event-schema Relay accepts into the generic eap item kafka.
k-fish
force-pushed
the
feat/ourlogs/adjust-log-item-type
branch
from
94acbee to
e551796
Compare
k-fish
commented
Mar 20, 2025
tests/integration/test_ourlogs.py
Outdated
Comment on lines
25
to
33
| "timestamp": start.timestamp(), | ||
| "trace_id": "5b8efff798038103d269b633813fc60c", | ||
| "span_id": "eee19b7ec3c1b174", | ||
| "level": "info", | ||
| "body": "Example log record", | ||
| "attributes": { | ||
| "boolean.attribute": {"bool_value": True}, | ||
| "sentry.severity_text": {"string_value": "info"}, | ||
| }, |
There was a problem hiding this comment.
@AbhiPrasad can you double check this looks right?
There was a problem hiding this comment.
Yup looks correct, I pushed up 2f8b6dd to test when we get fields missing.
Dav1dde
reviewed
Mar 20, 2025
Co-authored-by: David Herberth <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Now that we've mostly finalized the logs protocol with sdks (see develop doc for more info), we want to update Relay to allow the 'log' item type to be sent in this format.
SDK's will likely primarily use the
logItemType instead ofotel_logsince we don't need to change timestamp conventions, can send a simplifiedlevel(seeOurLogLeveladded in this PR).Schema changes
We've deprecated some fields in the protocol that only exist for OTEL:
severity_numberandseverity_text, we're coercing these tolevel, but we're keeping the original severity text and number as attributes as OTel allows custom severity text.observed_timestamp_nanosis always set by relay regardless of what is sent because Relay is the 'collector'. We have to leave this as an attribute as well since it's being used by the existing consumer for origin timestamp.timestamp_nanosbecomestimestamp: Timestamptrace_flags, this is unused, and the consumer doesn't even store it in the table. Will decide what to do with this later.Future work
ourlog_merge_otelfunction can be trimmed down since we won't need to fill in deprecated fields to send the same data to the kafka consumer.OurLogprotocol from json received from sdks to a generic EAP "trace items" kafka message that is essentially a couple fields (eg. traceid) + a KVMap forattributes.