Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce event segment to store multiple events under one key #7836

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

NingLin-P
Copy link
Contributor

Description

See the previous discussion starting from this comment in the thread.

Currently, all the events of the current block are stored in a single list StorageValue<Vec<EventRecord>> which brings problems:

  • To generate a storage proof to prove a specific event (e.g. PoV) we need to generate a storage proof of the whole list
  • When there are too many events, calculating storage root that includes such a big list can be expensive/slow

This PR introduces event segment to split the list into multiple smaller segments, each storing multiple events (but should still be much smaller than the single list) under one key in frame-system.

Integration

When EventSegmentSize is set to zero, event segment is disabled and the event is stored at Events as it is right now.

When EventSegmentSize > 0, the event segment is used to store event, be aware to:

  • Use migrate_from_events_to_event_segments storage migration to clear event from Events during runtime upgrade.
  • To prove a specific event with index i, generate storage proof of an item under key i / EventSegmentSize in EventSegments map.
  • To prove all the events of a specific block, generate storage proof of EventCount and the first (EventCount / EventSegmentSize) + 1 items in the EventSegments map, because other items in the map are from events of the previous block.

Review Notes

The event segment consists of:

  • A config item EventSegmentSize: Get<u32>, it is the max number of events stored in each segment.
    • When it is set to zero event segment is disabled and the event is stored at Events as it is right now.
    • While a larger segment leads toward the same issue as the StorageValue<Vec<EventRecord>>, a smaller segment brings more items to the Trie and may impact performance.
    • From the benchmark 100 maybe a good parameter.
  • A EventSegments: StorageMap<u32, Vec<EventRecord>>, replace Events: StorageValue<Vec<EventRecord>> to store event, starting from index 0 in the map each segment can fit EventSegmentSize events, once the segment is full, the following event will be stored at the next segment in index 1, etc.
  • A UnclearedEventCount: StorageValue<u32>, the total number of uncleared events from the previous blocks.
    • Due to performance impact, the event segment from the previous blocks is not cleared but instead overwritten by event deposited later.
    • Only the first EventCount number of events in the map are deposited by the current block.

@NingLin-P NingLin-P requested a review from a team as a code owner March 6, 2025 20:52
@NingLin-P
Copy link
Contributor Author

cc @bkchr @xlc

Signed-off-by: linning <[email protected]>
Signed-off-by: linning <[email protected]>
Signed-off-by: linning <[email protected]>
@NingLin-P
Copy link
Contributor Author

Kindly ping

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant