Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] upgrade source monday to cdk v6 #55712

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

dbgold17
Copy link
Contributor

@dbgold17 dbgold17 commented Mar 12, 2025

What

@brianjlai and I paired on this today.

We attempted to start migrating source monday to use v6 of the CDK (from v0 😓). To do so, we successfully migrated the activity logs stream from using a custom incremental cursor (IncrementalSingleSlice) to the CDK supported DatetimeBasedCursor and manually verified that the sync worked. A mock server test for this is forthcoming.

This work however is not yet complete as we noticed this version bump may have broken some other streams (investigation pending) and we'd ideally rip out some of the other custom python classes. We are coordinating with @tolik0 on this as he is also trying to upgrade source Monday with this PR

How

Review guide

User Impact

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

@dbgold17 dbgold17 requested a review from Copilot March 12, 2025 04:45
Copy link

vercel bot commented Mar 12, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 12, 2025 4:58am

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


david.gold seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR upgrades the source_monday integration to use CDK v6 by switching from a custom incremental cursor to the supported DatetimeBasedCursor and making other necessary adjustments.

  • Migrate activity_logs_stream to use DatetimeBasedCursor with appropriate datetime settings.
  • Update initialization and error handling in run.py.
  • Adjust extractor logic to yield records and update dependency versions in pyproject.toml.

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
source_monday/run.py Refactored initialization by extracting source creation and adding error handling.
pyproject.toml Updated Python version and CDK dependency version.
source_monday/source.py Updated constructor signature to accept catalog, config, and state.
source_monday/manifest.yaml Modified incremental sync configuration to use DatetimeBasedCursor and added new parameters.
source_monday/components.py Changed IncrementalSingleSlice to inherit from DeclarativeCursor and added a new select_state method.
source_monday/item_pagination_strategy.py Updated next_page_token parameters and logic for subpage handling.
source_monday/extractor.py Changed extract_records to yield records from a generator instead of returning a list.
Comments suppressed due to low confidence (1)

airbyte-integrations/connectors/source-monday/source_monday/manifest.yaml:121

  • The description URL appears to reference the Intercom API, which may be incorrect for the Monday source integration. Please verify that the documentation link is appropriate.
description: "https://developers.intercom.com/intercom-api-reference/reference/scroll-over-all-companies"


return result
yield from result
Copy link
Preview

Copilot AI Mar 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The extract_records function is annotated to return a List[Record] but yields records instead, which conflicts with its declared return type. Consider either updating the annotation to an appropriate generator type or accumulating results into a list before returning.

Copilot is powered by AI, so mistakes are possible. Review output carefully before use.

Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants