Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚨Source Fauna: Migrate to poetry #41051

Merged
merged 33 commits into from
Mar 14, 2025
Merged

Conversation

btkcodedev
Copy link
Collaborator

What

Migrating to poetry

Can this PR be safely reverted and rolled back?

  • YES 💚

@btkcodedev btkcodedev self-assigned this Jul 8, 2024
Copy link

vercel bot commented Jul 8, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
airbyte-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 13, 2025 9:43pm

@octavia-squidington-iii octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/fauna labels Jul 8, 2024
@natikgadzhi
Copy link
Contributor

Build fails because of AirbyteLogger — we need to pin this to the current verison of CDK it uses

Copy link

coderabbitai bot commented Jul 10, 2024

Important

Review skipped

Auto reviews are limited to specific labels.

Labels to auto review (1)
  • coderabbit-ai-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@btkcodedev
Copy link
Collaborator Author

btkcodedev commented Jul 15, 2024

Hi @ChristoGrab, Need a little help here, I've pinned the CDK with 0.1 which was extremely old, 0.80.0, 0.60.0, and even ^3
Nothing is working, Could you please comment on what I could test

@ChristoGrab
Copy link
Contributor

@btkcodedev Thanks for trying to push this one through, I know trying to find the right version can be a pain. Left a couple suggestions to try setting the CDK to "0.60.0" again and bumping the version of the "faunadb" dependency.

The unit test failure seems to be present in the existing connector, so let's not worry about that failure for this migration. It looks like the acceptance test failures are ones we can ignore as well, a lot of noise about more modern CDK expectations. As long as the PyAirbyte failure is resolved when we update the dependencies this time around I'm inclined to merge since the connector seems to at least function on a basic level in the tests.

@natikgadzhi
Copy link
Contributor

Devin, delete the Dockerfile plesae

ql: 100
sl: 100
connectorBuildOptions:
baseImage: docker.io/airbyte/python-connector-base:2.0.0@sha256:c44839ba84406116e8ba68722a0f30e8f6e7056c726f447681bb9e9ece8bd916
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to use python-connector-base:3.0.2 here. Look up the SHA in other connectors, Devin.

@natikgadzhi
Copy link
Contributor

natikgadzhi commented Mar 1, 2025

/format-fix

Format-fix job started... Check job output.

🟦 Job completed successfully (no changes).

@btkcodedev
Copy link
Collaborator Author

Really.. Devin.. I know this day would come😭

@natikgadzhi
Copy link
Contributor

Devin, bruh, remove the repeating remoteRegistries key in metadata.

@btkcodedev yeah, the thing struggles.

@natikgadzhi
Copy link
Contributor

@btkcodedev can you take care of the unit tests here? I'd like to get it over with but this is low priority

@btkcodedev
Copy link
Collaborator Author

The conversion from Dockerfile to CDK is a side effect for the database_test file since it couldn't locate the Docker version. I'll remove it now that it's handled in Poetry.

@btkcodedev
Copy link
Collaborator Author

Yep done. Unit tests cleared.

@btkcodedev btkcodedev requested a review from natikgadzhi March 13, 2025 22:06
@natikgadzhi natikgadzhi merged commit a0975f0 into master Mar 14, 2025
27 of 29 checks passed
@natikgadzhi natikgadzhi deleted the btkcodedev/faunaMigrtepoetry branch March 14, 2025 02:22
sc250072 added a commit to Teradata/airbyte that referenced this pull request Mar 14, 2025
* ✨ Source Intercom: adding a mock server test (airbytehq#54715)

* [Destination MS SQL V2] Correct Part Size, No buffered input stream (airbytehq#55252)

* [source-hibob] Changed check stream from payrolls to profiles airbytehq#55674 (airbytehq#55675)

* Add llms.txt to docs.airbyte.com (airbytehq#55261)

* ✨ TikTok Marketing Source: Add `Pixels`, `PixelInstantPageEvents`, `PixelEventsStatistics` streams (airbytehq#55669)

Co-authored-by: Octavia Squidington III <[email protected]>

* chore(ci): remove stale and unused workflows (airbytehq#55260)

* ci: for community CI, rename 'early ci' to 'pre-release checks', skip duplicated tests in 'connector tests' (airbytehq#55241)

* (feat: Salesloft) - Add emails_scoped_fields stream (airbytehq#55229)

* 🐛 Source Outreach: remove stream_state interpolation (airbytehq#55180)

Co-authored-by: Natik Gadzhi <[email protected]>
Co-authored-by: Octavia Squidington III <[email protected]>

* fix(source-stripe): disable progressive rollout (airbytehq#55682)

Signed-off-by: Artem Inzhyyants <[email protected]>

* fix(source-instagram): Disable cache for InstagramMediaChildrenTransformation (airbytehq#55685)

* [Destination MSSQL V2] Bulk Load Local Performance Test (airbytehq#55687)

* [Destination-S3] File Xfer Local Performance Test (airbytehq#55220)

* 🐛bug(source-hubspot): fix deals_archived and marketing_emails issues for CAT (airbytehq#54177)

* ✨Source Quickbooks: Migrate to manifest-only (airbytehq#55263)

* ✨ Source Zendesk Chat : Migrate to Manifest-only (airbytehq#47319)

* ✨ Source Facebook Marketing: Add `learning_stage_info` field to AdSets stream (airbytehq#50418)

Co-authored-by: Marcos Marx <[email protected]>
Co-authored-by: marcosmarxm <[email protected]>

* Source Sendgrid: Update manifest for adapting changes with AsyncRetriever (airbytehq#55185)

Co-authored-by: Octavia Squidington III <[email protected]>

* docs: fix broken markup in Python CDK Basic Concepts page (airbytehq#55699)

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: [email protected] <[email protected]>

* Source Zendesk Talk : Restore Unit Test (airbytehq#50956)

Co-authored-by: Octavia Squidington III <[email protected]>
Co-authored-by: Natik Gadzhi <[email protected]>

* chore: 2.0.0 release (airbytehq#55684)

* Adding EnrichedAirbyteValue and DeclaredField (airbytehq#55218)

Co-authored-by: Octavia Squidington III <[email protected]>

* Add Airbyte Academy section (airbytehq#49964)

* [source-faker] - Bump to stable 6.2.21 (airbytehq#55705)

* fix: remove duplicate breaking changes from destination-mssql metadata (airbytehq#55718)

* fix: restore definition ID (airbytehq#55720)

* ✨ Source Freshdesk : Migrate to Manifest-only (airbytehq#54687)

Co-authored-by: Octavia Squidington III <[email protected]>

* chore(source-s3): update base image to 4.0.0 and use caret dependencies (do not merge) (airbytehq#55202)

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: [email protected] <[email protected]>
Co-authored-by: Octavia Squidington III <[email protected]>
Co-authored-by: Aldo Gonzalez <[email protected]>

* fix(Source-LinkedIn-Ads): Update outdated schema (airbytehq#55724)

* Bump @babel/runtime-corejs3 from 7.23.6 to 7.26.10 in /docusaurus (airbytehq#55708)

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* 🐛 Source S3: Up CDK to fix schema type issue (airbytehq#55694)

* source-shipstation contribution from suhl79 (airbytehq#55738)

Co-authored-by: Octavia Squidington III <[email protected]>
Co-authored-by: Marcos Marx <[email protected]>

* destination-teradata: Upgrade JDBC driver (airbytehq#55183)

Co-authored-by: Marcos Marx <[email protected]>

* 🐛 Destination Databricks: Fix destination check test table collisions when multiple connections write to same schema. (airbytehq#55232)

Co-authored-by: Octavia Squidington III <[email protected]>

* (source-sendgrid) - Configure max concurrent async job count (airbytehq#55744)

* pass streams to debezium sources on cold start (airbytehq#55734)

* Destination S3 Data Lake: exclude invalid fields from identifier fields (airbytehq#55700)

* [source-mysql] pin to cdk 0.342 (airbytehq#55754)

* docs: add enterprise connector documentation (airbytehq#55751)

* Add UTM source (airbytehq#55733)

* Matteogp/docs sap hana update 1 (airbytehq#55696)

* Destination S3 Data Lake: Handle number in primary key (airbytehq#55755)

* Fix reversed assertions in MySQL source tests (airbytehq#55756)

* Update CDK to pass DestinationRecordRaw around (airbytehq#55737)

Co-authored-by: Octavia Squidington III <[email protected]>
Co-authored-by: Edward Gao <[email protected]>

* fix(ci): remove empty notify-on-push workflow file (airbytehq#55757)

* Update OTEL metrics, add new linting exceptions (airbytehq#55752)

* 11526 second pass through iceberg documentation (airbytehq#55736)

* [source-mysql] don't do sampling for source-mysql (airbytehq#55761)

Co-authored-by: Octavia Squidington III <[email protected]>

* 🚨Source Fauna: Migrate to poetry (airbytehq#41051)

---------

Signed-off-by: Artem Inzhyyants <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Maxime Carbonneau-Leclerc <[email protected]>
Co-authored-by: Johnny Schmidt <[email protected]>
Co-authored-by: tautvydas-v <[email protected]>
Co-authored-by: Ian Alton <[email protected]>
Co-authored-by: Tope Folorunso <[email protected]>
Co-authored-by: Octavia Squidington III <[email protected]>
Co-authored-by: Natik Gadzhi <[email protected]>
Co-authored-by: Aaron ("AJ") Steers <[email protected]>
Co-authored-by: Tyler B <[email protected]>
Co-authored-by: kyleromines <[email protected]>
Co-authored-by: Artem Inzhyyants <[email protected]>
Co-authored-by: Anatolii Yatsuk <[email protected]>
Co-authored-by: Aldo Gonzalez <[email protected]>
Co-authored-by: Dhroov Makwana <[email protected]>
Co-authored-by: jake horban <[email protected]>
Co-authored-by: Marcos Marx <[email protected]>
Co-authored-by: marcosmarxm <[email protected]>
Co-authored-by: btkcodedev <[email protected]>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: [email protected] <[email protected]>
Co-authored-by: Jonathan Pearlin <[email protected]>
Co-authored-by: Francis Genet <[email protected]>
Co-authored-by: Patrick Nilan <[email protected]>
Co-authored-by: Aldo Gonzalez <[email protected]>
Co-authored-by: Alfredo Garcia <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: suhl79 <[email protected]>
Co-authored-by: Satish Chinthanippu <[email protected]>
Co-authored-by: Sena Heydari <[email protected]>
Co-authored-by: Matt Bayley <[email protected]>
Co-authored-by: Edward Gao <[email protected]>
Co-authored-by: Matteo Palarchio <[email protected]>
Co-authored-by: Wenqi Hu <[email protected]>
Co-authored-by: Yue Li <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/fauna
Projects
Status: ✔️ Done
Development

Successfully merging this pull request may close these issues.

4 participants