Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(openapi): Adding subtype for openapi source #12873

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

gabe-lyons
Copy link
Contributor

This diff is so large because I had to refactor the openapi source to use MCP instead of MCE. The actual change is very minimal: just adding subtype.

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@@ -8,9 +8,6 @@
from typing import Dict, Iterable, List, Optional, Set, Union

import nest_asyncio
from okta.client import Client as OktaClient
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pylint insisted upon this in order to let me commit

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good with me, as long as ruff is happy with it


from datahub.ingestion.run.pipeline import Pipeline
from datahub.ingestion.source.identity.okta import OktaConfig
from okta.models import Group, User
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pylint insisted upon this in order to let me commit

]
}
},
"entityType": "dataset",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did visual validation that the metadata here is unchanged (save subtypes) and just re-organized.

@gabe-lyons
Copy link
Contributor Author

lint issues are unrelated.

@github-actions github-actions bot added the ingestion PR or Issue related to the ingestion of metadata label Mar 13, 2025
Copy link
Collaborator

@hsheth2 hsheth2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stamping, but I didn't review in depth

@@ -8,9 +8,6 @@
from typing import Dict, Iterable, List, Optional, Set, Union

import nest_asyncio
from okta.client import Client as OktaClient
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good with me, as long as ruff is happy with it

Copy link

codecov bot commented Mar 14, 2025

Codecov Report

Attention: Patch coverage is 78.57143% with 6 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...-ingestion/src/datahub/ingestion/source/openapi.py 77.77% 6 Missing ⚠️

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ingestion PR or Issue related to the ingestion of metadata pending-submitter-merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants