Skip to content

Commit a0975f0

Browse files
authored
🚨Source Fauna: Migrate to poetry (#41051)
1 parent c26d2ae commit a0975f0

File tree

10 files changed

+1845
-659
lines changed

10 files changed

+1845
-659
lines changed

airbyte-integrations/connectors/source-fauna/.dockerignore

-6
This file was deleted.

airbyte-integrations/connectors/source-fauna/Dockerfile

-38
This file was deleted.

airbyte-integrations/connectors/source-fauna/README.md

+55-7
Original file line numberDiff line numberDiff line change
@@ -120,22 +120,70 @@ python main.py read --config secrets/config.json --catalog integration_tests/con
120120

121121
### Locally running the connector docker image
122122

123-
#### Build
124123

125-
**Via [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md) (recommended):**
124+
125+
#### Use `airbyte-ci` to build your connector
126+
The Airbyte way of building this connector is to use our `airbyte-ci` tool.
127+
You can follow install instructions [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md#L1).
128+
Then running the following command will build your connector:
126129

127130
```bash
128-
airbyte-ci connectors --name=source-fauna build
131+
airbyte-ci connectors --name source-fauna build
132+
```
133+
Once the command is done, you will find your connector image in your local docker registry: `airbyte/source-fauna:dev`.
134+
135+
##### Customizing our build process
136+
When contributing on our connector you might need to customize the build process to add a system dependency or set an env var.
137+
You can customize our build process by adding a `build_customization.py` module to your connector.
138+
This module should contain a `pre_connector_install` and `post_connector_install` async function that will mutate the base image and the connector container respectively.
139+
It will be imported at runtime by our build process and the functions will be called if they exist.
140+
141+
Here is an example of a `build_customization.py` module:
142+
```python
143+
from __future__ import annotations
144+
145+
from typing import TYPE_CHECKING
146+
147+
if TYPE_CHECKING:
148+
# Feel free to check the dagger documentation for more information on the Container object and its methods.
149+
# https://dagger-io.readthedocs.io/en/sdk-python-v0.6.4/
150+
from dagger import Container
151+
152+
153+
async def pre_connector_install(base_image_container: Container) -> Container:
154+
return await base_image_container.with_env_variable("MY_PRE_BUILD_ENV_VAR", "my_pre_build_env_var_value")
155+
156+
async def post_connector_install(connector_container: Container) -> Container:
157+
return await connector_container.with_env_variable("MY_POST_BUILD_ENV_VAR", "my_post_build_env_var_value")
129158
```
130159

131-
An image will be built with the tag `airbyte/source-fauna:dev`.
160+
#### Build your own connector image
161+
This connector is built using our dynamic built process in `airbyte-ci`.
162+
The base image used to build it is defined within the metadata.yaml file under the `connectorBuildOptions`.
163+
The build logic is defined using [Dagger](https://dagger.io/) [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/pipelines/builds/python_connectors.py).
164+
It does not rely on a Dockerfile.
165+
166+
If you would like to patch our connector and build your own a simple approach would be to:
167+
168+
1. Create your own Dockerfile based on the latest version of the connector image.
169+
```Dockerfile
170+
FROM airbyte/source-fauna:latest
132171

133-
**Via `docker build`:**
172+
COPY . ./airbyte/integration_code
173+
RUN pip install ./airbyte/integration_code
134174

175+
# The entrypoint and default env vars are already set in the base image
176+
# ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
177+
# ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
178+
```
179+
Please use this as an example. This is not optimized.
180+
181+
2. Build your image:
135182
```bash
136183
docker build -t airbyte/source-fauna:dev .
184+
# Running the spec command against your patched connector
185+
docker run airbyte/source-fauna:dev spec
137186
```
138-
139187
#### Run
140188

141189
Then run any of the connector commands as follows:
@@ -178,4 +226,4 @@ You've checked out the repo, implemented a million dollar feature, and you're re
178226
4. Make the connector documentation and its changelog is up to date (`docs/integrations/sources/fauna.md`).
179227
5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
180228
6. Pat yourself on the back for being an awesome contributor.
181-
7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.
229+
7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,29 @@
11
data:
2+
ab_internal:
3+
ql: 100
4+
sl: 100
5+
connectorBuildOptions:
6+
baseImage: docker.io/airbyte/python-connector-base:3.0.2@sha256:73697fbe1c0e2ebb8ed58e2268484bb4bfb2cb56b653808e1680cbc50bafef75
27
connectorSubtype: database
8+
connectorTestSuitesOptions:
9+
- suite: unitTests
10+
- suite: acceptanceTests
11+
testSecrets:
12+
- fileName: config.json
13+
name: SECRET_SOURCE-FAUNA_CREDS
14+
secretStore:
15+
alias: airbyte-connector-testing-secret-store
16+
type: GSM
17+
- fileName: config-deletions.json
18+
name: SECRET_SOURCE-FAUNA_DELETION_CREDS
19+
secretStore:
20+
alias: airbyte-connector-testing-secret-store
21+
type: GSM
322
connectorType: source
423
definitionId: 3825db3e-c94b-42ac-bd53-b5a9507ace2b
5-
dockerImageTag: 0.1.1
24+
dockerImageTag: 0.1.2
625
dockerRepository: airbyte/source-fauna
26+
documentationUrl: https://docs.airbyte.com/integrations/sources/fauna
727
githubIssueLabel: source-fauna
828
icon: fauna.svg
929
license: MIT
@@ -18,26 +38,8 @@ data:
1838
oss:
1939
enabled: true
2040
releaseStage: alpha
21-
documentationUrl: https://docs.airbyte.com/integrations/sources/fauna
41+
supportLevel: community
2242
tags:
2343
- language:python
2444
- cdk:python
25-
ab_internal:
26-
sl: 100
27-
ql: 100
28-
supportLevel: community
29-
connectorTestSuitesOptions:
30-
- suite: unitTests
31-
- suite: acceptanceTests
32-
testSecrets:
33-
- name: SECRET_SOURCE-FAUNA_CREDS
34-
fileName: config.json
35-
secretStore:
36-
type: GSM
37-
alias: airbyte-connector-testing-secret-store
38-
- name: SECRET_SOURCE-FAUNA_DELETION_CREDS
39-
fileName: config-deletions.json
40-
secretStore:
41-
type: GSM
42-
alias: airbyte-connector-testing-secret-store
4345
metadataSpecVersion: "1.0"

0 commit comments

Comments
 (0)