|
2 | 2 |
|
3 | 3 | ## Upgrading to 2.0.0 |
4 | 4 |
|
5 | | -This version removes the Airbyte "raw" tables introduced in version [1.0.0](#upgrading-to-100). As such, |
| 5 | +This version removes the Airbyte "raw" tables introduced in version 1.0.0. As such, |
6 | 6 | any attempt to upgrade an existing connection will fail unless a "truncate refresh" is first executed. It is |
7 | 7 | recommended that you should create a new connection using this upgraded destination and delete the existing |
8 | 8 | connection and generated "raw" tables in the destination after performing a successful sync via the new connection. |
9 | 9 |
|
10 | 10 | In addition to removing the Airbyte "raw" tables, this version also introduces a new insert mode: bulk insert via |
11 | 11 | Azure Blob Storage. This mode is only supported with Microsoft SQL Server 2017 (14.x) and newer and in scenarios where |
12 | 12 | the database instance has access to Azure Cloud. This is a net-new configuration option and may be opted in to at any point after upgrading to |
13 | | -the new connection version. |
14 | | - |
15 | | -## Upgrading to 1.0.0 |
16 | | - |
17 | | -This version removes the option to use "normalization" with Microsoft SQL Server. It also changes |
18 | | -the schema and database of Airbyte's "raw" tables to be compatible with the new |
19 | | -[Destinations V2](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#what-is-destinations-v2) |
20 | | -format. These changes will likely require updates to downstream dbt / SQL models. After this update, |
21 | | -Airbyte will only produce the "raw" v2 tables, which store all content in JSON. These changes remove |
22 | | -the ability to do deduplicated syncs with Microsoft SQL Server. |
23 | | -If you are interested in the Microsoft SQL Server destination gaining the full features |
24 | | -of Destinations V2 (including final tables), click [[https://github.com/airbytehq/airbyte/discussions/37010]] |
25 | | -to register your interest. |
26 | | - |
27 | | -This upgrade will ignore any existing raw tables and will not migrate any data to the new schema. |
28 | | -For each stream, you should perform the following query to migrate the data from the old raw table |
29 | | -to the new raw table: |
30 | | - |
31 | | -```sql |
32 | | --- assumes your schema was 'default' |
33 | | --- replace `{{stream_name}}` with replace your stream name |
34 | | - |
35 | | -CREATE TABLE airbyte_internal.default_raw__stream_{{stream_name}} ( |
36 | | - _airbyte_raw_id VARCHAR(64) PRIMARY KEY, |
37 | | - _airbyte_data NVARCHAR(MAX), |
38 | | - _airbyte_extracted_at DATETIMEOFFSET(7) DEFAULT SYSDATETIMEOFFSET(), |
39 | | - _airbyte_loaded_at DATETIMEOFFSET(7), |
40 | | - _airbyte_meta NVARCHAR(MAX) |
41 | | -); |
42 | | - |
43 | | -INSERT INTO airbyte_internal.default_raw__stream_{{stream_name}} |
44 | | -SELECT |
45 | | - _airbyte_ab_id AS _airbyte_raw_id, |
46 | | - _airbyte_data as _airbyte_data, |
47 | | - _airbyte_emitted_at as _airbyte_extracted_at, |
48 | | - NULL as _airbyte_loaded_at, |
49 | | - NULL as _airbyte_meta |
50 | | -FROM airbyte._airbyte_raw_{{stream_name}} |
51 | | -``` |
52 | | - |
53 | | -**Airbyte will not delete any of your v1 data.** |
54 | | - |
55 | | -### Schema and the Internal Schema |
56 | | - |
57 | | -We have split the raw and final tables into their own schemas. For the Microsoft SQL Server destination, this means that |
58 | | -we will only write into the raw table which will live in the `airbyte_internal` schema. |
59 | | -The tables written into this schema will be prefixed with either the default database provided in |
60 | | -the `DB Name` field when configuring Microsoft SQL Server (but can also be overridden in the connection). You can |
61 | | -change the "raw" database from the default `airbyte_internal` by supplying a value for |
62 | | -`Raw Table Schema Name`. |
63 | | - |
64 | | -For Example: |
65 | | - |
66 | | -- Schema: `default` |
67 | | -- Stream Name: `my_stream` |
68 | | - |
69 | | -Writes to `airbyte_intneral.default_raw__stream_my_stream` |
70 | | - |
71 | | -Where as: |
72 | | - |
73 | | -- Schema: `default` |
74 | | -- Stream Name: `my_stream` |
75 | | -- Raw Table Schema Name: `raw_data` |
76 | | - |
77 | | -Writes to `raw_data.default_raw__stream_my_stream` |
| 13 | +the new connection version. |
0 commit comments