RADAR-base · pvannierop · Mar 9, 2025 · Mar 9, 2025 · Mar 10, 2025 · Mar 10, 2025
diff --git a/docs/upgrade.md b/docs/upgrade.md
@@ -2,9 +2,126 @@
 
 Run the following instructions to upgrade an existing RADAR-Kubernetes cluster.
 
-| :exclamation: Note                                                                                                                                                                                                                                |
-| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Upgrading the major version of a PostgreSQL image is not supported. If necessary, we propose to use a `pg_dump` to dump the current data and a `pg_restore` to restore that data on a newer version. Please find instructions for this elsewhere. |
+## Upgrade to RADAR-Kubernetes version 1.3.0
+
+This version introduces postgresql and timescaledb clusters managed by the ClounNativePG operator. The upgrade process
+involves:
+
+1. (Optional) Manual merge of existing _app-server_ and _upload-connect-backend_ databases into the _management_portal_
+   postgres database.
+   cluster.
+2. Automated import of the _management_portal_ database into the new CloudNativePG postgres cluster.
+3. (Optional) Automated import of the timescaledb databases (connected to the JDBC connector services) into the new
+   CloudNativePG timescaledb cluster.
+
+For migration perform the following steps:
+
+1. (Optional) For the _app-server_ and _upload-connect-backend_ databases perform a dump of the databases to the local machine,
+   create the new database in _management-portal_database and import the dump. For example, to dump the _app-server_
+   database run:
+
+### App-server database migration
+
+```shell
+kubectl exec radar-appserver-postgresql-0 -- bash -c "PGPASSWORD=<appserver database password> pg_dump -U postgres appserver" > appserver.sql
+kubectl exec -i postgresql-0 -- bash -c "PGPASSWORD=<management-portal database password> psql -U postgres -d appserver -t -c 'CREATE DATABASE appserver'" 
+cat appserver.sql | kubectl exec -i postgresql-0 -- bash -c "PGPASSWORD=<management-portal database password> psql -U postgres -d appserver" 
+````
+
+### Upload-connect-backend database migration
+
+```shell
+kubectl exec radar-upload-postgresql-0 -- bash -c "PGPASSWORD=<upload-connect-backend database password> pg_dump -U postgres upload_connect_backend" > uploadconnector.sql
+kubectl exec -i postgresql-0 -- bash -c "PGPASSWORD=<management-portal database password> psql -U postgres -d upload_connect_backend -t -c 'CREATE DATABASE uploadconnector'"
+cat uploadconnector.sql | kubectl exec -i postgresql-0 -- bash -c "PGPASSWORD=<management-portal database password> psql -U postgres -d uploadconnector"
+```
+
+### Kratos database migration
+
+```shell
+kubectl exec radar-kratos-postgresql-0 -- bash -c "PGPASSWORD=<kratos database password> pg_dump -U postgres kratos" > kratos.sql
+kubectl exec -i postgresql-0 -- bash -c "PGPASSWORD=<management-portal database password> psql -U postgres -d kratos -t -c 'CREATE DATABASE kratos'"
+cat kratos.sql | kubectl exec -i postgresql-0 -- bash -c "PGPASSWORD=<management-portal database password> psql -U postgres -d kratos"
+```
+
+Note: database passwords can be found in the `secrets.yaml` file.
+
+2. Activate import of the _management-portal_ database in the `production.yaml` file. Make sure include all the
+   databases that need to be imported. For example:
+
+```yaml
+cloudnative_postgresql:
+  _install: true
+  ...
+  cluster:
+    mode: recovery
+    recovery:
+      import:
+        databases:
+          - managementportal
+          - restsourceauthorizer
+          - appserver
+          - uploadconnector
+          - ...
+```
+
+Note: make sure to include all the databases that need to be imported, but to not include database nor present in the
+_management_portal_ database.
+
+3. Trigger the import of the _management_portal_ database enabling the _cloudnative-pg_ operator and _cn-postgresql_
+   services in `production.yaml`:
+
+```yaml
+cloudnative_pg:
+  _install: true
+  ...
+
+...
+
+cloudnative_postgresql:
+  _install: true
+  ...
+```
+
+And run:
+
+```shell
+helmfile sync
+```
+
+4. (Optional) Activate the import of any deployed timescaledb database in the `production.yaml` file. This can be achieved by
+   adding the import setting to the respective JDBC connector definitions. For instance, to migrate the
+   _data-dashboard-backend_ timescaledb, add the following to the `radar_jdbc_connector_data_dashboard_backend` section:
+
+```yaml
+radar_jdbc_connector_data_dashboard_backend:
+  _install: true
+  ...
+  radar-cloudnative-timescaledb:
+    cluster:
+      mode: recovery
+      import:
+        databases:
+          - data-dashboard
+        source:
+          host: data-dashboard-timescaledb-postgresql
+          username: postgres
+          database: postgres
+          sslMode: prefer
+          passwordSecret:
+            create: false
+            name: data-dashboard-timescaledb-postgresql
+            key: postgres-password
+```
+
+Equivalent changes should be made for the _realtime-dashboard_ and _grafana-metrics_ timescaledb databases. Make sure to
+update the example above with the respective values for `datsbases:`, `host:` and `passwordSecret.name:` fields.
+
+5. Redeploy the _radar-jdbc-connector_ services to trigger the import of the timescaledb databases:
+
+```shell
+helmfile sync
+```
 
 ## Upgrade to RADAR-Kubernetes version 1.2.0
 
@@ -16,31 +133,32 @@ Run the following instructions to upgrade an existing RADAR-Kubernetes cluster.
 
 ```yaml
     management_portal:
-        smtp:
-            enabled: true
-            host: smtp
-            port: 25
-            from: [email protected]
-            starttls: false
-            auth: true
+      smtp:
+        enabled: true
+        host: smtp
+        port: 25
+        from: [email protected]
+        starttls: false
+        auth: true
 ```
 
 ```yaml
     radar_appserver:
-        smtp:
-            enabled: true
-            host: smtp
-            port: 25
-            from: [email protected]
-            starttls: false
-            auth: true
+      smtp:
+        enabled: true
+        host: smtp
+        port: 25
+        from: [email protected]
+        starttls: false
+        auth: true
 ```
 
 4. Update _timescaledb_ database configuration:
 
 - Rename `timescaledb_username` to `grafana_metrics_db_username`
 - Remove `grafana_metrics_username` and `timescaledb_db_name` variables.
-- When using _realtime-dashboard_, add `realtime_dashboard_db_username` that points to the current value of `timescaledb_username`. 
+- When using _realtime-dashboard_, add `realtime_dashboard_db_username` that points to the current value of
+  `timescaledb_username`.
 
 5. For databases where data should persist after the update uncomment the respective `existingClaim` field. Example:
 
@@ -75,22 +193,36 @@ data_dashboard_db_password: <same password as timescaledb_password>
 realtime_dashboard_db_password: <same password as timescaledb_password>
 ```
 
-2. Rename the `grafana_metrics_password` secret to `grafana_metrics_db_password` and `timescaledb_password` to `data_dashboard_db_password`.
-
+2. Rename the `grafana_metrics_password` secret to `grafana_metrics_db_password` and `timescaledb_password` to
+   `data_dashboard_db_password`.
 
 ### MongoDB
-MongoDB has been updated to a new version and it's not compatible with the current version that has been installed in the cluster. There are two pathes forward:
-- Deleting the MongoDB and its volumes and then installing it and configuring Graylog again. This is the recommeneded approach since usually there is no important data is stored in MongoDB and the Graylog stack will be replaced in the next release.
-- Upgrading MongoDB cluster. If you have configured Graylog significantly, then it might be better to upgrade the MongoDB instead of reinstalling it. Options are:
-  - Manually exporting the databases and doing a clean reinstall as stated in the last step and the importing the data again.
-  - Following official MongoDB upgrade instructions for version [5.0](https://www.mongodb.com/docs/manual/release-notes/5.0-upgrade-replica-set/), [6.0](https://www.mongodb.com/docs/manual/release-notes/6.0-upgrade-replica-set/) and [7.0](https://www.mongodb.com/docs/manual/release-notes/7.0-upgrade-replica-set/).
+
+MongoDB has been updated to a new version and it's not compatible with the current version that has been installed in
+the cluster. There are two pathes forward:
+
+- Deleting the MongoDB and its volumes and then installing it and configuring Graylog again. This is the recommeneded
+  approach since usually there is no important data is stored in MongoDB and the Graylog stack will be replaced in the
+  next release.
+- Upgrading MongoDB cluster. If you have configured Graylog significantly, then it might be better to upgrade the
+  MongoDB instead of reinstalling it. Options are:
+    - Manually exporting the databases and doing a clean reinstall as stated in the last step and the importing the data
+      again.
+    - Following official MongoDB upgrade instructions for
+      version [5.0](https://www.mongodb.com/docs/manual/release-notes/5.0-upgrade-replica-set/), [6.0](https://www.mongodb.com/docs/manual/release-notes/6.0-upgrade-replica-set/)
+      and [7.0](https://www.mongodb.com/docs/manual/release-notes/7.0-upgrade-replica-set/).
 
 ## Upgrade to RADAR-Kubernetes version 1.1.x
-Before running the upgrade make sure to copy `environments.yaml.tmpl` to `environments.yaml` and if you've previously changed `environments.yaml` apply the changes again. This is necessary due to addition of `helmDefaults` and `repositories` configurations to this file.
+
+Before running the upgrade make sure to copy `environments.yaml.tmpl` to `environments.yaml` and if you've previously
+changed `environments.yaml` apply the changes again. This is necessary due to addition of `helmDefaults` and
+`repositories` configurations to this file.
 
 ## Upgrade to RADAR-Kubernetes version 1.0.0
 
-Before running the upgrade, compare `etc/base.yaml` and `etc/base.yaml.gotmpl` with their `production.yaml` counterparts. Please ensure that all properties in `etc/base.yaml` are overridden in your `production.yaml` or that the `base.yaml` default value is fine, in which case no value needs to be provided in `production.yaml`.
+Before running the upgrade, compare `etc/base.yaml` and `etc/base.yaml.gotmpl` with their `production.yaml`
+counterparts. Please ensure that all properties in `etc/base.yaml` are overridden in your `production.yaml` or that the
+`base.yaml` default value is fine, in which case no value needs to be provided in `production.yaml`.
 
 To upgrade the initial services, run
 
@@ -100,7 +232,9 @@ helm -n graylog uninstall mongodb
 kubectl delete -n graylog pvc datadir-mongodb-0 datadir-mongodb-1
 ```
 
-Note that this will remove your graylog settings but not your actual logs. This step is unfortunately needed to enable credentials on the Graylog database hosted by the mongodb chart. You will need to recreate the GELF TCP input source as during install.
+Note that this will remove your graylog settings but not your actual logs. This step is unfortunately needed to enable
+credentials on the Graylog database hosted by the mongodb chart. You will need to recreate the GELF TCP input source as
+during install.
 
 Then run
 
@@ -115,38 +249,47 @@ To update the Kafka stack, run:
 helmfile -f helmfile.d/10-base.yaml apply --concurrency 1
 ```
 
-After this has succeeded, edit your `production.yaml` and change the `cp_kafka.customEnv.KAFKA_INTER_BROKER_PROTOCOL_VERSION` to the corresponding version documented in the [Confluent upgrade instructions](https://docs.confluent.io/platform/current/installation/upgrade.html) of your Kafka installation. Find the currently installed version of Kafka with `kubectl exec cp-kafka-0 -c cp-kafka-broker -- kafka-topics --version`.
+After this has succeeded, edit your `production.yaml` and change the
+`cp_kafka.customEnv.KAFKA_INTER_BROKER_PROTOCOL_VERSION` to the corresponding version documented in
+the [Confluent upgrade instructions](https://docs.confluent.io/platform/current/installation/upgrade.html) of your Kafka
+installation. Find the currently installed version of Kafka with
+`kubectl exec cp-kafka-0 -c cp-kafka-broker -- kafka-topics --version`.
 When the `cp_kafka.customEnv.KAFKA_INTER_BROKER_PROTOCOL_VERSION` is updated, again run
 
 ```shell
 helmfile -f helmfile.d/10-base.yaml apply
 ```
 
-To upgrade to the latest PostgreSQL helm chart, in `production.yaml`, uncomment the line `postgresql.primary.persistence.existingClaim: "data-postgresql-postgresql-0"` to use the same data storage as previously. Then run
+To upgrade to the latest PostgreSQL helm chart, in `production.yaml`, uncomment the line
+`postgresql.primary.persistence.existingClaim: "data-postgresql-postgresql-0"` to use the same data storage as
+previously. Then run
 
 ```shell
 kubectl delete secrets postgresql
 kubectl delete statefulsets postgresql-postgresql
 helmfile -f helmfile.d/10-managementportal.yaml apply
 ```
 
-If installed, `radar-appserver-postgresql`, uncomment the `production.yaml` line `radar_appserver_postgresql.primary.existingClaim: "data-radar-appserver-postgresql-postgresql-0"`. Then run
+If installed, `radar-appserver-postgresql`, uncomment the `production.yaml` line
+`radar_appserver_postgresql.primary.existingClaim: "data-radar-appserver-postgresql-postgresql-0"`. Then run
 
 ```shell
 kubectl delete secrets radar-appserver-postgresql
 kubectl delete statefulsets radar-appserver-postgresql-postgresql
 helmfile -f helmfile.d/20-appserver.yaml apply
 ```
 
-If installed, to upgrade `timescaledb`, uncomment the `production.yaml` line `timescaledb.primary.existingClaim: "data-timescaledb-postgresql-0"`. Then run
+If installed, to upgrade `timescaledb`, uncomment the `production.yaml` line
+`timescaledb.primary.existingClaim: "data-timescaledb-postgresql-0"`. Then run
 
 ```shell
 kubectl delete secrets timescaledb-postgresql
 kubectl delete statefulsets timescaledb-postgresql
 helmfile -f helmfile.d/20-grafana.yaml apply
 ```
 
-If installed, to upgrade `radar-upload-postgresql`, uncomment the `production.yaml` line `radar_upload_postgresql.primary.existingClaim: "data-radar-upload-postgresql-postgresql-0"`. Then run
+If installed, to upgrade `radar-upload-postgresql`, uncomment the `production.yaml` line
+`radar_upload_postgresql.primary.existingClaim: "data-radar-upload-postgresql-postgresql-0"`. Then run
 
 ```shell
 kubectl delete secrets radar-upload-postgresql