Upgrade Procedure
Step-by-step procedure for upgrading a GridGain 8.9.39 cluster to 8.10 with Rolling Upgrades or offline, with verification, troubleshooting, and a recovery path back to 8.9.
This guide describes how to upgrade a GridGain 8.9.39 cluster to GridGain 8.10, and how to check that the upgrade worked.
Before you start, read What to Know Before Upgrading. It helps you choose between a rolling and an offline upgrade and lists the changes that this guide refers to.
The procedure has five phases: prerequisites, compatibility check, preparation, the upgrade itself, and verification.
A downgrade from 8.10 to 8.9 is not supported. To recover 8.9, see the Recovery Path.
Prerequisites
Confirm all of the following before you start:
The cluster runs GridGain 8.9.39 or an earlier version listed in Supported Starting Versions. Behavior can differ in places between releases. See the release notes for the releases between your version and 8.9.39 first.
Java 17 or later is installed on every server host. You can keep running the 8.9 nodes on their current Java version until you restart them. Each node switches to Java 17 when you restart it from the 8.10 distribution, so set
JAVA_HOMEfor the 8.10 scripts before you start the node.A valid GridGain license is installed, and it enables every feature that your configuration uses. See Per-Feature License Checks.
You chose an upgrade path and scheduled a downtime window if the path needs one.
Storage for a backup of the 8.9 data is available outside the work directory. Take the backup before you upgrade. After the upgrade is finished, it is the only way to recover 8.9. See Step 3.
The 8.10 distribution is unpacked next to the 8.9 one. Keep the 8.9 distribution until the upgrade is validated, because recovery needs it.
Check Compatibility
Most upgrade work is conditional. Each area below may or may not apply to your deployment. Review every area and record each item that applies:
Review the platform and module changes. Java 17, removed modules, upgraded dependencies, and packaging. See Platform, Modules, and Packaging Changes.
Review the configuration changes. Changed defaults, new limits, and removed system properties. See Configuration and Default Value Changes.
Review the security and license changes. New permissions, closed-by-default features, and per-feature license checks. See Security, Permission, and Licensing Changes.
Review the client changes. Thin client timeouts, DNS, transactions, .NET, and C++. See Client and Driver Changes.
Prepare the Distribution, Configuration, and Cluster
None of the following requires downtime, except where noted.
Preparation Tasks
Build the 8.10
libsfolder. Do not copy thelibsfolder of 8.9 into the 8.10 distribution.If you run only the modules that ship in
libsby default, use the 8.10libsfolder as shipped.Otherwise, compare your 8.9
libsandlibs/optionalfolders with a fresh 8.9 distribution to list the modules you added or moved by hand. Copy only those that you use and that still exist in 8.10.A module that you enabled in 8.9 by moving it into
libs, for exampleignite-opencensus, is not enabled in 8.10 until you do the same there.control-center-agentis inlibs/optional. If you use it, move it intolibs, and use Control Center 2026.2.These steps describe a ZIP installation. For other deployments:
Docker:
ignite-opencensusis no longer enabled by default. If your configuration usesOpenCensusTracingSpiorOpenCensusMetricExporterSpi, add it toOPTION_LIBS, or the node fails at startup withClassNotFoundException. See Enabling Modules.Kubernetes: with the GridGain Operator, set
option_libs, and check its value if you rely on a module that is no longer on the classpath by default. See option_libs. For a manual deployment, see Creating Pod Configuration.Maven or an embedded node: there is no
libs/optionalfolder to move. See Using Maven and Enabling Modules.
See Removed Modules and Integrations and Distribution and Packaging Changes.
Update applications and build files. Apply the changes from your work list: Spring, Hibernate, removed modules, and client changes.
Update the node configuration. Remove unsupported system properties, resolve new limits, and set the new properties that your features need.
Keep the persistent data in place. Native persistence only. Each node must keep its consistent ID and the same
storagePath,walPath, andwalArchivePath. If you do not set these properties, the default location is the work directory, and the work directory of a new distribution is empty. Point the new node at the old one by setting theIGNITE_WORK_DIRenvironment variable or the paths in the configuration.Prepare the 8.9 cluster. Rolling path: configure Rolling Upgrades and back up the data, as described in steps 1 to 3 of Upgrade With Rolling Upgrades. Offline path: back up the data as described in Upgrade Without Rolling Upgrades.
Check caches that must stay available. Configure at least one backup for them, or plan a maintenance window. See Application Behavior During a Rolling Upgrade.
When Is Preparation Complete?
If you have a staging cluster, rehearse the whole procedure there first.
Upgrade With Rolling Upgrades
Use this path to upgrade the nodes one at a time, without stopping the cluster for the upgrade itself. It needs Rolling Upgrades. The steps are the same for both editions, except for the backup. Ultimate Edition takes a snapshot. Enterprise Edition copies the persistence folders, which needs a short stop of the whole cluster. If you cannot use Rolling Upgrades, or you prefer to stop the cluster, see Upgrade Without Rolling Upgrades.
The procedure has three parts:
Steps 1–3 configure Rolling Upgrades on the 8.9 cluster and back up its data.
Steps 4–6 start the rolling upgrade, upgrade the nodes one at a time, and finish it.
Verify and Stabilize covers the checks that follow.
If authentication is enabled, pass the credentials to every command: control.sh --user <user> --password <password> ..., and on Ultimate Edition also snapshot-utility.sh ... -user=<user> -password=<password>.
Step 1: Configure Rolling Upgrades
Every 8.9 node, on both editions, must have rollingUpdatesEnabled=true in its GridGainConfiguration before the upgrade. Replace the license path with your own, and skip the properties that you already have:
GridGainConfiguration is an element of IgniteConfiguration.pluginConfigurations. Enabling Rolling Upgrades adds a small amount of data to every network message. See Performance Considerations.
Ultimate Edition: Add a Snapshot Configuration
To take a snapshot in step 3, also add a snapshotConfiguration to the GridGainConfiguration of every node:
Store the snapshots outside of the work directory, so that you can keep them when you remove the data. Nodes can use the same snapshotsPath, because each snapshot gets a subfolder for every node. This also works for nodes on different hosts if the folder is a shared file system that all hosts mount at the same path.
Step 2: Restart the Cluster
You cannot add rollingUpdatesEnabled one node at a time. A node with rollingUpdatesEnabled=true is not allowed to join a cluster whose nodes do not have it, even if all nodes run the same version.
If the property is already set on every node, skip this step. Otherwise, stop the whole cluster, add the properties from step 1 to the configuration of every node, and start the cluster again. This is a planned downtime. With native persistence, the data and the baseline are kept.
Step 3: Back Up the Data
Take a backup before you upgrade. After the rolling upgrade is finished, it is your only way to recover 8.9. See Recovery Path. Follow the part for your edition.
Ultimate Edition: Take a Full Snapshot
When the cluster is running with the new configuration, take a full snapshot on 8.9:
The command ends with Command [SNAPSHOT] successfully finished in <seconds> seconds. Like control.sh, snapshot-utility.sh connects to port 11211 of the local host by default. If your REST connector uses another port or host, pass -port=<port> and -host=<host>, for example snapshot-utility.sh snapshot -type=full -port=11300. Then list the snapshots and write down the ID of the one that you just took:
The line of the new snapshot shows its ID and the product version that took it:
You need the ID if you decide to recover the 8.9 state.
Enterprise Edition: Copy the Persistence Folders
Enterprise Edition has no snapshots. The backup is a copy of the persistence folders of every node. The upgrade itself runs without downtime, but the copy does not. It needs a short stop of the whole cluster.
Stop all nodes before you copy. Do not copy the folders of running nodes, and do not copy the nodes one at a time while the others keep working. Data changes between the moments that you copy each node, so a restore would mix primary partitions and backups from different moments, and the cluster would hold inconsistent data. Only a copy of stopped nodes, taken from a deactivated cluster, is consistent.
If you stopped the whole cluster in step 2, take the copy in that window, before you start the nodes again. Otherwise, stop the cluster for it now:
Deactivate the cluster with
control.sh --set-state INACTIVE --yes, and stop all nodes.On every node, copy these to a location outside the work directory:
The
dbfolder of the work directory. Always copy it, even if you set all three paths below. It holds the binary metadata, and the data cannot be read without it.The folders that
storagePath,walPath, andwalArchivePathpoint to, if you set them.The node configuration and the consistent ID.
Keep the copy of each node separate from the others, and note which node it came from.
Start the nodes again with the 8.9 distribution and the configuration from step 1. With native persistence, the cluster activates with the old baseline when every baseline node is back. Check it with
control.sh --baseline.
Step 4: Start the Rolling Upgrade
Without the --yes option, the command asks for confirmation. When you run it from a script, always pass --yes.
The command prints Rolling upgrade mode successfully enabled. Check the status:
Before you start, it shows Rolling upgrade is disabled. After start, it shows Rolling upgrade is enabled., and every node is listed under the nodes that are not updated:
Setting rollingUpdatesEnabled is not enough on its own. The status stays disabled until you run this command.
Start the first 8.10 node only after this command. Otherwise the node is rejected (see Troubleshooting).
From the first upgraded node on, use the control.sh of the 8.10 distribution for the --rolling-upgrade commands.
Step 5: Upgrade the Nodes One at a Time
For each node, in any order:
Stop the node and wait until its process has exited.
Start the node from the 8.10 distribution with the same configuration, work directory, and consistent ID.
Wait until the node has joined and rebalancing has finished. Look in the log of the new node for the
Finish exchange futuremessage and then for eitherCompleted (final) rebalancing(data moved to the node) orSkipping rebalancing (nothing scheduled)(no data changed while the node was down). Do not stop the next node before you see one of them. You can also usecontrol.sh --metric cluster. The valuecluster.Rebalancedisfalsewhile data moves andtruewhen it is done. For a few seconds after the node joins, it can still showtruebefore rebalancing starts, so wait until it showstrueon several checks in a row, a few seconds apart.Run
control.sh --rolling-upgrade status. After each node, one node moves from the not-updated list to the updated list. When the first node is upgraded, the status looks like this, andTarget versionnow shows the 8.10 build:
The status lists the first 8 characters of the node ID, and a restarted node gets a new ID, so check that the updated list grows by one and the not-updated list shrinks by one. To identify a node, take its new ID from the node log. Look for the line Local node [ID=...] in the startup banner, or locNode=... in the Topology snapshot line.
The new node writes a warning to its log that the build versions differ, but that they are compatible. This is expected.
Step 6: Finish the Rolling Upgrade
When the status shows none under the nodes that are not updated, finish the rolling upgrade:
If you run finish while an 8.9 node is still in the cluster, it is refused with Rolling Upgrade mode cannot be disabled and lists the nodes of each version. The exit code is 0 in this case too.
A successful finish prints Rolling upgrade mode successfully disabled. Afterward, control.sh --rolling-upgrade status shows Rolling upgrade is disabled., the initial version is the 8.10 build (for example Initial version: 8.10.x#<build>-sha1:<hash>), and Target version is N/A. New 8.10 features become available.
Continue with Verify and Stabilize.
Application Behavior During a Rolling Upgrade
While the upgrade is in progress, the cluster stays available. Clients that are connected to a node that is being stopped see errors for a short time. Plan for the following:
Applications must reconnect and repeat the failed operation. A JDBC connection that was open on a stopped node fails with
Failed to handle JDBC request because node is stopping, and you must open a new one. Give the driver more than one address (for examplejdbc:ignite:thin://host1:10800,host2:10800,host3:10800), so that a new connection does not go to the node that is down. The Java thin client reconnects and retries by itself.Caches without backups are unavailable for the partitions on the stopped node. A cache with
backups=0(the default) fails withCacheInvalidStateException, and the errors continue after the node returns until you reset the lost partitions. The log of the nodes names the affected cache, for exampleDetected lost partitions [grp=SQL_PUBLIC_NB, parts=[...]]. Configure at least one backup for the caches that must stay available, or plan a maintenance window for the others.With native persistence, the data is not lost. After the node is back, run
control.sh --cache reset_lost_partitions <cache_name>(or--all), and the cache works again. You can reset the caches once, after the last node is back. The cache of an SQL table is namedSQL_<schema_name>_<table_name>, for exampleSQL_PUBLIC_NB, unless you setCACHE_NAMEwhen you created the table.Two features stay off while an 8.9 node is in the cluster. A service with call interceptors is refused (
not all nodes in the cluster support service call interceptors), and the separate scale-up and scale-down baseline auto-adjust commands (control.sh --baseline scale_up_auto_adjustandscale_down_auto_adjust) fail. Both start working when the last 8.9 node is replaced. GridGain 8.9.39 nodes advertise these two features without implementing them, and 8.10 does not rely on that advertisement.
Each node needs time to stop, join, and rebalance. Only the data that changed while the node was down is moved.
Upgrade Without Rolling Upgrades
If you can stop the cluster, you do not need rollingUpdatesEnabled and you do not need control.sh --rolling-upgrade:
Install the 8.10 distribution and Java 17 or later. Do not copy the old
libsfolder, and review the removed modules.Start all nodes from the 8.10 distribution with the same configuration, work directory, and consistent IDs. You can start them together or one after another, in any order.
Continue with Verify and Stabilize.
With native persistence, the cluster activates by itself with the old baseline as soon as every baseline node is back, and all data is kept. You do not run any control.sh --rolling-upgrade command. To confirm, run control.sh --baseline. It shows Cluster state: active and every baseline node as ONLINE.
To recover 8.9 after an offline upgrade, restore the backup from step 1, as described in After Finish.
On Ultimate Edition, there is another way to upgrade: take the full snapshot on 8.9, start a new empty 8.10 cluster with the same number of nodes, and restore the snapshot into it with the 8.10 snapshot-utility.sh restore -id=<snapshot_id>. The 8.10 utility can restore snapshots that 8.9 created. Use this way to move to new hosts, or to keep the old 8.9 cluster untouched.
Verify and Stabilize
With the cluster running on 8.10, confirm it is healthy before you consider the upgrade complete:
Check the cluster and the data. Run
control.sh --baselineand compare the node list with your baseline. The output starts withCluster state: activeand lists every node asONLINE. Run a query from a client, compare row counts with the values before the upgrade, and runcontrol.sh --cache idle_verify. A healthy run ends withidle_verify check has finished, no conflicts have been found.Reset lost partitions. If you have caches without backups, look for
Detected lost partitionsin the node logs and reset them as described in Application Behavior During a Rolling Upgrade.Run the real workload. Run your regression suite or production workload end to end and confirm it passes.
Reconnect monitoring and tooling. Confirm that the nodes connect to Control Center 2026.2 (the agent must be in
libs), check the node logs for license errors, and update tooling that reads JMX object metrics.Re-establish backups. On Ultimate Edition, take a first snapshot on 8.10 and confirm it completes. On Enterprise Edition, plan your regular backups of the persistence folders.
Optional: switch to Lucene 10. If you plan to use it, switch every node after the upgrade is finished. See Lucene.
Troubleshooting
These are the problems that come up most often during the upgrade. If you hit something not covered here, contact MariaDB support. Problems with Connection to cluster failed, a refused finish, a restarted node missing from the status, and lost partitions are described where they occur. See Step 5, Step 6, and Application Behavior During a Rolling Upgrade.
Rolling Upgrade mode cannot be enabled. Consider changing 'GridGainConfiguration.rollingUpdatesEnabled' configuration property. on control.sh --rolling-upgrade start
rollingUpdatesEnabled is not set on the nodes. The status also stays disabled until you run start, even when the property is set.
Set rollingUpdatesEnabled=true on every node and restart the cluster (Step 2), then run start --yes.
Rolling updates are not properly configured when a node starts
The node and the cluster it joins differ in rollingUpdatesEnabled. Either the node has it set and the cluster does not, or the cluster has it set and the node does not. The second case also hits applications that start a thick client (a node in client mode) while the rolling upgrade path is configured.
If the cluster lacks the property, stop the whole cluster, add the property to every node, and start the cluster again (Step 2). If a thick client lacks it, add rollingUpdatesEnabled=true to the GridGainConfiguration of the client.
GridGain snapshots are not configured (error 4400) from snapshot-utility.sh
snapshotConfiguration is not set in the GridGainConfiguration of the node.
Add snapshotConfiguration with a snapshotsPath to every node and restart the cluster (Step 1).
The node has been rejected due to disabled Rolling Upgrade when a node starts
Either an 8.10 node joins before you ran start, or an 8.9 node joins after finish.
In the first case, run control.sh --rolling-upgrade start --yes before you start the 8.10 node. In the second case, an 8.9 node cannot rejoin after finish. See Recovery Path.
A node starts on an empty baseline with none of your caches
The node does not use the same consistent ID and persistence directories as the node it replaces. The work directory of the new distribution is empty.
Stop the node. Set the same consistent ID, and point storagePath, walPath, and walArchivePath, or IGNITE_WORK_DIR, at the existing data. Start the node again.
ignite.sh, ignite.bat, control.sh, or control.bat refuses to start
JAVA_HOME points at a JDK older than 17.
Set JAVA_HOME to Java 17 or later for the 8.10 scripts.
Recovery Path
A downgrade from 8.10 to 8.9 is not supported. An 8.9 node cannot join a cluster after the rolling upgrade is finished, and data that 8.10 writes may use formats that 8.9 does not read. For example, an 8.9 node does not start if the data has a PARTITIONED cache with a vector index: 8.9 supports a vector index only on a REPLICATED cache, and the node stops with Failed to create vector index. Only REPLICATED cache mode is supported.
The supported recovery depends on when you decide:
Before you finish the rolling upgrade, you revert it. No backup is needed. See Before Finish.
After
finish, or after an offline upgrade, you restore the backup that you took before the upgrade. See After Finish.
If you use data center replication, read Upgrade and Recovery to 8.9 before you recover.
The backup that you can restore depends on your edition:
Ultimate
A full snapshot with snapshot-utility.sh
Restore the snapshot into an 8.9 cluster: Recover From a Snapshot.
Enterprise
A copy of the db folder, and of storagePath, walPath, and walArchivePath if you set them, of every node
Put the copies back and start 8.9: Recover From a Copy of the Persistence Folders.
Before Finish
While the rolling upgrade is still in progress, you can revert it (see Reverting Rolling Upgrades). Revert each node that you already upgraded. Stop the node and start it again from the 8.9 distribution with the same configuration, work directory, consistent ID, and Java version. The node rebalances from the 8.10 nodes, and tables, columns, and indexes that you created through 8.10 nodes stay visible.
After all nodes are back on 8.9, the status still shows the target version 8.10 and lists all nodes as not updated. Run control.sh --rolling-upgrade finish --yes to leave the rolling upgrade mode. The initial version stays 8.9.x.
After Finish
After finish, an 8.9 node cannot join the cluster any more. This section also applies after an offline upgrade, where there is no finish. Use one of the two recoveries below. Both return the cluster to the state at the time of the backup.
Recover From a Snapshot
Ultimate Edition only. Use the full snapshot that you took before the upgrade:
Stop all nodes.
Remove the data of the nodes, or move it to another location if you may need the 8.10 data. If you set
storagePath,walPath, andwalArchivePath, remove the folders they point to. Thedbfolder of the work directory then holds only small binary metadata, and it can stay. If you did not set them, remove thedbfolder of the work directory. Keep the snapshot folder.Start the nodes from the 8.9 distribution, with the 8.9 configuration. It must still have a
snapshotConfigurationwhosesnapshotsPathpoints at the folder that holds your snapshots. Activate the cluster withcontrol.sh --set-state ACTIVE --yes.Find the snapshot ID. Run the 8.9
snapshot-utility.sh list, and choose the snapshot that you took before the upgrade. ItsPRODUCT VERSIONSshows 8.9.x. A snapshot whosePRODUCT VERSIONSshows 8.10.x was taken after the upgrade, so it does not hold the state that you validated on 8.9.Restore the snapshot with the 8.9
snapshot-utility.sh restore -id=<snapshot_id>. A successful restore printsRestored: true.Check the data. Compare row counts with the values before the upgrade and run
control.sh --cache idle_verify.
Recover From a Copy of the Persistence Folders
Enterprise Edition only. Use the copies that you took before the upgrade, as described in Step 3. To recover the 8.9 state, restore the copies and start the cluster on 8.9:
Stop all nodes.
On every node, move the current
dbfolder, and thestoragePath,walPath, andwalArchivePathfolders if you set them, aside. Remove them instead if you do not need the 8.10 data. Copy the backup of that same node back to the same paths.Start the nodes from the 8.9 distribution, with the 8.9 configuration and the same consistent IDs. When every baseline node is back, the cluster activates with the old baseline.
Check the data. Run
control.sh --baseline, compare row counts with the values before the upgrade, and runcontrol.sh --cache idle_verify.
Restore the copies of all nodes together, and only copies that were taken in the same stop of the cluster. Do not restore only some of the nodes, and do not mix copies from different moments.
This page is: Copyright © 2026 MariaDB. All rights reserved.
Last updated
Was this helpful?

