Mariadb Master-Master recovery in the 4 node from manual backup with Mariabackup

Hi,

i have the question. I have cluster Master-Master with 4 servers (MariaDB 10.3.18) Centos 7 64bit

im started Galera_new_cluste new cluster in Server 1. After that I run the command /usr/bin/mariabackup --backup --galera-info --no-timestamp --target-dir=/var/lib/mysql/backup/

after the complete backup in the first server, I transfer to another node-server this backup: rsync -avr root@$server1:/var/lib/mysql/backup/ /var/lib/mysql/backup/ then I run: mariabackup --prepare --galera-info --no-timestamp --target-dir=/var/lib/mysql/backup/ and mariabackup --copy-back --galera-info --no-timestamp --target-dir=/var/lib/mysql/backup/

I building the grastate.dat with dates from the xtrabackup_galera_info file. and started node 1 with command systemctl start MariaDB

and after sereverals seconds MariaDB has remove all dates from the catalog with recovering datas and try the self sync the db (20 Gb) with the crash by the began. Please advise me, where is mistake ?

2019-10-21 8:39:18 2 [Note] WSREP: Prepared SST request: mariabackup|10.0.39.74:4444/xtrabackup_sst1 2019-10-21 8:39:18 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification. 2019-10-21 8:39:18 2 [Note] WSREP: REPL Protocols: 9 (4, 2) 2019-10-21 8:39:18 2 [Note] WSREP: Assign initial position for certification: 12847514, protocol version: 4 2019-10-21 8:39:18 0 [Note] WSREP: Service thread queue flushed. 2019-10-21 8:39:18 2 [Note] WSREP: IST receiver addr using tcp:10.0.39.74:4568 2019-10-21 8:39:18 2 [Note] WSREP: Prepared IST receiver, listening at: tcp:10.0.39.74:4568 2019-10-21 8:39:18 0 [Note] WSREP: Member 3.0 (api-db-74) requested state transfer from '*any*'. Selected 0.0 (api-db-61)(SYNCED) as donor. 2019-10-21 8:39:18 0 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 12847534) 2019-10-21 8:39:18 2 [Note] WSREP: Requesting state transfer: success, donor: 0 2019-10-21 8:39:18 2 [Note] WSREP: GCache history reset: 74c6b91c-e219-11e9-b779-1e7d34f8244b:0 -> 74c6b91c-e219-11e9-b779-1e7d34f8244b:12847514 WSREP_SST: [INFO] Proceeding with SST (20191021 08:39:18.363) WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | mbstream -x; RC=( ${PIPESTATUS[@]} ) (20191021 08:39:18.365) WSREP_SST: [INFO] Cleaning the existing datadir and innodb-data/log directories (20191021 08:39:18.368) removed ‘/var/lib/mysql/mariadb/ibdata1’ removed...... removed ‘/var/lib/mysql/mariadb/ib_logfile1’ WSREP_SST: [INFO] Waiting for SST streaming to complete! (20191021 08:39:19.834) 2019-10-21 18:41:18 0 [Note] WSREP: (98e3914c, 'tcp:0.0.0.0:4567') turning message relay requesting off Terminated WSREP_SST: [ERROR] Removing /var/lib/mysql/mariadb.sst/xtrabackup_galera_info file due to signal (20191021 18:42:44.571) WSREP_SST: [ERROR] Parent mysqld process (PID:24513) terminated unexpectedly. (20191021 18:42:44.579) WSREP_SST: [ERROR] Cleanup after exit with status:32 (20191021 18:42:44.585) 2019-10-21 18:42:44 0 [ERROR] WSREP: Process completed with error: wsrep_sst_mariabackup --role 'joiner' --address '10.0.39.74' --datadir '/var/lib/mysql/mariadb/' --parent '24513' --mysqld-args --wsrep_start_position=00000000-0000-0000-0000-000000000000:-1: 32 (Broken pipe) 2019-10-21 18:42:44 0 [ERROR] WSREP: Failed to read uuid:seqno and wsrep_gtid_domain_id from joiner script. 2019-10-21 18:42:44 0 [ERROR] WSREP: SST failed: 32 (Broken pipe) 2019-10-21 18:42:44 0 [ERROR] Aborting

2019-10-21 18:42:44 0 [Warning] WSREP: 0.0 (api-db-61): State transfer to 3.0 (api-db-74) failed: -22 (Invalid argument) 2019-10-21 18:42:44 0 [ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():775: Will never receive state. Need to abort.

my server.cnf file:

[galera] wsrep_on=ON binlog_format=row default_storage_engine=InnoDB innodb_autoinc_lock_mode=2 query_cache_size=0 query_cache_type=0 datadir=/var/lib/mysql/mariadb innodb_file_per_table innodb_flush_log_at_trx_commit=2 wsrep_provider=/usr/lib64/galera/libgalera_smm.so wsrep_cluster_address="gcomm:10.0.39.61" wsrep_cluster_name='MyCluster' wsrep_node_address='10.0.39.74'

wsrep_node_name='db74' wsrep_sst_method=mariabackup wsrep_sst_auth=sst:password

Best regards, Alex