ClickHouse is currently at the design stage of creating some universal backup solution. Some custom backup strategies are:
- Each shard is backed up separately.
- FREEZE the table/partition. For more information, see Alter Freeze Partition.
- This creates hard links in shadow subdirectory.
- rsync that directory to a backup location, then remove that subfolder from shadow.
- Cloud users are recommended to use Rclone.
- Always add the full contents of the metadata subfolder that contains the current DB schema and clickhouse configs to your backup.
- For a second replica, it’s enough to copy metadata and configuration.
- Data in clickhouse is already compressed with lz4, backup can be compressed bit better, but avoid using cpu-heavy compression algorythms like gzip, use something like zstd instead.
The tool automating that process clickhouse-backup.
Last modified 2021.08.12: General corrections and updates. (876b7b6)