Clean up orphaned objects on s3

Clean up orphaned objects left in an S3-backed ClickHouse tiered‐storage

Problems

  • TRUNCATE and DROP TABLE remove metadata only.
  • Long-running queries, merges or other replicas may still reference parts, so ClickHouse delays removal.
  • There are bugs in Clickhouse that leave orphaned files, especially after failures.

Solutions

  • use our utility for garbage collection - https://github.com/Altinity/s3gc
  • or create a separate path in the bucket for every table and every replica and remove the whole path in AWS console
  • you can also use clickhouse-disk utility to delete s3 data:
clickhouse-disks --disk s3 --query "remove /cluster/database/table/replica1"