mirror of
https://gitverse.ru/kpa39l/garage-s3-cluster.git
synced 2026-09-29 09:15:07 +00:00
estorozhenko
60 lines
2.8 KiB
Markdown
60 lines
2.8 KiB
Markdown
---
|
|
name: garage-s3-cluster
|
|
description: Garage S3 cluster status checks on vps01/bigbox/vps02.
|
|
---
|
|
|
|
# Garage S3 cluster (vps01 / bigbox / vps02)
|
|
|
|
Garage = self-hosted S3-compatible storage cluster, replication_factor=3, read/write_quorum=2.
|
|
Runs in Docker on bigbox from /opt/garage. Bucket "obsidian" holds obsidian-vault backups.
|
|
|
|
## Topology (WireGuard 10.8.0.0/24)
|
|
- vps01: 10.8.0.1:3901 (hostname 5599453-kpa39l, zone vps01)
|
|
- bigbox: 10.8.0.2:3901 (zone home) — /opt/garage, wg0 = 10.8.0.2/24
|
|
- vps02: 10.8.0.4:3901 (zone vps02)
|
|
- S3 API (incl. admin API): 0.0.0.0:3900 on every node; RPC: WG IP:3901
|
|
- Config: /opt/garage/garage.toml (rpc_secret, admin_token, bootstrap_peers inside)
|
|
|
|
## Status check — the reliable way
|
|
Container `dxflrs/garage:v2.1.0` is SCRATCH: no shell, no CLI in PATH.
|
|
`docker exec garage garage ...` and `docker exec garage sh ...` both fail with
|
|
"executable file not found". The CLI binary lives at `/garage` INSIDE the image — run it
|
|
with `--entrypoint`, mounting BOTH config and meta (the node key lives in meta/; skipping
|
|
the meta mount gives "Unable to read node key. It will be generated..."):
|
|
|
|
```
|
|
docker run --rm \
|
|
-v /opt/garage/garage.toml:/etc/garage.toml:ro \
|
|
-v /opt/garage/meta:/var/lib/garage/meta \
|
|
--entrypoint /garage dxflrs/garage:v2.1.0 status
|
|
```
|
|
|
|
Subcommands that work at top level: `status` (health table — HEALTHY NODES),
|
|
`stats` (block manager + cluster-wide usage), `bucket list`.
|
|
Pitfall: `garage cluster status` does NOT exist in v2.1 — "Found argument 'cluster' which
|
|
wasn't expected". There is no `cluster` subcommand; status/stats are top-level.
|
|
|
|
Healthy signals in `garage stats`: "resync queue length: 0", "blocks with resync errors: 0",
|
|
MklTodo/GcTodo/InsQueue all 0.
|
|
|
|
## Admin HTTP API (port 3900) — do not rely on it
|
|
- `Authorization: Bearer <admin_token>` → "Unsupported authorization method" (S3-style XML error)
|
|
- `X-Garage-Admin-Token: <admin_token>` → AccessDenied "anonymous access"
|
|
- `/v2/status`, `/v1/status`, `/cluster/status` all same behavior.
|
|
- The CLI route above is the dependable path; no working HTTP admin call was found.
|
|
|
|
## Files in /opt/garage (bigbox)
|
|
- docker-compose.yml — service garage, network_mode: host, volumes: garage.toml, meta/, data/
|
|
- garage.toml — full config; admin_token duplicated here and in admin_token file
|
|
- admin_token — admin token (65 hex chars); secret — RPC secret (not for admin API)
|
|
- cli/ — BROKEN: garage-v2.1.0-linux-x86_64.tar.gz is a 10-byte "Not Found" placeholder. Ignore; use the in-image CLI above.
|
|
- meta/, data/ — LMDB storage (db_engine = "lmdb")
|
|
|
|
## Reachability probe (over WG)
|
|
```
|
|
for ip in 10.8.0.1 10.8.0.2 10.8.0.4; do
|
|
timeout 3 bash -c "echo > /dev/tcp/$ip/3901" 2>/dev/null && echo "$ip:3901 OK" || echo "$ip:3901 FAIL"
|
|
done
|
|
```
|
|
|
|
Script: `scripts/garage-status.sh` — runs the CLI status command with correct mounts. |