# Docker IP shift: nginx stale upstream → WS 502 → Converse spinner

Symptom (2026-08-30, chat.nixg.ru): page loads fine, the login form flashes briefly,
then Converse shows only the pulsing white "connecting" circle forever. No config
change was made — the break appeared after container recreation.

## Root cause

`icq-webchat`'s nginx resolves `proxy_pass http://prosody:5280` ONCE at config load and
caches the IP for the life of the process. Docker bridge IPs are assigned at container
creation; after `docker compose up` recreates containers (or adds a service) any
container can move to a different IP. In this incident:

- webchat started 42h ago → cached the OLD mapping (prosody = 172.27.0.2)
- prosody recreated 26h ago, slidgram created 22h ago → IPs shifted
- new mapping: slidgram = 172.27.0.2, webchat = 172.27.0.3, prosody = 172.27.0.4

Every `/xmpp-websocket` request was proxied to Slidge's port 5280 (closed) →
Connection refused → 502 → client never connects → spinner forever.

## Diagnostic trail (fast)

```bash
docker logs icq-webchat --tail 40 | grep -E 'xmpp-websocket|refused'
# → connect() failed (111: Connection refused) while connecting to upstream
#   upstream: "http://172.27.0.2:5280/xmpp-websocket"   ← STALE IP in the error line
docker compose ps --format '{{.Name}}\t{{.Status}}'   # "Up X hours" reveals who was recreated
# get ACTUAL container IPs on the compose network:
docker network inspect icq_default --format '{{range .Containers}}{{.Name}} {{.IPv4Address}}{{"\n"}}{{end}}'
```

Compare the upstream IP in the nginx error with the current IP of `prosody`: mismatch =
stale DNS cache. Note: `docker logs icq-prosody` can show 0 lines because Prosody logs
to the mounted volume — read `/opt/icq/logs/prosody.log` instead (it is the live log,
debug-level, presence/roster traffic from the Telegram bridge is visible there).

## Fix

```bash
cd /opt/icq && docker compose restart webchat   # nginx re-resolves "prosody" → new IP
```

Verify: `docker logs icq-webchat --since 2m | grep -cE '502|refused'` → 0. The user must
F5 the tab — an already-open tab stuck on the spinner does NOT auto-reconnect.

## Permanent hardening (avoid recurrence)

In `webchat/nginx.conf`, force per-request DNS via Docker's embedded resolver + a
variable in `proxy_pass` (the documented nginx pattern: a static hostname in proxy_pass
is resolved only at config load; a variable makes nginx consult the resolver per request):

```nginx
location /xmpp-websocket {
    resolver 127.0.0.11 valid=30s;           # Docker embedded DNS, re-resolve every 30s
    set $prosody prosody:5280;
    proxy_pass http://$prosody/xmpp-websocket;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $host;
    proxy_read_timeout 3600s;
    proxy_send_timeout 3600s;
}
```

## Related

- Blank page (NOTHING renders, ESM `import.meta` SyntaxError) → `conversejs-blank-page-and-auth-testing.md`
- Caddy bind-mount inode trap (edit silently ignored) → `webchat-conversejs-and-caddy-trap.md`
- This is the same class as any nginx-in-Docker stale-upstream issue; the tell is the IP
  in the 502 line belonging to a different container than the one named in proxy_pass.