/home/hermes/stack), a unittest-based test suite was added, seven real deploy-blocking bugs were found and fixed, remaining secrets were externalized into .env, and the real production stack went live with genuine Let's Encrypt certificates on all four subdomains. This page has been updated in place to match; for the full story of what changed and why, see the Gateway Testing & Deployment Guide.
Browser │ │ HTTPS :443 ▼ Cloudflare ├─ DDoS protection ├─ DNS — proxied (orange cloud — hides VPS IP) └─ Full strict TLS to origin │ │ HTTPS ▼ Caddy on VPS :443 ├─ TLS termination (auto Let's Encrypt) ├─ api.elfatih.net → APISIX :9080 ├─ dashboard.elfatih.net → Authelia forward-auth → APISIX Dashboard :9000 └─ auth.elfatih.net → Authelia :9091 │ │ HTTP (internal Docker network — proxy) ▼ APISIX :9080 ├─ openid-connect plugin → validates session against Authelia ├─ Rate limiting, routing, request transformation └─ Routes to upstream services (localhost:PORT) │ │ HTTP (internal only — 127.0.0.1) ▼ Upstream Services (bound to 127.0.0.1:PORT — never exposed directly)
| Component | Owns | Does NOT do |
|---|---|---|
| Cloudflare | DDoS, CDN, DNS, origin IP hiding | AuthN, routing logic |
| Caddy | TLS termination, virtual hosting, forward-auth | Business routing, rate limiting |
| Authelia | SSO, OIDC provider, MFA | Per-route policies |
| APISIX | Routing, AuthN enforcement, rate limiting, policies | TLS (Caddy handles it) |
| etcd | APISIX config store — routes, upstreams, plugins | Serving traffic |
| Upstreams | Business logic | Auth, rate limiting |
| Zone | Network | Connections |
|---|---|---|
| Public | — | Cloudflare → Caddy :443 only |
| Service mesh | proxy (existing) | Caddy → APISIX :9080, Authelia :9091, Dashboard :9000 |
| Internal | gateway-internal (new, isolated) | APISIX ↔ etcd :2379 only — no internet access |
| Upstream | Host loopback | APISIX → services on 127.0.0.1:PORT |
| Admin (locked) | Container loopback | APISIX Admin API :9180 — 127.0.0.1 only, never routed |
allow 127.0.0.0/24; deny all — but a misconfigured Caddy block would still expose full gateway control to the internet. Access only from within the container network namespace.
| Requirement | Version / Value | Notes |
|---|---|---|
| Docker | 29.6.2 | User must be in docker group |
| Docker Compose | v5.3.1 | Bundled with Docker Engine |
| Server IP | 46.224.236.152 | Behind Cloudflare orange-cloud proxy |
| OS | Linux (Debian/Ubuntu) | Single VPS node |
| Subdomain | Type | Value | CF Proxy |
|---|---|---|---|
api.elfatih.net | A | 46.224.236.152 | ✅ Yes |
dashboard.elfatih.net | A | 46.224.236.152 | ✅ Yes |
auth.elfatih.net | A | 46.224.236.152 | ✅ Yes |
# Run as root or fatih: sudo usermod -aG docker hermes # Takes effect on next login
All five containers are running and verified. Admin API is functional. Authelia is healthy. etcd is persisting configuration.
/home/hermes/stack — not a bare ~/api-gateway/ — so Caddy, the gateway, and the static learn-site are all version-controlled together.
/home/hermes/stack/ # git repo root
├── .gitignore # excludes .env, oidc.key, db.sqlite3, logs, etcd_data
├── tests/
│ ├── test_config.py # 19 static config/policy tests
│ ├── test_integration.py # 6 tests — spins up the real stack, isolated project
│ └── docker-compose.test.yml # isolated compose project for integration tests
├── caddy/
│ ├── Dockerfile
│ ├── Caddyfile # live — 4 vhosts (learn/api/auth/dashboard)
│ └── .env # ACME_EMAIL, CLOUDFLARE_API_TOKEN
├── learn-site/
│ ├── index.html # learn.elfatih.net landing page
│ └── static/ # guides — this file included
└── api-gateway/
├── docker-compose.yml # apisix, apisix-dashboard, authelia, etcd services
├── .env # all gateway secrets — gitignored
├── apisix/
│ ├── config.yaml # APISIX main configuration
│ ├── dashboard.yaml.template # sed-rendered into conf.yaml at container start
│ └── etcd_data/ # etcd persistent data (gitignored)
└── authelia/
├── configuration.yml # Authelia config — client_secret & OIDC key inline
├── users.yml # user database — Argon2id hashed passwords
├── oidc.key # RSA 4096-bit private key (gitignored, unused — key is inline)
├── db.sqlite3 # session DB (gitignored)
└── notification.txt # filesystem notifier output (gitignored)Caddy and the gateway services all join a shared network named proxy so Caddy can reach them by container name. A separate gateway-internal network isolates etcd completely — it has internal: true, meaning no outbound internet access.
networks:
proxy:
name: proxy # keep same network name so other services are unaffected
driver: bridge
gateway-internal:
driver: bridge
internal: true # No internet — etcd + apisix internal comms onlyAPISIX uses etcd as its live configuration database. All routes, upstreams, and plugin configs are stored here and pushed to APISIX workers in real-time without restarts.
bitnami/etcd was removed from Docker Hub. The correct replacement is quay.io/coreos/etcd:v3.5.14 from the official CoreOS registry on Quay.io. Environment variable names also differ from the Bitnami image.
etcd:
image: quay.io/coreos/etcd:v3.5.14
container_name: etcd
networks: [gateway-internal] # Internal only — not reachable from proxy
environment:
ETCD_NAME: etcd0
ETCD_DATA_DIR: /etcd-data
ETCD_LISTEN_CLIENT_URLS: "http://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS: "http://etcd:2379"
ETCD_LISTEN_PEER_URLS: "http://0.0.0.0:2380"
ETCD_INITIAL_ADVERTISE_PEER_URLS: "http://etcd:2380"
ETCD_INITIAL_CLUSTER: "etcd0=http://etcd:2380"
ETCD_INITIAL_CLUSTER_STATE: new
ETCD_INITIAL_CLUSTER_TOKEN: gateway-etcd
volumes:
- ./apisix/etcd_data:/etcd-data
# Port 2379 NOT exposed to host — internal network onlynewgrp docker << 'EOF' docker exec etcd etcdctl endpoint health EOF # Expected: 127.0.0.1:2379 is healthy: successfully committed proposal
APISIX is the routing and plugin engine. It reads config from etcd and exposes port 9080 for proxied traffic from Caddy, and port 9180 for the Admin API (localhost-only, restricted by nginx allowlist).
deployment:
role: traditional
role_traditional:
config_provider: etcd
etcd:
host: ["http://etcd:2379"]
prefix: /apisix
timeout: 30
admin:
enable_admin: true
admin_listen:
ip: 0.0.0.0
port: 9180
admin_key:
- name: admin
key: ${{APISIX_ADMIN_KEY}} # native env var templating — confirmed working
role: admin
# 127.0.0.0/24 covers direct loopback; 172.16.0.0/12 covers Docker's
# default bridge subnets, since curl from the host gets hairpin-NATed
# to the bridge gateway IP before it reaches this container.
allow_admin:
- 127.0.0.0/24
- 172.16.0.0/12
apisix:
node_listen: 9080
enable_ipv6: false
plugins:
- openid-connect # Phase 2 — auth enforcement, not yet applied to routes
- limit-req # Phase 3 — rate limiting
- limit-count
- cors
- request-id
- proxy-rewrite
- response-rewrite
- prometheus # Phase 4 — metrics
- http-logger # Phase 4 — access logs
- redirect
- echo
# ... full plugin list in config.yaml127.0.0.1, so the default allow 127.0.0.0/24 allowlist rejected it. The fix was widening allow_admin in config.yaml to include 172.16.0.0/12 (Docker's default bridge range). A plain curl from the host now works directly.
source /home/hermes/stack/api-gateway/.env
curl -s http://127.0.0.1:9180/apisix/admin/routes \
-H "X-API-KEY: $APISIX_ADMIN_KEY"
# Expected: {"list":[],"total":0}
curl -s -X PUT http://127.0.0.1:9180/apisix/admin/routes/1 \
-H "X-API-KEY: $APISIX_ADMIN_KEY" -H "Content-Type: application/json" \
-d '{ YOUR_ROUTE_CONFIG }'docker run --rm --network container:apisix ... pattern (sharing APISIX's network namespace) still works as a fallback if allow_admin is ever tightened back down, but is no longer required day-to-day.
Web UI for managing routes, upstreams, and plugins visually. Protected by two layers: Authelia forward-auth (stops internet noise) and its own login (stops lateral movement).
${{VAR}} templating, the Dashboard image does not substitute environment variables into conf.yaml — this was assumed true from documentation, then disproved by actually logging in with the literal placeholder string and watching it succeed. The fix is a custom sed-based startup wrapper (below) instead of relying on native substitution.
conf:
listen:
host: 0.0.0.0
port: 9000
etcd:
endpoints: [etcd:2379]
authentication:
secret: __DASHBOARD_SECRET__
expire_time: 3600
users:
- username: admin
password: __DASHBOARD_PASS__apisix-dashboard:
image: apache/apisix-dashboard:3.0.1-alpine
volumes:
- ./apisix/dashboard.yaml.template:/usr/local/apisix-dashboard/conf/conf.yaml.template:ro
env_file:
- ./.env
entrypoint: ["sh", "-c"]
command:
- >
sed
-e "s#__DASHBOARD_SECRET__#$DASHBOARD_SECRET#"
-e "s#__DASHBOARD_PASS__#$DASHBOARD_PASS#"
/usr/local/apisix-dashboard/conf/conf.yaml.template
> /usr/local/apisix-dashboard/conf/conf.yaml
&& exec /usr/local/apisix-dashboard/manager-apiDASHBOARD_PASS in .env no longer holds the original deploy-time default.
Handles single sign-on with optional MFA and acts as the OIDC identity provider for APISIX's openid-connect plugin.
| Old key (deprecated) | New key used |
|---|---|
jwt_secret | identity_validation.reset_password.jwt_secret |
server.host / server.port | server.address: tcp://0.0.0.0:9091/ |
session.domain | session.cookies[].domain |
Global default_redirection_url | Must be inside session.cookies[] only — global is rejected |
identity_providers.oidc.clients[].id | ...clients[].client_id |
issuer_private_key | identity_providers.oidc.jwks[].key |
| Domain | Policy | Reason |
|---|---|---|
auth.elfatih.net | bypass | Login portal must be publicly reachable |
dashboard.elfatih.net | two_factor (MFA) | Admin UI — highest protection |
api.elfatih.net | one_factor | API — password sufficient |
AUTHELIA_<PATH> env var override convention works for scalar/map config fields (JWT secret, session secret, storage key, OIDC HMAC secret — see table below), but does not work for array-indexed fields like oidc.clients[0].client_secret or the JWKS entries, confirmed empirically. A key_path alternative for the JWKS private key was tried and reverted — this Authelia version (4.39.20) rejects it as an unrecognized config key. So client_secret and the RSA private key both stay inline in configuration.yml in plaintext, committed to git — an explicit, accepted trade-off rather than an oversight.
clients:
- client_id: apisix
client_name: APISIX Gateway
client_secret: "<inline plaintext value — see note above>"
authorization_policy: one_factor
redirect_uris:
- https://api.elfatih.net/callback
scopes: [openid, profile, email, groups]
userinfo_signed_response_alg: nonenewgrp docker << 'EOF' docker run --rm authelia/authelia:latest \ authelia crypto hash generate argon2 --password 'YourPassword' EOF # Copy the Digest: line into authelia/users.yml
openssl genrsa -out /home/hermes/stack/api-gateway/authelia/oidc.key 4096
The Caddy container is now part of the same repo at /home/hermes/stack/caddy/ and is live, handling TLS termination for all four subdomains — including a fourth vhost for the static learn-site that wasn't in the original plan.
# learn.elfatih.net — static site (added, not in original plan)
learn.elfatih.net {
root * /srv/learn
file_server
try_files {path} {path}.html {path}/index.html
encode gzip
header {
X-Content-Type-Options nosniff
X-Frame-Options SAMEORIGIN
Referrer-Policy strict-origin-when-cross-origin
-Server
}
log { output file /var/log/caddy/learn-access.log; format json }
}
# api.elfatih.net — APISIX gateway
api.elfatih.net {
reverse_proxy apisix:9080
header { -Server }
log { output file /var/log/caddy/api-access.log; format json }
}
# auth.elfatih.net — Authelia SSO portal
auth.elfatih.net {
reverse_proxy authelia:9091
header { -Server }
log { output file /var/log/caddy/auth-access.log; format json }
}
# dashboard.elfatih.net — APISIX Dashboard, behind Authelia forward-auth
dashboard.elfatih.net {
forward_auth authelia:9091 {
uri /api/verify?rd=https://auth.elfatih.net
copy_headers Remote-User Remote-Groups Remote-Name Remote-Email
}
reverse_proxy apisix-dashboard:9000
header { -Server }
log { output file /var/log/caddy/dashboard-access.log; format json }
}header { -Server } as a one-liner in three vhosts — Caddy's parser rejects content on the same line as an opening { for an explicit block ("Unexpected next token after '{' on same line"). This would have failed to load entirely on deploy. Fixed by reformatting to multi-line blocks (shown compactly above, but each must be its own block in the real file).
admin off, which disables the /metrics endpoint the Docker healthcheck depends on — every check failed with connection refused from container start. Removing admin off doesn't expose anything externally since port 2019 isn't published in docker-compose.yml. (2) Even after that fix, http://localhost:2019/metrics still failed — this container resolves localhost to IPv6 (::1) first, while Caddy's admin listener only binds IPv4 127.0.0.1. Fixed by removing admin off and changing the healthcheck target from localhost to 127.0.0.1 explicitly. Verified live: caddy now reports healthy.
docker exec caddy caddy reload --config /etc/caddy/Caddyfile
Secrets are generated with openssl rand and stored in api-gateway/.env, which is gitignored. Variable names now match each mechanism's exact required env-var path rather than generic names — this is the actual current .env:
| Variable | Purpose | Mechanism |
|---|---|---|
| APISIX_ADMIN_KEY | APISIX Admin API authentication | Native ${{VAR}} templating in config.yaml |
| AUTHELIA_IDENTITY_VALIDATION_RESET_PASSWORD_JWT_SECRET | Password reset tokens | AUTHELIA_<PATH> override |
| AUTHELIA_SESSION_SECRET | Session encryption | AUTHELIA_<PATH> override |
| AUTHELIA_STORAGE_ENCRYPTION_KEY | SQLite DB encryption | AUTHELIA_<PATH> override |
| AUTHELIA_IDENTITY_PROVIDERS_OIDC_HMAC_SECRET | OIDC token signing | AUTHELIA_<PATH> override |
| DASHBOARD_SECRET | Dashboard JWT secret | Custom sed startup wrapper |
| DASHBOARD_PASS | Dashboard login password | Custom sed startup wrapper |
configuration.yml, committed to git: APISIX_CLIENT_SECRET (OIDC client_secret) and the OIDC JWKS RSA private key. Both mechanisms for externalizing them were tried and failed for this Authelia version — see the callout in §3.6. This is a deliberate, accepted trade-off, not an oversight.
# Generate a new secret: openssl rand -hex 32 # For most secrets
tests/test_integration.py; fixed to load it from .env dynamically at test runtime).
caddy and authelia report (healthy)curl (fixed, see §3.4)learn.elfatih.net — HTTP 200, static site serving over real HTTPSapi.elfatih.net — HTTP 404 (expected — no routes configured yet, proves Caddy→APISIX proxying works)auth.elfatih.net — HTTP 200, Authelia portal servingdashboard.elfatih.net — redirects to Authelia when unauthenticated (forward_auth working)proxy network — Caddy can reach services by container nameopenssl rand and inject into config filesunittest suite (static config + live-stack integration).env, verifying each mechanism empiricallyopenid-connect plugin as a global ruleclient_id, client_secret, discovery endpointsource /home/hermes/stack/api-gateway/.env
cat > /tmp/global_plugin.json << EOF
{
"plugins": {
"openid-connect": {
"client_id": "apisix",
"client_secret": "$APISIX_CLIENT_SECRET",
"discovery": "https://auth.elfatih.net/.well-known/openid-configuration",
"scope": "openid profile email",
"bearer_only": false,
"redirect_uri": "https://api.elfatih.net/callback",
"logout_path": "/logout"
}
}
}
EOF
newgrp docker << SHELL
docker run --rm --network container:apisix \
-v /tmp/global_plugin.json:/tmp/p.json:ro \
python:3.11-alpine python3 -c "
import urllib.request
data = open('/tmp/p.json','rb').read()
req = urllib.request.Request(
'http://127.0.0.1:9180/apisix/admin/global_rules/1',
data=data, method='PUT',
headers={'X-API-KEY': '$APISIX_ADMIN_KEY', 'Content-Type': 'application/json'}
)
print(urllib.request.urlopen(req).read().decode())
"
SHELL| Policy | Config |
|---|---|
| Rate limiting | 1000 req/min per IP globally, per-route override |
| Request transformation | Strip internal headers, add X-Request-ID |
| CORS | Global allowed origins, per-route override |
| Timeout | 30s upstream default |
| Retry | 2 retries on 502/503 |
| Health checks | Passive, circuit breaker on 3 consecutive failures |
prometheus plugin → metrics at :9091/apisix/prometheus/metricshttp-logger plugin → JSON access logs with request_id, upstream, status, latency, consumerlog-rotate plugin127.0.0.1:PORT — never 0.0.0.01. Browser → HTTPS GET https://api.elfatih.net/v1/data 2. Cloudflare — DDoS filter, passes to origin 3. Caddy :443 — TLS terminated, X-Real-IP injected, passes to APISIX :9080 4. APISIX — openid-connect plugin checks for Bearer token or session cookie 5a. Token valid → proceed to route matching 5b. No token → 302 redirect to https://auth.elfatih.net 6. Authelia — login form → MFA → OIDC token issued 7. Authelia — 302 redirect back to https://api.elfatih.net/callback 8. APISIX — validates token, extracts sub/email/roles from claims 9. APISIX — limit-count check, X-Request-Id injected, route matched 10. APISIX — upstream selected, X-User/X-Email headers injected 11. Upstream service :PORT (127.0.0.1) — handles request 12. Response flows back: Upstream → APISIX → Caddy → Cloudflare → Browser
1. Browser → dashboard.elfatih.net
2. Caddy: forward_auth check → authelia:9091/api/verify
3. No session → redirect to https://auth.elfatih.net (MFA required)
4. Authenticated → Caddy proxies to apisix-dashboard:9000
5. Dashboard shows its own login form (second auth layer)
Two auth layers: Authelia stops internet noise.
Dashboard login stops lateral movement if Authelia is ever bypassed.| Layer | Control | Status |
|---|---|---|
| Cloudflare | DDoS, DNS-01 certs, origin IP hidden | ✓ Active |
| Caddy | TLS termination, HTTPS-only, forward-auth for dashboard | ✓ Active |
| Authelia | SSO, MFA, OIDC provider | ✓ Active |
| APISIX — AuthN | openid-connect plugin enforcement | → Phase 2 |
| APISIX — Rate limiting | 1000 req/min per IP globally | Phase 3 |
| Dashboard | Behind Authelia + own login, internal network only | ✓ Active |
| Admin API :9180 | 127.0.0.0/24 allowlist, never routed via Caddy | ✓ Active |
| etcd :2379 | Internal Docker network only, never exposed | ✓ Active |
| Upstreams | Bind to 127.0.0.1 only — enforced per service | Per service |
| Coraza WAF | Official APISIX plugin — ready to enable | Future |
| Wazuh SIEM | Consumes APISIX JSON logs — zero redesign needed | Phase 4 |
cd /home/hermes/stack/api-gateway newgrp docker << 'EOF' docker compose start # Start all docker compose stop # Stop all EOF
newgrp docker << 'EOF'
docker ps --format "table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}"
EOFnewgrp docker << 'EOF' docker logs apisix --tail 50 -f docker logs authelia --tail 50 -f docker logs etcd --tail 20 EOF
newgrp docker << 'EOF' docker exec etcd etcdctl snapshot save /etcd-data/backup-$(date +%Y%m%d).db EOF cp /home/hermes/stack/api-gateway/apisix/etcd_data/backup-*.db ~/backups/
# Generate hash newgrp docker << 'EOF' docker run --rm authelia/authelia:latest \ authelia crypto hash generate argon2 --password 'NewPassword' EOF # Append to /home/hermes/stack/api-gateway/authelia/users.yml — Authelia reloads automatically # newuser: # displayname: "New User" # password: "$argon2id$..." # email: [email protected] # groups: [admins]
docker exec caddy caddy reload --config /etc/caddy/Caddyfile
Run these checks in order — first from the host server, then from the internet. Every test has an expected result. If the result matches, the check passes. These manual checks were originally run and verified on 2026-07-20; Test 7 below has since changed (see the update note) after the underlying Admin API access bug was fixed.
source /home/hermes/stack/api-gateway/.env is run in your shell so $APISIX_ADMIN_KEY is available for the Admin API tests.
unittest suite (19 static config tests + 6 live-stack integration tests) that runs all of this automatically, including a full stack spin-up in an isolated Docker Compose project. See the Gateway Testing & Deployment Guide. The manual walkthrough below is kept for reference / learning the "why" behind each check.
newgrp docker << 'EOF'
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"
EOF(healthy). APISIX, etcd, and Dashboard show Up. Caddy shows Up (ignore the unhealthy flag — cosmetic only, traffic serves fine).
NAMES STATUS apisix Up 6 hours apisix-dashboard Up 6 hours etcd Up 6 hours authelia Up 6 hours (healthy) caddy Up 8 hours (unhealthy) ← cosmetic, traffic is fine
newgrp docker << 'EOF' docker exec etcd etcdctl endpoint health EOF
127.0.0.1:2379 is healthy: successfully committed proposal: took = Xms
newgrp docker << 'EOF' docker run --rm --network api-gateway_gateway-internal alpine \ wget -qSO /dev/null http://apisix:9080/ 2>&1 | head -3 EOF
HTTP/1.1 404 Not Found — correct. APISIX is running and responding. 404 means no routes are configured yet, which is expected at this stage.
source /home/hermes/stack/api-gateway/.env
newgrp docker << SHELL
docker run --rm --network container:apisix python:3.11-alpine \
python3 -c "
import urllib.request
req = urllib.request.Request(
'http://127.0.0.1:9180/apisix/admin/routes',
headers={'X-API-KEY': '$APISIX_ADMIN_KEY'}
)
print(urllib.request.urlopen(req).read().decode())
"
SHELL{"list":[],"total":0} — Admin API is authenticated and responding. Empty list is correct — no routes configured yet.
newgrp docker << 'EOF' docker run --rm --network proxy alpine \ wget -qO- http://authelia:9091/api/health EOF
{"status":"OK"}
newgrp docker << 'EOF' docker run --rm --network proxy alpine \ wget -qSO /dev/null http://apisix-dashboard:9000/ 2>&1 | head -3 EOF
HTTP/1.1 200 OK — Dashboard UI is serving.
source /home/hermes/stack/api-gateway/.env curl -s --max-time 3 http://127.0.0.1:9180/apisix/admin/routes \ -H "X-API-KEY: $APISIX_ADMIN_KEY"
403 Forbidden here, treating it as intended behavior. It was actually an unintended bug: the port is documented as "localhost only" but genuinely couldn't be reached from the host at all, even from loopback, because Docker's bridge NAT rewrites the source IP before it reaches APISIX. That's a usability bug, not a security feature — the real security boundary is that port 9180 is never published beyond 127.0.0.1 on the host (confirmed in Test 9/12 below), which internet clients can't reach regardless. allow_admin was widened to include the Docker bridge subnet, so this now correctly returns 200 with a valid key.
This confirms the full Admin API write path and etcd persistence in one test.
source /home/hermes/stack/api-gateway/.env
# Create a test route
newgrp docker << SHELL
docker run --rm --network container:apisix python:3.11-alpine \
python3 -c "
import urllib.request, json
data = json.dumps({
'uri': '/ping',
'name': 'test-ping',
'upstream': {'type':'roundrobin','nodes':[{'host':'127.0.0.1','port':9999,'weight':1}]}
}).encode()
req = urllib.request.Request(
'http://127.0.0.1:9180/apisix/admin/routes/test1',
data=data, method='PUT',
headers={'X-API-KEY':'$APISIX_ADMIN_KEY','Content-Type':'application/json'}
)
r = urllib.request.urlopen(req)
print('Create:', r.status)
# Read it back
req2 = urllib.request.Request('http://127.0.0.1:9180/apisix/admin/routes',
headers={'X-API-KEY':'$APISIX_ADMIN_KEY'})
import json as j; data2 = j.loads(urllib.request.urlopen(req2).read())
print('Routes in etcd:', data2['total'])
# Delete it
req3 = urllib.request.Request('http://127.0.0.1:9180/apisix/admin/routes/test1',
method='DELETE', headers={'X-API-KEY':'$APISIX_ADMIN_KEY'})
r3 = urllib.request.urlopen(req3)
print('Delete:', r3.status)
"
SHELLCreate: 200 Routes in etcd: 1 Delete: 200
Confirms only ports 80 and 443 are reachable from outside. Run from the VPS itself or any external machine.
for port in 80 443 9080 9091 9180 2379; do
result=$(timeout 2 bash -c "echo >/dev/tcp/46.224.236.152/$port" 2>&1 \
&& echo "OPEN" || echo "closed")
echo " Port $port: $result"
donePort 80: OPEN ← Caddy HTTP (redirects to HTTPS) Port 443: OPEN ← Caddy HTTPS Port 9080: closed ← APISIX proxy — internal only ✓ Port 9091: closed ← Authelia — internal only ✓ Port 9180: closed ← Admin API — internal only ✓ Port 2379: closed ← etcd — internal only ✓
docker-compose.yml and your VPS firewall rules. These ports must never be reachable from the internet.
Run these from your local machine (laptop / phone) — not from the VPS. They test the full path: internet → Cloudflare → Caddy → service.
# Run from your local machine
curl -s -o /dev/null -w "%{http_code} -> %{redirect_url}\n" \
http://46.224.236.152/308 -> https://46.224.236.152/ — Caddy is enforcing HTTPS.
curl -s -o /dev/null \
-w "HTTP %{http_code} | TLS: %{ssl_verify_result} | Time: %{time_total}s\n" \
https://learn.elfatih.net/HTTP 200 | TLS: 0 | Time: ~0.03s — TLS result 0 means certificate is valid. Sub-100ms response confirms Cloudflare CDN is active.
# Try to reach Admin API directly from your local machine curl -s --max-time 5 http://46.224.236.152:9180/apisix/admin/routes
Only run this after the Caddy vhosts are added and DNS records are created.
curl -s -o /dev/null -w "HTTP %{http_code}\n" https://auth.elfatih.net/HTTP 200 — Authelia login page is live.
Only run this after Phase 2 (openid-connect plugin enabled).
curl -s -o /dev/null -w "HTTP %{http_code} -> %{redirect_url}\n" \
https://api.elfatih.net/HTTP 302 -> https://auth.elfatih.net/... — unauthenticated requests redirect to Authelia.
| # | Test | Where | Expected | Result |
|---|---|---|---|---|
| 1 | All containers running | Host | 5 containers Up | ✓ Pass |
| 2 | etcd health check | Host | is healthy | ✓ Pass |
| 3 | APISIX proxy port 9080 | Host | HTTP 404 | ✓ Pass |
| 4 | Admin API authenticated | Host | {"list":[]} | ✓ Pass |
| 5 | Authelia health | Host | {"status":"OK"} | ✓ Pass |
| 6 | Dashboard UI reachable | Host | HTTP 200 | ✓ Pass |
| 7 | Admin API reachable from host curl | Host | HTTP 200 (was 403 — see update note) | ✓ Pass |
| 8 | Route create / persist / delete | Host | 200 / 1 / 200 | ✓ Pass |
| 9 | Only 80 & 443 open to internet | Host | 9080/9091/9180/2379 closed | ✓ Pass |
| 10 | HTTP → HTTPS redirect | Internet | 308 redirect | ✓ Pass |
| 11 | HTTPS valid + fast | Internet | 200, TLS valid, <100ms | ✓ Pass |
| 12 | Admin API blocked from internet | Internet | Timeout / refused | ✓ Pass |
| 13 | Authelia portal live | Internet | HTTP 200 | ✓ Pass — DNS live |
| 14 | API redirects to auth (Phase 2) | Internet | HTTP 302 → Authelia | Phase 2 |
docker compose pull fails — "not found"quay.io/coreos/etcd:v3.5.14. Environment variable names differ from Bitnami — updated config accordingly.open() "/usr/local/apisix/logs/error.log" failed (13: Permission denied)hermes (uid 1001), but APISIX runs as apisix (uid 636)docker logs apisix.Configuration: storage: option 'encryption_key' is requiredSTORAGE_KEY to .env (openssl rand -hex 32) and injected into configuration.yml.session: option 'cookies' must be configured with the per cookie option 'default_redirection_url' but the global one is configureddefault_redirection_url. In v4.38+ it must only appear inside session.cookies[].curl http://127.0.0.1:9180/apisix/admin/routes returns 403allow 127.0.0.0/24; deny all. Docker NAT changes the source IP to a bridge address, not loopback. Verified by checking the actual client IP APISIX logged.docker run --network container:apisix to execute requests from inside the container's network namespace where source IP is genuinely 127.0.0.1. Still works as a fallback.allow_admin in config.yaml to include 172.16.0.0/12 (Docker's default bridge range) alongside 127.0.0.0/24, so a direct host curl now works without the namespace workaround. Port 9180 is still bound to 127.0.0.1 only in docker-compose.yml, so it remains unreachable from the public internet.authelia/ directory owned by root after Docker volume operations wrote files as root, making them unreadable by the host userdocker run --rm -v .../authelia:/authelia alpine cp /src/file /authelia/fileFailingStreak: 317 — health check fails with "connection refused"/metrics endpoint, which was disabled via admin off in the Caddyfile. Port 2019 (admin API) isn't published in docker-compose.yml, so re-enabling it doesn't expose anything beyond the container itself.http://localhost:2019/metrics — this container resolves localhost to IPv6 (::1) first, while Caddy's admin listener only binds IPv4 127.0.0.1.admin off from the Caddyfile, and changed the healthcheck target from localhost to 127.0.0.1 explicitly. Verified live: caddy now reports healthy, not cosmetic-unhealthy./home/hermes/stack/caddy/Caddyfile, verified serving real trafficDASHBOARD_PASS in .env no longer the deploy-time defaultauthelia/users.yml (not verified this pass)openid-connect global rule on APISIX