Read-only audit

Blinkin URL Migration Findings

Prepared 2026-05-19 for the AWS to Hetzner migration rehearsal.

Situation

The current rehearsal surface is app-upco.blinkin.de. It can authenticate into the restored Blinkin tenant and show restored production data. The problem appears when opening a companion: generated links still point to blinkin.blinkin.io.

This is expected from the restored data model: published companion/app links use tenant.domain as the canonical public host.

Confirmed Root Cause

P0: The app-upco candidate resolves the Blinkin tenant, but that tenant still reports domain: blinkin.blinkin.io. That value drives companion URL generation and sends users back to the old AWS-backed host. 
{
  "id": 126,
  "name": "Blinkin",
  "domain": "blinkin.blinkin.io",
  "auth0Id": "org_O7TERzIkkx3IMBw8"
}

Inventory Snapshot

54 tenant rows in candidate DB
42 null or empty domain rows
12 tenant domains still on .blinkin.io
ID Tenant Current Domain Rehearsal Status
126Blinkinblinkin.blinkin.ioMap to app-upco.blinkin.de for candidate rehearsal.
133Wilo SEwilo.blinkin.ioNeeds explicit DNS, proxy, and Auth0 decision.
175Fraunhoferfraunhofer.blinkin.ioNeeds explicit DNS, proxy, and Auth0 decision.
217Telekomtelekom.blinkin.ioNeeds explicit DNS, proxy, and Auth0 decision.
220TAPGIGtapgig.blinkin.ioNeeds explicit DNS, proxy, and Auth0 decision.
221Fortinetfortinet.blinkin.ioNeeds explicit DNS, proxy, and Auth0 decision.
225Bosch Industriekesselbosch.blinkin.ioNeeds explicit DNS, proxy, and Auth0 decision.
231Acmedemo.blinkin.ioNeeds explicit DNS, proxy, and Auth0 decision.
232Huber & Rannerhuberranner.blinkin.ioNeeds explicit DNS, proxy, and Auth0 decision.
341Gebr Heinemannheinemann.blinkin.ioNeeds explicit DNS, proxy, and Auth0 decision.
345Periperi.blinkin.ioNeeds explicit DNS, proxy, and Auth0 decision.
347muehlhausmoersemmy.blinkin.ioNeeds explicit DNS, proxy, and Auth0 decision.

Live Boundary Checks

app-upco.blinkin.de -> 178.105.30.180
app-prod.blinkin.de -> 178.105.98.166
blinkin.blinkin.io -> AWS NLB / 63.178.127.147, 3.126.66.33, 18.199.229.247
telekom.blinkin.io -> AWS NLB / 63.178.127.147, 18.199.229.247, 3.126.66.33

https://app-upco.blinkin.de/__healthz -> 200 text/plain
https://app-upco.blinkin.de/houston/tenants/current -> 200 application/json
https://blinkin.blinkin.io/houston/tenants/current -> 200 text/html

GitHub Runbook Cross-Check

The exact runbook is now available as the authoritative checklist: Tenant Subdomains and Public Media Asset Fix Runbook. Initial GitHub checkout/search did not locate it by title, but the supplied text has been preserved in the repo evidence set and reconciled against the app-upco findings.

Found Doc What It Proves Limit
2026-05-09T19-14-55Z-apps-auth0-and-studio-media-fix.md Staging Auth0 callback and Studio media base were fixed for *.blinkin.de. It explicitly did not touch blinkin.io production Auth0, DNS, or runtime files.
2026-05-10-prod-clean-rebuild-runbook.md Prod rebuild must keep prod-only DB, storage, Auth0, domains, and secrets separate. DNS/Auth0 cutover is a separate approval gate, not part of the rebuild itself.
2026-05-10-staging-url-architecture.md Tenant hosts such as acme.blinkin.de represent real org/Auth0/tenant identity. The doc treats .de as safe staging/rebuild and .io as current production shape.
Cross-check: the supplied runbook is correct for the .de Hetzner production/candidate surface: public tenant hosts become <tenant>.blinkin.de, stale .blinkin.io generated links are eliminated, and media is resolved through the Azure account root. If the final business goal is to keep customer-facing .io, do not blindly apply the .io -> .de tenant-domain migration as the final cutover. Move DNS/proxy/Auth0 behind the .io hosts instead.

Runbook Checklist Added

The preserved checklist adds a second risk class to this migration: public media. The media fix is separate from tenant-domain canonicalization.

Risk Required Rule Verification
Tenant URLs For the .de clone, generated companion URLs must use the selected canonical suffix. SQL has zero %.blinkin.io rows; copied/opened companion URLs do not contain .blinkin.io.
Staging wildcard Prefer <tenant>.staging.blinkin.de unless Cloudflare routes <tenant>.blinkin.de to staging. dig and curl -I hit the intended staging VM, not production.
Media base Use https://picassoprod.blob.core.windows.net, not a value ending in /images. No public JSON or HTML contains /images/images/ or root-relative uploaded media paths.

Immediate Safe Fix For Rehearsal

Candidate-only dry-run SQL for the Blinkin tenant: 

BEGIN;

UPDATE tenant
SET domain = 'app-upco.blinkin.de'
WHERE id = 126
  AND domain = 'blinkin.blinkin.io';

SELECT id, name, domain, auth0_id
FROM tenant
WHERE id = 126;

-- ROLLBACK; -- keep this while reviewing
-- COMMIT;   -- only after human approval and companion-link verification

Do Not Bulk-Map Enterprises Yet

Risk: Changing telekom.blinkin.io to telekom.blinkin.de without routing proof can break enterprise login and public companion access.
  1. DNS must exist and point to the intended Hetzner candidate or proxy.
  2. Caddy/reverse proxy must accept that host and route app/API traffic correctly.
  3. Auth0 callback, logout, and web origin settings must allow that host.

Final .io Cutover Model

If the final goal is to keep customer-facing .io domains, the cutover is mainly DNS/proxy/Auth0 plus final sync, not a blind tenant-domain rewrite.

  1. Move .io DNS from AWS to Hetzner.
  2. Configure Hetzner Caddy/proxy for every .io host.
  3. Confirm Auth0 production settings for callbacks, logout URLs, and web origins.
  4. Freeze old production and perform one final database/media sync.