# 1. Set credentials (see below)# 2. Verify everything is in place
./scripts/verify-secrets.sh
# Expected: ✅ All critical checks passed!# 3. Deploy
./scripts/setup.sh
./scripts/bootstrap.sh
Why: Backstage catalog refresh, ArgoCD repo sync, DORA metrics, GitHub Org
Team/User identity sync (catalog.providers.githubOrg — see
ADR-0004)
Scopes needed:repo, read:org, read:user, gist
read:user (or user:email) is easy to miss and the failure is non-obvious:
the org-sync GraphQL query always requests each member's email when
authenticating with a classic PAT, and GitHub rejects the whole query, not
just that field, without the scope — symptom is INSUFFICIENT_SCOPES in the
backend logs and zero synced Users/Groups (nobody can sign in, since sign-in
requires a synced User entity).
# Set in local/.envecho"GITHUB_TOKEN=ghp_YOUR_TOKEN">>local/.env
Also update Secrets Manager after bootstrap:
123456
# bootstrap.sh does this automatically if GITHUB_TOKEN is in local/.env# To update manually:
awssecretsmanagerget-secret-value--secret-ididp-mvp/backstage--regionus-east-1\--querySecretString--outputtext|python3-c"import sys,json,os; d=json.load(sys.stdin); d['GITHUB_TOKEN']=os.environ['GITHUB_TOKEN']; print(json.dumps(d))"|awssecretsmanagerput-secret-value--secret-ididp-mvp/backstage--secret-stringfile:///dev/stdin
Get it:
1. https://github.com/settings/developers → New OAuth App
2. Fill in:
- Application name:Backstage IDP
- Homepage URL:http://localhost:3000 (update after deploy)
- Callback URL:http://YOUR_BACKSTAGE_ALB_URL/api/auth/github/handler/frame (Use a placeholder — update after bootstrap prints the real ALB URL)
3. Copy Client ID and generate Client Secret
123
# Set in local/backstage/.envAUTH_GITHUB_CLIENT_ID=your-client-id
AUTH_GITHUB_CLIENT_SECRET=your-client-secret
After bootstrap: Update the OAuth app callback URL with the real ALB hostname.
Status: ☐ App created / ☐ Credentials set in local/backstage/.env / ☐ Callback URL updated post-deploy
3. ANTHROPIC_API_KEY (optional — required for AI features)¶
Why: Powers KAgent AI agents and the idp-assistant chatbot
# Set in local/.envANTHROPIC_API_KEY=sk-ant-YOUR_KEY
# Also create in Secrets Manager (bootstrap.sh reads from local/.env)
awssecretsmanagercreate-secret\--nameidp-mvp/kagent\--secret-string"{\"ANTHROPIC_API_KEY\":\"sk-ant-YOUR_KEY\"}"\--regionus-east-1
# Set in terraform/terraform.tfvars before running terraform applydatadog_api_key="YOUR_DD_API_KEY"datadog_app_key="YOUR_DD_APP_KEY"
Populates both idp-mvp/datadog (Datadog Agent) and idp-mvp/backstage (Backstage /datadog
proxy + dd-trace) Secrets Manager secrets — scripts/bootstrap.sh installs the Agent
automatically once these are set.
Why: the Backstage On-Call and Issues entity tabs. Without these the tabs render
their empty state; nothing else breaks.
Get it: a read-only PagerDuty REST API key, and for Jira a Base64 encoding of
email:api_token.
1234
# Set in terraform/terraform.tfvars before running terraform applypagerduty_token="YOUR_PAGERDUTY_READ_ONLY_KEY"jira_token="BASE64_OF_email:api_token"jira_url="https://your-company.atlassian.net"
All three land in the idp-mvp/backstage Secrets Manager secret and reach Backstage
on EKS as PAGERDUTY_TOKEN, JIRA_TOKEN and JIRA_URL. pagerduty_token and
jira_token default to REPLACE_ME; jira_url defaults to empty, which leaves
app-config.aws.yaml pointing at an RFC 2606 .invalid host that never resolves.
Tools:
[ ] aws sts get-caller-identity → works, shows correct account
[ ] terraform -version → ≥ 1.5
[ ] kubectl version → installed
[ ] helm version → ≥ 3.x
[ ] docker info → running, buildx available
[ ] gh auth status → authenticated
[ ] jq --version → installed
[ ] python3 --version → installed
AWS:
[ ] AWS region confirmed (default: us-east-1)
[ ] Sufficient EC2 quota: 4× t3.medium + 4× t3.large in your region
[ ] S3 Terraform state bucket + DynamoDB lock table created BY HAND
(nothing in the repo creates them — setup.sh does NOT, despite what
this line used to claim. terraform/main.tf hardcodes the bucket name
and table, so `terraform init` fails at Phase 1 without them.)
Credentials:
[ ] GITHUB_TOKEN set in local/.env (scope: repo, read:org, read:user)
[ ] AUTH_GITHUB_CLIENT_ID set in local/backstage/.env
[ ] AUTH_GITHUB_CLIENT_SECRET set in local/backstage/.env
[ ] ANTHROPIC_API_KEY in idp-mvp/kagent Secrets Manager (if using AI)
[ ] datadog_api_key / datadog_app_key set in terraform.tfvars (if using Datadog)
[ ] pagerduty_token set in terraform.tfvars (if using the On-Call tab)
[ ] jira_token / jira_url set in terraform.tfvars (if using the Issues tab)
Verification:
[ ] ./scripts/verify-secrets.sh passes with ✅ All critical checks passed!
The OAuth callback URL must exactly match the Backstage ALB URL:
123456
# Get the real URL
kubectlgetingressbackstage-nbackstage-ojsonpath='{.status.loadBalancer.ingress[0].hostname}'# Update your GitHub OAuth app at:# https://github.com/settings/developers → your app → Edit# Callback URL: http://<above-hostname>/api/auth/github/handler/frame
awssecretsmanagerget-secret-value--secret-ididp-mvp/kagent\--regionus-east-1--querySecretString--outputtext|jq.
# Verify ANTHROPIC_API_KEY value is correct (starts with sk-ant-)# Update if needed:
awssecretsmanagerput-secret-value\--secret-ididp-mvp/kagent\--secret-string'{"ANTHROPIC_API_KEY":"sk-ant-NEW_KEY"}'\--regionus-east-1
kubectlrolloutrestartdeployment/kagent-controller-nkagent
# 1. Personalise FIRST (first time only — replaces moatazeldebsy and the other# placeholders, creates the Terraform state bucket, writes .idp-config.env).# This has to come before verify-secrets.sh: that script looks secrets up by# "<cluster-name>/kagent" and friends, and the cluster name is one of the# values setup.sh substitutes. Run it the other way round on a fresh clone and# every check reports a missing secret under the template's default name.
./scripts/setup.sh
# → answer "skip" if you want to review before deploying# 2. Verify secrets (now that the cluster name is yours)
./scripts/verify-secrets.sh
# 3. Deploy the core platform (~40–70 min). AI/ML is opt-in:# --with-ai adds KAgent, MLflow, Langfuse and the MCP servers# --adp implies --with-ai and adds the agentic development platform# Without them, no Langfuse RDS or MLflow/Langfuse S3 buckets are provisioned# either, so skipping the AI layer is a real cost saving rather than just# skipping the workloads.
./scripts/bootstrap.sh
# 4. Update the GitHub OAuth callback URL with the printed ALB hostname# 5. Run validation
./scripts/validate-deployment.sh
To add the AI/ML stack after the fact, re-run with --with-ai (or --adp), or
call bootstrap-ai.sh directly. Note the --aws flag on the latter: without it
the script targets your local Kind context, not EKS.
12
./scripts/bootstrap.sh--with-ai# or --adp
./scripts/bootstrap-ai.sh--aws--adp--region<region>--cluster<cluster-name>
Removing AI infrastructure is deliberately explicit. Omitting --with-ai on a
cluster that already has it keeps it — otherwise a forgotten flag would drop
the Langfuse database. Pass --remove-ai-infra to actually destroy it.
See docs/DEPLOYMENT_GUIDE.md for full details, known issues, and troubleshooting.