Administrators

A Phalanx administrator is someone who is responsible for defining a Phalanx environment, installing Phalanx in that environment, and maintaining the resulting cluster. Administrators operate infrastructure, manage secrets, and are involved in the deployment, configuration, and Argo CD synchronization of applications.

Initial install

• Phalanx requirements
• Hostnames and TLS
• Set up secrets management
• Installing a Phalanx environment

Procedures

• Upgrade windows and sequence
• Syncing Argo CD in an environment
• Sync secrets for an environment
• Audit secrets for an environment
• Updating the pull secret for an environment
• Migrating to the new secrets management system

Troubleshooting

• Troubleshooting the Rubin Science Platform
  • PostgreSQL cannot mount its persistent volume
  • Spawner menu missing images, nublado stuck pulling the same image
  • Spawner menu missing images, cachemachine stuck pulling the same image
  • Spawning a notebook fails with a pending error
  • User gets permission denied from applications
  • You need privileged access to the filestore
  • User pods don’t spawn, reporting “permission denied” from Moneypenny
  • Login fails with “bad verification code” error
  • User keeps logging in through the wrong identity provider

Infrastructure

• Using Google Kubernetes Engine
  • Getting GKE Kubernetes credentials
  • Managing GCP resources with Terraform
• Filestore
  • Privileged access to the filestore
• Kubernetes kubelet nodeStatusMaxImages setting for Nublado
  • Setting nodeStatusMaxImages
  • Background

Reference

• Command-line interface