Phalanx Git repository structure#

Phalanx is an open source Git repository hosted at lsst-sqre/phalanx. This page provides an overview of this repository’s structure, for both application developers and environment administrators alike. For background on Phalanx and its technologies, see Overview of the Phalanx platform concepts first.

Key directories#

applications directory#

Browse applications/ on GitHub

Every Phalanx application has its own sub-directory within applications named after the application itself (commonly the name is also used as a Kubernetes Namespace). A Phalanx application is itself a Helm chart. Helm charts define Kubernetes templates for the application deployment, values for the templates, and references to any sub-charts from external repositories to include as a sub-chart. See the Helm documentation for details on the structure of Helm charts.

Per-environment Helm values#

Phalanx Helm charts in Phalanx include the per-environment configuration, in addition to a common set of defaults. A chart’s defaults are located in its main values.yaml file. The per-environment values files, named values-environment.yaml, override those default values for the application’s deployment in the corresponding environments.

Applications based on third-party charts#

Note that some applications are based entirely (or primarily) on third-party open source charts. In this case, the application’s Helm chart includes that external chart as a dependency through its Chart.yaml. See the Helm documentation on chart dependencies.

environments directory#

Browse environments/ on GitHub

The environments directory is where environments are defined (an environment is a distinct Kubernetes cluster).

The environments/templates directory contains a Helm template per application, like this one for the noteburst application:

environments/templates/noteburst-application.yaml#
{{- if .Values.applications.noteburst -}}
apiVersion: v1
kind: Namespace
metadata:
  name: "noteburst"
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: "noteburst"
  namespace: "argocd"
  finalizers:
    - "resources-finalizer.argocd.argoproj.io"
spec:
  destination:
    namespace: "noteburst"
    server: "https://kubernetes.default.svc"
  project: "default"
  source:
    path: "applications/noteburst"
    repoURL: {{ .Values.repoUrl | quote }}
    targetRevision: {{ .Values.targetRevision | quote }}
    helm:
      parameters:
        - name: "global.host"
          value: {{ .Values.fqdn | quote }}
        - name: "global.baseUrl"
          value: "https://{{ .Values.fqdn }}"
        - name: "global.vaultSecretsPath"
          value: {{ .Values.vaultPathPrefix | quote }}
      valueFiles:
        - "values.yaml"
        - "values-{{ .Values.name }}.yaml"
{{- end -}}

The template defines a Kubernetes Namespace and an Argo CD Application for each Phalanx application. Application resources direct Argo CD to deploy and synchronize the corresponding application Helm chart from the Phalanx applications directory.

Notice that these templates are wrapped in a conditional, which controls whether an application is deployed in a given environment. The values.yaml file in the environments directory defines boolean variables for each application. Only some required applications are enabled by default; the rest are disabled by default. Each environment then has a file named values-environment.yaml that defines environment-specific settings and enables the applications that should be deployed to that environment.

installer directory#

Browse installer/ on GitHub

This directory contains a script named install.sh. The arguments to this are the name of the environment, the Vault RoleID, and the Vault SecretID (see Secrets management overview for more details on Vault). This installer script is the entry point for setting up a new environment. It can also be run on an existing environment to update it. See the environment bootstrapping documentation for details.

charts directory#

Browse charts/ on GitHub

This directory contains Helm charts shared by multiple Phalanx applications that are not generally useful enough to warrant separate publication in a proper Helm chart repository.

In some cases, several Phalanx applications should use common Helm templates to avoid duplication. The best way to do this within Helm is to use a subchart. This can be done by publishing a separate Helm chart in lsst-sqre/charts, but publication as a Helm chart implies that the chart may be useful outside of Phalanx. Sometimes these shared subcharts are merely artifacts of code organization and deduplication within Phalanx, and should not have an independent existence outside of Phalanx. In those cases, they’re maintained in the charts directory.

See Sharing subcharts between applications for details.

docs directory#

Browse docs/ on GitHub

This directory contains the Sphinx documentation that you are reading now. See Contributing to the documentation.

src directory#

Browse src/ on GitHub

This directory contains the source of the Phalanx command-line tool (see Command-line interface). The Python dependencies for that command-line tool are managed in the requirements directory.

starters directory#

Browse starters/ on GitHub

This directory contains templates for contributing new applications to Phalanx. See Create new Helm chart from template.

tests directory#

Browse tests/ on GitHub

This directory primarily contains tests for the Phalanx command-line tool. However, it also contains some tests written in Python that check the consistency of the Phalanx configuration, and therefore runs for all changes, not just those that change the source of the command-line tool.

Branches#

The default branch is main. This default branch is considered the source of truth for fullly synchronized Phalanx environments.

Updates to Phalanx are introduced as pull requests on GitHub. Repository members create branches directly in lsst-sqre/phalanx (see the Data Management workflow guide) External collaborators should fork Phalanx and create pull requests.

It is possible (particularly in non-production environments) to deploy applications from branches of Phalanx, which is useful for debugging new and updating applications before updating the main branch. You can learn how to do this in Deploying from a branch for development.

Test and formatting infrastructure#

The Phalanx repository uses two levels of testing and continuous integration.

Pre-commit performs file formatting, linting, and schema checking, both on your local editing environment (when configured) and verified in GitHub Actions. In one check, Pre-commit regenerates Helm chart documentation for applications with helm-docs. See the .pre-commit-config.yaml file for configuration details. Learn how to set up Pre-commit in your local editing environment in Setting up a Phalanx development environment.

Second, GitHub Actions runs a CI workflow (.github/workflows/ci.yaml). This workflow has four key jobs:

  • Linting with Pre-commit, mirroring the local editing environment.

  • Static validation of Helm charts with the helm/chart-testing-action GitHub action.

  • Python tests of both the Phalanx command-line tool and of the Phalanx configuration.

  • An integration test of a Phalanx environment in a minikube.

Next steps#

Start working with Phalanx:

  • If you are a developer looking to integrate your application into Phalanx, see the Developers section to get started.

  • If you are an administrator looking to create a new environment or operate an existing one, see the Administrators section.