CMP Development

PyPE is scheduled to go away in March 2024. 

UGS PyPE apps are to be moved to the new CMP platform before PyPE goes away.

There is thorough documentation for this, created by the CMP team at ITS. These are Cliff's Notes.

ITS documentation: /wiki/spaces/utcmp/pages/55773494

Local Development

Things you need on your computer to get started

• • Install Docker Desktop to run locally. Technically you should have a paid subscription. These are covered at the Pro level, and subscriptions are managed by the individual with ProCard access. (Talk to the business office people for ProCard access.)
  • MySQL connection: there's something in the Docker local dev GitHub readme that talks about the ISP(?) in Docker and the connection; without making this modification in Docker, you may get intermittent MySQL connection errors when starting up the local environment. If you get the errors, just try again; they're no big deal.

Things you need in your application to get started

• • To create a base image in Docker, you need multiple files added to a PyPE application
  • Because the base image is very generic, you need to make sure all extra packages you need are in the requirements.txt.
  • The Docker-local-dev-2023 branch of each of our projects should have what you need - except...
  • You'll need local_settings.env, which works like ye olde local_settings.py. Here's what should be in there, minus the actual values:

PYPE_DEPLOYED = False

LDAP_PASSWORD =
LDAP_BIND_DN = 


DOCREPO_PASSWORD =


SECRET_KEY =
NOTIFICATION_ENVIRONMENT = 'TEST'

PETITION_DB_USER = 
PETITION_DB_PASSWORD =
UGSDB_DB_USER =
UGSDB_DB_PASSWORD = 


SITE_DOMAIN = 'local.utexas.edu:8000'

Making it go

• • Open your terminal to point at the root level of the project
  • At the command line: make runserver

Deployment to the CMP

Extra stuff you need access to

• • Be aware: UGS uses Artifactory credentials connected to a guest EID in the ugst GitHub organization. This is to prevent disaster when John wins the lottery and Leslie moves to Spain. The credentials for the Guest EID are in Stache. If someone is reading this and we've already abandoned UC WebDev, the Guest EID is connected to the UGS WebDev email address in ID Manager.
  • You need to be in the AD group. Adding a layer of complexity, the UC WebDev AD group is "ugs", whereas the GitHub organization is "ugst."
  • You'll need to be able to access GitHub (ugst), Rancher, Splunk, Backstage, and Harbor. I think joining the AD group should cover this access; if not, ITS should take care of it.

Steps before deployment

GitHub

• • Set up the organization
    • There are unique tokens at the org level that get shared with selected repos. For UC, all repos have access.
      • GH_ACTIONS_PAT - associated with Guest EID 
      • PIP_CONFIG - associated with Guest EID
      • CMP_API_TOKEN - in Stache from CMP team
      • HARBOR_TOKEN - in Stache from CMP team
      • HARBOR_USER - in Stache from CMP team (ours was missing; it was emailed to me)
    • I don't think PIP_CONFIG will need repo-level changes, but it can be done at the repo level if needed. (Repo-level stuff supersedes org-level stuff in GitHub)
  • Set up the repo
    • Set up environments: development, quality, production

Your project's files

• • Add critical files to the project - some of these will help automate GitHub connecting the repo to other pieces of the CMP machinery. Some of these include:
    • .github directory - its contents are critical to GitHub allowing you to run a workflow (deploy) the project. This folder should be in Master to get seen by GitHub, even if you want to deploy a different branch. Adding the directory to Master won't harm Master
    • Workflow
    • Manifests/environment/properties.yml - this is where you specify the name of the secrets file that will be stored in Rancher, in the RUNTIME_SECRETS
      • Make sure to fill in everything you need for all environments (including prod!) in all properties.yaml files before you're ready to deploy to prod! This means deciding the name of your secrets file ahead of time.
    • catalog_info.yml
    • gunicorn.conf.py - this was added in Petitions to avoid inexplicable 504 errors for the first few hours after deployment to qual, and seemed to work.
  • Commit and push these changes to github. Remember that the .github directory and contents need to be in Master for things to work automagically

Ready to deploy

For PyPE apps

• • Route the application and environment to CMP from PyPE.

First-ever deployment for the repo and the environment: you still don't have secrets in Rancher

• • You still don't have secrets in Rancher, and Rancher can't create the box to keep the secrets until it knows such a repo/environment combo exists. This box is called the "namespace."
  • To get Rancher to make the namespace, you need to deploy in GitHub and let the deployment fail. Then Rancher will have a namespace for you to put the project secrets.
  • To deploy:
    • Go to the Repo's main page in GitHub, and click on the "Actions" tab, then select "Build and Deployment" on the left.
    • Then you want to "Run workflow" on the right. Clicking allows you to choose the branch and environment, and then run (or deploy). This will take a while.
  • Then go to Rancher and look for the secrets. From the upper left home menu, click on the UT thing, and then on more resources/core/secrets
  • Click "create", and choose opaque secret
  • Make sure you make the secret with the same name as the RUNTIME_SECRETS, and it's in the right namespace. New namespaces might not appear; SHIFT + refresh on the browser can populate them in the create dropdown.
  • Put the secret stuff from local_settings.env in here. It's the sorts of things like database names and passwords, SECRET_KEYs, and (AFAIK) NOTIFICATION_ENVIRONMENT, since that's set by environment.
  • Everything in the Rancher secrets is pulled in from settings.py with os.environ.get().
  • Now you have secrets in Rancher! 

Deployment

• • To deploy:
    • Go to the Repo's main page in GitHub, and click on the "Actions" tab, then select "Build and Deployment" on the left.
    • Then you want to "Run workflow" on the right. Clicking allows you to choose the branch and environment, and then run (or deploy). This will take a while, because it still hasn't built this image before.
    • Once you see a green check mark, you can click on "Deployments" on the left sidebar and watch the deployment complete
    • A green check mark here indicates completion.
    • If you encounter errors in Build and Deployment, click in to the failed part of the workflow for the stack trace.
    • If you update secrets in Rancher, I think you need to re-deploy to pull the new secrets in to the project.