The forgeops-ng command

The forgeops-ng command, the next generation of the forgeops command is now available in the technical preview status. The forgeops-ng command simplifies ForgeOps deployment, makes it more deterministic, and helps in managing the ForgeOps deployment life-cycle.

It lets you:

Use Kustomize natively so users can update and use overlays as expected
Generate a Kustomize overlay manually when you need the overlay
Build and manage Docker images per overlay to allow different images in an environment
Create and manage each ForgeOps deployment configuration
Apply the environment configuration changes using either Kustomize or Helm

In the technology preview, the name forgeops-ng is used. In the next major ForgeOps release, the -ng suffix may be dropped. Whereupon, this will be the forgeops replacing the current forgeops command. If you start using forgeops-ng before then, please be aware of and plan for that name change.

Features in forgeops-ng

Discrete overlays

The current forgeops command has the following limitations:

It generates a Kustomize overlay every time it runs.
It overwrites any post-deployment changes in Kustomize overlays.
It ignores the customizations in kustomization.yaml file during deployment. Instead, it uses the preconfigured patch files.

When using forgeops-ng, the overlay management is not automated, and needs to be manually triggered by an administrator.

It is recommended to create an overlay for each environment, such as test, stage, and prod. It is also recommended to create an overlay for each single-instance environment, such as test-single, stage-single, and prod-single. The single-instance overlays help you develop file-based configuration changes, export them, and build new images.

Per overlay image-defaulter: Each overlay includes an image-defaulter component. When using Kustomize, you can develop and build your images in your single-instance environment. Once you are happy with it, you can copy the image-defaulter’s kustomization.yaml file into your running overlay.
Sub-overlays: The overlays are composed of sub-overlays to install or delete individual components. Each platform product has its own sub-overlay, and there are sub-overlays to handle shared pieces. You can run kubectl apply -k or kubectl delete -k to apply or delete an overlay or a sub-overlay.
Require specifying the overlay: Another benefit of discrete overlays is the ability to target a specific overlay when running the forgeops-ng commands. If you forget to specify the overlay, the forgeops-ng command exits with a prompt to specify an overlay.

Setup

The forgeops-ng command is developed using Python. Run the forgeops-ng configure command to ensure the required packages are set up:

$ cd /path/to/forgeops/bin
$ ./forgeops-ng configure

Workflow

The workflow of forgeops-ng is designed to be production first. The previous forgeops tool is designed as a demonstration, and is not intended to be used in production. The field was seeking a production workflow and tooling to support it.

The new workflow has three distinct steps:

1. Create an environment:

This step is used to manage the overlay and values files on an ongoing basis. Only the requested changes are incorporated, so the customizations are not impacted.
2. Build images for an environment:

The build step assembles the file-based configuration changes into container images, and updates the image-defaulter and values files for the targeted environment.
3. Apply an environment:

The apply step lets you deploy the image you configured.

It is recommended that you start with a single-instance deployment to develop your AM and IDM configuration, so you can export them and build your custom container images.

1. Create an environment

The first thing you do is create an environment using the forgeops-ng env command. You need to specify an FQDN (--fqdn) and an environment name (--env-name).

Previously, the t-shirt sized overlays called small, medium, and large were provided, along with the default overlay cdk. With forgeops-ng, a single-instance overlay replaces cdk and is provided in the kustomize-ng/overlay/default directory.

You can use --small, --medium, and --large flags to configure your overlay, and the forgeops-ng env command populates your environment with the size you requested.

For example, the following command creates a medium-sized stage deployment with an FQDN of stage.iam.example.com:

$ cd /path/to/forgeops
$ ./bin/forgeops-ng env --fqdn stage.iam.example.com --medium --env-name stage

The default deployment size is single-instance. The following example command creates a single-instance environment: [Notice that --single-instance is not specified.]:

$ cd /path/to/forgeops
$ ./bin/forgeops-ng env --fqdn stage.iam.example.com --env-name stage-single

You will find the environment specification files in the ENV_NAME specific files. For example, in kustomize-ng/overlay/small and helm/small directories.

2. Build images for an environment

Use the forgeops-ng build command to create a new container image. The forgeops-ng build command applies the config profile from the build docker/am/config-profiles/profile and docker/idm/config-profiles/profile to build AM and IDM container images and push the images to your container registry. It also updates the image-defaulter and values files for the targeted environment.

To build new AM and IDM images for our stage environment using the stage-cfg profile, run the command:

$ ./bin/forgeops-ng build --env-name stage \
 --config-profile stage-cfg \
 --push-to my.registry.com/my-repo/stage am idm

You can then use forgeops apply or helm install to apply the environment via Helm or Kustomize to deploy.

3. Apply an environment

Use the overlay created with forgeops env command to deploy your environment:

In a Helm-based deployment, you specify the Helm values file in the helm install or helm upgrade command:
```
$ helm upgrade --install ...
```
In a Kustomize-based deployment, you have two options:
- Using the kubectl apply command, for example:
  $ kubectl apply -k /path/to/forgops/kustomize-ng/overlay/my-overlay
- Using the forgeops-ng apply command, for example:
  $ ./bin/forgeops-ng apply --env-name stage

Command reference

The forgeops-ng command is a Bash wrapper script that calls appropriate scripts in bin/commands. These scripts are written in either Bash or Python. All the bash scripts support the new --dryrun flag which display the command that would be run and enable you to inspect it before actually running the command. The Python scripts env and info do not support --dryrun.

Helm Support

Both Kustomize and Helm are supported by the forgeops-ng command. Use the forgeops-ng env command to generate Helm values file and Kustomize overlays for each environment. The forgeops-ng build command updates the Helm values file and the Kustomize image-defaulter overlay file for the specified environment.

The values.yaml file contains all the Helm values. The other files group the different values so that you can use them individually if you need to.

Custom paths

By default, forgeops-ng uses the docker, kustomize-ng, and helm directories. You can set up your own locations separately and specify the appropriate flags on the command line or set the appropriate environment variable in the path/to/forgeops/forgeops-ng.conf file.