Opscotch packager app in 3.1.1
Opscotch 3.1.1 standardizes app packaging around the packager app. This is the supported way to produce production-ready Opscotch app packages, and it is the path to future-safe packages going forward.
The packager is no longer just a CLI-shaped tool. It is an Opscotch app with an HTTP API.
The packager app is the supported app packaging mechanism for Opscotch.
In practice, that means:
- apps need to be packaged to run in a production runtime
- the packager app is the supported way to produce those packages
You can start the packager from your own Opscotch agent by referencing the .oapp from bootstrap, or by using the packaged Docker image that wraps that same setup. In this post, the examples use the Docker path because it is the easiest way to show the flow end to end.
What 3.1.1 provides
Opscotch 3.1.1 packaging is built around a different trust posture. It is designed so that a package can be created once, distributed to multiple deployments, and verified at deploy time.
The main outcomes are:
- multi-recipient packaging
- authenticated encryption
- signatures
- embedded licenses
- deploy-time verification
- package once, run many
This also means the end customer can assert package authorship, and can add their own signature endorsements and repackage when that is part of their delivery process.
Why this model exists
The goal is to make Opscotch app packages distributable, trustable, and secure without making packaging operationally awkward.
In plain English: the cryptographic parts are there so the package can be distributed broadly while still proving who packaged it, who signed it, and who it was encrypted for. That is what enables a package-once, distribute-many model instead of treating each deployment as an isolated packaging event.
Packager app (API-driven) quick flow
The quickest way to understand the packager is to run it and look at the main outcomes it exposes.
Start the Docker container
The packager app is published as an .oapp and can be referenced directly from bootstrap. For example, the app can be loaded from:
https://github.com/opscotch/opscotch-apps/releases/download/opscotch-packager-1.0/opscotch-packager-1.0.oapp
We also publish a self-contained Docker image, so for this post we will use the containerized path.
The packager needs access to the resource directories that workflow apps pull from, so those directories are mounted into the container and made available to the running packager app.
The opscotch-community/resources directory is one of those dependencies, so you need to have it downloaded locally before starting the container.
In most cases, you will also have your own resources in addition to the community resources. The Docker container makes provision for two additional resource paths, exposed as local-resources1 and local-resources2.
docker run \
-p 39575:39575 \
-v "<path to opscotch community>/resources:/opscotch-community-resources" \
-v "/path/to/local-resources-1:/local-resources1" \
-v "/path/to/local-resources-2:/local-resources2" \
ghcr.io/opscotch/opscotch-packager-standalone:latest
When the container starts, the packager will be available on port 39575.
Package a workflow
Use POST /workflow-package to package a workflow. This is the main packaging flow for workflows and the endpoint most readers should care about first.
Supporting functions around it are:
POST /workflow-compile: validate a workflow, remove comments, and load resourcesPOST /workflow-hash: return the workflow hash after validation, comment removal, and resource loadingPOST /workflow-inspect: inspect a packaged workflowGET /key-gen?purpose=...: generate keys for signing, authenticated encryption, anonymous encryption, or symmetric encryptionPOST /package-envelope: add an additional package envelope around an existing package
At a high level, this endpoint:
- prepares the workflow
- hashes it
- applies signatures
- encrypts for one or more recipients
- emits the packaged app artifact
The main outcome is an .oapp package file. In the request:
packageIdidentifies the package - this must match thepackaging.packageIdin the bootstrap. This asserts that the.oappis the expected packagekeysprovides the cryptographic keys referenced by the requestsignersdeclares which signing keys should signencryptionKeyslists the recipient key ids for the package, and the corresponding public keys are provided inkeyspackagerIdentityPrivateKeyidentifies the authenticated secret key used as the packager identity
POST /workflow-package uses a multipart form upload. In the curl example below, -F 'body=...' sends the JSON metadata, and -F 'stream=@...' uploads the workflow file that the packager will process. In the packaging flow, the response is the packaged artifact itself, so the example writes the output to an .oapp file.
Example:
curl "http://localhost:39575/workflow-package" \
-H "Accept: application/octet-stream" \
-F "body=$(cat <<EOF
{
"packageId": "my-app",
"keys": [
{
"id": "owner-key",
"purpose": "sign",
"type": "secret",
"keyHex": "<128-char signing secret key>"
},
{
"id": "recipient-1",
"purpose": "authenticated",
"type": "public",
"keyHex": "<64-char authenticated public key>"
},
{
"id": "packager-identity",
"purpose": "authenticated",
"type": "secret",
"keyHex": "<64-char authenticated secret key>"
}
],
"signers": [
{
"key": "owner-key",
"signText": "signed by owner"
}
],
"encryptionKeys": ["recipient-1"],
"packagerIdentityPrivateKey": "packager-identity"
}
EOF
)" \
-F 'stream=@/path/to/workflow.config.json;type=application/octet-stream' \
--output my-app.oapp
This my-app.oapp is now:
- packaged
- signed by
owner-keywhich is verifiable by the customer - encrypted and only openable by 'recipient-1' if and only if recipient-1 asserts that the package was produced by the 'packager-identity'
Encrypt bootstrap content
Use POST /bootstrap-encrypt when you need to encrypt bootstrap content.
In the request:
encryptionKeyslists the authenticated public key ids that should be able to decrypt it, and the corresponding public keys are provided inkeyspackagerIdentityPrivateKeyidentifies the authenticated secret key used to encrypt itkeysprovides the key material referenced by those idsstreamis the bootstrap content being encrypted
This endpoint validates the bootstrap payload and returns an encrypted bootstrap value.
Example:
curl "http://localhost:39575/bootstrap-encrypt" \
-H "Accept: application/octet-stream" \
-F "body=$(cat <<EOF
{
"keys": [
{
"id": "packager-identity",
"purpose": "authenticated",
"type": "secret",
"keyHex": "<64-char authenticated secret key>"
},
{
"id": "target1",
"purpose": "authenticated",
"type": "public",
"keyHex": "<64-char authenticated public key>"
},
{
"id": "target2",
"purpose": "authenticated",
"type": "public",
"keyHex": "<64-char authenticated public key>"
}
],
"packagerIdentityPrivateKey": "packager-identity",
"encryptionKeys": ["target1", "target2"]
}
EOF
)" \
-F 'stream=@./bootstrap.json;type=application/octet-stream'
This is the same multipart request pattern used in POST /workflow-package: JSON metadata in body and bootstrap content in stream.
The response is a Base64 encoded encrypted bootstrap value.
When using an encrypted bootstrap, you must provide a OPSCOTCH_BOOTSTRAP_SECRETKEY environment variable in this format <sender public key hex>/<decoder private key hex>
Encrypt string content
Use POST /string-encrypt when you need to encrypt a string. In the request:
encryptionKeyslists the authenticated public key id that should be able to decrypt it, and the corresponding public key is provided inkeyspackagerIdentityPrivateKeyidentifies the authenticated secret key used to encrypt itkeysprovides the key material referenced by those ids
string-encrypt is a single-recipient flow, so encryptionKeys should contain just one target id.
Example:
echo secret | curl "http://localhost:39575/string-encrypt" \
-H "Accept: text/plain" \
-F "body=$(cat <<EOF
{
"keys": [
{
"id": "packager-identity",
"purpose": "authenticated",
"type": "secret",
"keyHex": "<64-char authenticated secret key>"
},
{
"id": "target1",
"purpose": "authenticated",
"type": "public",
"keyHex": "<64-char authenticated public key>"
}
],
"packagerIdentityPrivateKey": "packager-identity",
"encryptionKeys": ["target1"]
}
EOF
)" \
-F 'stream=@-;type=application/octet-stream'
This is the same multipart request pattern used in POST /workflow-package: JSON metadata in body and encryption content in stream.
The response is an encoded string. In current packager versions this is a versioned prefix followed by Base64 data. It can be used in authentication host data properties and requires that you provide an OPSCOTCH_STRING_SECRETKEYS environment variable. Because you can have multiple encoded strings, they may have different decode keys, so this variable is a list of keys in this format <sender public key hex>/<decoder private key hex>,<another sender public key hex>/<another decoder private key hex>
Other available functions
The packager app also exposes:
GET /health: simple health checkPOST /build-workflows: build a workflow from a manifest that includes and customizes multiple workflow files
For this post, the important point is that the API is best understood by outcome:
- package a workflow
- encrypt bootstrap content
- encrypt string content
The remaining endpoints support those outcomes or help inspect results.