API Reference
Overview
This documentation describes the functionality that is available for configuration authors.
Please see the main sections
- Bootstrap schema: Documentation describing the bootstrap schema.
- Workflow schema: Documentation describing the workflow schema.
- Workflow Javascript Contexts: Documentation describing the javascript functionality for workflow execution.
- Testing schema: Documentation describing the testing schema.
- Error Codes: Documentation for all known opscotch error codes.
opscotch configuration model
The opscotch model is relatively straight forward, and uses repeated patterns - once you grasp these you should be able to understand the entire structure.
Here is the outline of the entire structure of the two configurations: the bootstrap and the workflow. Note that not every field is required, so be sure to check the detailed descriptions.
The bootstrap schema overview
[
{
"enabled" : true,
"agentPrivateKey": "",
"deploymentId": "",
"remoteConfiguration": "",
"remoteConfigurationAuth": "",
"remoteConfigurationTimeout" : 0,
"frequency": 0,
"reloadTimeout" : 0,
"keys" : [
{
"id" : "",
"keyHex" : "",
"metadata" :
"purpose" : "",
"type" : ""
}
],
"licenseHost" : "",
"licensePoolId" : "",
"packaging" : {
"additionalSigners" : [],
"packageId" : "",
"packagerIdentities" : [],
"requiredAdditionalSignerCount" : 0,
"requiredSigners" : [
{
"id" : "",
"description" : ""
}
]
},
"errorHandling": {
"enableLocalLogging": true,
"metrics": {
"enabled": true,
"routingToken": "",
"outputUrl": "",
"outputAuthorization": ""
},
"logs": {
"enabled": true,
"routingToken": "",
"outputUrl": "",
"outputAuthorization": ""
},
"redactionPatterns": [
""
]
},
"workflow" : {
"metricOutput": {
"enabled": true,
"routingToken": "",
"outputUrl": "",
"outputAuthorization": ""
},
"outputs": [
{
"id": "",
"routingToken": "",
"type": "",
"outputUrl": "",
"typeProperties": {
"": {
"": ""
}
}
}
],
"errorHandling": {
"enableLocalLogging": true,
"metrics": {
"enabled": true,
"routingToken": "",
"outputUrl": "",
"outputAuthorization": ""
},
"logs": {
"enabled": true,
"routingToken": "",
"outputUrl": "",
"outputAuthorization": ""
}
}
},
"allowDeploymentAccess" : [
{
"id" : "",
"deploymentId" : "",
"access" : "",
"anyDeployment" : false
}
],
"allowExternalHostAccess": [
{
"authenticationHost" : true,
"id" : "",
"host": "",
"httpTimeout" : 0,
"transport": "",
"inProcServerId": "",
"inProcDeploymentId": "",
"headers": {
"": ""
},
"allowList" : [
{
"method" : "",
"uriPattern" : ""
}
],
"data" : {}
}
],
"allowFileAccess": [
{
"id" : "",
"directoryOrFile" : "",
"patterns" : [""],
"LIST" : true,
"READ" : true,
"WRITE" : true,
"DELETE" : true
}
],
"allowHttpServerAccess": [
{
"id" : "",
"port" : 0,
"tcp" : false,
"inProcOnly" : false,
"serveFromPackagedAsset" : {
"packagedAssetFileId": ""
}
}
],
"persistenceRoot" : "",
"data": {},
"agentControl" : {
"workflowCanShutdownAgent": true
}
}
]
The workflow schema overview
{
"workflows": [
{
"enabled": true,
"name": "",
"steps": [
{
"enabled": true,
"debug": true,
"type": "",
"stepId": "",
"trigger" : {
"deploymentAccess" : {
"ids" : [ "" ],
"noTrace" : false
},
"fileWatcher": {
"bootstrapFileId": "",
"eventSplitter": "",
"patterns": [ "" ],
"splitAtEnd": false
},
"tcp": {
"server" : "",
"delimiter" : "",
"stripDelimiter" : true
},
"http": {
"method": "",
"path": "",
"server" : ""
},
"timer" : {
"delay" : 0,
"period" : 0
},
"runOnce" : true
},
"authenticationProcessor": {
"script": "",
"resource": "",
"processors" : [
{
"script": "",
"resource": ""
"data": {}
}
],
"data": {}
},
"splitGenerator": {
"script": "",
"resource": "",
"processors" : [
{
"script": "",
"resource": ""
"data": {}
}
],
"data": {}
},
"urlGenerator": {
"script": "",
"resource": "",
"processors" : [
{
"script": "",
"resource": ""
"data": {}
}
],
"data": {}
},
"payloadGenerator": {
"script": "",
"resource": "",
"processors" : [
{
"script": "",
"resource": ""
"data": {}
}
],
"data": {}
},
"itemResultProcessor": {
"script": "",
"resource": "",
"processors" : [
{
"script": "",
"resource": ""
"data": {}
}
],
"data": {}
},
"resultsProcessor": {
"script": "",
"resource": "",
"processors" : [
{
"script": "",
"resource": ""
"data": {}
}
],
"data": {}
},
"httpStatusHandling": {
"" : {
"script": "",
"resource": "",
"processors" : [
{
"script": "",
"resource": ""
"data": {}
}
],
"data": {}
}
},
"httpConnectFailedProcessor": {
"script": "",
"resource": "",
"processors" : [
{
"script": "",
"resource": ""
"data": {}
}
],
"data": {}
},
"httpTimeout" : 0,
"singleThreaded" : "",
"persistenceFile" : "",
"data": {}
}
],
"data" : {},
"defaultStepProperties" : {
< ... any step properties ... >
}
}
],
"defaultStepProperties" : {
< ... any step properties ... >
}
}
Bootstrap schema
JSON Schema in the Bootstrap model:
AgentControl
Controls behaviour that allows workflows to influence the agent lifecycle.
JSON Properties
The following properties are available to set on the AgentControl JSON schema.
workflowCanShutdownAgent
Allow workflows to request that the agent shuts down.
Defaults to false
{
"workflowCanShutdownAgent": true
}
Bootstrap
A configuration that is loaded into the agent at startup and determines how the agent loads the main configuration is loaded.
Please note that the bootstap file that is authored is an ARRAY of these objects ie:
[
{
"agentPrivateKey": ...
"deploymentId": ...
}
]
It will be provided to the agent as an argument, either via a base64 encoded json text or a path to a json file
JSON Properties
The following properties are available to set on the Bootstrap JSON schema.
agentControl
Controls what workflows are permitted to do to the running agent.
{
"agentControl": {
"workflowCanShutdownAgent": true
}
}
agentPrivateKey
Required The private key for the agent to decrypt the remote configuration.
This should be a base64 encoded private key in pkcs8 format
This is how to create an appropriate key
1. Create key pair
openssl genrsa -out keypair.pem 2048
2. Extract public part
openssl rsa -in keypair.pem -pubout -out public.key
3. Extract private part
openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in keypair.pem -out private.key
4. Base64 the contents of this file
cat private.key | base64 -w0
{
"agentPrivateKey": "LS0tLS1CRUdJTiBQUklWQV...."
}
allowDeploymentAccess
A list of Deployment Access Permitters that describe cross-deployment access within the same agent.
{
"allowDeploymentAccess": [
{
"id" : "appBridge",
"deploymentId" : "remote-deployment",
"access" : "call"
},
{
"id" : "receiveAny",
"access" : "receive",
"anyDeployment" : true
}
]
}
References:
- (Blog)
allowExternalHostAccess
A list of Host objects that describes access to external http(s) services that can be called from the workflow configuration.
{
"allowExternalHostAccess": [
{
"id" : "myHost",
"host": "https://www.example.com",
"headers": {
"Content-Type": "application/json;charset=UTF-8",
"Accept": "application/json, text/plain, *\/*"
},
"allowList" : [
{ "method" : "GET", "uriPattern" : "/this/is/allowed.*"},
{ "method" : "POST", "uriPattern" : "/this/.*\/allowed"}
],
"data" : {
}
}
}
]
References:
allowFileAccess
A list of File Permitters that describe access to files and directories.
When a directory is specified the permissions apply to all files in that directory.
Available permissions are:
- LIST
- READ
- WRITE
- DELETE
{
"allowFileAccess": [
{
"id" : "myTxtFiles",
"directoryOrFile" : "/path/to/a/directory",
"pattern" : ".*\\.txt",
"READ" : true
}
]
}
allowHttpServerAccess
data
A JSON Object for adding properties.
These properties will be merged with the WorkflowConfiguration.data configuration field and be available to all child elements in the main configuration tree.
{
"data": {
"somedata" : {
"abc: 123
},
"someotherdata" : [ 1, 2, 3]
}
}
deferLoading
When set to true, step executors are initialized lazily to reduce startup time for large configurations.
At startup, only trigger wiring is performed. The rest of step setup is deferred until first execution.
Defaults to false
{
"deferLoading" : true
}
References:
- (Blog)
deploymentId
Required A Service Provider defined value that must be unique per deployed agent configuration.
This property is used by Service Provider downstream processor to enrich the transmitted data.
{
"deploymentId": "a7d2f8"
}
enabled
Disable is bootstrap by setting enabled : false
Defaults to true
{
"enabled": false
}
errorHandling
Determines the behavior of error handling while loading the main configuration
{
"errorHandling": {
...
}
}
frequency
Sets how often the agent checks for configuration updates in milliseconds.
For URL Bootstrap.remoteConfiguration, this is the HTTP polling period. For file Bootstrap.remoteConfiguration, this controls whether the file watch/update mechanism is enabled.
Setting frequency to 0 disables all configuration updating after startup.
In this mode the configuration is loaded once at startup and no HTTP polling or file watching is started.
Defaults to 60000 (60 seconds)
{
"remoteConfiguration": "http://www.example.com",
"remoteConfigurationAuth" : "xodsjfujrhdjfk",
"frequency" : 60000
}
References:
- (Blog)
keys
Optional List of keys made available to the agent.
The set of keys will be made available to the agent for cryptographic functions like encryption and signing.
The keys can be used by the packaging subsystem and can also be made available in workflows via the crypto context
{
"keys": [
{
"id" : "mySignKey",
"type" : "secret",
"purpose" : "sign",
"keyHex" : "DEADBEEF"
}
]
}
licensing
Define licensing options
{
"licenseHost" : "https://your.opscotch.license.app",
"licenseHostPoolId" : "anotherPool"
}
packaging
Used to declare the required packaging authenticity before deploying the package.
{
"packaging": {
"packageId" : "thePackage"
}
}
persistenceRoot
Defines the filesystem storage root for persistence
{
"persistenceRoot": "/storage"
}
reloadTimeout
Defines the maximum time in milliseconds to wait for a workflow to terminate running flows before loading a new version
Defaults to 10000 (10 seconds)
{
"reloadTimeout": 1000
}
remoteConfiguration
Required A URL or file path to the main configuration.
The main configuration does not exist, the agent will wait until it becomes available. When referencing a file, its important to remember that the path is relative to the opscotch working directory, NOT the bootstrap file
{
"remoteConfiguration": "http://www.example.com",
"remoteConfigurationAuth" : "xodsjfujrhdjfk",
"frequency" : 60000
}
remoteConfigurationAuth
When using a URL remoteConfiguration, setting this property will set the "Authorization" header
{
"remoteConfiguration": "http://www.example.com",
"remoteConfigurationAuth" : "xodsjfujrhdjfk",
"frequency" : 60000
}
remoteConfigurationTimeout
When using a URL remoteConfiguration, setting this property will set the http timeout period in milliseconds
When frequency > 0, remoteConfigurationTimeout must be less than frequency.
Defaults to 1000 (1 second)
{
"remoteConfiguration": "http://www.example.com",
"remoteConfigurationAuth" : "xodsjfujrhdjfk",
"frequency" : 60000,
"remoteConfigurationTimeout" : 1000
}
startupPriority
Controls startup activation ordering across deployments in the same bootstrap file. Lower numbers activate first.
Defaults to 100
{
"startupPriority" : 1
}
References:
- (Blog)
workflow
Determines metric and log outputs for the workflows
{
"workflow": {
...
}
}
DeploymentAccessPermitter
Defines cross-deployment access within the same agent.
Use access "call" on the caller deployment and "receive" on the receiver deployment.
JSON Properties
The following properties are available to set on the DeploymentAccessPermitter JSON schema.
access
Required Access mode: "call" or "receive".
{
"access": "call"
}
anyDeployment
When access is "receive", allow calls from any deployment.
{
"anyDeployment": true
}
Defaults to false
deploymentId
Required when access is "call". The deploymentId of the remote bootstrap. When access is "receive" and anyDeployment is true, this can be omitted.
{
"deploymentId": "remote-deployment"
}
id
Required A unique string that identifies this deployment access link.
{
"id": "appBridge"
}
startupWaitTimeoutMs
When access is "call", maximum time in milliseconds to wait for the target deployment to finish startup loading when it is still initializing.
Defaults to 1000
ErrorHandling
A configuration defining if and how to send error metrics or log to a remote system
JSON Properties
The following properties are available to set on the ErrorHandling JSON schema.
enableLocalLogging
Enable or disables logging of errors and information to the agent standard out
Defaults to true
{
"enableLocalLogging": true
}
logs
Defines how error logs are sent to a remote system.
Error logs are brief messages from a fault in the agent and are sent to a remote system for attention.
Error log configuration only apply to the process configuration they are defined for. For example the Bootstrap.errorHandling defines error handling while loading configurations, and the WorkflowOutputs.errorHandling defines error handling during the execution of configuration steps.
{
"logs": {
...
}
}
metrics
Defines how error metrics are sent to a remote system.
Error metrics represent a fault in the agent and are sent to a remote system for attention.
Error metric configuration only apply to the process configuration they are defined for. For example the Bootstrap.errorHandling defines error handling while loading configurations, and the WorkflowOutputs.errorHandling defines error handling during the execution of configuration steps.
{
"metrics": {
...
}
}
redactionPatterns
Defines regex patterns to apply redaction to log statements
Log statements from will have these redaction patterns applied before logging.
{
"redactionPatterns": [
"[0-9]+"
]
}
FilePermitter
Describes controlled access to files and directories.
NOTE: The permission constraints are additional to the OS filesystem permissions: setting a file as readable here does not imply that it is readable from the filesystem.
Regular Expressions can be used to further restrict access to specific files.
JSON Properties
The following properties are available to set on the FilePermitter JSON schema.
DELETE
When true files will be deletable.
Defaults to false
{
"DELETE": true
}
directoryOrFile
A path to a directory or file.
The directory or file does not need to exist. If a directory is specified it must end in a / (or the OS file separator)
Directory example:
{
"directoryOrFile": "/path/to/directory/"
}
File example:
{
"directoryOrFile": "/path/to/directory/or/file"
}
id
An id that must be unique to this bootstrap.
Use this id in the workflow step file trigger.
{
"id": "myTextFiles"
}
LIST
When true files will be listable.
Defaults to false
{
"LIST": true
}
patterns
A list of (Java) Regular Expressions that can be used to further restrict access to files in the directory
{
"patterns": [
".*\\.txt$",
".*\\.log$"
]
}
READ
When true files will be readable.
Defaults to false
{
"READ": true
}
required
When true the file or directory must exist otherwise an error is thrown.
Defaults to false
{
"required": true
}
WRITE
When true files will be writable.
Defaults to false
{
"WRITE": true
}
Host
A HTTP(S) host that the agent/app is expected to communicate with. Apps CAN NOT call any host not referenced by a Host record.
JSON Properties
The following properties are available to set on the Host JSON schema.
allowList
A list of allowed paths that may be called on this host.
A list of regex that will be matched to URL path. Matching results will be allowed, non-matching results will be blocked and logged.
{
"allowList": [
{ "method" : "GET", "uriPattern" : "/this/is/allowed"},
{ "method" : "POST", "uriPattern" : "/this/.*\/allowed"}
]
}
authenticationHost
When set to true the host can only be used by an authentication context,
when set to false (default) the host can only be used by a non-authentication context.
{
"authenticationHost": true
}
Defaults to false
data
A JSON Object with Additional properties that will accessible during authentication steps.
You can use environment variables in the form of ${NAME} anywhere in the data structure:
the environment variable will be substituted prior to parsing.
Strings in the data structure (after environment variable substitution) can be encrypted using the opscotch packager. Values will be decrypted at access time.
Encryption is done via the packaging tool and requires the agent public key.
{
"data": {
"somedata" : {
"abc: 123
},
"someotherdata" : [ 1, 2, 3],
"someencryptedstring" : {
"abc" : "some encrypted value"
}
}
}
headers
A object of HTTP headers to be sent with every call.
{
"headers": {
"Content-Type": "application/json;charset=UTF-8",
"Accept": "application/json, text/plain, *\/*"
}
}
host
Required when transport = "http" A URL that is used as the base for constructing URLs to call.
{
"host": "https://www.example.com:8080"
}
httpTimeout
Default http timeout period in milliseconds for steps that call this host.
{
"httpTimeout": 10000
}
id
Required A unique string that identifies this host in other configurations.
{
"id": "myHost"
}
inProcDeploymentId
When transport is inProc, optionally route to a different deploymentId.
{
"inProcDeploymentId": "remote-deployment"
}
References:
- (Blog)
inProcServerId
When transport is inProc, identifies the bootstrap http server to route to internally.
{
"inProcServerId": "myAPI"
}
References:
- (Blog)
transport
HostAllow
Defines allowed http access patterns
JSON Properties
The following properties are available to set on the HostAllow JSON schema.
method
HTTP Method to match on. One of:
- GET
- HEAD
- POST
- PUT
- DELETE
- OPTIONS
- PATCH
{
"method": "GET"
}
uriPattern
Regex pattern to match on the URI.
{
"uriPattern": ".*"
}
HttpServerPermitter
Creates an server bound to the specified port.
Steps for this bootstrap using a http or tcp trigger can bind to this server.
A special feature is being able to serve static content from a zip file - see serveFromPackagedAsset
JSON Properties
The following properties are available to set on the HttpServerPermitter JSON schema.
bindAddress
Bind address for the server.
Defaults to "127.0.0.1", use "0.0.0.0" for all interfaces
{
"bindAddress": "127.0.0.1"
}
id
Required A unique string that identifies this server in other configurations.
{
"id": "myAPI"
}
inProcOnly
When true, this server will not bind to a network port and will only be available for in-process routing.
{
"inProcOnly": true
}
Defaults to false
References:
- (Blog)
port
Required when inProcOnly false. A port on the host that this server will bind to. Normal port binding rules apply.
{
"port": 1234
}
serveFromPackagedAsset
Describes how to serve content from a zip file.
{
"serveFromPackagedAsset": {
...
}
}
tcp
Enables raw TCP triggers on this port when set to true.
Defaults to false
{
"tcp": true
}
References:
- (Blog)
Key
Defines a cryptographic key made available to agent internal cryptographic functions (encryption and signing) and also to workflow/app cryptographic functions.
Keys must have a specified purpose and type that imply the required key length, which is verified before use.
Keys defined in the bootstrap are made available to the workflow context and also to internal agent operations like packaging and security.
Keys can be added to a workflow context, during a workflow context, via the crypto context.registerKey(...).
Once a key is added, it cannot be retrieved or removed, except for keys added during a workflow context are removed when that context ends.
When using keys, they can be selected by id or purpose/type. Keys are only ever used by reference and are never readable in the workflow context.
See the crypto context for more information.
JSON Properties
The following properties are available to set on the Key JSON schema.
id
Required A unique identifier used to reference a specific key
{
"id" : "thisIsMyKey"
}
keyHex
Required The hex encoded key
metadata
Additional metadata used solely for ease of key management in the bootstrap. This metadata is not loaded into the agent.
purpose
Required The intended purpose of the key.
The purpose is used to filter and verify the key properties for the intended usage.
Values can be one of:
sign: used for signing or signing verification- primitive: Ed25519
- secret key length: 64
- public key length: 32
authenticated: used for mutually authenticated encryption- primitive: X25519 (key exchange) + XSalsa20-Poly1305
- secret key length: 32
- public key length: 32
symmetric: used for symmetric encryption- primitive: XSalsa20-Poly1305
- secret key length: 32
anonymous: used for anonymous public key encryption- primitive: X25519 (key exchange) + XSalsa20-Poly1305
- secret key length: 32
- public key length: 32
{
"purpose" : "sign"
}
type
Required public or secret corresponding to the purpose
{
"type" : "secret"
}
Licensing
Define how to obtain a license for the workflow.
Licenses are issued to your organization by an opscotch license vendor.
Organization licenses are installed into an opscotch licensing app which might be separate service or might be running on the same agent as your workflows.
See the opscotch licensing app for more information.
References:
- (Blog)
JSON Properties
The following properties are available to set on the Licensing JSON schema.
basicAuth
Set the licensing basic auth header
{
"basicAuth" : "bm90IHNlY3JldA=="
}
licenseHost
An http(s) host running an opscotch licensing app where licenses can be obtained from.
Defaults to null. When null, remote licensing setup is not configured for the deployment.
{
"licenseHost" : "https://your.opscotch.license.app"
}
licenseHostPoolId
A licensing pool id configured on the opscotch licensing app.
Defaults to "default".
{
"licenseHostPoolId" : "anotherPool"
}
Output
Configuration for how to send data to a remote system.
JSON Properties
The following properties are available to set on the Output JSON schema.
enabled
Enables or disables the sending of data.
Defaults to false
{
"enabled": true
}
id
Optional A unique id used to reference this output from workflows.
{
"id": "my-output"
}
outputAuthorization
Set the "Authorization" header.
{
"outputAuthorization" : "xodsjfujrhdjfk"
}
outputUrl
Required The URL that data is transmitted to.
{
"outputUrl": "http://www.example.com"
}
persistenceRoot
If set, defines the location to store output queue files
{
"persistenceRoot" : "/tmp/output-queues"
}
routingToken
Required A Service Provider defined value that must be unique per deployed agent.
This property is used by Service Provider downstream processor to direct and enrich the transmitted data.
{
"routingToken": "dogurj"
}
type
Optional The output type. When set to otel, an OTEL sender is used.
{
"type": "otel"
}
typeProperties
Optional Type-specific properties. When a type is set, its properties should be stored under a key matching the type name.
{
"type": "otel",
"typeProperties": {
"otel": {
"aProperty": "aValue"
}
}
}
Packaging
Packaging applies required security constraints to package (app) loading.
Packages can be signed by the packaging entity and the signatures can be verified before loading. Signatures can be required or optional, and can have a required number of signatures.
Packages can be encrypted with mutually authenticated encryption meaning that the packager added keys specifically for this deployment, and this deployment has the matching key.
JSON Properties
The following properties are available to set on the Packaging JSON schema.
additionalSigners
Optional Used in conjunction of the requiredAdditionalSignerCount, define a list of additional signers by id that can also sign the package.
The id must match a single key id in keys.
This property is used to provide a pool of additional signers to sign the package.
{
"keys" : [
{
"id" : "trusted-party-1",
"purpose" : "sign",
"type" : "public",
"keyHex" : "..."
},
{
"id" : "trusted-party-1",
"purpose" : "sign",
"type" : "public",
"keyHex" : "..."
}
],
"packaging" : {
"additionalSigners": [
{
"id": "trusted-party-1",
"description" : "A trusted party"
},
{
"id": "trusted-party-2",
"description" : "Another trusted party"
}
]
}
}
packageId
Optional However, if using opscotch packaging app then its required. The packageId must match the packageId in the packaged file.
This property is used to ensure only the packageId specified is laoded.
{
"packageId": "a7d2f8"
}
packagerIdentities
Optional A list of public key ids of packaging identities that proves the authenticity of the package.
The key identified by the id, must be in the keys list.
Note that a package can be encrypted with multiple keys and each decryption key pair must be present in the keys list.
{
"packagerIdentities": [
"trusted-party-1", "trusted-party-2"
]
}
requiredAdditionalSignerCount
Optional Set the number of additional signers required from the additionalSigners list.
This property is used to ensure that the packageId is signed by a minimum number of specified keys.
Defaults to 0
{
requiredAdditionalSignerCount : 2
}
requiredSigners
Optional Define a list of required signers by id. The id must match a single key id in Bootstrap.keys
This property is used to ensure that the package is signed by specified keys
{
"keys" : [
{
"id" : "trusted-party-1",
"purpose" : "sign",
"type" : "public",
"keyHex" : "..."
},
{
"id" : "trusted-party-1",
"purpose" : "sign",
"type" : "public",
"keyHex" : "..."
}
],
"packaging" : {
"requiredSigners": [
{
"id": "trusted-party-1",
"description" : "A trusted party"
},
{
"id": "trusted-party-2",
"description" : "Another trusted party"
}
]
}
}
References:
- (Blog)
ServePackagedAsset
Defines how to serve static files from a packaged asset (zip file)
JSON Properties
The following properties are available to set on the ServePackagedAsset JSON schema.
packagedAssetFileId
Identifies a FilePermitter in the bootstrap to serve from.
THe FilePermitter should be a path to a zip file.
{
"packagedAssetFileId": "myZipFile"
}
WorkflowConfiguration
The configuration that describes the main activities of the agent
JSON Properties
The following properties are available to set on the WorkflowConfiguration JSON schema.
data
A JSON Object for adding properties.
These properties will be merged with the Bootstrap.data configuration field and be available to all child elements.
{
"data": {
"somedata" : {
"abc: 123
},
"someotherdata" : [ 1, 2, 3]
}
}
defaultStepProperties
Default properties applied to all steps in all workflows.
These properties are merged into each step and can be overridden by the step.
{
"defaultStepProperties": {
"debug": true
}
}
workflows
Required A list of Workflow describing the agent tasks
{
"workflows" [
...
]
}
WorkflowOutputs
Determines the behaviour of error handling and metrics for the workflow
JSON Properties
The following properties are available to set on the WorkflowOutputs JSON schema.
errorHandling
Determines the behavior of error handling while running the main configuration
{
"errorHandling": {
...
}
}
metricOutput
Metrics represent a data point that is interesting, generated by the agent and are sent to a remote system for analysis.
{
"metricOutput": {
...
}
}
outputs
Additional outputs that can be referenced by id from workflows.
{
"outputs": [
{
"id": "my-otel",
"type": "otel",
"routingToken" : "anId",
"outputUrl": "https://collector:4317",
"typeProperties": {
"otel": {
"protocol": "grpc"
}
}
}
]
}
References:
- (Blog)
Workflow schema
JSON Schema in the Workflow model:
DeploymentAccessTrigger
Describes how to bind to a deployment access trigger.
References:
- (Blog)
JSON Properties
The following properties are available to set on the DeploymentAccessTrigger JSON schema.
ids
Identifies the bootstrap deployment access ids to listen to (local to this deployment).
{
"ids": ["appBridge"]
}
noTrace
When true these executions will be omitted from tracing (monitoring)
{
"noTrace" : true
}
FileWatchingTrigger
Defines a trigger that fires when new lines are added to a file.
Each event (split by the lineSplit property) will be sent to the step processor. A sample event:
{
"log": {
"file": {
"path": "/path/to/file.txt"
},
"offset": 4130
},
"message": "this is a line from a file",
"input": {
"type": "log"
},
"host": {
"name": "hostname",
"ip": "[fe80:0:0:0:a6d7:feea:f601:902%wlp1s0, 192.168.0.27]"
},
"agent": {
"type": "opscotch",
"version": "3.0.0"
},
"ecs": {
"version": "1.12"
}
}
JSON Properties
The following properties are available to set on the FileWatchingTrigger JSON schema.
bootstrapFileId
Defines the allowed file access from the bootstrap
This must match an id in a bootstrap.allowFileAccess.id
{
"bootstrapFileId": "myTestFiles"
}
eventSplitter
Defines the (Java) regular expressions used to split events.
The delimiter will be present on the event message. The events that are collected from this trigger will be passed into the step processor as a list. The event has select field from the ECS for base, log, host, agent.
{
"eventSplitter": "\\n"
}
noTrace
When true these executions will be omitted from tracing (monitoring)
{
"noTrace" : true
}
patterns
A list of (Java) regular expressions used for file selection
{
"patterns": [
".+\\.txt$",
".+\\.log$"
]
}
splitAtEnd
When true the delimiter is at the end of the line, when false the delimiter will be at the start of the line.
Defaults to true
{
"splitAtEnd": false
}
HttpRequestTrigger
Describes how to bind to a http request
A sample event:
{
"uri": "/api/users/123",
"method": "POST",
"path": "/api/users/123",
"query": "foo=bar",
"body": "{\"name\": \"test\"}",
"headers": {
"Content-Type": ["application/json"],
"Authorization": ["Bearer token123"]
}
}
JSON Properties
The following properties are available to set on the HttpRequestTrigger JSON schema.
method
The HTTP method to listen for.
Defaults to GET
{
"method": "POST"
}
multiPartUploadByteLimit
Limits multipart uploads to this number of bytes This must be set to enable multi-part uploads.
{
"multiPartUploadByteLimit" : 100000
}
NOTE: setting this enables multipart uploads for this endpoint
noTrace
When true these executions will be omitted from tracing (monitoring)
{
"noTrace" : true
}
path
A regular expression that matches against the URI path
{
"path": "/api/.*"
}
server
Identifies the bootstrap http server to listen to
{
"server": "myAPI"
}
JavaScriptSource
A JavaScript that is executed during the processing of a Step
JSON Properties
The following properties are available to set on the JavaScriptSource JSON schema.
data
A JSON Object for adding properties.
Used to pass parameters and data to the script.
These properties will be merged with all other data fields in the json path, available to the step script elements.
{
"data": {
"somedata" : {
"abc: 123
},
"someotherdata" : [ 1, 2, 3],
}
}
resource
A file path to the script to execute.
Must be supplied if JavaScriptSource.script is not supplied
{
"resource": "scripts/myscript.js"
}
script
The script to execute.
A json-escaped script. Must be supplied if JavaScriptSource.resource is not supplied
{
"script": "console.log(\\"hello world\\");"
}
JavascriptProcessor
JSON Properties
The following properties are available to set on the JavascriptProcessor JSON schema.
processors
A list of JavaScriptSource that will be executed in order
{
"processors" : [
{
"resource" : "..."
},
{
"resource" : "..."
}
]
}
Persistence
Defines a cache/memory/key-value store that the Step has access to.
Required if using various functions. If required and not defined an error will be logged
This is currently a BETA feature and is unsupported.
JSON Properties
The following properties are available to set on the Persistence JSON schema.
hexSymmetricKey
obfuscate
persistenceFile
saveRatio
PersistentQueueParams
This is currently a BETA feature and is unsupported.
JSON Properties
The following properties are available to set on the PersistentQueueParams JSON schema.
maxFileBytes
HARD CODED TO 5MB
maxMemoryBytes
HARD CODED TO 1MB (1024 * 1024)
queuefilePrefix
HARD CODED TO persistenceFile + "-queue-" + deploymentId + "-" + stepId
Step
The Step is the primary configuration item for the agent configuration.
This configuration describes how to perform a single step or action in a Workflow.
There are different types of steps, however they all have the same execution structure:
- Prepare
- Execute
- Process
Steps take three forms of input:
- The response from a call to a Host (
context.getMessageBodyAsString()) - The message passed from another Step (
context.getPassedMessageAsString()) - Data from the configuration (
context.getData())
The different types of Steps allow for specialised behaviour:
-
"scripted" type: this is the default type with the following flow:
-
Optionally generate a URL (urlGenerator)
-
Optionally (requires a url to have been set) generate a payload (payloadGenerator)
-
Optionally perform authentication steps (authenticationProcessor, call other steps to perform authentication)
-
Optionally call out to a named Bootstrap Host with the generated URL and payload
-
Optionally handle specific HTTP status codes with httpStatusHandling
OR
-
Process the response or perform after-call processing (resultsProcessor)
Perhaps:
- Send a message to another Step
- Produce some metrics
-
-
"scripted-split-aggregate" type: this is a modified "scripted" type that has the following workflow of:
-
transforming the input into a list (splitGenerator)
-
calling a URL with each item in the list:
-
Optionally generate a URL (urlGenerator)
-
Optionally (requires a URL to have been set) generate a payload (payloadGenerator)
-
Optionally perform authentication steps (authenticationProcessor, call other steps to perform authentication)
-
Optionally call out to a named Bootstrap Host with the generated URL and payload
-
Optionally handle specific HTTP status codes with httpStatusHandling
OR
-
Optionally Process each item's response (itemResultProcessor)
-
Collect the result into a response list
-
-
The list of collected responses can then be processed and per normal (resultsProcessor)
Perhaps:
- Send a message to another Step
- Produce some metrics
Please note that in the case that one of the HTTP requests fail, it is your responsibility to handle this case by using the
httpStatusHandling,itemResultProcessor, andresultsProcessor
-
-
"scripted-auth" type: this is identical to "scripted" except that it has an alternative javascript context - it is not allowed to send metrics or call to non-authentication steps, but has access to the bootstrap data which may contain secrets.
-
"httpFile" type: this is a specialized step that serve static content from a zip file by mapping a url path into a zip file path. This type REQUIRES a
httptrigger to be defined, an only allowsGETrequests
At least one step in a Workflow will define a Step.trigger that will trigger the start of Workflow processing.
These forms of input and different type allows for the construction of flexible pipelines that should be sufficient to perform most tasks.
JSON Properties
The following properties are available to set on the Step JSON schema.
authenticationProcessor
This processor runs directly before an http call. It is not valid to have an authenticationProcessor without an urlGenerator.
This processor is expected to modify the context, generally by calling authentication steps and/or applying headers.
{
"authenticationProcessor": {
...
}
}
DEFAULT_HTTP_TIMEOUT
data
A JSON Object for adding properties.
Used to pass parameters and data to the JavaScriptSource elements.
These properties will be merged with all other data fields in the json path, available to the step JavaScriptSource elements.
{
"data": {
"somedata" : {
"abc: 123
},
"someotherdata" : [ 1, 2, 3],
}
}
debug
Enables or disables the debug logs for this step. Defaults to false.
{
"debug": true
}
enabled
Enables or disables the execution of this step. Defaults to true.
{
"enabled": true
}
httpConnectFailedProcessor
Defines how to handle HTTP connection failures using a standard processor
{
"httpConnectFailedProcessor": {
...
}
}
httpStatusHandling
A set of JavascriptProcessor to process depending on the http status.
The key is the HTTP status either as a discrete number ie "401" or a range "400-499" and the value is a (standard processor)(#JavascriptProcessor).
When a status code matches a status handler and the handler is executed the resultProcessor or itemProcessor
will not be executed.
Note: When supplying ANY httpStatusHandling you will also need explicitly declare success codes ie 200 as the resultsProcessor will not be called
{
"httpStatusHandling": {
"301-302" : {
...
},
"401" : {
...
}
}
}
httpTimeout
Set the http timeout period in milliseconds
Set to 0 to disable the read timeout and wait indefinitely for response data.
Defaults to 10000 (10 seconds)
{
"httpTimeout" : 10000
}
itemResultProcessor
When using Step.type "scripted-split-aggregate", defines if and how to process items before they're added to the list.
{
"itemResultProcessor": {
...
}
}
payloadGenerator
Defines how to generate the call payload, including setting headers.
This requires the urlGenerator to be present and will throw a validation exception.
If this processor is not used, then the HTTP body will be removed.
{
"payloadGenerator": {
...
}
}
persistenceFile
Defines a cache/memory/key-value store that the Step has access to.
Required if using various functions. If required and not defined an error will be logged
{
"persistenceFile": "persistenceFile.data"
}
resultsProcessor
Required Defines how to process the result of the call to the host. Note: You might setup result processing via the httpStatusHandling property - you will still need to define a resultsProcessor though, even if it is an empty string.
{
"resultsProcessor": {
...
}
}
singleThreaded
Determines the behaviour when the step is called while it's still executing.
Options are:
- "none" - step is not single threaded and can be run simultaneously (default)
- "return" - step is single threaded: if the step is already running, any calls to start the step will return immediately without running
Future options are likely : queue - execution will wait for the running execution to complete
splitGenerator
When using Step.type "scripted-split-aggregate", defines how to generate the list to be operated on.
{
"splitGenerator": {
...
}
}
stepId
Required Sets the id of the step.
Must be unique.
{
"stepId": "the-first-step-in-my-route"
}
trigger
Defines how a workflow is started.
{
"trigger": {
...
}
}
type
Sets the type of the step.
Either "scripted", "scripted-split-aggregate", "scripted-auth" as "httpFile" as defined above.
Defaults to "scripted"
{
"type": "scripted-split-aggregate"
}
urlGenerator
Required Defines how to generate the URL
This processor should not attempt to set the payload (the current internal logic may result in unexpected results). To set the payload, use the payloadGenerator
{
"urlGenerator": {
...
}
}
TcpTrigger
Describes how to bind to a TCP socket
References:
- (Blog)
JSON Properties
The following properties are available to set on the TcpTrigger JSON schema.
batchSize
Configures the number of frames to batch before triggering the workflow. The payload passed to the workflow will be a JSON array string containing each frame as a UTF-8 string.
Defaults to 1
{
"batchSize": 10
}
delimiter
Configures delimiter based framing, emitting a message each time the delimiter is encountered.
{
"delimiter": "\\n"
}
fixedLength
Configures fixed length framing, emitting a message every time the configured number of bytes are read.
{
"fixedLength": 1024
}
maxBatchWindowMillis
Configures the maximum time to wait (in milliseconds) before flushing a batch. When set, the batch will be emitted at most after this window, even if batchSize is not reached.
Defaults to 0 (disabled)
{
"maxBatchWindowMillis": 250
}
noTrace
When true these executions will be omitted from tracing (monitoring)
server
Identifies the bootstrap http server to listen to
{
"server": "myTCP"
}
stripDelimiter
When true the configured delimiter will be removed from the emitted payload.
Defaults to true
{
"stripDelimiter": false
}
Timer
Defines a repeating, fixed-rate period trigger
JSON Properties
The following properties are available to set on the Timer JSON schema.
delay
Defines a delay in milliseconds before the first invocation.
Defaults to 0.
{
"delay": 60000
}
noTrace
When true these executions will be omitted from tracing (monitoring)
{
"noTrace" : true
}
period
Defines a delay in milliseconds between invocations.
Defaults to 1000.
{
"period": 60000
}
Trigger
Defines the starting condition for a workflow
JSON Properties
The following properties are available to set on the Trigger JSON schema.
deploymentAccess
Defines a trigger that fires when a deployment access request is received.
{
"deploymentAccess": {
...
}
}
fileWatcher
Defines a trigger that fires when new lines are added to a file.
{
"fileWatcher": {
...
}
}
http
Defines a trigger that fires when a matching http request is received.
{
"http": {
...
}
}
runOnce
Sets trigger to run only once.
Defaults to false
{
"runOnce": true
}
runOnceNoTrace
When true these executions will be omitted from tracing (monitoring)
{
"runOnceNoTrace" : true
}
tcp
Defines a trigger that fires when data is received on a TCP socket.
{
"tcp": {
...
}
}
timer
Defines a repeating period trigger
{
"timer": {
...
}
}
Workflow
Defines a set of Steps that the agent will perform in order to generate metrics
JSON Properties
The following properties are available to set on the Workflow JSON schema.
data
A JSON Object for adding properties.
Used to pass parameters and data to the JavaScriptSource elements.
These properties will be merged with all other data fields in the json path, available to the step JavaScriptSource elements.
{
"data": {
"somedata" : {
"abc: 123
},
"someotherdata" : [ 1, 2, 3],
}
}
defaultStepProperties
Default properties applied to all steps in this workflow.
These properties are merged into each step and can be overridden by the step.
{
"defaultStepProperties": {
"debug": true
}
}
enabled
Enable or disables the loading of the workflow.
Defaults to true
{
"enabled": false
}
name
Required The user friendly name of the workflow.
{
"name": "my workflow"
}
steps
Workflow scripting contexts
Scripting context objects in the Workflow modek:
- AgentControlContext
- AuthenticationJavascriptContext
- ByteBufferHandle
- ByteContext
- ByteReader
- CryptoContext
- DiagnosticsContext
- Doc
- FilesContext
- JavascriptContext
- JavascriptStateContext
- LicensingContext
- PersistentQueueContext
- TimestampManager
AgentControlContext
Allows workflows to request lifecycle actions on the running agent.
This is currently in beta and not supported
Methods
The following methods are available to call on the context.agentControl() object.
shutdownAgent
shutdownAgent()
Request that the agent begins shutting down.
Workflows must enable bootstrap.agentControl.workflowCanShutdownAgent or this will throw an error.
AuthenticationJavascriptContext
This is a specialised Javascript Context for authentication steps.
Most of the methods on Javascript Context are still available, the methods listed below are specialisations.
sendMetric and
Methods
- addSplitReturnItem
- addSystemError
- addUserError
- bytes
- counter
- crypto
- delta
- diagnosticLog
- end
- getAllErrors
- getAuthenticationPropertiesFromStep
- getBody
- getData
- getFirstError
- getHeader
- getMessageBodyAsString
- getPassedMessageAsString
- getPersistedItem
- getProperty
- getRestrictedDataFromHost
- getStepProperties
- getSystemErrors
- getTimestamp
- getTimestampManager
- getUserErrors
- hasSystemErrors
- hasUserErrors
- hash
- isErrored
- jsonPath
- licensing
- mergeJsonStrings
- queue
- regexMatch
- removeAllHeaders
- removeHeader
- sendToStep
- sendToStepAndForget
- setAuthenticationPropertiesOnStep
- setBody
- setCounter
- setData
- setHeader
- setHttpMethod
- setMessage
- setPersistedItem
- setProperty
- setUrl
- sleep
- xmlToJson
addSplitReturnItem
addSplitReturnItem(String item)
When the current step is a "scripted-split-aggregate" type (see Step) the processed item must be added to the returned aggregate using this method.
context.addSplitReturnItem("hello world");
addSystemError
addSystemError(String error)
Adds an error not intended for a user outside the agent. It will also mark the context as in an error state.
System errors would generally be thrown or handled.
addUserError
addUserError(String error)
Adds an error intended for a user outside the agent. It will also mark the context as in an error state.
User errors would generally make their way to a human.
bytes
bytes() : ByteContext
Obtain a Byte context for working with bytes
counter
counter(String name, Double add) : Double
Atomically add to (or subtract from) a thread safe atomic counter and return the updated value
To just get the value pass 0 for the add parameter add
var value = context.counter("myCounter", 0); // just get the value
var added = context.counter("myCounter", 10); // 10 is added and the final value returned
var subtracted = context.counter("myCounter", -10); // 10 is subtracted and the final value returned
crypto
crypto() : CryptoContext
Obtain a Cryptography context for working with cryptography
delta
delta(String key, Double currentValue) : Double
Returns the differences between the current value and the last value stored for given key
delta = context.delta("myTs", 12);
diagnosticLog
diagnosticLog(String message)
Write a diagnostic log. This will only be emitted in development mode, or in Production mode if the step is enabled via a "enableDiagnostics" message
context.diagnosticLog("Hello World");
end
end()
Terminates the running flow and returns immediately.
context.end();
getAllErrors
getAllErrors() : List of java.lang.String
Retrieves a list of all error messages associated with the current context.
getAuthenticationPropertiesFromStep
getAuthenticationPropertiesFromStep(String stepId, String key) : String
ACCESSES RESTRICTED DATA
Gets the authentication properties from another step. These properties can expire.
getBody
getBody() : String
A shorthand for: check getMessageBodyAsString()then check getPassedMessageAsString()
body = context.getBody();
@return the contents eiter passed into the step or set by the step.
getData
(2 variants)
getData() : String
data = JSON.parse(context.getData());
getData(String key) : String
data = context.getData(key);
getFirstError
getFirstError(List of java.lang.String errors) : String
Retrieves the first error message from a given list of errors, if available.
getHeader
getHeader(String name) : String
Get a header from the previous request/response - note: it always returns an array string
headerArray = context.getHeader("aHeader");
header = JSON.parse(headerArray)[0];
getMessageBodyAsString
getMessageBodyAsString() : String
Get the current message body, this could be the response from an http call for example
body = context.getMessageBodyAsString();
getPassedMessageAsString
getPassedMessageAsString() : String
Get the message body that was passed to this step.
passedMessage = context.getPassedMessageAsString();
getPersistedItem
getPersistedItem(String key) : String
Gets an item from the step persistence
item = context.getPersistedItem("myItem");
getProperty
getProperty(String key) : Object
Gets a property from this running context
property = context.getProperty("aKey");
getRestrictedDataFromHost
getRestrictedDataFromHost(String host) : String
ACCESSES RESTRICTED DATA
Gets the json string of restricted data for a host name
hostData = JSON.parse(context.getRestrictedDataFromHost("myhost"));
getStepProperties
getStepProperties() : Object of java.lang.Object
Returns the set properties for the Step. These properties will persist after requests but not over config reloads/restarts
property = context.getStepProperties().put("myProperty", "ok");
getSystemErrors
getSystemErrors() : List of java.lang.String
Retrieves a list of system-specific error messages associated with the current context.
getTimestamp
getTimestamp() : Integer
Get the current UTC timestamp (note that the timestamp might be set by a test or other mechanism) in milliseconds since 1970-01-01T00:00:00Z (UTC)
ts = context.getTimestamp();
date = new Date(context.getTimestamp())
getTimestampManager
getTimestampManager() : TimestampManager
The timestamp manager allows for persistence of timestamps
tm = context.getTimestampManager();
getUserErrors
getUserErrors() : List of java.lang.String
Retrieves a list of user-specific error messages associated with the current context.
hasSystemErrors
hasSystemErrors() : Boolean
Checks if there are any system-specific errors in the current context.
hasUserErrors
hasUserErrors() : Boolean
Checks if there are any user-specific errors in the current context.
hash
hash(String toHash) : String
Note: This is a non-cryptographic hash and is intended to be used for obfuscation.
When hashing a value, the original and the hash are emitted to the log to be referred to at a later date. The hash can be used in metrics etc so that the original value is not transmitted.
The intention is that the raw value can remain in the originating network and the hash value can be transmitted out of network.
hashedValue = context.hash("hello world");
isErrored
isErrored() : Boolean
Indicates whether the current context is in an errored state.
jsonPath
jsonPath(String json, String expression) : String
Returns the result from a JSONPath expression
licensing
licensing() : LicensingContext
Obtain licensing information.
This is currently a BETA feature and is unsupported.
mergeJsonStrings
mergeJsonStrings(String one, String two) : String
Returns a merged json string
queue
queue() : PersistentQueueContext
Returns the high performance persistent queue
regexMatch
regexMatch(String regex, String input) : List of java.lang.String
Returns the regex matches
removeAllHeaders
removeAllHeaders()
Remove all headers from the current state
context.removeAllHeaders();
removeHeader
removeHeader(String name)
Remove a header from the current state
context.removeHeader("aHeader");
sendToStep
(3 variants)
sendToStep(String stepName, String body) : JavascriptStateContext
Send a message to another step and return a completed context upon completion
var completedContext = context.sendToStep("myStep", "theBody");
sendToStep()
Send a message to another deployment step and return a completed context upon completion.
var completedContext = context.sendToStep("appBridge", "myStep", "theBody");
References:
-
(Blog)
sendToStep()
Send a message to another deployment step with headers (key pair available from context.getHeader(...)) and return a completed context upon completion.
var completedContext = context.sendToStep("appBridge", "myStep", "theBody", { "header1" : "abc"});
References:
-
(Blog)
sendToStepAndForget
(4 variants)
sendToStepAndForget(String stepName, String body)
Send a message to another step and returns immediately. The step called will be processed in another thread, and you will not be able to access to the result.
context.sendToStep("myStep", "theBody");
sendToStepAndForget()
Send a message to another deployment step and returns immediately.
context.sendToStepAndForget("appBridge", "myStep", "theBody");
References:
-
(Blog)
sendToStepAndForget(String stepName, String body, Object of java.lang.String headers)
Send a message to another step with headers (key pair available from context.getHeader(...) and returns immediately.
The step called will be processed in another thread, and you will not be able to access to the result.
context.sendToStep("myStep", "theBody", { "header1" : "abc"});
sendToStepAndForget()
Send a message to another deployment step with headers (key pair available from context.getHeader(...)) and returns immediately.
The step called will be processed in another thread, and you will not be able to access to the result.
context.sendToStepAndForget("appBridge", "myStep", "theBody", { "header1" : "abc"});
References:
-
(Blog)
setAuthenticationPropertiesOnStep
setAuthenticationPropertiesOnStep(String stepName, Long expiresInMs, String key, String authenticationProperties)
ACCESSES RESTRICTED DATA
Sets the authentication properties from another step. These properties can expire.
setBody
setBody(Object body)
An alias for setMessage(...)
context.setBody("hello");
setCounter
setCounter(String name, Double value)
Atomically set the counter to an absolute value
setData
setData(String data)
Set the JSON data for this step
context.setData(JSON.stringify(data));
setHeader
setHeader(String name, Object value)
Add a header to the next http request
context.setHeader("aHeader", "aValue");
setHttpMethod
setHttpMethod(String method)
Set the HTTP method for the next http action
This is generally used when a method other than the default is required.
The HTTP method will default to GET unless there is a payload body then it will default to POST
context.setHttpMethod("POST");
setMessage
setMessage(Object message)
Sets the message body for this running exchange
context.setMessage("hello world");
setPersistedItem
setPersistedItem(String key, String value)
Sets an item to the step persistence
context.setPersistedItem("myItem", "theValue");
setProperty
setProperty(String key, Object value)
Sets a property onto this running context to be available in downstream step execution
This differs from setPersistedItem in that setPersistedItem sets onto the STEP to be available on THIS STEP
in future runs, while setProperty makes an item available to OTHER steps
context.setProperty("aKey", "aValue");
setUrl
setUrl(String hostref, String path)
Set the url for the next http action with reference to a named host
The path can start with a / even if the named host ends in a / - the result will be a single '/'
context.setUrl("myHost", "/a/path");
sleep
sleep(Integer ms)
Force the currently running script to sleep for the given milliseconds
context.sleep(100);
xmlToJson
xmlToJson(String xml) : String
Returns a json string from an xml string
ByteBufferHandle
The ByteBufferHandle is a convenience "label" to make it clear that you're dealing with an internal byte buffer.
When you see a method taking or returning a ByteBufferHandle you know that you are dealing with a byte buffer.
See byte context
ByteContext
A low-level byte buffer manipulation interface providing minimal primitives for binary data operations.
Most operations either take or return a ByteBufferHandle. The byte buffer handle is a reference to a byte buffer in the agent - you will never interact with the buffer itself, rather you delegate the functionality to the agent.
The ByteContext offers the following categories of functionality:
- conversion to and from text encoded bytes (base64, hex)
- creating new byte arrays (create, createFromByteArray, createFromString)
- byte array manipulation (concat, copy, getSize, resize, slice, release)
- byte compression (gzip, gunzip, zip, unzip)
- writing to byte buffer (writeByte, writeBytes)
- reading from byte buffer (readByte, reader.available, reader.read, reader.readAll)
All high-level operations (multi-byte integers, strings, etc.) should be implemented in JavaScript using these primitives.
References:
Methods
The following methods are available to call on the context.bytes() object.
- base64ToBinary
- binaryToBase64
- binaryToHex
- concat
- copy
- create
- createFromByteArray
- createFromString
- getSize
- gunzip
- gzip
- hash
- hexToBinary
- readByte
- reader
- release
- resize
- sha256
- slice
- split
- toString
- unzip
- writeByte
- writeBytes
- zip
base64ToBinary
base64ToBinary(String base64) : ByteBufferHandle
Converts a Base64 encoded String into a binary buffer.
let buffer = byteContext.base64ToBinary("aGVsbG8="); // "Hello"
binaryToBase64
binaryToBase64(ByteBufferHandle buffer) : String
Converts buffer contents to a Base64 encoded string.
let buffer = byteContext.createFrom([72, 101, 108, 108, 111]); // "Hello"
let b64 = byteContext.binaryToBase64(buffer);
console.log(b64); // "aGVsbG8="
binaryToHex
binaryToHex(ByteBufferHandle buffer) : String
Converts buffer contents to a hexadecimal string representation.
let buffer = byteContext.createFrom([72, 101, 108, 108, 111]); // "Hello"
let hex = byteContext.binaryToHex(buffer);
console.log(hex); // "48656C6C6F"
concat
concat(ByteBufferHandle[] buffers) : ByteBufferHandle
Combines two buffers into a new buffer containing all bytes from both.
let header = byteContext.createFrom([0xFF, 0xFE]); // Magic bytes
let data = byteContext.createFrom([1, 2, 3, 4]); // Payload
let packet = byteContext.concat([header, data]);
// packet contains: [0xFF, 0xFE, 1, 2, 3, 4]
copy
copy(ByteBufferHandle sourceBuffer) : ByteBufferHandle
Creates a deep copy of an existing buffer.
let original = byteContext.create(10);
let copy = byteContext.copy(original);
// Modifying copy won't affect original
copy = byteContext.writeByte(copy, 0, 255);
console.log(byteContext.readByte(original, 0)); // 0 (unchanged)
create
create(Integer size) : ByteBufferHandle
Creates a new buffer of the specified size, initialized with zeros.
let buffer = byteContext.create(1024); // 1KB buffer of zeros
console.log(byteContext.getSize(buffer)); // 1024
createFromByteArray
createFromByteArray(byte[] bytes) : ByteBufferHandle
Creates a new buffer from a byte array.
// Create from existing data
let bytes = [0x48, 0x65, 0x6C, 0x6C, 0x6F]; // "Hello" in ASCII
let buffer = byteContext.createFrom(bytes);
console.log(byteContext.getSize(buffer)); // 5
createFromString
createFromString(String string) : ByteBufferHandle
Creates a new buffer from a string.
// Create buffer from UTF-8 string
let text = "Hello, 世界!"; // Unicode text
let buffer = byteContext.createFrom(text);
getSize
getSize(ByteBufferHandle buffer) : Integer
Returns the current size of a buffer in bytes.
let buffer = byteContext.create(42);
console.log(byteContext.getSize(buffer)); // 42
gunzip
gunzip(ByteBufferHandle buffer) : ByteBufferHandle
Decompresses GZIP-compressed buffer contents.
let compressed = byteContext.gzip(originalBuffer);
let decompressed = byteContext.gunzip(compressed);
gzip
gzip(ByteBufferHandle buffer) : ByteBufferHandle
Compresses buffer contents using GZIP compression.
let compressed = byteContext.gzip(originalBuffer);
let decompressed = byteContext.gunzip(compressed);
hash
hash(ByteBufferHandle buffer) : Integer
Computes a CRC32 hash of the buffer contents.
let buffer = byteContext.createFrom([1, 2, 3, 4]);
let hash = byteContext.hash(buffer);
console.log(hash); // Unsigned 32-bit CRC value as a number
hexToBinary
hexToBinary(String hex) : ByteBufferHandle
Converts a hexadecimal string to binary buffer data.
let buffer = byteContext.hexToBinary("48656C6C6F"); // "Hello" in hex
readByte
readByte(ByteBufferHandle fromBuffer, Integer offset) : Integer
Reads a single byte from the buffer at the specified offset.
let buffer = byteContext.create(5);
buffer = byteContext.writeByte(buffer, 2, 170); // 0xAA
let value = byteContext.readByte(buffer, 2);
console.log(value); // 170
reader
(2 variants)
reader(ByteBufferHandle buffer) : ByteReader
Creates a sequential reader for efficient forward-only reading of buffer data. This is more efficient than random access for sequential operations.
let reader = byteContext.reader(buffer);
// examples
let byte = reader.readByte();
let int32 = reader.readInt32BE();
let bytes = reader.readBytes(length);
reader() : ByteReader
Reads from the context stream
release
release(ByteBufferHandle[] buffers)
Securely releases (erase) buffers from memory.
resize
resize(ByteBufferHandle buffer, Integer newSize) : ByteBufferHandle
Changes the size of a buffer. If enlarged, new bytes are zero-filled. If shrunk, excess bytes are discarded.
let buffer = byteContext.create(10);
buffer = byteContext.resize(buffer, 20); // Grow to 20 bytes (zeros added)
buffer = byteContext.resize(buffer, 5); // Shrink to 5 bytes
sha256
sha256(ByteBufferHandle buffer) : ByteBufferHandle
Computes a SHA-256 digest of the buffer contents.
slice
slice(ByteBufferHandle sourceBuffer, Integer offset, Integer length) : ByteBufferHandle
Extracts a portion of a buffer without modifying the original.
let buffer = byteContext.create(100);
let middle = byteContext.slice(buffer, 25, 50); // Extract bytes 25-74
console.log(byteContext.getSize(middle)); // 50
// Extract from offset to end
let tail = byteContext.slice(buffer, 80, byteContext.getSize(buffer) - 80);
split
split(ByteBufferHandle buffer, ByteBufferHandle delimiter, Integer includeDelimiter) : ByteBufferHandle[]
Splits a buffer by a delimiter buffer.
let buffer = byteContext.createFrom([1, 2, 3, 2, 4]);
let delimiter = byteContext.createFrom([2]);
let parts = byteContext.split(buffer, delimiter, 0); // [[1], [3], [4]]
toString
toString(ByteBufferHandle buffer) : String
unzip
unzip(ByteBufferHandle buffer) : ByteBufferHandle
Decompresses ZIP-compressed buffer contents.
let compressed = byteContext.zip(originalBuffer);
let decompressed = byteContext.unzip(compressed);
writeByte
writeByte(ByteBufferHandle buffer, Integer offset, Integer value)
Writes a single byte to the buffer at the specified offset.
let buffer = byteContext.create(10);
buffer = byteContext.writeByte(buffer, 0, 0xFF); // Write 255
buffer = byteContext.writeByte(buffer, 1, 0x00); // Write 0
// Build a 32-bit big-endian integer
function writeInt32BE(buffer, offset, value) {
buffer = byteContext.writeByte(buffer, offset, (value >>> 24) & 0xFF);
buffer = byteContext.writeByte(buffer, offset + 1, (value >>> 16) & 0xFF);
buffer = byteContext.writeByte(buffer, offset + 2, (value >>> 8) & 0xFF);
buffer = byteContext.writeByte(buffer, offset + 3, value & 0xFF);
return buffer;
}
writeBytes
writeBytes(ByteBufferHandle toBuffer, Integer offset, ByteBufferHandle sourceBuffer, Integer sourceOffset, Integer length)
Copies bytes from one buffer to another. This is the most efficient way to move large amounts of data between buffers.
let source = byteContext.createFrom([1, 2, 3, 4, 5]);
let dest = byteContext.create(10);
// Copy bytes 1-3 from source to position 2 in dest
byteContext.writeBytes(dest, 2, source, 1, 3);
// dest now contains: [0, 0, 2, 3, 4, 0, 0, 0, 0, 0]
zip
zip(ByteBufferHandle buffer) : ByteBufferHandle
Compresses buffer contents using ZIP compression.
let compressed = byteContext.zip(originalBuffer);
let decompressed = byteContext.unzip(compressed);
ByteReader
A sequential reader for efficient forward-only reading of buffer data.
Methods
The following methods are available to call on the context.bytes().reader(...) object.
available
available() : Integer
Returns the number of bytes that can be read from the current position.
let available = reader.available();
read
read(Integer length) : ByteBufferHandle
Reads the specified number of bytes from the current position into a new buffer.
let buffer = reader.read(20);
console.log(context.bytes().getSize(buffer)); // 20
readAll
readAll() : ByteBufferHandle
Reads all the available bytes from the current position into a new buffer.
let buffer = reader.readAll();
CryptoContext
The Crypto context provides cryptographic operations for signing, encryption and random byte generation.
It works with the byte context, and all parameters and returns are byte buffer handles.
The opscotch cryptography functions use the libsodium implementation, which is a high-grade industry standard.
Some notes about keys:
- Keys need to be a specific kind and size for each function and must be registered (either in the bootstrap or during a workflow) before used - see registerKey.
- You can generate a key for a specific function using generateKeyPair
- You can validate a key for a given function using validateKey
- For key requirements see Key
References:
- (Blog)
The following cryptographic functions are provided for use within workflows:
- Public key signatures
- Used to create and verify digital signatures that ensure message authenticity and integrity. A private key is used to generate a unique signature for a message, while the corresponding public key allows anyone to verify the signature. A verified signature proves it was created by the owner of the private key, and the message hasn't been tampered with.
- methods:
- sign
- verifySignature
- Algorithm used: Ed25519
- You can provide your own Ed25519 keys
- Authenticated public key encryption
- Used to encrypt messages that can only be decrypted by a specific intended recipient and authenticates the sender's identity. The recipient can verify the message came from the claimed sender, while ensuring message confidentiality. This provides both encryption and authentication in a single operation.
- methods:
- encryptPublicKey
- decryptPublicKey
- Algorithms used:
- Key exchange: X25519
- Encryption: XSalsa20
- Authentication: Poly1305
- You can provide your own X25519 keys
- Anonymous public key encryption
- Used for one-way encryption where only the recipient can decrypt the message using their private key. The sender remains anonymous as no authentication is performed. This provides message confidentiality without revealing the sender's identity.
- methods:
- encryptAnonymous
- decryptAnonymous
- Algorithms used:
- Key exchange: X25519
- Encryption: XSalsa20 stream cipher
- You can provide your own X25519 keys
- Symmetric secret key encryption
- Used when both parties share a single secret key that is used for both encryption and decryption. The same key must be securely shared between parties beforehand. This provides fast and secure encryption for parties who have already established a shared secret key.
- methods:
- encryptSymmetric
- decryptSymmetric
- Algorithm used: XSalsa20 stream cipher
- You can provide your own 32-byte secret key
- Random byte generation
- Generate cryptographically random byte array
- methods:
- randomBytes
- Hahsing
- Generate cryptographically secure hash values
- methods:
- hash
- hmacSha256
- Key management
- Various functions for working with keys:
- methods:
- generateKeyPair
- registerKey
- validateKey
Methods
The following methods are available to call on the context.crypto() object.
- decryptAnonymous
- decryptPublicKey
- decryptSymmetric
- encryptAnonymous
- encryptPublicKey
- encryptSymmetric
- generateKeyPair
- hash
- hmacSha256
- passwordHash
- randomBytes
- registerKey
- sign
- validateKey
- verifySignature
decryptAnonymous
decryptAnonymous(ByteBufferHandle payloadBytes, String registeredPublicKeyId, String registeredPrivateKeyId) : ByteBufferHandle
Decrypts a payload using anonymous public-key encryption.
decryptPublicKey
decryptPublicKey(ByteBufferHandle payloadBytes, ByteBufferHandle nonceBytes, String registeredPublicKeyId, String registeredPrivateKeyId) : ByteBufferHandle
Decrypts a payload using authenticated public-key encryption.
decryptSymmetric
decryptSymmetric(ByteBufferHandle encryptedBytes, ByteBufferHandle nonceBytes, String registeredSharedKeyId) : ByteBufferHandle
Decrypts a payload using symmetric decryption.
encryptAnonymous
encryptAnonymous(ByteBufferHandle payloadBytes, String registeredPublicKeyId) : ByteBufferHandle
Encrypts a payload using anonymous public-key encryption.
This will be used when the sender uses the recipient public key to encrypt and the payload is only openable by the holder of the secret key. There is no sender identity or verification.
encryptPublicKey
encryptPublicKey(ByteBufferHandle payloadBytes, ByteBufferHandle nonceBytes, String registeredPublicKeyId, String registeredPrivateKeyId) : ByteBufferHandle
Encrypts a payload using authenticated public-key encryption. This is used when the recipient knows the sender and vice versa. The payload will only be openable by the intended recipient and will only be openable if verified from the sender.
encryptSymmetric
encryptSymmetric(ByteBufferHandle payloadBytes, ByteBufferHandle nonceBytes, String registeredSharedKeyId) : ByteBufferHandle
Encrypts a payload using symmetric encryption.
generateKeyPair
generateKeyPair(String purpose) : ByteBufferHandle[]
Generate a new public/private key pair for the given type
hash
hash(ByteBufferHandle input) : ByteBufferHandle
Generates a consistent hash for the same input
hmacSha256
hmacSha256(ByteBufferHandle key, ByteBufferHandle bytes) : ByteBufferHandle
Generates an HMAC using SHA-256.
passwordHash
(2 variants)
passwordHash(ByteBufferHandle password) : ByteBufferHandle[]
Given a user-provided password, create a safe, high-entropy key/hash you can use for encryption or password verification
This version of the function uses the minimum safe parameters for the lowest memory and CPU usage. Using higher resource settings will make the calculation take longer, which means any attacks on a password will also take longer.
See notes on the other passwordHash function.
passwordHash(ByteBufferHandle password, ByteBufferHandle salt, Integer hashLength, Integer pwhashAlg, Integer pwhashOpsLimit, Integer pwhashMemLimit) : ByteBufferHandle[]
Given a user-provided password, create a safe, high-entropy key/hash you can use for encryption or password verification
This version of the function allows for the control of memory and CPU usage. Using higher resource settings will make the calculation take longer, which means any attacks on a password will also take longer.
NOTE: This function returns an array of byte buffer handles. The first one is the generated (likely secret) key/hash. The second one is a byte representation of a json string with the following object that is required to repeat the process to get the same outcome:
{
"pwhashAlg" : 0,
"pwhashOpsLimit" : 0,
"pwhashMemLimit" : 0,
"pwhashSalt" : "<base64 encoded string>",
"keyHash" : "<base64 encoded string>",
}
The keyHash property is a 'generic hash' of the generated key.
randomBytes
randomBytes(Integer length) : ByteBufferHandle
Generates random bytes that are cryptographically secure.
registerKey
registerKey(String purpose, String type, ByteBufferHandle key) : String
Registers a cryptographic key and returns a registered key id for use in the current context.
WARNING: THE INPUT KEY BYTES ARE ZEROED OUT - THIS DESTROYS THE INPUT.
If you need the input key you must make a copy with context.bytes().copy(KEY)
Public and secret keys need to be registered separately.
The purpose is important and can be one of the following cryptographic primitives:
- sign - used for
signandverifySignatureusing public/secret keys - authenticated - used for
encryptPublicKeyanddecryptPublicKeyfor mutually authenticated public-key encryption - symmetric - used for
encryptSymmetricanddecryptSymmetric- this only uses a secret key - anonymous - used for
encryptAnonymousanddecryptAnonymous- this is the classic public/private key encryption
The type will either be public or secret depending on the key being registered.
Each permutation of purpose and type will have a specific required key length in bytes
and an error will be thrown if the byte length (not the hex character length) does not match.
const keyId = context.crypto().registerKey("sign", "secret", context.bytes().hexToBinary("6250E4....0E4"));
sign
sign(ByteBufferHandle payloadBytes, String registeredKeyId) : ByteBufferHandle
Signs a payload using a secret key and returns the signature.
validateKey
validateKey(String id, String purpose, String type) : Boolean
Determines if a given key id matches a registered key with the given purpose and type
verifySignature
verifySignature(ByteBufferHandle signatureBytes, ByteBufferHandle messageBytes, String registeredPublicKeyId) : Boolean
Verifies that a signature matches a message using a public key.
DiagnosticsContext
An interface for interacting with OpenTelemetry
Methods
The following methods are available to call on the context.diagnostic() object.
errored
(2 variants)
errored(String error)
Mark the trace as errored and records the error message
errored(String type, String error)
Mark the trace as errored and records the error type and message
event
event(String event)
Add an event to a trace
setAttribute
(3 variants)
setAttribute(String key, String value)
Set an attribute onto the trace
setAttribute(String key, Integer value)
Set an attribute onto the trace
setAttribute(String key, Double value)
Set an attribute onto the trace
Doc
Documentation and schema validation for a JavaScript step.
The Doc model allows JavaScript steps to declare input/output/data schemas for validation, describe their purpose, and define execution logic.
The doc model is available in the JavaScript resource as doc
Usage in JavaScript:
doc.inSchema({
"type": "object",
"properties": {
"name": { "type": "string" }
}
});
doc.outSchema({
"type": "object",
"properties": {
"result": { "type": "string" }
}
});
doc.description("This step processes user input");
doc.run(() => {
// Step logic here
});
Methods
asUserErrors
asUserErrors() : Doc
Marks errors as user error when the user is expected to resolve the error.
When enabled, validation errors will be added the user error context rather than the system error context
dataSchema
dataSchema(Object schema) : Doc
Defines the JSON Schema for the data object validation.
The schema is used to validate the data object (passed via JavaScriptSource.data) before the step executes. If validation fails, a user-friendly error is returned.
description
description(String description) : Doc
Provides a human-readable description of what the step does.
This description is used for documentation and may be displayed in error messages or API documentation.
inSchema
inSchema(Object schema) : Doc
Defines the JSON Schema for input validation.
The schema is used to validate the incoming request body before the step executes. If validation fails, a user-friendly error is returned.
outSchema
outSchema(Object schema) : Doc
Defines the JSON Schema for output validation.
The schema is used to validate the step's output before it's returned. If validation fails, a user-friendly error is returned.
run
run(Object run) : Doc
Sets the execution function for the step.
This function is called when the step executes. It should contain the main logic for the step.
FilesContext
The FileContext provides workflow access to files.
Only files defined in the bootstrap allowFileAccess property can be accessed.
Methods
The following methods are available to call on the context.files(fileId) object.
copy
copy(String srcPath, String dstFileId, String dstPath, Boolean mkdir, Boolean overwrite)
Copies a file from one location to another
delete
delete(String file)
Delete a file
list
list(String path) : String
Lists the directory
move
move(String srcPath, String dstFileId, String dstPath, Boolean mkdir, Boolean overwrite)
Moves a file from one location to another
read
read(String file) : String
Read the contents of the file as a string
reader
reader(String file) : ByteReader
Creates a byte reader on the contents of the file
write
write(String file, String body)
Write the string contents to a file
writeBinary
writeBinary(String file, ByteBufferHandle buffer, Integer offSet)
Write the binary contents from the byte buffer to a file
JavascriptContext
This is the entry point to the agent functionality from javascript.
Methods
The following methods are available to call on the context object.
- addSplitReturnItem
- addSystemError
- addUserError
- agentControl
- bytes
- counter
- crypto
- delta
- diagnostic
- diagnosticLog
- end
- files
- getAllErrors
- getBody
- getData
- getFirstError
- getHeader
- getMessageBodyAsString
- getPassedMessageAsString
- getPersistedItem
- getProperty
- getStepProperties
- getStream
- getSystemErrors
- getTimestamp
- getTimestampManager
- getUserErrors
- hasSystemErrors
- hasUserErrors
- hash
- isErrored
- jsonPath
- licensing
- mergeJsonStrings
- queue
- regexMatch
- removeAllHeaders
- removeHeader
- sendMetric
- sendToStep
- sendToStepAndForget
- setBody
- setCounter
- setData
- setHeader
- setHttpMethod
- setHttpMultipartType
- setMessage
- setPersistedItem
- setProperty
- setStream
- setStreamFromString
- setUrl
- sleep
- xmlToJson
addSplitReturnItem
addSplitReturnItem(String item)
When the current step is a "scripted-split-aggregate" type (see Step) the processed item must be added to the returned aggregate using this method.
context.addSplitReturnItem("hello world");
addSystemError
addSystemError(String error)
Adds an error not intended for a user outside the agent. It will also mark the context as in an error state.
System errors would generally be thrown or handled.
addUserError
addUserError(String error)
Adds an error intended for a user outside the agent. It will also mark the context as in an error state.
User errors would generally make their way to a human.
agentControl
agentControl() : AgentControlContext
Obtain an Agent Control context for influencing the agent lifecycle.
This is currently in beta and not supported
bytes
bytes() : ByteContext
Obtain a Byte context for working with bytes
counter
counter(String name, Double add) : Double
Atomically add to (or subtract from) a thread safe atomic counter and return the updated value
To just get the value pass 0 for the add parameter add
var value = context.counter("myCounter", 0); // just get the value
var added = context.counter("myCounter", 10); // 10 is added and the final value returned
var subtracted = context.counter("myCounter", -10); // 10 is subtracted and the final value returned
crypto
crypto() : CryptoContext
Obtain a Cryptography context for working with cryptography
delta
delta(String key, Double currentValue) : Double
Returns the differences between the current value and the last value stored for given key
delta = context.delta("myTs", 12);
diagnostic
diagnostic() : DiagnosticsContext
Obtain a Trace context for adding data to sent traces
diagnosticLog
diagnosticLog(String message)
Write a diagnostic log. This will only be emitted in development mode, or in Production mode if the step is enabled via a "enableDiagnostics" message
context.diagnosticLog("Hello World");
end
end()
Terminates the running flow and returns immediately.
context.end();
files
files(String id) : FilesContext
Request access to the bootstrap file permission
getAllErrors
getAllErrors() : List of java.lang.String
Retrieves a list of all error messages associated with the current context.
getBody
getBody() : String
A shorthand for: check getMessageBodyAsString()then check getPassedMessageAsString()
body = context.getBody();
@return the contents eiter passed into the step or set by the step.
getData
(2 variants)
getData() : String
data = JSON.parse(context.getData());
getData(String key) : String
data = context.getData(key);
getFirstError
getFirstError(List of java.lang.String errors) : String
Retrieves the first error message from a given list of errors, if available.
getHeader
getHeader(String name) : String
Get a header from the previous request/response - note: it always returns an array string
headerArray = context.getHeader("aHeader");
header = JSON.parse(headerArray)[0];
getMessageBodyAsString
getMessageBodyAsString() : String
Get the current message body, this could be the response from an http call for example
body = context.getMessageBodyAsString();
getPassedMessageAsString
getPassedMessageAsString() : String
Get the message body that was passed to this step.
passedMessage = context.getPassedMessageAsString();
getPersistedItem
getPersistedItem(String key) : String
Gets an item from the step persistence
item = context.getPersistedItem("myItem");
getProperty
getProperty(String key) : Object
Gets a property from this running context
property = context.getProperty("aKey");
getStepProperties
getStepProperties() : Object of java.lang.Object
Returns the set properties for the Step. These properties will persist after requests but not over config reloads/restarts
property = context.getStepProperties().put("myProperty", "ok");
getStream
getStream() : ByteReader
Get the stream on the context.
getSystemErrors
getSystemErrors() : List of java.lang.String
Retrieves a list of system-specific error messages associated with the current context.
getTimestamp
getTimestamp() : Integer
Get the current UTC timestamp (note that the timestamp might be set by a test or other mechanism) in milliseconds since 1970-01-01T00:00:00Z (UTC)
ts = context.getTimestamp();
date = new Date(context.getTimestamp())
getTimestampManager
getTimestampManager() : TimestampManager
The timestamp manager allows for persistence of timestamps
tm = context.getTimestampManager();
getUserErrors
getUserErrors() : List of java.lang.String
Retrieves a list of user-specific error messages associated with the current context.
hasSystemErrors
hasSystemErrors() : Boolean
Checks if there are any system-specific errors in the current context.
hasUserErrors
hasUserErrors() : Boolean
Checks if there are any user-specific errors in the current context.
hash
hash(String toHash) : String
Note: This is a non-cryptographic hash and is intended to be used for obfuscation.
When hashing a value, the original and the hash are emitted to the log to be referred to at a later date. The hash can be used in metrics etc so that the original value is not transmitted.
The intention is that the raw value can remain in the originating network and the hash value can be transmitted out of network.
hashedValue = context.hash("hello world");
isErrored
isErrored() : Boolean
Indicates whether the current context is in an errored state.
jsonPath
jsonPath(String json, String expression) : String
Returns the result from a JSONPath expression
licensing
licensing() : LicensingContext
Obtain licensing information.
This is currently a BETA feature and is unsupported.
mergeJsonStrings
mergeJsonStrings(String one, String two) : String
Returns a merged json string
queue
queue() : PersistentQueueContext
Returns the high performance persistent queue
regexMatch
regexMatch(String regex, String input) : List of java.lang.String
Returns the regex matches
removeAllHeaders
removeAllHeaders()
Remove all headers from the current state
context.removeAllHeaders();
removeHeader
removeHeader(String name)
Remove a header from the current state
context.removeHeader("aHeader");
sendMetric
(7 variants)
sendMetric(String key, Double value)
Send a metric
context.sendMetric("theKey", 123);
sendMetric(String routingToken, Integer timestamp, String key, Double value, Object of java.lang.String metadata)
Send a metric
context.sendMetric("theRoutingToken", theTimestamp, "theKey", 123, { "someMeta" : "Data"});
sendMetric()
Send a metric to a specific output id
context.sendMetric("my-output", "theKey", 123);
References:
-
(Blog)
sendMetric(String outputId, Integer timestamp, String key, Double value)
Send a metric to a specific output id
context.sendMetric("my-output", theTimestamp, "theKey", 123);
sendMetric(String outputId, String key, Double value, Object of java.lang.String metadata)
Send a metric to a specific output id
context.sendMetric("my-output", "theKey", 123, { "someMeta" : "Data"});
sendMetric(Integer timestamp, String key, Double value)
Send a metric
context.sendMetric(theTimestamp, "theKey", 123);
sendMetric(Integer timestamp, String key, Double value, Object of java.lang.String metadata)
Send a metric
context.sendMetric(theTimestamp, "theKey", 123, { "someMeta" : "Data"});
sendToStep
(4 variants)
sendToStep(String stepName, String body) : JavascriptStateContext
Send a message to another step and return a completed context upon completion
var completedContext = context.sendToStep("myStep", "theBody");
sendToStep()
Send a message to another deployment step and return a completed context upon completion.
var completedContext = context.sendToStep("appBridge", "myStep", "theBody");
References:
-
(Blog)
sendToStep(String stepName, String body, Object of java.lang.String headers) : JavascriptStateContext
Send a message to another step with headers (key pair available from context.getHeader(...) and return a completed context upon completion
var completedContext = context.sendToStep("myStep", "theBody", { "header1" : "abc"});
sendToStep()
Send a message to another deployment step with headers (key pair available from context.getHeader(...)) and return a completed context upon completion.
var completedContext = context.sendToStep("appBridge", "myStep", "theBody", { "header1" : "abc"});
References:
-
(Blog)
sendToStepAndForget
(4 variants)
sendToStepAndForget(String stepName, String body)
Send a message to another step and returns immediately. The step called will be processed in another thread, and you will not be able to access to the result.
context.sendToStep("myStep", "theBody");
sendToStepAndForget()
Send a message to another deployment step and returns immediately.
context.sendToStepAndForget("appBridge", "myStep", "theBody");
References:
-
(Blog)
sendToStepAndForget(String stepName, String body, Object of java.lang.String headers)
Send a message to another step with headers (key pair available from context.getHeader(...) and returns immediately.
The step called will be processed in another thread, and you will not be able to access to the result.
context.sendToStep("myStep", "theBody", { "header1" : "abc"});
sendToStepAndForget()
Send a message to another deployment step with headers (key pair available from context.getHeader(...)) and returns immediately.
The step called will be processed in another thread, and you will not be able to access to the result.
context.sendToStepAndForget("appBridge", "myStep", "theBody", { "header1" : "abc"});
References:
-
(Blog)
setBody
setBody(Object body)
An alias for setMessage(...)
context.setBody("hello");
setCounter
setCounter(String name, Double value)
Atomically set the counter to an absolute value
setData
setData(String data)
Set the JSON data for this step
context.setData(JSON.stringify(data));
setHeader
setHeader(String name, Object value)
Add a header to the next http request
context.setHeader("aHeader", "aValue");
setHttpMethod
setHttpMethod(String method)
Set the HTTP method for the next http action
This is generally used when a method other than the default is required.
The HTTP method will default to GET unless there is a payload body then it will default to POST
context.setHttpMethod("POST");
setHttpMultipartType
setHttpMultipartType(String textPartName, String contentType, String binaryPartName, String binaryContentType, String binaryFileName)
Enables multipart HTTP requests and sets the content type and part names.
setMessage
setMessage(Object message)
Sets the message body for this running exchange
context.setMessage("hello world");
setPersistedItem
setPersistedItem(String key, String value)
Sets an item to the step persistence
context.setPersistedItem("myItem", "theValue");
setProperty
setProperty(String key, Object value)
Sets a property onto this running context to be available in downstream step execution
This differs from setPersistedItem in that setPersistedItem sets onto the STEP to be available on THIS STEP
in future runs, while setProperty makes an item available to OTHER steps
context.setProperty("aKey", "aValue");
setStream
setStream(ByteReader reader)
Sets the body content from the provided byte buffer.
context.setStream(buffer);
@param reader name containing the body content to be set
setStreamFromString
setStreamFromString(String streamString)
Sets a stream from a string
setUrl
setUrl(String hostref, String path)
Set the url for the next http action with reference to a named host
The path can start with a / even if the named host ends in a / - the result will be a single '/'
context.setUrl("myHost", "/a/path");
sleep
sleep(Integer ms)
Force the currently running script to sleep for the given milliseconds
context.sleep(100);
xmlToJson
xmlToJson(String xml) : String
Returns a json string from an xml string
JavascriptStateContext
The Javascript State Context is a reduced version of the JavascriptContext containing readonly functions.
This context is returned from context.sendToStep(..) and contains the completed context of the step that was sent to.
Methods
The following methods are available to call on the var returnedContext = context.sendToStep(...) object.
- getAllErrors
- getBody
- getFirstError
- getMessageBodyAsString
- getProperty
- getStepProperties
- getSystemErrors
- getUserErrors
- hasSystemErrors
- hasUserErrors
- isErrored
getAllErrors
getAllErrors() : List of java.lang.String
Retrieves a list of all error messages associated with the current context.
getBody
getBody() : String
A shorthand for: check getMessageBodyAsString()then check getPassedMessageAsString()
body = context.getBody();
@return the contents eiter passed into the step or set by the step.
getFirstError
getFirstError(List of java.lang.String errors) : String
Retrieves the first error message from a given list of errors, if available.
getMessageBodyAsString
getMessageBodyAsString() : String
Get the current message body, this could be the response from an http call for example
body = context.getMessageBodyAsString();
getProperty
getProperty(String key) : Object
Gets a property from this running context
property = context.getProperty("aKey");
getStepProperties
getStepProperties() : Object of java.lang.Object
Returns the set properties for the Step. These properties will persist after requests but not over config reloads/restarts
property = context.getStepProperties().put("myProperty", "ok");
getSystemErrors
getSystemErrors() : List of java.lang.String
Retrieves a list of system-specific error messages associated with the current context.
getUserErrors
getUserErrors() : List of java.lang.String
Retrieves a list of user-specific error messages associated with the current context.
hasSystemErrors
hasSystemErrors() : Boolean
Checks if there are any system-specific errors in the current context.
hasUserErrors
hasUserErrors() : Boolean
Checks if there are any user-specific errors in the current context.
isErrored
isErrored() : Boolean
Indicates whether the current context is in an errored state.
LicensingContext
Obtain licensing information.
This is currently a BETA feature and is unsupported.
Methods
isRegistered
isRegistered(String licenseKey) : Boolean
PersistentQueueContext
A high performance implementation of persistent FIFO queue.
The persistence context is got via context.queue().
Once the queue is obtained, items can be added and removed over multiple runs.
The queue will store a small amount of items in memory and overflow to disk.
Queue items will persist over agent restarts.
Methods
push
push(String item)
Add an item to the end of the persistent queue.
context.queue().push("adding item");
pushArray
pushArray(java.util.Collection<String> array)
Add the contents of an array of item to the end of the persistent queue.
context.queue().pushArray(["adding", "multiple", "items"]);
returnArray
returnArray(java.util.Collection<String> array)
Return the array items to the head of the queue
returnItem
returnItem(String item)
Return the item to the head of the queue
take
take(Integer count) : List of java.lang.String
Request the given amount of items from the queue or an empty array when no items available.
NOTE: the returned amount may not be the entire available items. The method should be called repeatedly until the desired amount is collected. This function waits for a short period for data to be loaded from file and may not load all the data.
var items = context.queue().take(1000);
TimestampManager
Looks after timestamps for metrics, for example recording the last known timestamp for a metric
TimestampManager is local to the step.
Timestamps can have a named key, or if not provided with a key, will just store with a default key.
Methods
firstTimestamp
firstTimestamp() : Long
get
(2 variants)
get() : Long
get(String key) : Long
minutesAgo
minutesAgo(Long timestamp) : Integer
set
(2 variants)
set(String timestamp)
Sets the timestamp without a key see: set(String key, String timestamp)
set(String key, String timestamp)
Sets the timestamp for the given key
setIfLastest
setIfLastest(String key, String timestamp)
Only performs the set operation if the passed timestamp is later than the previously stored.
opscotch testing model
The opscotch testing model describes how to define tests that are executed against a running opscotch agent.
The test runner configuration is passed to the test runner and describes the locations of tests and resource files. The test configuration describes each individual test.
Here is the outline of the entire structure of the opscotch testing models: the runner configuration and the test configuration.
Note that not every field is required, so be sure to check the detailed descriptions.
The test runner schema overview
{
"resourceDir": "",
"resourceDirs": [
""
],
"testDir": "",
"testDirs": [
""
],
"license": ""
}
The test schema
{
"ignore": true,
"fromDirectory": "",
"testThisStep": "",
"useThisTestConfigFile": "",
"useThisTestBootstrapFile": "",
"mockEndpointResponses" : [
{
"whenThisIsCalled": "",
"expectedPayload": "",
"expectedHeaders": {
"": [""]
},
"returnThisFile": "",
"returnThisStatusCode" : 0,
"mockedHeaders" : {
"": [""]
},
"thisManyTimes" : 0
}
],
"theseMetricsShouldHaveBeenSent" : [
{
"timestamp": 0,
"key": "",
"value": 0.0,
"dimensionMap" : {
"": ""
}
}
],
"theseLogsShouldHaveBeenSent" : [
""
],
"logsShouldNotContain" : [
""
],
"makeTheseCalls" : [
{
"url": "",
"expectedResponse": "",
"expectedSize": 0,
"method": "",
"headers": [
[ "" , "" ]
],
"body": "",
"timeoutSeconds" : 0
}
],
"timestampOverride": 0,
"secondsToWaitForCompletion" : 0
}
Test schema
JSON Schema in the Test model:
HttpCall
Make an http call and assert results
JSON Properties
The following properties are available to set on the HttpCall JSON schema.
body
The http body to send
expectedResponse
The expected response
expectedResponseCode
The expected HTTP response code from the request.
This property is optional and defaults to 200, which usually indicates a successful HTTP response.
expectedSize
The expected size of the response
headers
The http headers to use
method
The http method to use
Defaults to 'get'
timeoutSeconds
The allowed timeout of the http call in seconds
Defaults to 10
url
The url to call
MockHttpResponse
Instructs the mock http server what urls to expect and how to respond.
These requests and responses are required to have been called by the test, otherwise the test will fail.
JSON Properties
The following properties are available to set on the MockHttpResponse JSON schema.
delay
The number of milliseconds to delay the response This will default to 0
{
"delay" : 0
}
expectedHeaders
If set, will assert that expectedHeader's key/value pair is found within the HTTP header. As headers can have multiple values the value is represented as an array.
{
"expectedHeaders" : {"some": ["thing"] }
}
expectedPayload
If set will assert that the exact payload was received.
{
"expectedPayload": "{\"some\":\"thing\"}"
}
mockedHeaders
If set will return these headers with the response. As headers can have multiple values the value is represented as an array.
{
"mockedHeaders" : {
"Set-Cookie": ["JSESSIONID=A","X-CSRF-TOKEN=B"]
}
}
returnThisBody
The response body to return directly.
{
"returnThisBody" : "{\"ok\":true}"
}
returnThisFile
The file path of the response to return.
{
"returnThisFile" : "test.response.summary.json"
}
returnThisStatusCode
The status code to return
{
"returnThisStatusCode" : 302
}
thisManyTimes
The number of times this endpoint should be invoked This will default to 1
{
"thisManyTimes" : 2
}
whenThisIsCalled
Required The complete url that the mock server will listen for.
The "mockserver" host is replaced out to the actual host and port for the mock server.
{
"whenThisIsCalled" : "http://mockserver/controller/restui/licenseRule/getAllRulesSummary"
}
withMethod
The method expected ie GET, POST etc
This will default to GET and if an expectedPayload is provided it will default to POST
{
"withMethod" : "GET"
}
SendMetricVerification
Defines expected metrics to have been published from the test. If these metrics are not received the test will fail.
Pay special note to the timestamp field.
{
"theseMetricsShouldHaveBeenSent": [
{
"key": "mymetric",
"value": 67.0
}
]
}
JSON Properties
The following properties are available to set on the SendMetricVerification JSON schema.
dimensionMap
Assert that received and expected dimensions are exact.
{
"dimensionMap" : {
"ad" : "ad_mh"
}
}
key
Required Assert the exact key on the metric.
{
"key" : "abcde"
}
timestamp
If set will assert the exact timestamp on the metric, otherwise the timestamp will be ignored during assertion.
{
"timestamp" : 1593658546131
}
toThisRoute
value
Required Assert the exact value on the metric.
{
"value" : "1234.56"
}
TestConfig
Describes how to execute a test and verify results
JSON Properties
The following properties are available to set on the TestConfig JSON schema.
data
A JSON Object for adding properties.
Used to pass parameters and data to the JavaScriptSource elements.
These properties will be merged with all other data fields in the json path, available to the step JavaScriptSource elements.
{
"data": {
"somedata" : {
"abc: 123
},
"someotherdata" : [ 1, 2, 3],
}
}
fromDirectory
Required The relative path from the test directory to the test resources (test workflow configurations, mock response files etc).
{
"fromDirectory" : "/mytest/"
}
ignore
If true the test will not be run.
{
"ignore" : true
}
logsShouldNotContain
These logs are not expected and the test will fail if they are generated by the workflow configuration.
This is useful to ensure that secrets are not logged.
{
"logsShouldNotContain" : [
...
]
}
makeTheseCalls
Make http calls to the agent and assert the response
{
"makeTheseCalls" : [
...
]
}
mockEndpointResponses
Required Sets up a http server with mocked responses.
{
"mockEndpointResponses" : [
...
]
}
secondsToWaitForCompletion
Determines how long to wait for logs and metric assertions before the test is stopped
Defaults to 10
{
"secondsToWaitForCompletion" : 20
}
testThisStep
Required The name of the step in the workflow configuration to trigger to start the test.
{
"testThisStep" : "start-application"
}
theseLogsShouldHaveBeenSent
These logs are expected and the test will fail if they are not generated by the workflow configuration
{
"theseLogsShouldHaveBeenSent" : [
...
]
}
theseMetricsShouldHaveBeenSent
Required These metrics are expected and the test will fail if they are not generated by the workflow configuration
{
"theseMetricsShouldHaveBeenSent" : [
...
]
}
timestampOverride
If set (to a millisecond timestamp) the system clock will be set to this time.
{
"timestampOverride" : 1593658546131
}
useThisTestBootstrapFile
The bootstrap configuration to include for the test.
{
"useThisBootstrapFile" : "test.bootstrap.json"
}
useThisTestConfigFile
Required The workflow configuration file to load for the test.
{
"useThisTestConfigFile" : "test.config.json"
}
TestRunnerConfig
JSON configuration required for defining test configurations against agents.
This configuration file is passed to the test runner as a command line argument and tells it where to find the tests and resource files.
JSON Properties
The following properties are available to set on the TestRunnerConfig JSON schema.
concurrency
Number of tests to run at the same time This will default to half the number of processors available or 1 if there is only one. The maximum value is the number of processors available -2 or 1 if there are less than 3 processors available
debug
If true, will enable debug for the test
iterations
Speficies the number of times to run the tests Defaults to 1.
license
Required The license key
resourceDir
Deprecated: use resourceDirs
Required The path to the configuration resource directory
resourceDirs
Required The paths to the configuration resource directories
stopOnFailure
Indicates whether the test runner should stop executing tests after the first failure is encountered.
Defaults to false, meaning the test runner will continue executing all configured tests, regardless of whether a failure occurs.
If set to true, the test runner will immediately terminate testing upon the first failed test. Currently running tests will continue till completion.
testDir
Deprecated: use testDirs
Required The path to the configuration test directory
testDirs
Required The paths to the configuration test directories
opscotch Error Codes
Most errors will have an opscotch error code that may be referenced in logs, metrics or messages. Please see the list below for descriptions of error codes.
AJC
AJC1: Can not decrypt encrypted data
Occurs when encrypted data fails to decrypt in authentication context.
AJC2: Authentication route target not permitted
Authentication flows may call only scripted-auth steps.
AS
AS2: Invalid arguments, expecting bootstrap
The agent was started with unexpected arguments. Please check the documentation.
AS3: Agent error occurred before startup
Please report this error to opscotch
BC
BC1: No buffer exists for key
The requested byte buffer does not exist in this context
BC2: Could not gunzip
Check that the buffer contains gzipped data
BC4: Could not unzip
Check that the buffer contains zipped data
BC5: Buffer offset overrun
Requested offset is greater than the buffer size
BC6: Buffer length overrun
Requested length is greater than the buffer size
BC7: Invalid buffer byte indexing
Requested offset and length is greater than the buffer size
BL
BL1: Failed to load bootstrap
Generic bootstrap load failure. See additional error codes.
BL10: Missing environment variable in bootstrap
A bootstrap contains ${VAR} placeholders that are not set in the environment
BL11: Missing B2 bootstrap decryption key
A B2/ encrypted bootstrap requires OPSCOTCH_BOOTSTRAP_SECRETKEY to be set
BL2: Duplicate deploymentId
Bootstrap loaded with a deploymentId that is already loaded.
BL3: Failed to load bootstrap - Decryption
Failed to decrypt using internal opscotch keys.
BL4: Duplicate host id
Bootstrap hosts must have unique ids.
BL5: Duplicate host id
Bootstrap hosts must have unique ids.
BL6: Required file does not exist
A bootstrap allowFileAccess is marked as required but does not exist
BL7: Duplicate key id
bootstrap.keys contains duplicate key ids
BL8: HTTP bootstrapping not supported
use local file for bootstrapping
BL9: Invalid agent private key
Check that the bootstrap.agentPrivateKey is correct
BS
BS1: Malformed url
The url specified in Output.outputUrl is malformed.
BS2: Failed send for metrics or logs
Unexpected send failure; data likely lost.
CC
CC10: No key found for verifySignature
The passed key or buffer id could not be found
CC11: No key found for encryptSymmetric
The passed key or buffer id could not be found
CC12: Payload must be at least 1 byte
Check the passed payload size
CC13: Could not encrypt using encryptSymmetric
Check the passed key or buffer id
CC14: No key found for decryptSymmetric
The passed key or buffer id could not be found
CC15: Payload must be at least 1 byte
Check the passed payload size
CC16: Could not decrypt using decryptSymmetric
Check the passed key or buffer id
CC17: No public key found for encryptAsymmetric
The passed key or buffer id could not be found
CC18: No secret key found for encryptAsymmetric
The passed key or buffer id could not be found
CC19: Payload must be at least 1 byte
Check the passed payload size
CC2: No key found matching criteria
The passed key or buffer id is null
CC20: Could not encrypt using encryptPublicKey
Check the passed key or buffer id
CC21: No public key found for decryptPublicKey
The passed key or buffer id could not be found
CC22: No secret key found for decryptPublicKey
The passed key or buffer id could not be found
CC23: Payload must be at least 1 byte
Check the passed payload size
CC24: Could not decrypt using decryptPublicKey
Check the passed key or buffer id
CC25: No public key found for encryptAnonymous
The passed key or buffer id could not be found
CC26: Payload must be at least 1 byte
Check the passed payload size
CC27: Could not encrypt using encryptAnonymous
Check the passed key or buffer id
CC28: No public key found for decryptAnonymous
The passed key or buffer id could not be found
CC29: No secret key found for decryptAnonymous
The passed key or buffer id could not be found
CC3: Key add failed
required: id, purpose, type, keyHex
CC30: Payload must be at least 1 byte
Check the passed payload size
CC31: Could not decrypt using decryptAnonymous
Check the passed key or buffer id
CC34: The passed key bytes are null
Check the passed key bytes
CC35: The passed byte array size are incorrect
Check the passed key bytes
CC36: Could not generate password hash
Check the error message for more details
CC37: Could not generate HMAC SHA-256
Check the passed key or buffer id
CC4: Key add failed
Incorrect key length, check the key length matches the key 'purpose'
CC5: Key validation failed
required: id, purpose, type
CC7: No key found for sign
The passed key or buffer id could not be found
CC8: Payload must be at least 1 byte
Check the passed payload size
CC9: Could not generate signature
Check the passed key or buffer id
CD
CD1: The agent private key is invalid
Decoded but not a valid RSA PKCS8 private key.
CD2: Could not read the packaged configuration
The bootstrap settings are wrong, or the package is incompatible with this agent version.
CD3: Could not decrypt config
Workflow configuration could not be decrypted.
CD4: Workflow JSON file format is not supported
Attempted to load a raw workflow json file.
CD5: Workflow config is not valid
Check packager version
CD6: Can not load a config that was packaged with an incompatible packager
Check packager version
CD9: The workflow file is empty and will not be loaded
This is generally not a problem and can happen when updating the file and the writing process truncates the file
CDE
CDE1: Workflow Configuration not loaded
Signature could not be validated even after decryption.
CP
CP1: Workflow configuration load failed
Generic workflow configuration load failure.
CP2: Workflow configuration load failed
Generic workflow configuration load failure.
CP3: Invalid agentPrivateKey in bootstrap
Base64 decode of agentPrivateKey failed.
CP4: agentPrivateKey required in Prod Mode
Missing agentPrivateKey when running in Prod Mode.
CR
CR1: Failed loading configuration
Failure after JSON validation when loading steps.
CR2: Step requested does not exist
Requested step has not been loaded.
CR3: Workflow did not finish active runs in time
Reload forced while runs were still active.
CRW
CRW1: Exception while checking license
Problem with the license checking mechanism.
CW
CW1: Malformed url
Invalid URL in bootstrap remoteConfiguration.
CW2: Invalid bootstrap remote configuration settings
remoteConfigurationTimeout must be less than frequency.
DA
DA1: No bootstrap.allowDeploymentAccess with id
Workflow attempted deployment access with a missing allowDeploymentAccess id.
DA2: Deployment access denied
Target deployment does not allow access from caller deployment.
DA3: Deployment not available
Target deployment is not running or has no active configuration.
FC
FC1: Can not read file
The requested file could not be read. Check file system permissions
FC10: Not permitted to write
The requested destination file is not permitted to be written. Check bootstrap.allowFileAccess.WRITE.
FC11: Can not write file
The requested destination file could not be written to. Check file system properties.
FC12: Can not write file
The requested destination file could not be written to. Check file system properties.
FC13: Can not delete file
The requested destination file could not be deleted. Check file system properties.
FC2: Not permitted to read file
The requested file is not permitted to be read. Check bootstrap.allowFileAccess.READ.
FC3: Can not read file
The requested file could not be read. Check file system permissions
FC4: Not permitted to read file
The requested file is not permitted to be read. Check bootstrap.allowFileAccess.READ.
FC5: Not permitted to read file
The requested file is not permitted to be read. Check bootstrap.allowFileAccess.READ.
FC6: Not permitted to move file without delete permission
The requested file is not permitted to be deleted while moving. Check bootstrap.allowFileAccess.DELETE.
FC7: Source file does not exist
The requested file does not exist
FC8: Not permitted to write
The requested destination file is not permitted to be written. Check bootstrap.allowFileAccess.WRITE.
FC9: Destination file exists and no overwrite
The destination file exists and overwrite was specified.
FW
FW1: Failed to load file
remoteConfiguration refers to a file that could not be loaded.
FW2: Failed to read file
remoteConfiguration file could not be read.
FW3: Failed to load file
File read but contents could not be processed.
FW4: Can not load zero length file
remoteConfiguration refers to a zero length file.
HF
HF1: URI pattern already exists
URI pattern already exists on this HTTP server.
HFS
HFS2: Http request trigger is required
Add http request trigger to the http file serving step
HFS3: Only GET method is permitted
Only GET method is permitted on the http file serving step
HFS4: Requires allowHttpServerAccess
Http file serving steps requires bootstrap.allowHttpServerAccess
HFS5: Requires allowHttpServerAccess.servePackagedAsset
Http file serving steps requires bootstrap.allowHttpServerAccess.servePackagedAsset
HFS6: Requires bootstrap.allowFileAccess
Http file serving steps requires bootstrap.allowFileAccess
HP
HP1: Not on allow list
HTTP address not on host.allowList for path/method.
HP2: Authentication Failed
authenticationProcessor failed to complete.
HP3: Url not set
urlGenerator used but URL was not set.
HP4: Error processing HTTP response
Exception during HTTP response processing.
HP5: Error while making http request from step
HTTP request failed.
HP6: Multipart request missing body or stream
When using multipart requests, both body and stream must be set.
HP7: Multipart response content type not supported
Before returning the response, ensure that you perform context.removeHeader('Content-Type') then specifically add your expected content type context.setHeader('Content-Type', myType)
HT
HT1: No bootstrap.allowHttpServerAccess with id
Workflow trigger attempted to attach to bootstrap.allowHttpServerAccess.id that does not exist
HT2: Invalid HTTP method
Invalid HTTP method specified in workflow trigger
HT7: Client disconnected
Http client disconnect be connection was closed
HT8: Multipart request received but httpRequestTrigger.multiPartUploadByteLimit is not set
If expecting multipart uploads ensure httpRequestTrigger.multiPartUploadByteLimit is set appropriately
HT9: Multipart upload exceeds size limit of httpRequestTrigger.multiPartUploadByteLimit
If expecting multipart uploads ensure the size of the uploaded file does not exceed the httpRequestTrigger.multiPartUploadByteLimit
HW
HW1: Could not get Workflow configuration
HTTP resource fetch failed.
HW2: Could not get Workflow configuration
Decode failed after HTTP fetch.
IT
IT11: Can not start file watcher
File watching is not supported on this OS or filesystem
IT12: unable to load file
An error occurred while loading a file from the file water for the step.trigger.fileWatcher
IT13: unable to load file
An error occurred while loading a file from the file water for the step.trigger.fileWatcher
IT2: Path is invalid
Check the step.trigger.fileWatcher paths
IT3: eventSplitter pattern is invalid
Check the step.trigger.fileWatcher.eventSplitter pattern
IT4: unable to load files
An error occurred while starting the file water for the step.trigger.fileWatcher
IT5: No bootstrap.allowFileAccess with id
Check the bootstrap.allowFileAccess pattern for the step.trigger.fileWatcher.bootstrapFileId
IT6: bootstrap.allowFileAccess has an invalid pattern
Check the bootstrap.allowFileAccess pattern for the step.trigger.fileWatcher.bootstrapFileId
IT7: fileTrigger.pattern has an invalid pattern
Check the step.trigger.fileWatcher.patterns
IT8: eventSplitter has an invalid pattern
Check the step.trigger.fileWatcher.eventSplitter pattern
IT9: unable to load file
An error occurred while loading a file from the file water for the step.trigger.fileWatcher
JC
JC1: Messages failed to load
Workflow configuration messages failed to load.
JC10: getTimestamp
Error during context.getTimestamp().
JC11: getTimestampManager
Error during context.getTimestampManager().
JC12: setProperty
Error during context.setProperty().
JC13: getStepProperties
Error during context.getStepProperties().
JC14: getProperty
Error during context.getProperty().
JC15: setPersistedItem
Error during context.setPersistedItem().
JC16: getPersistedItem
Error during context.getPersistedItem().
JC17: setMessage
Error during context.setMessage().
JC18: addSplitReturnItem
Error during context.addSplitReturnItem().
JC19: getData
Error during context.getData().
JC2: getPassedMessageAsString
Error during context.getPassedMessageAsString().
JC20: setHeader
Error during context.setHeader().
JC21: getHeader
Error during context.getHeader().
JC22: delta
Error during context.delta().
JC23: regexMatch
Error during context.regexMatch().
JC24: mergeJsonStrings
Error during context.mergeJsonStrings().
JC25: xmlToJson
Error during context.xmlToJson().
JC26: jsonPath
Error during context.jsonPath().
JC27: removeHeader
Error during context.removeHeader().
JC28: setHttpMethod
Error during context.setHttpMethod().
JC29: Malformed URL
Invalid URL passed to context.setUrl().
JC3: getMessageBodyAsString
Error during context.getMessageBodyAsString().
JC30: Host not found
hostref not found in bootstrap host property.
JC31: Host not permitted
Host referenced from the wrong context.
JC32: host-ref can not be null
hostref missing when calling context.setUrl().
JC33: files
Error during context.files().
JC34: queue
Error during context.queue().
JC35: counter
Error during context.counter().
JC36: setCounter
Error during context.setCounter().
JC37: setBody
Error during context.setBody().
JC38: setStream
Error during context.setStream().
JC39: setStreamFromString
Error during context.setStreamFromString().
JC4: sendToStep
Error during context.sendToStep().
JC40: sendToStepAndForget
Error during context.sendToStepAndForget().
JC41: getBody
Error during context.getBody().
JC42: metric key is empty
The metric can not be empty
JC43: Workflow initiated shutdown is disabled
Set bootstrap.agentControl.workflowCanShutdownAgent to true to allow shutdownAgent()
JC44: setHttpMultipartType
Error during context.setHttpMultipartType().
JC45: removeAllHeaders
Error during context.removeAllHeaders().
JC5: sendMetric
Error during context.sendMetric().
JC6: getAnomalyDetection
Error during context.getAnomalyDetection().
JC7: sendMetric
Error during context.sendMetric().
JC8: sendMetric
Error during context.sendMetric().
JC9: diagnosticLog
Error during context.diagnosticLog().
JSS
JSS1: docs.run(...) is not a function
Ensure that a function is passed to docs.run()
JSS2: Error parsing schema
Check that a valid jsonschema is passed
JSS3: Error parsing schema
Check that a valid jsonschema is passed
JSS4: Unknown schema type
Check that a valid jsonschema is passed
OHC
OHC1: Unhandled Exception
Unhandled exception during HTTP response processing.
OHC2: Unhandled Exception
Unhandled exception during HTTP failure.
OMR
OMR1: Error while closing Agent Metric Sender
Error occurred while closing Agent Metric Sender.
PS
PS4: Persistence directory does not exist
Ensure that the bootstrap.persistenceRoot is set to a directory that exists
PS5: Persistence seems to be encrypted but not decryption key provided
Ensure that the step.persistence.hexSymmerticKey is set
PS6: Persistence seems to be encrypted but decryption failed with key provided
Ensure that the step.persistence.hexSymmerticKey is the same used to encrypt
PV
PV1D1: Unable to read package
Unable to read package, check the packager is up to date with the agent.
PV1D2: Could not decrypt package
Check that the agent private key matches the packaging public key
PV1D3: Could not verify the author
The workflow configuration could not be verified, check the packager is up to date with the agent
PV1D4: The agent private key is invalid
Expected a base64 encoded RSA PKCS8
PV1D5: Unable to read package
Unable to read package, check the packager is up to date with the agent.
PV1D6: Unable to read package
Unable to read package, check the packager is up to date with the agent.
PV2D17: Unable to read package
Unable to read package, check the packager is up to date with the agent.
SE
SE1: Could not create persistence
persistenceFile defined but file could not be created.
SE10: Error validating output against outSchema
Check you output matches the declared schema
SE11: Failed to load resource
Check the name is correct and resource directories are accessible
SE2: Ending Run
Run ended prematurely and was not completed.
SE3: Unhandled Exception - Please report this to opscotch support
Run ended prematurely and exception handler failed.
SE4: Workflow configuration was updated
Run did not complete before configuration update.
SE5: Ending Run
Likely caused by JavaScript in executing step processor.
SE6: Persistence requested but not setup
Processor called persistence but it was not set up.
SE7: TimestampManager requires persistence
TimestampManager requires persistence, but it was not set up
SE8: Error validating input against inSchema
Check you input matches the declared schema - if this is occurring in an authentication processor, note that the authentication processor runs after the urlGenerator and payloadGenerator
SE9: Error validating input against dataSchema
Check you data matches the declared schema
SM
SM1: Secure memory allocation failed
Could not allocate secure memory region.
SM2: Secure memory not locked
mlock unavailable; secure memory is not page-locked and may be swapped.
T
T2: OpenTelemety endpoint and type must be set
Check OPSCOTCH_OTEL_PROPERTIES environment variable
TT
TT1: No bootstrap.allowHttpServerAccess with id
Tcp trigger attempted to attach to bootstrap.allowHttpServerAccess.id that does not exist
TT2: Invalid TCP framing configuration
tcpTrigger requires either fixedLength or delimiter
TT3: TCP not permitted on server
Enable tcp on bootstrap.allowHttpServerAccess entry
TT4: Error processing TCP payload
Exception while handling TCP trigger