YAML Command Files

Command files use a YAML structure to accommodate the nested structure of HTTP requests. Files contain command sequences and the commands themselves, which are HTTP requests.

Upload YAML command files in the Errthquake portal on the Test Assets page.

Command Sequences

All sequences are optional. For example, a command file that only defines a GlobalInitSeq is valid, and would configure a test that simply runs the global initialization sequence.

GlobalInitSeq

The global initialization sequence. This sequence is run once prior to running any other commands. Environment variables, secrets, and regular expression captures can be used in the GlobalInitSeq, but CSV input cannot.

InitSequence

The initialization sequence for each virtual user. Virtual users run in parallel, so after any GlobalInitSeq is complete, the InitSequence will begin for the requested number of virtual users.

LoopSequence

The loop sequence for each virtual user. After the InitSequence is complete, the LoopSequence runs for each virtual user.

LoopCount

The LoopCount is a partner configuration to the LoopSequence defining the number of times the LoopSequence should run.

Configuring Command Sequences

Command sequences are structured as a comma-separated list of commands and pause directives that define time between commands.

Command Sequence Examples

GlobalInitSeq referencing an "auth" command, a "cleanup" command, and two 500 millisecond pauses.

GlobalInitSeq: "auth,pause-500ms,cleanup,pause-500ms"

InitSequence referencing an "auth" command, a 100 millisecond pause, an "upload" command, and a 2 second pause.

InitSequence: "auth,pause-100ms,upload,pause-2s"

LoopSequence and LoopCount referencing a "download" command and a 200 millisecond pause, to be run 20 times.

LoopSequence: "download,pause-200ms"

LoopCount: 20

Pause Directives

Pause directives are defined by prefixing pause- to a duration string. Errthquake is written in Golang and uses Go duration formats. Valid time units are ns, us (or µs), ms, s, m, and h.

If the overall test time dictates that the test finishes while a command sequence is paused, the sequence will end without running the pending command.

Commands

Commands are HTTP requests configured in YAML format. The name of the command is itself the primary attribute. In this example commands named "home" and "docs" are defined.

Commands:
  home:
    Url: https://myservice.com/

  docs:
    Url: https://docs.myservice.com/

If the overall test time dictates that the test finishes while a command request is in progress, the test engine will wait for the response before ending the test.

Command Attributes

The command attributes build an HTTP request.

Url: The request URL. Any querystring parameters should be included here.
- Querystring values hardcoded in the command configuration should be URL encoded, but values read from CSV test input will be URL encoded by the test engine.
Headers: Optional attribute/value map of HTTP headers
Method: Optional HTTP method. All methods are supported, GET is the default.
BodyText: Optional request body text.
- The test engine will URL encode the body text if the request includes a Content-Type header with value "application/x-www-form-urlencoded".
BodyParams: Optional attribute/value map used when building the request body with a querystring format.
- Values hardcoded in the command configuration should be URL encoded, but values read from CSV test input will be URL encoded by the test engine.
- An "@<filename>" attribute/value pair will load a test object file in the request body.
RegexVals: Optional set of variables to be assigned by capturing response text with regular expressions.
- Use a <variable>: <regex with capture> format, for example TOKEN: '"token":"(\S+?)"'.
- The Errthquake test engine executes Golang regular expressions, use this format in your definitions.
Errthquake currently has a 30 second connection timeout, a 30 second request timeout, and a 90 second idle connection timeout.

Command Variables

CSV Input Variables

Commands can reference CSV input variables by including a {$n} formatted substring in an attribute, where n is the field number in the CSV input.

January,1,2025

Commands:
  calendar:
     Url: https://calendar.myservice.com/?month={$1}&date={$2}&year={$3}

Environment Variables and Secrets

Commands can reference environment variables and secrets by including a {$VAR} formatted substring in an attribute.

Commands:
  adminauth:
    Url:  https://login.myservice.com/?username={$ADMINUSER}&password={$ADMINPASS}

Workspace scoped environment variables and secrets can be defined in the Errthquake portal.

Captured Regular Expression Values

Commands can reference captured regular expression values by includng a {%VAR} formatted substring in an attribute.

Commands:
  readqueue:
    Url:  https://queue.myservice.com/next
    RegexVals:
      ENTRY: '"next_entry":"(\S+?)"'

  readentry:
    Url: https://queue.myservice.com/entries/{%ENTRY}

Test Configuration Example

GlobalInitSeq: "authcmd,pause-500ms,admincleanup,pause-500ms"

InitSequence: "authcmd,pause-500ms,metadata,pause-500ms,upload,pause-2s"

LoopSequence: "view,pause-200ms"

LoopCount: 20

Commands:
  adminauth:
    Url:  https://auth.myservice.com/token
    Method: POST
    Headers:
      Content-Type: "application/x-www-form-urlencoded"
    BodyText: 'username={$ADMINUSER}&password={$ADMINPASS}&grant_type=password&audience=my-app'
    RegexVals:
      TOKEN: '"id_token":"(\S+?)"'

  admincleanup:
    Url: https://api.myservice.com/content
    Headers:
        Authorization: "Bearer {%TOKEN}"
    Method: DELETE

  userauth:
    Url:  https://auth.myservice.com/token
    Method: POST
    Headers:
      Content-Type: "application/x-www-form-urlencoded"
    BodyText: 'username={$1}&password={$2}&grant_type=password&audience=my-app'
    RegexVals:
      TOKEN: '"id_token":"(\S+?)"'

  metadata:
    Url:  https://api.myservice.com/content
    Headers:
      Authorization: "Bearer {%TOKEN}"
    Method: POST
    BodyText: '{ "permalink": "{$3}", "title": "{$4}", "description": "{$5}", "price": "{$6}" }'

  upload:
    Url:  https://api.myservice.com/content/{$3}
    Headers:
      Authorization: "Bearer {%TOKEN}"
    Method: PUT
    BodyParams:
      file: "@CoverArt.jpg"

  view:
    Url: https://api.myservice.com/content/{$3}

The example includes

{$ADMINUSER} and {$ADMINPASS} references to environment variables and secrets
{$1}..{$6} references to CSV test input fields
{%TOKEN} value assigned by regular expression capture.
Uploading a file named CoverArt.jpg

Example CSV Input for this Test

user1,password1,abbeyroad,The Beatles,Abbey Road,42.58
user2,password2,thriller,Michael Jackson,Thriller,25.71
user3,password3,nightattheopera,Queen,A Night at the Opera,39.23
user4,password4,darksideofthemoon,Pink Floyd,The Dark Side of the Moon,48.99
user5,password5,ziggystartdust,David Bowie,Ziggy Stardust,31.84
user6,password6,somegirls,The Rolling Stones,Some Girls,37.62
user7,password7,hwy61revisited,Bob Dylan,Highway 61 Revisited,29.38
user8,password8,purplerain,Prince,Purple Rain,21.97

In the example configuration above, the BodyText in the upload command expands to:

{ "permalink": "abbeyroad", "title": "The Beatles", "description": "Abbey Road", "price": "42.58"}

A Note on Security

Errthquake stores assets on encrypted file systems, but test configurations and inputs are viewable as plain text in the web portal.

Virtual users in a load test typically access accounts in test tier applications rather than production, and typically control disposable resources. Password security may be a secondary concern if an account's scope is sufficiently limited.

When load test passwords need to be secured, defining several passwords in Errthquake Secrets and using them for test accounts is a better strategy.