Package io.nosqlbench.engine.api.activityconfig.yaml

Class OpTemplate

java.lang.Object
io.nosqlbench.engine.api.activityconfig.yaml.OpTemplate
All Implemented Interfaces:
io.nosqlbench.engine.api.util.Tagged
Direct Known Subclasses:
OpData, OpDef
public abstract class OpTemplate extends Object implements io.nosqlbench.engine.api.util.Tagged

The OpTemplate is the developer's view of the operational templates that users provide in YAML or some other structured format.

Terms

Within this documentation, the word OpTemplate will refer to the template API and semantics. The word user template will refer to the configuration data as provided by a user.

OpTemplates are the native Java representation of the user templates that specify how to make an executable operation. OpTemplates are not created for each operation, but are used to create an mostly-baked intermediate form commonly known as a ready op. It is the intermediate form which is used to create an instance of an executable op in whichever way is the most appropriate and efficient for a given driver.

This class serves as the canonical documentation and API for how user templates are mapped into a fully resolved OpTemplate. User-provided op templates can be any basic data structure, and are often provided as part of a YAML workload file. The description below will focus on structural rules rather than any particular encoding format. The types used are fairly universal and easy to map from one format to another.

A long-form introduction to this format is included in the main NoSQLBench docs at docs.nosqlbench.io under the Designing Workloads section.

A few structural variations are allowed -- No specific form enforced. The reasons for this are: 1) It is generally obvious what as user wants to do from a given layout. 2) Data structure markup is generally frustrating and difficult to troubleshoot. 3) The conceptual domain of NB op construction is well-defined enough to avoid ambiguity.

Type Conventions

For the purposes of simple interoperability, the types used at this interface boundary should be limited to common scalar types -- numbers and strings, and simple structures like maps and lists. The basic types defined for ECMAScript should eventually be supported, but no domain-specific objects which would require special encoding or decoding rules should be used.

Standard Properties

Each op template can have these standard properties:
  • name - every op template has a name, even if it is auto generated for you. This is used to name errors in the log, to name metrics in telemetry, and so on.
  • description - an optional description, defaulted to "".
  • statement - An optional string value which represents an opaque form of the body of an op template
  • params - A string-object map of zero or more named parameters, where the key is taken as the parameter name and the value is any simple object form as limited by type conventions above.
  • bindings - A map of binding definitions, where the string key is taken as the anchor name, and the string value is taken as the binding recipe.
  • tags - A map of tags, with string names and values
The user-provided definition of an op template should capture a blueprint of an operation to be executed by a native driver. As such, you need either a statement or a set of params which can describe what specific type should be constructed. The rules on building an executable operation are not enforced by this API. Yet, responsible NB driver developers will clearly document what the rules are for specifying each specific type of operation supported by an NB driver with examples in YAML format.

OpTemplate Construction Rules

The available structural forms follow a basic set of rules for constructing the OpTemplate in a consistent way.

  1. A collection of user-provided op templates is provided as a string, a list or a map.
  2. All maps are order-preserving, like LinkedHashMap
  3. For maps, the keys are taken as the names of the op template instances.
  4. The content of each op template can be provided as a string or as a map.
    1. If the op template entry is provided as a string, then the OpTemplate is constructed as having only a single statement property (in addition to defaults within scope). as provided by OpTemplate API.
    2. If the op template entry is provided as a map, then the OpTemplate is constructed as having all of the named properties defined in the standard properties above. Any entry in the template which is not a reserved word is assigned to the params map as a parameter, in whatever structured type is appropriate (scalar, lists, maps).

    Example Forms

    The valid forms are shown below as examples.

    One String Statement

    
 statement: statement

    List of Templates

    
 statements:
   - statement1
   - statement2

    List of Maps

    
 statements:
   - name: name1
     stmt: statement body
     params:
       p1: v1
       p2: v2

    List Of Condensed Maps

    
 statements:
   - name1: statement body
     p1: v1
     p2: v2

  • Field Details

  • Constructor Details

    • OpTemplate

      public OpTemplate()

  • Method Details

    • getDesc

      public abstract String getDesc()
      Returns:
      a description for the op template, or an empty string

    • getName

      public abstract String getName()
      Returns:
      a name for the op template, user-specified or auto-generated

    • getTags

      public abstract Map<String,String> getTags()
      Return a map of tags for this statement. Implementations are required to add a tag for "name" automatically when this value is set during construction.
      Specified by:
      getTags in interface io.nosqlbench.engine.api.util.Tagged
      Returns:
      A map of assigned tags for the op, with the name added as an auto-tag.

    • getBindings

      public abstract Map<String,String> getBindings()

    • getParams

      public abstract Map<String,Object> getParams()

    • getParamsAsValueType

      public <T> Map<String,T> getParamsAsValueType(Class<? extends T> type)

    • removeParamOrDefault

      public <V> V removeParamOrDefault(String name, V defaultValue)

    • getParamOrDefault

      public <V> V getParamOrDefault(String name, V defaultValue)

    • getParam

      public <V> V getParam(String name, Class<? extends V> type)

    • getOptionalStringParam

      public <V> Optional<V> getOptionalStringParam(String name, Class<? extends V> type)

    • getOptionalStringParam

      public Optional<String> getOptionalStringParam(String name)

    • getParsed

      public Optional<io.nosqlbench.virtdata.core.templates.ParsedStringTemplate> getParsed(Function<String,String>... rewriters)
      Parse the statement for anchors and return a richer view of the StmtDef which is simpler to use for most statement configuration needs.
      Returns:
      an optional ParsedStringTemplate

    • getParsed

      public Optional<io.nosqlbench.virtdata.core.templates.ParsedStringTemplate> getParsed()

    • getOp

      public abstract Optional<Map<String,Object>> getOp()

    • asData

      public Map<String,Object> asData()

    • getStmt

      public Optional<String> getStmt()
      Legacy support for String form statements. This is left here as a convenience method, however it is changed to an Optional to force caller refactorings.
      Returns:
      An optional string version of the op, empty if there is no 'stmt' property in the op fields, or no op fields at all.

    • getParamReader

      public io.nosqlbench.nb.api.config.params.Element getParamReader()

    • size

      public int size()
      Returns:
      the size of remaining fields from the op template and the params map.

    • remainingFields

      public Map<String,Object> remainingFields()
      Returns:
      the map of all remaining fields from the op template and the params map.

    • assertConsumed

      public void assertConsumed()