# RFC for new protocol converter configuration ## **Configuration Overview** The configuration is divided into several sections: 1. **Basic Settings**: Essential configurations for beginners. 2. **Conditions**: Advanced rules for tag processing. 3. **Custom Protocol Settings**: Protocol-specific overrides. 4. **Advanced Processing**: For custom schemas and complex processing. --- ## **1. Basic Settings** These settings are intended for beginners and cover the essential configurations required to get started. ### **1.1. Default Protocol Settings** ```yaml tagProcessor: defaultProtocolSettings: opcua # Set the default protocol (e.g., opcua, modbus) ``` - **Purpose**: Specifies the default protocol to apply standard settings automatically. - **Usage**: Set to the protocol of the devices you are interfacing with. ### **1.2. General Options** ```yaml tagProcessor: defaultProtocolSettings: opcua # Basic grouping groupTo: enterprise: "YourEnterprise" # Set your enterprise name site: "DefaultSite" # Set your site name ``` - **Purpose**: Defines the basic grouping hierarchy for your tags. - **Usage**: Replace `"YourEnterprise"` and `"DefaultSite"` with your actual enterprise and site names. ### **1.3. Renaming Tags** ```yaml tagProcessor: renaming: "OriginalTagName1": "NewTagName1" "OriginalTagName2": "NewTagName2" ``` - **Purpose**: Simplifies complex or vendor-specific tag names. - **Usage**: Provide a mapping of original tag names to new, simplified names. ### **1.4. Regex-Based Replacements** ```yaml tagProcessor: replaceAll: tagFolder: "/[^a-zA-Z0-9_\\/]/g": "" # Removes special characters except '/' ``` - **Purpose**: Sanitizes tag folders or names using regex patterns. - **Usage**: Specify the regex pattern and replacement string. --- ## **2. Conditions** This section allows advanced users to define conditional rules for tag processing. ### **2.1. Conditions Structure** ```yaml conditions: - if: groupTo: : tagOverrides: : ``` - **Purpose**: Applies specific processing rules when certain conditions are met. - **Components**: - **if**: A bloblang condition expression that evaluates to `true` or `false`. Boolean operators such as `&&` and `||` can be used to apply multiple conditions. - **groupTo**: Specifies the grouping hierarchy based on the condition. - **tagOverrides**: Optionally overrides tag properties such as the tagFolder or the tagName. - **Edge Cases**: the conditions are executed from top to bottom and a message / a tag will be matched against all conditions. It starts with the “basic grouping” (see `tagProcessor.groupTo`) and then every condition and its overrides are executed in order. ### **2.2. Examples** #### **2.2.1. Grouping by Modbus Slave ID** ```yaml conditions: - if: meta("modbus_slave_id") == 10 groupTo: area: "AreaA" line: "Line1" ``` - **Explanation**: If the `modbus_slave_id` is `10`, group the tags under `AreaA` and `Line1`. #### **2.2.2. Overriding Tag Names** ```yaml conditions: - if: meta("modbus_address") == "B123" tagOverrides: tagName: "Temperature" ``` - **Explanation**: If the `modbus_address` is `"B123"`, override the tag name to `"Temperature"`. --- ## **3. Custom Protocol Settings** Advanced users can override default protocol settings with custom configurations. ### **3.1. Custom Protocol Settings Structure** ```yaml customProtocolSettings: tagFolder: meta("") tagName: meta("") tagType: meta("") ``` - **Purpose**: To specify protocol-specific metadata for tag properties. - **Usage**: Replace `` with the appropriate metadata keys for your protocol. ### **3.2. Example for OPC UA** ```yaml customProtocolSettings: tagFolder: meta("opcua_tag_group") tagName: meta("opcua_tag_name") tagType: meta("opcua_tag_type") ``` - **Explanation**: Overrides the default OPC UA tag folder, name, and type with the specified metadata. --- ## **4. Advanced Processing** For scenarios requiring custom schemas or complex processing logic, advanced users can enable Bloblang scripting. When `advancedProcessing` is enabled, all other configurations are ignored. ### **4.1. Enabling Advanced Processing** ```yaml advancedProcessing: enabled: true script: | # Add your Bloblang script here meta kafka_topic = "umh.v1.._historian" # required value root = { # required value as well "timestamp_ms": 123, "yourTagName": this.content().string() } ``` - **Purpose**: Allows the use of Bloblang scripts for custom processing. - **Usage**: Set `enabled` to `true` and provide your Bloblang script under `script`. ### **4.2. Important Notes** - **Override Behavior**: When `advancedProcessing` is enabled, all other settings in the configuration are ignored. - **Fallback Option**: Use this option if you need to implement custom schemas not supported by the default configuration. --- ## **5. The Output** ### **5.1. Expected Inputs** The tag processor outputs messages in the `_historian` schema, formatted as JSON. **Message Structure:** - `this.timestamp_ms`: Unix timestamp in milliseconds. - `this.`: The tag value, enforced with ``. - `this.`: Additional metadata key-value pairs. - `meta("kafka_topic")`: The Kafka topic, structured according to ISA-95 hierarchy (enterprise, site, area, etc.). **Example Output:** ```json { "timestamp_ms": 1633072800000, "Temperature": 75.5, "serial_number": "12345", "meta": { "kafka_topic": "enterprise/site/area/line" } } ``` **Example Metadata:** ``` kafka_topic: "umh.v1...._historian.. ``` - **Explanation**: - **`timestamp_ms`**: The timestamp of the data point. - **`Temperature`**: The tag name with its value. - **`serial_number`**: An example of additional metadata. - **`meta.kafka_topic`**: The topic where the message will be published. ### **5.2. Custom Schemas** - If you require a custom schema that differs from the `_historian` schema, you must use **Advanced Processing** with Bloblang scripting. - **Note**: When using advanced processing, ensure your script outputs the data in your desired format. --- ## **6. Configuration Flow** To make the configuration intuitive, it's arranged in the following order: 1. **Basic Settings**: Start with default protocol settings and basic grouping, renaming and regex replacements. 2. **Conditions**: Define any conditional processing rules. 3. **Custom Protocol Settings**: Override protocol defaults if necessary. 4. **Advanced Processing**: Use Bloblang scripting for custom needs (overrides all previous settings). --- ## **7. Example Configuration** Here is an example that incorporates all the discussed elements: ```yaml tagProcessor: defaultProtocolSettings: opcua # Basic Setting groupTo: # Basic Grouping enterprise: "MyEnterprise" site: "MainSite" renaming: # Tag Renaming "ComplexTagName1": "SimpleName1" "ComplexTagName2": "SimpleName2" replaceAll: # Regex Replacements tagFolder: "/[^a-zA-Z0-9_\\/]/g": "" # Sanitize tag folders tagName: "/[^a-zA-Z0-9_]/g": "" # Sanitize tag names conditions: # Advanced Conditions - if: always() groupTo: area: "DefaultArea" tagOverrides: tagFolder: meta("opcua_tag_group") tagName: meta("opcua_tag_name") tagType: meta("opcua_tag_type") tagMetadata: serial_number: "SN12345" - if: meta("modbus_slave_id") == 10 groupTo: area: "ProductionArea" line: "AssemblyLine1" customProtocolSettings: # Protocol Overrides tagFolder: meta("opcua_tag_group") tagName: meta("opcua_tag_name") tagType: meta("opcua_tag_type") advancedProcessing: # Advanced Processing (Disabled) enabled: false script: | root = this # Custom Bloblang script ``` --- ## **8. Templating and Custom Variables ** To enhance flexibility and usability, the configuration supports templating using the Go templating language. This allows IT personnel or system integrators to create templates with custom variables. OT personnel or end-users can then fill in these variables via the Management Console UI under the **Connections** section. By utilizing templating, you can create more complex configurations that remain maintainable by non-IT personnel, ensuring that advanced setups are still accessible and manageable. ### **8.1. Purpose of Templating** - **Simplify Configuration for End-Users**: Templating abstracts the complexity of the configuration file, allowing end-users to input only the necessary variables. - **Reuse and Scalability**: Templates can be reused across multiple deployments, promoting consistency and reducing configuration errors. - **Customization**: Enables the creation of customized configurations tailored to specific use cases without modifying the underlying logic. ### **8.2. How Templating Works** - **Template Creation**: An IT professional or system integrator creates a configuration template using Go templating syntax, defining placeholders for variables. - **Variable Definition**: OT personnel or end-users provide values for these variables via the Management Console UI. - **Template Rendering**: The system processes the template, replacing placeholders with actual values to generate the final configuration. ### **8.3. Defining Custom Variables** Custom variables are defined by the template creator and are intended to be filled out by the end-user. **Example of Custom Variables Input in Management Console UI:** ```yaml customVariables: siteName: "TestSite" opcuaFolderNodeID: "ns=2;i=5567" ``` - **Explanation**: - **`siteName`**: The name of the site, provided by the end-user. - **`opcuaFolderNodeID`**: The OPC UA folder Node ID to filter tags. ### **8.4. Creating a Protocol Converter Template** Here's how a template might look with custom variables: ```yaml tagProcessor: defaultProtocolSettings: opcua # Basic Setting groupTo: # Basic Grouping enterprise: "MyEnterprise" site: "{{ .customVariables.siteName }}" conditions: - if: meta("opcua_nodeid") != "{{ .customVariables.opcuaFolderNodeID }}" tagOverrides: tagValue: deleted() ``` - **Template Components**: - **`{{ .customVariables.siteName }}`**: Placeholder that will be replaced with the actual `siteName` provided by the user. - **`{{ .customVariables.opcuaFolderNodeID }}`**: Placeholder for the `opcuaFolderNodeID` variable. - **Condition Logic**: Deletes tags that do not match the specified OPC UA Node ID, effectively filtering tags. ### **8.5. Using the Template** **Steps for End-Users:** 1. **Select Template**: Choose the appropriate template in the Management Console under **Connections**. 2. **Provide Variables**: Input values for the custom variables as prompted by the UI. - Example: ```yaml siteName: "FactoryA" opcuaFolderNodeID: "ns=2;i=5567" ``` 3. **Save Configuration**: The system renders the template with the provided variables, generating the final configuration. ### **8.6. Benefits of Templating** - **User-Friendly**: Allows non-IT personnel to configure complex settings without dealing with YAML syntax. - **Consistency**: Ensures standardized configurations across different deployments. - **Efficiency**: Saves time by reusing templates for similar setups, reducing duplication of effort. - **Maintainability**: Updates to the template automatically apply to all instances using it (when re-rendered), simplifying maintenance. ### **8.7. Advanced Templating Features** Templating supports advanced features that can further enhance configuration flexibility. #### **8.7.1. Conditional Logic** You can include conditional statements in your templates. **Example:** ```yaml {{ if .customVariables.enableFiltering }} conditions: - if: meta("opcua_nodeid") != "{{ .customVariables.opcuaFolderNodeID }}" tagOverrides: tagValue: deleted() {{ end }} ``` - **Explanation**: The `conditions` section is included only if `enableFiltering` is `true`. #### **8.7.2. Looping Constructs** You can iterate over lists or maps. **Example:** ```yaml groupTo: enterprise: "MyEnterprise" site: "{{ .customVariables.siteName }}" {{ range $key, $value := .customVariables.additionalGroupings }} {{ $key }}: "{{ $value }}" {{ end }} ``` - **Explanation**: Adds additional grouping levels based on user-provided key-value pairs. ### **8.8. Example Scenario** **Scenario**: A company has multiple sites with different OPC UA configurations but wants to maintain a consistent data processing structure. - **Template Creation**: IT creates a template with variables for `siteName` and `opcuaFolderNodeID`. - **End-User Input**: OT personnel at each site input their specific `siteName` and `opcuaFolderNodeID` via the UI. - **Outcome**: Each site has a customized yet consistent configuration, and OT personnel didn't need to understand the underlying YAML or templating syntax. ### **8.9. FAQs on Templating** **Q1: Can end-users modify the template?** - **A**: Generally, end-users provide variable values via the UI but do not modify the template itself. Template modifications are intended to be performed by IT personnel or system integrators. **Q2: What happens if a variable is missing or invalid?** - **A**: The system should validate inputs and prompt the user to correct any missing or invalid variables before rendering the final configuration. **Q3: Can templates include advanced processing sections?** - **A**: Yes, templates can include any configuration sections, including `advancedProcessing`. However, keep in mind that enabling `advancedProcessing` will override other configurations. **Note**: For detailed guidance on Go templating syntax, refer to the [Go Templating Documentation](https://golang.org/pkg/text/template/). --- ## **10. Additional Notes** - **Order of Processing**: The configuration is processed in the order presented. Enabling `advancedProcessing` will override all other settings, including those defined in templates. - **Error Handling**: Ensure all metadata keys used in `meta("")` and all custom variables are available and correctly formatted; otherwise, processing may fail. - **Documentation**: Refer to the UMH Management Console documentation for detailed explanations of each configuration option and templating guidelines. --- ## **11. FAQs** ### **Q1: What happens if both `customProtocolSettings` and `advancedProcessing` are enabled?** - **Answer**: If `advancedProcessing.enabled` is set to `true`, all other configurations, including `customProtocolSettings`, are ignored. Only the Bloblang script in `advancedProcessing.script` is executed. ### **Q2: Can I use `conditions` to perform tag renaming and replacements?** - **Answer**: Yes, you can use `conditions` along with `tagOverrides` to rename tags or modify tag properties based on specific conditions. ### **Q3: How do I ensure that the outputs conform to the expected schema?** - **Answer**: By default, the tag processor outputs messages in the `_historian` schema. Ensure your tag names, types, and metadata are correctly configured. If using advanced processing, your Bloblang script should output data in the desired format. --- **Feel free to reach out if you have any questions or need further assistance with your configuration.**