Using End-to-End Monitoring

This section talks about the Dashboard, Alert and Environment Groups and Stages features of End-to-End Monitoring.

Dashboard

The dashboard for End-to-End Monitoring gives you a collective view of all the business transactions carried out within your cloud platform. The default Dashboard view includes the transaction widgets and the default All Transactions group. The All Transactions group contains all the transactions within the application and cannot be deleted.

Note: When you perform sorting, the All transactions group remains on the top of the list.


  • – An alert icon to notify you of any new transaction notifications.
  • – User help that includes the help content for this application.
  • – Use the link provided here to log off from the application.
  • Time range – Below is the drop-down list for time range selection. By default, it is set to half an hour prior from the current system time. For example, if current system time is 3:30 PM, then the time range shows as 3 PM to 3 30 PM. Click next to the time range to select a specific duration.
  • Time zone – This is displayed next to the Time range field. Any time stamp displayed in End-to-End Monitoring is based on the user’s registered time zone specified in Software AG Cloud.
    Note that not all the time zones in Software AG Cloud are supported in End-to-End Monitoring. If a time zone in Software AG Cloud is not supported, then the time stamp in End-to-End Monitoring defaults to the Pacific Standard Time (PST) time zone.
  • Transaction widgets

    The following widgets are available on the dashboard:

    Creating a Group of Transactions

    Creating a group of transactions allows you to categorize the transactions based on your requirements.

    1. In the Dashboard page, click Create Group.

    2. In the Create group dialog box, provide the following details:

      Parameter Description
      Name Type a name for this group.

      Note: The group name cannot be edited after you save this form.
      Environment Provide the environment where this group will be created.
      Stage Provide the stage under which this group will be created.

      For webMethods.io Integration Cloud tenants this can be either of the following options:

      • Development

      • Test

      • Pre-Live

      • Live

      However, in case of Software AG cloud tenants, the stage is automatically mapped to the environment. Hence, the stage option is not available during group creation in Software AG cloud tenants.
      Status Group the transactions on the basis of their status.

      Note: Click Select All to select all the listed options.

      It can be a combination of:

      • Success

      • Failed

      Starts with Group the transactions on the basis of the application from where the business flow starts.

      You can choose one or more products when creating the group. For example, you can have a group where you have one transaction starting from webMethods.io API and another transaction starting from webMethods.io Integration.

      Note: Click Select All to select all the listed options.

      You can select from the following:

      • webMethods.io API

      • webMethods.io B2B

      • Note: Currently, End-to-End Monitoring support for B2B is very basic. You may not get complete trace for all types of B2B transactions. For example, transactions invoked by the AS2 protocol may not provide you the complete trace results.

      • webMethods.io Integration

      • Integration Cloud

      • webMethods Integration Server
      Duration (ms) Group the transactions on the basis of their execution time in milliseconds.

      You can select from the following:

      • – Select this operator to group the transactions that have an execution time which is lesser than or equal to the value specified here.

      • – Select this operator to group the transactions that have an execution time which is greater than or equal to the value specified here.

      • – Select this operator to group the transactions that have an execution time which is between the values specified here. For example, if you specify 20 and 30 (ms), then both the values 20 and 30 are also taken into consideration.
      Product(s) involved Group the transactions on the basis of the product(s) involved in a business transaction.

      Use the drop-down list to select one or more products. The group includes all the transactions for the selected product. If you select more than one product, it will list the transactions for the standalone products along with the transactions that involve both the products. For example, if you select webMethods.io API and webMethods.io Integration, the group will include the following:

      Note: Click Select All to select all the listed options.

      • all transactions that involve only webMethods.io Integration

      • all transactions that involve only webMethods.io API

      • all transactions that involve both webMethods.io Integration and webMethods.io API.

      Transaction name(s) Group the transactions on the basis of their names. Click + to add more than one entry.

      Note: Click X to remove the added names.
      Error Message(s) Group the transactions on the basis of an error message. Click + to add more than one entry.
    3. Click Save.

    Viewing the Transactions List

    1. On the Dashboard page, select the group for which you want to view the transactions. On the Transactions list page, you will see all the transactions for the selected time frame and the set filter(s).

    2. Click on the top-right corner of the screen to open the Show Columns dialog box.

    3. Select the parameters for which you want to view the transaction details and click Save. The transactions list shows the following parameters:

      Parameter Description
      Status Status of the transaction. Currently, we have the following states:

      • – Transaction successful.

      • – Transaction failed.
      Name Name of the transaction. This name is based on the starting point of the transaction. For example, if the transaction starts from webMethods.io API, then this name would be the name of the API.
      Duration (ms) Total time taken by the transaction to complete.
      Starts with The application from which the transaction starts. For example, webMethods.io Integration.
      Started on Start time for the transaction.
      Error message Message with which the transaction failed.
      Trace ID This is a unique identifier for the transaction.

    This is an example of the transaction list view:

    You can also filter the transactions from this page. For more information, see Filtering the Business Transactions.

    Editing a Transactions Group

    1. On the Dashboard page, select in the row for the transaction that you want to edit.

      Note: Editing a transaction group impacts all the rules associated with it.

    2. Make the necessary changes and click Update.

    Deleting a Transactions Group

    1. On the Dashboard page, select in the row for the transaction that you want to delete.

      Note: Deleting a transaction group deletes all the rules and the rule violations associated with it.

    2. Click Delete. On successful deletion, you will see a confirmation message on the screen.

    Filtering the Business Transactions

    You can filter the business transactions within any group to view a custom set of transactions.

    Note: Filters that were set during group creation are already applied when you open the group.

    To filter the transactions

    1. On the Transactions list page, click as shown in the following example:

    2. In the Filter dialog box, you can choose from the following filters:

      Note: Filters set during group creation are greyed out and not available for selection.

      Value Description
      Status Select the transaction status. You can select from the following and click Save:

      • Success – Lists all the completed business transactions.

      • Failed – Lists all the failed transactions.
      Name Type the name of the business transaction and click Save. To add more than one transaction name, click +.
      Duration Select from the following operators using the drop-down list and provide a valid time in milliseconds:

      • – Select this operator to filter the transactions that have an execution time which is lesser than or equal to the value specified here.

      • – Select this operator to filter the transactions that have an execution time which is greater than or equal to the value specified here.

      • – Select this operator to group the transactions that have an execution time which is between the values specified here. For example, if you specify 20 and 30 (ms), then both the values 20 and 30 are also taken into consideration.
      Transaction starts with Filter the transactions on the basis of the application from where the business flow starts. Select the checkbox next to the product. You can select more than one product.
      Error message Filter the transactions based on the error message with which it failed. Type the error message and click Save. For example, server not found. You can filter based on more than one error message by clicking+next to the error message box.
      Product(s) involved Filter the transactions based on the products that are involved with this transaction. Select the checkbox next to the product and click Save. You can select more than one product.
      Trace ID Filter the transactions based on this unique identifier associated with a transaction. Enter the trace ID and click Save. You can filter based on more than one Trace ID by clicking + next to the trace ID box.
    3. The applied filters show up on the top of the page as shown in the following example:

      You can close a particular filter by selecting the filter and clicking X adjacent to the filter name.

    4. After you apply the filters and retrieve the list of transactions, you can further sort the list by clicking the column headings. Sort in ascending or descending order by toggling the column header.

    Business Transaction Details Page

    The Business Transaction Details page provides the following information about the transaction:

    Parameter Description
    Name Transaction name.
    Trace ID Distinct identifier for the transaction generated internally by the application.
    Status Status of the transaction.
    Start period Time at which the transaction started.
    End period Time at which the transaction ended.
    Duration (ms) Total time taken by the transaction to execute. This includes the Total transit time and the Processing time of the transaction. Total transit time is the time from which the first product in the cross-product transaction receives the request from the client until it sends back the response to the client, excluding the processing time taken by each product. This is dependent on network latency and other factors such as Software AG products on different tenants, in a cross-product transaction.

    This is an example of a business transactions details page:

    Duration

    Duration is the total time taken by a transaction from the time a client initiates a request to the time a response is received by the client. Let us consider the following scenario where the duration for a transaction to complete is 200ms. In the following image, 200ms is the time difference between INT22 and INT11. The time taken by the transaction within the API component which is 50ms is already taken into account as part of this 200ms value. End-to-End Monitoring records the duration only from the time a client request enters the first component of the cloud platform (INT11) and the time the response exits the last component of the cloud platform (INT22).

    Note: The application does not note the time the client sends the request (T1) and the time the client receives the response (T2), as this happens outside the Software AG environment.

    Business Flow Map

    This provides a logical representation of the business flow showing the path taken by the transaction through the various cloud components. Also, the processing time for the transaction within each component is visible.

    A legend is available to identify the status of a transaction. An example of a business flow is as follows:

    Viewing the Component Details for a Transaction

    To view the component details for a transaction

    1. On the Application dashboard, select the transaction group for which you want to view the details.

    2. On the Business Transaction Details page, click the component from the business flow map for which you want to view the details. The details open in a separate pane and includes the following information:

      Field Description
      Tenant name Provides the tenant of the transaction.
      Status Provides the status of the transaction. Status can be:

      • Success

      • Failed
      Cause of failure Provides the reason for the error to occur. If a transaction is successful, then this parameter is not visible.
      Processing time Time spent by the transaction within this component.
      Stage Phase at which the failure has occurred during the execution of a transaction. Stage represents an activity in the life cycle of the transaction.
      Environment group The mapped environment group where the failure has occured during the execution of the transaction.
      More details Click this URL to open the details of the component highlighted in the Business Flow Map. Currently, this link is available for B2B and WebMethods Integration Cloud nodes only. Clicking this link redirects you to an instance of the respective component in a new tab. For example, if you click on an Integration Cloud component, you are redirected to the instance of webMethods Integration Cloud in a new tab.

    This is an example of component details pane:

    This is an example of the More details page of a workflow:

    Alert

    The Alert page lists all the Rule violations and the Rule list. From the Alert page, you can create rules. These rules are a set of conditions. When these conditions are met by a group of transactions, the rule violation occurs. You can configure a rule such that it triggers an alert in the form of an on-screen notification or email or both. The rules that you create are applicable to all the transactions of a group with which it is associated. For information on creating a rule, see Creating a Rule.

    Viewing Alerts

    To view the alerts

    1. Click at the top of the screen to open the Rule violations pane. Here, you will see a list of all the new rule violations along with the older ones. If there are new rule violations, you will see a number with . For example, two new rule violations are indicated by at the top of the screen.

    2. In the Rule violations pane, click View all to go to the Rule violations section of the Alert page.

    3. In the Rule violations tab, you will see all the rules violations listed in a table format.

    Creating a Rule

    Note: The rules are evaluated by the system for every 10 minutes interval.

    The application triggers alerts when a group of transactions violates the conditions defined in a rule associated with it.

    To create a rule

    1. On the Alert page, click Rule list.

    2. Select Create rule.

      Note: The application provides you a set of default rules. For more information, see Default Rules.

    3. In the Create rule dialog box, provide the following details:

      • In the General information section, provide the details for the following and click Next.

        Value Description
        Name Provide a name for the rule.

        Note: You cannot change the rule name once you save the rule.
        Description Provide a meaningful description for the rule.
        Environment Select the environment that the rule is applicable to, from the drop-down list. You can select only one environment at a time.
        Stage Provide the stage to which this rule is applicable.

        For webMethods.io Integration Cloud tenants this can be either of the following options:

        • Development

        • Test

        • Pre-Live

        • Live

        However, in case of Software AG cloud tenants, the stage is automatically mapped to the environment. Hence, the stage option is not available during rule creation in Software AG cloud tenants.
        Group Select the group from the drop-down list. This group list will only contain the groups associated with the tenant selected in the Environment drop-down.

        Note: Currently, you can associate a rule only to a single group. However, a group can be associated with multiple rules.

      This is an example of the General information section:

      • In the Rule & Action section, provide the details for the following and click Next.

        Value Description
        KPI This is the key performance index. Select from the following options:

        • Error count

        • Error rate

        • Average response time
        Time range The time range on the basis of which this rule should trigger an alert when violated. Select from the following options:

        • In last 30 mins

        • In last 1 hour

        • In last 12 hours

        • In last 24 hours
        Operator Select the operator on the basis of which this rule should trigger an alert when violated. Select from the following options:

        • Less than (<) – lesser than the value specified in the Value field.

        • Greater than (>) – greater than the value specified in the Value field.

        • Equals to (=) – equal to the value specified in the Value field.

        Note: Value could be a number, percentage, or time and changes on the basis of the KPI you select.
        Value This field varies on the basis of your selection in the KPI field. The value field changes on the basis of the KPI you select:

        • For Error count, it is set to Value(number).

        • For Error rate, it is set to Value(%).

        • For Average response time, it is set to Value(ms).
      • In the Action section, select your notification preference and click Next. You can select either or both of the following options:

      • Value Description
        Show app notification Select this option to get on-screen app notifications.
        Send email Select this option to subscribe for email alerts whenever a group violates a rule. Provide the recipient email address. You can enter more than one email address separated by a comma.

      This is an example of the Rule & Action section:

      • In the Summary section, verify all the details provided by you and click Save.

    Default Rules

    The application provides you with a set of pre-defined rules. By default, these rules are disabled. These pre-defined rules are as follows:

    Note: Other than the Name, you can modify all the other values as per your requirements.

    Name Description Group KPI Time Range Operator Value Show app notification Send email
    Average response time is out of compliance The average response time for all transactions in last 1 hour is greater than 1000 ms. All Transactions Average response time In last 1 hour > 100 (ms) True False
    Error count is out of compliance The error count for all transactions in last 1 hour is more than 100. All Transactions Error count In last 1 hour > 10 True False
    Error rate is out of compliance The error rate for all transactions in last 1 hour exceeds 10%. All Transactions Error rate In last 1 hour > 10% True False

    Note: Each of these default rules will be available for every environment in your tenant. For instance, if you have four environments then the Rule list will show 12 default rules.

    Working with Rule Violations

    The Rule violations page provides details of all the violations. Use this page to search for specific violations and to view all the violations within the various groups for a specified duration. The following table provides you details of all the actions possible in the Rule violations page:

    Action Description
    Show all Use this drop-down list to filter all the violations on the basis of duration or status. The values include:

    • All open

    • All resolved

    • Last 1 hour

    • Last 12 hours

    • Today

    • Yesterday

    • Last 7 days
    This is the refresh button. Click this to refresh the list of rule violations. It shows the time when the list was last refreshed.
    This is the column view settings button. Click this to customize the columns you want to view for the rule violations.
    Select this drop-down list to filter the rule violations on the basis of group name. You also get a search box when you click this drop-down list. Use this search box to type a group name that you want to search. To search the rule violations across all the available groups, click Select All.

    Note: You can select more than one group.
    Search by name Type a rule name in this search box to filter all the rule violations on the basis of rule name.

    To sort the rule violations, click on the header row of any column. You can sort in both ascending and descending order.

    Working with Rule List

    The application lists all the rules that you create in the Rule list page. The following table provides you details of all the actions possible in the Rule list page:

    Action Description
    Create rule Use this action button to create a new rule. For more information, see Creating a Rule.
    To edit a rule, click this action button on the row of the corresponding rule.

    Note: You cannot modify the rule name.
    To delete a rule, click this action button on the row of the corresponding rule.
    Select this toggle button in the Status column to enable or disable a rule. indicates that a rule is enabled and indicates that a rule is disabled.

    Note: When you use this button, you will see corresponding system messages on screen to indicate if a rule is enabled or disabled successfully.
    This is the column view settings button. Click this to customize the columns you want to view for the listed rules. In the Show Columns dialog, select the columns you want to view and click Save.
    Select this drop-down list to filter the rules on the basis of group name. You also get a search box when you click this drop-down list. Use this search box to type a group name that you want to search.

    Note: You can select more than one group.
    Search by name Type a rule name in this search box to filter the rules on the basis of rule name.

    To sort the rules, click on the header row of any column. You can sort in both ascending and descending order.

    Where can I find a detailed Step-by-Step Tutorial?

    For a detailed tutorial click Step-by-Step Tutorial

    Environment Groups and Stages

    Get a granular view of the transaction data by filtering the transactions based on the environment group and the related stages.

    Environment Groups and Stages: End-to-End Monitoring from Software AG on Vimeo.

    Overview

    Customers implement their solutions with Software AG Cloud products spanning multiple environments (one product installed in each environment) as well as each environment having stages like Development, Test and Production. This feature helps in providing a more granular view of the transaction data by allowing you to filter the transactions based on the environment group and the related stages. Using the environment group, you can view the transactions that span across more than one environment. Using the stages filter, you can further filter the transaction data based on the different stages the environments of the selected environment group are mapped with.

    For example, let us assume there are two environments T1 and T2. A transaction starts from T1 and moves to T2. If the tenant is logged into T1, then the transaction details can be traced as the transaction starts from T1. However, if the tenant is logged into T2, then the transaction details are unavailable. To solve this, we have the Environment groups using which you can get the transaction details.

    Creating an Environment Group

    When you access End-to-End Monitoring from the App Switcher, you will see the following screen providing you the details on how to create environment groups. Select Next to start the environment creation process. This Environment group creation flow window will appear until you disable it using the Don’t show this message again checkbox.

    You can skip this step and create the Environment groups at a later stage, by clicking the Later option.

    Note: If you are a non-Software AG Cloud user, contact your Application Administrator to create the Environment groups.

    If you selected Later on the onboarding screen, you can also create environment groups by following these steps:

    1. Select End-to-End Monitoring. Go to User Profile > Administrator > Manage environment groups.

    2. On the Manage environment groups page, click Create.

    3. In the Create environment group window, provide an environment group name.

      Note: You cannot change this name once you create the environment group.

    4. In Filter by stage, select the stages based on which you want to filter the environments. You can select more than one stage. Stages may include the following:

      • Development
      • Testing
      • Production

      The application auto-populates the environments list based on your selection of Stages.

    5. Select all the environments that you want to include in your group and click Save.

      Note: After you select an environment, it will no longer be available for selection when you create another environment group. You can map an environment to a single environment group only.

    6. On successful creation of the environment group, the application shows you a confirmation message and you will see the newly created environment group in the Environment group listing table.

    Editing an Existing Environment Group

    1. Go to User Profile > Administrator > Manage environment groups.

    2. On the Manage environment groups page, click in the row of the environment group that you want to edit.

    3. Make the required edits.

      Note: You cannot edit the environment group name.

    4. After you make the changes, click Save.

    Deleting an existing Environment Group

    1. Go to User Profile > Administrator > Manage environment groups.

    2. In the Manage environment groups section, click in the row of the environment group that you want to delete.

      Note: The deletion of an environment group is a permanent action.

    3. On successful deletion, the application shows you a confirmation message and the environment group listing is updated.

    Note:

    • Environment groups are currently specific to the region of the logged-in environment. Any listing shows environment groups from that specific region only.

    • After you add an environment to an environment group, it is no longer available for addition to any other environment group. You will receive the below prompt when attempting to create an environment group when all environments have already been mapped.

    Non-Software AG Cloud tenants

    If you are a webMethods Integration Cloud (WMIC) tenant, the environment creation process is different from the above mentioned steps. Non-Software AG Cloud tenants will need to contact Software AG Cloud Operations with the following questionnaire to assist in the creation of the environment groups.

    1. What is the Account Name?

      The account name links the various environment groups together and is usually the customer name or the master account name in Software AG Cloud.

    2. What are the environment groups, and the environments within each group?

      Specify a list of environment groups and the list of environments within each environment group and the stages (Development, Test, Pre-Live, Live) each environment belongs to. The list of environments within an environment group are used to filter the information when a given environment group is selected as part of the stages filter. All the environment groups will be linked to the same account name. Environment is also referred to as tenants in webMethods Integration Cloud (WMIC) terminology.

    Note:

    • A webMethods Integration Cloud (WMIC) tenant can have multiple stages. However, an API Gateway(APIGW) tenant can have only one stage.
    • If information regarding environments and environment groups needs to be updated, contact Software AG Global support with information in the above questionnaire.
    • Only four pre-defined stages are available for Non-Software AG Cloud tenants:
      • Development
      • Test
      • Pre-Live
      • Live

    This table explains some valid and invalid configurations. You can use the same table template when you contact Software AG global support. Consider all these configurations for a given Account ID UPS:

    Valid and Invalid Configurations
    Tenant Name Environment Group Tenant Type Stage Valid/Reason for Invalidation
    Tenant1 Logistics WMIC
    • Development
    • Test
    • Pre-Live
    • Live
    Valid
    Tenant2 Logistics APIGW
    • Development
    Valid
    Tenant3 Sales WMIC
    • Development
    • Test
    • Pre-Live
    • Live
    Valid
    Tenant4 Sales APIGW
    • Live
    Valid
    Tenant5 Logistics APIGW
    • Pre-Live
    • Live
    Invalid

    APIGW tenants can be mapped to a single stage only.
    Tenant6 Logistics WMIC
    • Production
    • Test
    Invalid

    Production stage for WMIC is invalid. Valid options are:
    • Development
    • Test
    • Pre-Live
    • Live
    Tenant4 Logistics APIGW
    • Live
    Invalid

    Tenant4 is already mapped to the Sales group, hence it is unavailable for the other groups.

    Dashboard and Alert Changes

    The main dashboard page of End-to-End Monitoring contains a listing of all groups. There are new column entries available on this table as a part of the environment groups and stages feature. To view these entries, click . You can now add the following additional columns for table sorting purposes:

    On Dashboard page All groups table, click .

    You will notice the additional column entries:

    On Alert page Rule violations table, click .

    You will notice the same additional column entries as Dashboard page above.

    On Alert page Rule list table, click .

    You will notice the same additional column entries as Dashboard page above.

    Masthead Filters

    After environments have been grouped as per your need, this feature provides filters on the Mast head to conveniently refine information listed on the Dashboard or Alert pages as per the Environment group or Stage in view or Both.

    Dashboard page All groups table listing filtered as per Logistic Environment group on the masthead:

    Dashboard page All groups table listing filtered as per Development Stage in view on the masthead:

    Alert page Rule list table listing filtered as per Sales Environment group on the masthead:

    Where can I find a detailed Step-by-Step Tutorial?

    For a detailed tutorial click Step-by-Step Tutorial - Stages Feature

    Example - Environment Groups and Stages

    Consider a scenario where the Environment groups in your account are mapped in the following manner:

    Environment group Environment (Stage)
    Fin1 devuhm02 tenant in Development stage
    Logistics devuhm123 tenant in Development stage
    prodenv1 tenant in Production stage
    Marketing envstage1 tenant in PreStage stage

    Now, on the masthead when you select All Environment group and All Stage in view:

    You will notice that all dashboard page widgets and All groups listing automatically update information as per the masthead selections.

    Next, create a group test2 which is linked to environment devuhm02 and save the group. Change the masthead selections to Fin1 Environment group:

    You will now notice that the four widgets reflect new data and the new listing only shows test2 apart from All transactions.

    You will notice similar filtering capabilities on Alert page as well:

    Next, create a group test1 which is linked to environment prodenv1 and save the group. Change the masthead selections to Logistics Environment group:

    You will now notice that the four widgets reflect new data and the new listing only shows test1 apart from All transactions.

    Corresponding changes in Alert page:

    Migration - Environment Groups and Stages

    Consider a scenario where you have an account with groups prior to account mapping. None of these groups are mapped to an environment group or stage. Let us take the below journey of migrating these groups to an environment groups and stages mapped account.

    Let us start with creating two groups, Old_Success and Old_fail with the below selected criteria:

    You will notice at this stage that the masthead is empty and does not provide any options related to environment groups or stages:

    Now let us create environment group mapping with the below criteria:

    Environment group Environment (Stage)
    Fin1 devuhm02 tenant in Development stage
    Logistics devuhm123 tenant in Development stage
    prodenv1 tenant in Production stage
    Marketing envstage1 tenant in PreStage stage

    Let us now select edit button against Old_Success and Old_fail groups and select Development stages for both groups:

    The masthead filters are now successfully able to filter the Old_Success and Old_fail groups. Hence, these groups have been successfully migrated to the new mapping.

    Hybrid Monitoring

    A hybrid integration consists of a transaction that spans multiple platforms and products including both cloud and on-premises setups. Software AG’s End-to-End Monitoring allows you to track your integrations from the cloud through to your on-premises installation to give you a complete view of your integration landscape regardless of the topology of your IT infrastructure.

    A hybrid integration use case is one where the transaction starts in a Software AG cloud product webMethods.io Integration and this cloud product calls a service on an on-premises Software AG product Integration Server to complete the transaction.

    Hybrid monitoring is the capability that monitors the end-to-end hybrid transaction and displays it in the End-To-End Monitoring user interface.

    The End-To-End Monitoring node on the Software AG installer allows you to setup the on-premises monitoring agents that enable the end-to-end transaction monitoring of hybrid integration.

    Note:
    For this release hybrid monitoring is supported for transactions from Integration Server and Microservices Runtime. In the future we will add support for all our products and even your own products, through an open tracing API.

    Pre-requisites

    1. Contact Software AG global support to enable the End-to-End Monitoring node in Software AG Installer. Download the Software AG Installer version 10.7.

    2. Install Integration Server or Microservices Runtime, if not done already. You will need to have hybrid transaction configured for Integration Server 10.7 and Microservices Runtime 10.7.

    3. Install the latest fixes for all installed Software AG products using Software AG update manager.

    4. Configure webMethods Cloud tenant connection in the Integration Server Administration UI.

    Configuration

    Following hybrid monitoring configurations are required in Integration Server and Microservices Runtime.

    Integration Server

    1. Make the following selections during Software AG Installer setup. If you have Integration Server pre-installed, only select End-to-End Monitoring.

    2. Post installation, copy the plugin jar file:

      • From directory:
        <INSTALL-DIR>\E2EMonitoring\agent\plugins\uhm-onpremise-is-plugin.jar
      • To directory:
        <INSTALL-DIR>\IntegrationServer\lib\jars\
    3. Copy log4j properties file:

      • From directory:
        <INSTALL-DIR>\E2EMonitoring\agent\config\e2ecustomlog4j2.properties
      • To directory:
        <INSTALL-DIR>\IntegrationServer\instances\default\
    4. Add the following lines at the end of file:
      <INSTALL-DIR>\profiles\IS_default\configuration\custom_wrapper.conf

      • Microsoft Windows

        wrapper.java.additional.501=-Xbootclasspath/a:..\..\..\E2EMonitoring\agent\uhm-apm-agent.jar;"%JAVA_BOOT_CLASSPATH%"
        wrapper.java.additional.502=-javaagent:..\..\..\E2EMonitoring\agent\uhm-apm-agent.jar=logging.dir=.\logs\
      • Linux

        wrapper.java.additional.501=-Xbootclasspath/a:../../../E2EMonitoring/agent/uhm-apm-agent.jar:"%JAVA_BOOT_CLASSPATH%"
        wrapper.java.additional.502=-javaagent:../../../E2EMonitoring/agent/uhm-apm-agent.jar=logging.dir=./logs/                                                                                                    
    5. Open the file: <INSTALL-DIR>\profiles\IS_default\configuration\custom_wrapper.conf, edit the property Dlog4j.configurationFile and append e2ecustomlog4j2.properties to it.

      • Microsoft Windows:

        • Existing

          wrapper.java.additional.204=Dlog4j.configurationFile="<INSTALL-DIR>\profiles\IS_default\configuration\logging\log4j2.properties,.tc.custom.log4j2.properties"
        • Appended

          wrapper.java.additional.204=Dlog4j.configurationFile="<INSTALL-DIR>\profiles\IS_default\configuration\logging\log4j2.properties,.tc.custom.log4j2.properties,e2ecustomlog4j2.properties"
      • Linux:

        • Existing

          wrapper.java.additional.204=-Dlog4j.configurationFile="<INSTALL-DIR>/profiles/IS_default/configuration/logging/log4j2.properties,.tc.custom.log4j2.properties"
        • Appended

          wrapper.java.additional.204=-Dlog4j.configurationFile="<INSTALL-DIR>/profiles/IS_default/configuration/logging/log4j2.properties,.tc.custom.log4j2.properties,e2ecustomlog4j2.properties"

    Microservices Runtime

    1. Make the following selections during Software AG Installer setup. If you have Microservices Runtime pre-installed, only select End-to-End Monitoring.

    2. Post installation, copy the plugin jar file:

      • From directory:
        <INSTALL-DIR>\E2EMonitoring\agent\plugins\uhm-onpremise-is-plugin.jar
      • To directory:
        <INSTALL-DIR>\IntegrationServer\lib\jars\
    3. Copy log4j properties file:

      • From directory:
        <INSTALL-DIR>\E2EMonitoring\agent\config\e2ecustomlog4j2.properties
      • To directory:
        <INSTALL-DIR>\IntegrationServer\
    4. Add the following lines at the end of file:

      • Microsoft Windows

        <INSTALL-DIR>\IntegrationServer\bin\setenv.bat

        set JAVA_UHM_OPTS=-javaagent:"../E2EMonitoring/agent/uhm-apm-agent.jar=logging.dir=./logs/" -Xbootclasspath/a:"../E2EMonitoring/agent/uhm-apm-agent.jar"
        set JAVA_CUSTOM_OPTS=%JAVA_CUSTOM_OPTS% %JAVA_UHM_OPTS%
        set JAVA_CUSTOM_OPTS=%JAVA_CUSTOM_OPTS% -Dlog4j.configurationFile="./e2ecustomlog4j2.properties"
      • Linux

        <INSTALL-DIR>\IntegrationServer\bin\setenv.sh

        JAVA_UHM_OPTS="-javaagent:../E2EMonitoring/agent/uhm-apm-agent.jar=logging.dir=./logs/ -Xbootclasspath/a:../E2EMonitoring/agent/uhm-apm-agent.jar"
        JAVA_CUSTOM_OPTS="${JAVA_CUSTOM_OPTS} ${JAVA_UHM_OPTS}"
        JAVA_CUSTOM_OPTS="${JAVA_CUSTOM_OPTS} -Dlog4j.configurationFile=./e2ecustomlog4j2.properties"

    What about multiple runtimes?

    There are scenarios when multiple runtimes like multiple Integration Server cluster nodes or Integration Server installations are pointing to the same end-to-end agent jar file. In such situations, you need to make hybrid monitoring configuration changes in each runtime. This is applicable to both Integration server and Microservices Runtime.

    Setting up what to trace

    You now need to choose the on-premise services that you wish to trace in End-to-End Monitoring. This is done by changing the auditing setting of each service. You can choose to trace both top level services or private sub-level services depending on how much information you want to trace. However, take caution not to trace too much as this will consume your cloud storage and effectively reduce the number of historical transactions that you can view. It can also potentially impact the performance of your integrations and increase network traffic. Note that you do not need to trace every sub-service to ensure that the transaction is monitored*, any errors are propagated up the chain until they are caught by an audited service.

    Note:

    • * Excludes asynchronous services that are invoked using messaging or sub-threads. In which case you need to have these services audited. They are considered top level services in their own right.
    • This setup piggybacks the on-premises service auditing feature.
    1. In Software AG Designer, for the on-premises Integration Server services associated with hybrid transaction including nested services, enable auditing with the following values:

      Enable auditing: Always
      Log on: Error, Success and start

    2. Here is an example of a hybrid transaction in the End-to-End Monitoring user interface:

    What are the configurable properties?

    Following are the configurable properties in the End-to-End Monitoring agent available at <INSTALL-DIR>\E2EMonitoring\agent\config

    Property Description
    agent.trace_transfer_interval=${SW_AGENT_TRACE_TRANSFER_INTERVAL:180000} This property defines the interval between each write operation to the file. The same property is used for data send interval through the gRPC channel. Default value is 18000 ms. It is recommended to reduce this time interval to shorter segment trace writes for testing purposes.
    collector.establish_cloud_communication=${SW_AGENT_COLLECTOR_ESTABLISH_CLOUD_COMMUNICATION:true} This property defines connectivity with the cloud collector. Default value is true, which means data is sent to the cloud collector.

    Note:
    Hybrid Monitoring data transfer from on-premises Integration Server or Microservices Runtime to cloud takes place through the tenant endpoint configured under webMethods Cloud Tenant Connections on port 11800.

    Troubleshooting

    Where can I find the log files?
    How can I view traces information in the log files?
    1. Navigate to the file e2ecustomlog4j2.properties at location:

      • Integration Server

        <INSTALL-DIR>\IntegrationServer\instances\default
      • Microservices Runtime

        <INSTALL-DIR>\IntegrationServer
    2. Apply DEBUG logging level to the following properties:

      appender.FILE_OUT.filter.threshold.level = DEBUG
      appender.FILE_OUT1.filter.threshold.level = DEBUG
      appender.FILE_OUT2.filter.threshold.level = DEBUG
      logger.20.level = DEBUG
      logger.21.level = DEBUG
      logger.22.level = DEBUG


    3. Here is a sample of the on-premises transaction trace:

      2021-08-06T19:49:50,328 DEBUG [e2eAgentAppender] - 
      [ENTRY]SpanInfo [traceId=[26655816.49441.45841902585488], segmentId:26655969.276.162825958992328, spanId:0, serviceName=forTesting:javaAsyncService1, startTime=1628259589923, duration=4ms, component=ONPREMISEIS, tenant=VMWODEPLAT01:5555, status=Success, requestID=5745b59f-089a-4b13-9038-f82188d68a32, parentLandscape=WMIO]
    Are there exception cases where hybrid transaction traces won’t work?

    Where can I find a detailed Step-by-Step Tutorial?

    For a detailed tutorial click Step-by-Step Tutorial - Hybrid Monitoring