Connector Builder - Node.js CLI

webMethods.io Integration is an integration platform as a service (iPaaS) that lets you connect web apps, on-premises systems, and devices to create integrations and automate tasks.

Create custom connectors with our developer platform. Know how it works and how to install it. In addition, learn how to create a connector, add triggers, actions, lookups, and authentications to it, deploy it, and share it with other users.

You can also create custom web connectors using the Connector Builder.

What's a Connector Builder App

The Connector Builder app is a custom Node.js application that you build using a web application’s APIs. So, for example, if you have a private API or an API that is not in webMethods.io Integration yet, you can create custom webMethods.io Integration actions and triggers for those APIs. Once you have created these custom actions and triggers, you can use them like any other action on the webMethods.io integration platform.

How the Connector Builder app works

Once you deploy the app on webMethods.io Integration, it is validated by the webMethods.io Integration platform. If the app contains any errors, you will see a relevant error message and error details on the console. Once the app is validated successfully, it will be made available to you locally. You can then share this app with specific users or publish it to webMethods.io Integration to make it available to all the users of webMethods.io Integration. Below is the list of basic terms you should be familiar with before you start creating your first Connector Builder app:

Connector Builder vs Node.js Block

Both the Connector Builder and the existing Node.js Block of webMethods.io Integration serve the same purpose; they let you create custom connectors. The major difference is that the Connector Builder works more like a programming project where you have to create your application from scratch whereas Node.js Block offers you a ready-to-use interface where you just have to enter the core logic for your application.

However, the Node.js block lets you create only actions, while the Connector Builder lets you create actions and triggers along with the custom authentications. You can choose any one of these options that fits your requirements and start creating your apps.

Requirements

The app can be developed using any Node version between 16.15.0 and 18.12.1.

If you have installed any other Node version, for example, v18.12.1 on your system, you can choose from one of the following options:

Installing Connector-builder Package

In order to get started with the Connector Builder, you will first need to install it on your system. To do so, run the following command:

npm install -g @webmethodsio/wmiocli

Once this is done, you can start building your app.

Quick Setup Guide

If you are familiar with how an iPaaS Connector Builder works, you can use the quick start guide below to start building your connectors. (For a detailed explanation of these steps, see Creating a Local App.

  1. Install the Connector Builder globally.

    npm install -g @webmethodsio/wmiocli

  2. Login to webMethods.io Integration.

    wmio login

    On running this command, you will be prompted to enter your tenant URL, email ID associated with the tenant, and the tenant developer key. You can get developer key by navigating to profile icon and then selecting the Profile option. Once you enter these details, you will be logged in to webMethods.io Integration.

    At this point your Connector Builder is installed and ready to go.

  3. Create a new app with minimum required files.

    wmio init

    You will be prompted to provide a name for the app.

  4. Move into newly created directory.

    cd <app_name>

  5. Install all libraries needed for your Connector Builder app.

    npm install

    You should now have a local working app. You can now start adding codes for your actions/triggers/lookups and authentication.

  6. Add action, trigger, and lookup. To add new action in your application, use:

    wmio create action <action_name> or wmio action <action_name>

    When you execute this command, a new action folder will be created in your app’s directory with a <action_name.js> file in it. You can then add your action code in that file’s execute function and update the input and output schema based on your requirements.

    To add new lookup in your application, use:

    wmio create lookup <lookup_name> or wmio lookup <lookup_name>

    When you execute this command, a new lookup folder will be created in your app’s directory with a <lookup_name.js> file in it. You can then add your lookup code in that file.

    To add new trigger in your application, use:

    wmio create trigger <trigger_name> or wmio trigger <trigger_name>

    When you execute this command, a new trigger folder will be created in your app’s directory with a <trigger_name.js> file in it. You can add your trigger code in that file.

    Note: The action/trigger/lookup name should consist of only alphanumeric characters and underscores.

  7. Create authentication for actions/triggers/lookup.

    wmio auth

    After this, you will be prompted to select the authentication type (basic/apikey/oauth/custom). When you select an authentication type, an authentication.js file will be created in your app’s directory. You can then add authentication logic to this authentication.js file.

    Note: If you want to implement Noauth for your custom connectors, refer to the Authentications section.

  8. Once all codes are added, test the app.

    This is an optional step but we recommend that you perform it. The testing will be done against the mock data you have provided in your code.

    wmio test

  9. Deploy the app to webMethods.io Integration.

    wmio deploy

    Once you have deployed the app, refresh the browser window of webMethods.io Integration UI. The deployed app will be added in the Connectors panel under the Services tab and will be available for you to use locally. You can then download this app. If you want to make your app globally available, kindly contact Global Support.

    Note: Refer to the Help section to get more details about the Connector Builder commands.

In the next section, we will understand how to create a Connector Builder app in detail.

Creating a Local App

Once you have installed the Connector Builder package, you can install the Connector Builder and set up your authentication to create a working app. It will be private and visible in your Connectors panel under the Services tab. Before we go through the steps to create a local app, let’s understand the important blocks that make the app.

Triggers

Triggers read data from the external app and execute the related workflow in webMethods.io Integration. Learn more about Triggers and how they work.

The basic structure for creating a trigger using the Connector Builder is given below. You can add your custom code in this structure to create your own triggers.

If you want to create a polling trigger, refer to the structure given below:

module.exports =
{
 name: "sample_trigger",
 label: "Sample Trigger",

 input:
 {
   type: 'object',
   title: 'Sample Trigger',
   description: "Short description",
   properties:
   {
     event:
     {
       type: 'string',
       enum: ['sample_trigger']
     },
      polling: {
              type: 'boolean',
              default: true,
              options: 
                     {
                       hidden: true
                     }
                },
   }
 },

 output:
 {
   "sample_trigger":
   {
     type: 'object',
     properties: {}      
   }
 },

 mock_data: {}, // add sample output data that will be used for testing

 execute: function(input, options, output)
 {
   // will be called every 5 minutes
   // your code here
 },

 activate: function(input, options, output)
 {
   // This function will be called whenever user activates the trigger/workflow. 
  // This function is useful in scenarios where you save cursor to figure out newly added/changed data in the
  // 3rd party services. You can update your cursor so that trigger won't fetch data of the period 
  // during which the trigger was deactivated.
 },

 validate: function(input, options, output)
 {
   // This function will be called when the trigger is created. 
   // This functions validates all inputs provided by the user and sends an error if any of the details (including auth) are incorrect. In case of an error, trigger server won't create the trigger.
 }

}

If you want to create a webhook trigger, refer to the structure given below:

module.exports =
{
 name: "sample_trigger",
 label: "Sample Trigger",

 input:
 {
   type: 'object',
   title: 'Sample Trigger',
   description: "Short description",
   properties:
   {
     event:
     {
       type: 'string',
       enum: ['sample_trigger']
     },
      polling: {
              type: 'boolean',
              default: false,
              options: 
                     {
                       hidden: true
                     }
                },
   }
 },

 output:
 {
   "sample_trigger":
   {
     type: 'object',
     properties: {}      
   }
 },

 mock_data: {}, // add sample output data that will be used for testing

 execute: function(input, options, output)
 {
   // will be called every 5 minutes
   // your code here
 },

 activate: function(input, options, output)
 {
   // This function will be called whenever user activates the trigger/workflow. 
  // This function is useful in scenarios where you save cursor to figure out newly added/changed data in the
  // 3rd party services. You can update your cursor so that trigger won't fetch data of the period 
  // during which the trigger was deactivated.
 },

 validate: function(input, options, output)
 {
   // This function will be called when the trigger is created. 
   // This functions validates all inputs provided by the user and sends an error if any of the details (including auth) are incorrect. In case of an error, trigger server won't create the trigger.
 }

}

Actions

Actions define the tasks to be performed. The basic structure for creating an action is given below. You can add your custom code in this structure to build your own actions.

module.exports =
{
 title: "sample_action",
 description: "",
 version: "v1",
 

 input:
 {
   title: 'sample_action',
   type: 'object',
   properties: {}
 },

 output:
 {
   title: 'output',
   type: 'object',
   properties: {}
 },

 execute: function(input, output)
 {
   // your code here
 }

}

Authentications

Most of the action/triggers need some sort of authentication from users to function. webMethods.io Integration provides some methods to help add authentication for your actions and triggers. These methods are listed below:

Once you have created the custom OAuth, you need to deploy it to webMethods.io Integration. To do so, run the following command:

`wmio oauth deploy`

Lookups

Lookup helps you autofill fields with the data from your account. It retrieves a list of records associated with the specified property or account. It is important to define the lookup function in the action source code when creating the individual scaffoldings. The basic structure for creating lookup is given below. You can add your custom code in this structure to create your own lookups.

Note: ‘Any’ type field is not supported in the input schema for the lookup modules.

module.exports =
{
   'name':'sample_lookup',  
   'label': 'label',
   'search': 'false',
   

   'execute': function (input, options, output)
   {
     // your code goes here
   }

}

Example

Let’s see how this works in practice. Suppose you want to create a new action for Evernote service. Follow the steps given below to achieve this:

Note: Ensure that you have installed Node.js version 4.x.x or higher on your machine and you are currently in the Connector Builder directory before attempting to create an app.

  1. Install the Connector Builder globally.

    npm install -g @webmethodsio/wmiocli

  2. Configure deploy key and log in to webMethods.io Integration.

    wmio login

    You will be prompted to enter the tenant URL, email associated with your tenant, and tenant developer key. To retrieve the tenant developer key, login to webMethods.io Integration UI and navigate to Profile icon > Settings.

    Once you have entered these details, you will be logged in to your webMethods.io Integration account. Your Connector Builder should be installed and ready to go at this point. Next, we’ll create our first app - Evernote.

  3. Create a directory for your app with the minimum required files.

    wmio init

    You will be prompted to provide a name and description for your app which will be displayed in the Services tab.

    Once you have entered these details, it will automatically download the required files for your app and will create an app directory inside the Connector Builder folder.

  4. Move into the newly created app directory.

    cd evernote

  5. Install all the libraries needed for your app.

    npm install

    You should now have a working local app. You can start adding codes for your actions and triggers.

  6. Add new action.

    wmio create action <action_name> or wmio action <action_name>

    Example: wmio create action create_issue or wmio action create_issue.

    You will notice that the action scaffolding has been created in your Evernote directory and it contains the newly created javascript file, evernote-notebook-create-.js. This file already contains the basic structure you need to follow to create a new action. You can now write your custom code for the action you want to create in this file. Similarly, you can add scaffoldings and relevant codes for lookups and triggers using the:

    wmio create lookup or wmio lookup <lookup_name>

    and

    wmio create trigger <trigger_name> or wmio trigger <trigger_name>

  7. Every action/trigger needs authentication. You can add authentication to your action/trigger using the following command:

    wmio auth

    You will be prompted to select the type of authentication method (Basic/OAuth/API Key/Custom) you want to use. Once you have selected an authentication method, an authentication.js file will be created in your app directory, in which you can add the authentication logic for your actions and/or triggers.

  8. Once all the codes are added, you can test your app:

    wmio test

    The testing will be done against the mock data you provided in the code. This is an optional step but we recommend performing it to ensure that the code you have written is working as expected. At this stage, the directory structure of your app will look something like this:

    EvernoteConnector
    ├── action
    │ ├── v2
    │ │ ├── evernote-notebook-create.js
    │ │ ├── evernote-note-update.js
    │ │ └── evernote-tag-create.js
    │ └── v3
    │ ├── evernote-notebook-getlist.js
    │ ├── evernote-note-create.js
    │ └── evernote-note-get.js
    ├── authentication.js
    ├── build
    │ └── build.zip
    ├── icon
    │ └── icon.png
    ├── index.json
    ├── lookup
    │ ├── get_notebook_list_lookup.js
    │ ├── get_note_list_lookup.js
    │ └── get_stacks_lookup.js
    └── package.json
  9. Deploy the app.

    wmio deploy

    Use this command to deploy your app to webMethods.io Integration.

    Note: For security reasons, we have disabled a few modules for app deployment. So, if you come across any ‘access denied for module’ error while deploying your app, please move the ‘require’ statement inside the function where you want to use it, in the package.json file.

    For example, if your current package.json file structure looks something like this:

    var request = require('request');
    ...
    ...
    
    
    execute: function (input, output) {
    // business logic
    }
    You need to change it to the following structure:
    
    ...
    ...
    
    execute: function (input, output) {
    var request = require('request');
    // business logic
    }

    You need to change it to the following structure:

    ...
    ...
    
    execute: function (input, output) {
    var request = require('request');
    // business logic
    }

    Once deployed, it’ll be automatically registered with webMethods.io Integration and will be available to you locally in the Connectors panel under the Services tab.

If you want to make this app globally available to all users, kindly contact Global Support.

Note: Exporting workflows containing locally deployed custom connectors is not supported. So ensure to make the relevant custom connectors globally available first before creating the workflows to export them successfully.

In case the workflow you want to export contains locally deployed custom connectors, peform the following steps:

  1. Make the relevant custom connectors globally available by contacting Global Support.
  2. Replace the globally available custom connectors with local ones.
  3. Reconfigure the custom connector mapping.
  4. Save and export the workflow.

Adding Custom Icon

You can add a custom icon for your application. To do so, navigate to your application directory, and place the required icon inside the icon folder.

Note: The custom icon should be a square, up to 128x128 pixels, and less than 1 MB.

Error Validations

webMethods.io Integration has implemented a number of test validations for Connector Builder. They are listed in the table given below:

Error Response Description
NO_CONNECTORS No connectors are created yet.
INVALID_COMMAND_EXECUTION Entered command input is invalid.
ILLEGAL_CHARS Special characters are not allowed in connector/action/trigger/lookup name.
UNAUTHORIZED You are not logged in. Use the wmio login command.
UNDEPLOYED_CONNECTOR Connector is not yet deployed. Use the wmio deploy command.
INVALID_COMMAND Entered command does not exist. Try using wmio help command to see the list of supported commands.
HISTORY_EMPTY History is not available.
SWAGGER_IMPORT_FAILURE Swagger version is not supported.
INVALID_CONNECTOR_NAME Special characters are not allowed in the connector name.
USER_LOGGED_IN_ERROR You are already logged in to webMethods.io Integration. Use wmio logout command to logout, and then relogin with a different account.
INVALID_EMAIL Entered email address is invalid.
INVALID_PASSWORD Entered password is Invalid.
LOGOUT_FAILURE You are logged in to webMethods.io Integration.
INVALID_SET_PARAMS Entered command is invalid. Try using wmio set lookup.
EMPTY_LOOKUP_LINK_FIELDS No fields available to link lookup.
LOOKUP_NOT_FOUND No lookup found in your current application.
VERSION_FAILURE No connector is registered yet. Use wmio deploy command to deploy your connector.
INVALID_AUTH Error validating authentication.
NO_AUTH_FILE Authentication file is missing .
CONNECTOR_VERSION_NOT_FOUND Current application does not have any version.
CONNECTOR_VERSION_ERROR Connector version must be greater than the currently deployed version.
LOOKUP_EXIST_ERROR Lookup module already exists.
INVALID_TRIGGER Invalid trigger entry found in the index.
INVALID_OAUTH_PARAM Invalid command parameter. Try using wmio oauth deploy.
OAUTH_MANDATORY_FIELD Missing mandatory field in the oauth.json file.
OAUTH_EMPTY_FIELD Oauth field must have at least one property.
OAUTH_FILE_ERROR Unable to find oauth.json in your application directory.
OAUTH_PARSE_ERROR Failed to parse oauth.json.
OAUTH_SCOPE_FAIL Scope property should be an object.
LOOKUP_ATTACH_ERROR Select at least one component to attach lookup.
LOOKUP_ENTRY_EXIST Lookup entry already exists for the specified field.
LOOKUP_DEPENDENCY Lookup needs dependencies of other input fields.
NO_FIELDS No fields found.
DETACH_ERROR No lookup found to detach.
DETACH_ACTION_FAIL No actions to detach skipping.
DETACH_TRIGGER_FAIL No triggers to detach skipping.
LOOKUP_ENTRY_NOT_EXIST No entry exists for selected lookup.
NAME_TOO_SHORT Name for {{type}} {{name}} is too short.
POSTMAN_FILE_ERROR File {{file}} not found.
INVALID_EXPORT Invalid Postman collection export file.
POSTMAN_PARSE_ERROR Failed to parse JSON.
POSTMAN_SCHEMA_ERROR No schema definition found in the exported json.
POSTMAN_SUCCESS Imported {{total}} actions successfully.
POLLING_PROMPT Is your trigger polling.
VERSION_PROMPT Provide version for your {{type}} {{name}}.
VERSION_ERROR Version should begin with prefix `v`, for example, v1 got {{version}}.
OAUTH_REQUIRED_INVALID Required field should be a plain js object, for example, { title: ‘Domain’, id: ‘domain’, description: ‘abc’, default: ‘mydomain’}.
ENCRYPT_OAUTH Encrypting …
INVALID_HOST Invalid Tenant URL.
OAUTH_PREPROCESS_INVALID Pre-processing hook {hook} params should be plain object.

Downloading your App

Once you have deployed the app, you can download its ZIP file in your current directory. To do so, use this command:

wmio download

When you run this command, a drop down list will appear on the screen from which you can select the app you want to download.

App Versioning

Each time you update your app, a new version is created. You can get a list of all the versions of your app using this command:

wmio versions

Deleting the Deployed Action or Trigger

If you want to delete any deployed action or trigger, you can do so by following the steps below:

  1. Download the app that contains the action or trigger you want to delete.

  2. Unzip the downloaded file.

  3. Delete the file associated with the action/trigger from the action/trigger folder and delete the name of the action/trigger from the index.json file.

  4. Deploy the app using the wmio deploy command.

This will remove the specified action/trigger from your app.

Supported Modules

You can use any npm module for developing connectors using the Node.js CLI framework. However, before using the npm module of your choice in CLI, check the module for potential security vulnerabilities and ensure the module is safe to use.

Note: We do not support C++ modules and modules that need to be compiled on binary. This is because, to make these modules work, they should be compiled based on target machine architecture.

Unpublishing the App

When you deploy an app, all the actions and triggers associated with it are by default published.

In order to unpublish the published app, use the following command:

wmio unpublish

This will also unpublish any triggers and actions associated with the app.

Deploying Connectors to Data Centers

You can deploy a connector within the same data center or across different data centers.

Each connector has an App ID and GUID. In order to import and export a workflow containing a custom CLI connector, webMethods.io Integration uses a GUID.

App ID: App ID is the application ID of a connector and can be different for each environment or data center in which the connector is deployed. For example, if a Google Tasks connector is deployed on a development environment as well as staging environment, then the App ID for Google Tasks will be different for both environments.

GUID: GUID is the unique ID associated with a connector and always remains the same across all environments and data centers. It is important to keep the connector GUID same across all environments and data centers for successful import and export operation.

Deploying a connector across different data centers

Perform the following steps to deploy a connector across different data centers:

Step 1: Login to the CLI app with your source tenant credentials.

Step 2: Run the following command:

wmio migrate {domain} {email} {developer key}

where,
domain - domain name of the destination tenant
email - email ID associated with the destination tenant
developer key - developer key associated with the destination tenant. To obtain the developer key, login to the relevant tenant and then navigate to Profile Icon > Profile option.`

Deploying a connector in the same data center across different tenants

Perform the following steps to deploy a connector in the same data center across different tenants:

Step 1: Login to the CLI app with the destination tenant credentials.

Step 2: Open the index.json file and remove the appid key from it.

Step 3: Run the following command:

wmio deploy

Note: Do not change or remove the connector GUID, otherwise your import and export operation will throw an error.

Creating Custom Connectors using Postman Collections

Collections in the Postman tool, house a set of similar API requests as a group. These existing collections or set of APIs can be used to create custom connectors through the Connector Builder. To do so, follow the steps given below:

  1. Go to the Postman tool and navigate to the Collections section. You will see the list of existing collections.

  2. Locate the collection for which you want to create custom actions. Click on the ellipsis icon (three tiny dots) given beside its name and click Export.

  3. You will be prompted to specify the collection version.

  4. Specify the location where you want to save the exported collection.

    Note: It is recommended that you save the collection inside the app directory for ease of use.

  5. Open the command prompt, navigate to your app directory, and enter the following command:

    wmio postman <collection_file_name.json>

    This will create an action file for each API call included in the specified collection. You can then redeploy your app to start using these apps like any other connector in the Connectors panel.

Getting User Info

To get information about the currently logged in user, use the following command:

wmio whoami

Help

To get the list of all the webMethods.io Integration-CLI commands along with their details, run the following in the command prompt:

wmio help or wmio

Given below is the list of all Connector Builder commands: