Daggy Sources

How to setup data sources for Daggy

Supported formats

Daggy supports yaml and json formats. In yaml repeated nodes can be grouped by yaml standard.

Data sources template

YAML

sources:
    host1:
        #....
    host2:
        #....
    #....
    hostN:
        #....
    

JSON

{
    "sources": {
        "host1" : {
        },
        "host2" : {
        },
        "hostN" : {
        }
    }
}

Each config conatins map of hosts. Host (data source) parameters is next:

Parameter	Type	Description	Is Requiered
type	string	Type of connection to host. Daggy supportes local and ssh host connection types	Yes
commands	array	Array of commands for simultaneous launch	Yes
parameters	map	Connection parameters	Required for ssh2 type
host	string	Host address	No. For ssh2 is 127.0.0.1 by default
reconnect	boolean	true, if need reconnect connection	No

Daggy Sources Types

Daggy supportes local and ssh2 (remote) host connection types.

Local type

YAML

sources:
    localhost:
        type: local
        commands:
            pingYa:
                exec: ping ya.ru
                extension: log

JSON

 {
    "sources": {
        "localhost": {
            "commands": {
                "pingYa": {
                    "extension": "log",
                    "exec": "ping ya.ru"
                }
            },
            "type": "local"
        }
    }
}

SSH2 type

YAML

remotehost:
    type: ssh2
    host: 192.168.1.9
    restart: false
    parameters:
      user: muxa
      key: /home/muxa/.ssh/id_rsa
    commands:
      pingYa:
        command: ping ya.ru
        extension: log

JSON

{
    "localhost": {
        "type": "ssh",
        "host": "192.169.1.9",
        "restart": false,
        "parameters": {
            "user": "muxa",
            "key": "/home/muxa/.ssh/id_rsa"
        },
        "commands": {
            "pingYa": {
                "command": "ping ya.ru",
                "extension": "log"
            }
        }
    }
}

SSH2 type additional parameters

• host - remote host ip address or url

• connection - map of ssh connection parameters. Includes next parameters

Connection Parameter	Type	Description	Default value
user	string	ssh2 connection login	Current user
passphrase	string	ssh2 connection password. If field is not empty, then will be used password authentication for ssh2 connection. In other case, will be used public key authentication
key	string	path to private key for ssh2 connection	~/.ssh/id_rsa
keyphrase	string	passphrase for private key file
port	integer	ssh2 connection port	22
timeout	integer	limit to establish ssh2 connection, in milliseconds	1000

Commands

YAML

commands:
      pingYa:
        exec: ping ya.ru
        extension: log
        restart: false

JSON

"commands": {
            "pingYa": {
                "command": "ping ya.ru",
                "extension": "log"
            }
        }

Each command must contain:

• name - unique within host command identifier, using in command output file name template.

• exec - shell script

• extension - extension for command output file

• restart - restart command if it finished

Using environment and other variables in Data Aggregation Sources

Mustache syntax

YAML/JSON Data Aggregation Sources are support mustache syntax:

• env_* - template for environment variable, where * is environment variable name.

• output_folder - output folder path

Example of using mustache syntax in Daggy Sources

YAML

aliases:  
    - &my_commands
        pingYa:
            exec: ping ya.ru
            extension: log
        pingGoo:
            exec: ping goo.gl
            extension: log
        
    - &ssh_auth
        user: {{env_USER}}
        passphrase: {{env_PASSWORD}}
            
sources:
    localhost:
        type: local
        commands: *my_commands
    remotehost:
        host: 192.168.1.9
        type: ssh2
        parameters: *ssh_auth
        commands: *my_commands
    remotehost2:
        host: 192.168.1.9
        type: ssh2
        parameters: *ssh_auth
        commands: *my_commands
    remotehost3:
        host: 192.168.1.9
        type: ssh2
        parameters: *ssh_auth
        commands: *my_commands

JSON

{
  "sources": {
    "remotehost3": {
      "host": "192.168.1.9", 
      "type": "ssh2", 
      "commands": {
        "pingYa": {
          "extension": "log", 
          "exec": "ping ya.ru"
        }, 
        "pingGoo": {
          "extension": "log", 
          "exec": "ping goo.gl"
        }
      }, 
      "parameters": {
        "user": "{{env_USER}}", 
        "passphrase": "{{env_PASSWORD}}"
      }
    }, 
    "remotehost2": {
      "host": "192.168.1.9", 
      "type": "ssh2", 
      "commands": {
        "pingYa": {
          "extension": "log", 
          "exec": "ping ya.ru"
        }, 
        "pingGoo": {
          "extension": "log", 
          "exec": "ping goo.gl"
        }
      }, 
      "parameters": {
        "user": "{{env_USER}}", 
        "passphrase": "{{env_PASSWORD}}"
      }
    }, 
    "remotehost": {
      "host": "192.168.1.9", 
      "type": "ssh2", 
      "commands": {
        "pingYa": {
          "extension": "log", 
          "exec": "ping ya.ru"
        }, 
        "pingGoo": {
          "extension": "log", 
          "exec": "ping goo.gl"
        }
      }, 
      "parameters": {
        "user": "{{env_USER}}", 
        "passphrase": "{{env_PASSWORD}}"
      }
    }, 
    "localhost": {
      "commands": {
        "pingYa": {
          "extension": "log", 
          "exec": "ping ya.ru"
        }, 
        "pingGoo": {
          "extension": "log", 
          "exec": "ping goo.gl"
        }
      }, 
      "type": "local"
    }
  }
}