Skip to content

Latest commit

 

History

History
959 lines (668 loc) · 18.3 KB

File metadata and controls

959 lines (668 loc) · 18.3 KB

Templates

Templates define a single application configuration template. Templates are stored under the /etc/confd/templates directory by default.

Templates are written in Go's text/template.

Template Functions

map

creates a key-value map of string -> interface{}

{{$endpoint := map "name" "elasticsearch" "private_port" 9200 "public_port" 443}}

name: {{index $endpoint "name"}}
private-port: {{index $endpoint "private_port"}}
public-port: {{index $endpoint "public_port"}}

specifically useful if you use a sub-template and you want to pass multiple values to it.

base

Alias for the path.Base function.

{{with get "/key"}}
    key: {{base .Key}}
    value: {{.Value}}
{{end}}

exists

Checks if the key exists. Return false if key is not found.

{{if exists "/key"}}
    value: {{getv "/key"}}
{{end}}

get

Returns the KVPair where key matches its argument. Returns an error if key is not found.

{{with get "/key"}}
    key: {{.Key}}
    value: {{.Value}}
{{end}}

gets

Returns all KVPair, []KVPair, where key matches its argument. Returns an error if key is not found.

{{range gets "/*"}}
    key: {{.Key}}
    value: {{.Value}}
{{end}}

getv

Returns the value as a string where key matches its argument or an optional default value. Returns an error if key is not found and no default value given.

value: {{getv "/key"}}

With a default value

value: {{getv "/key" "default_value"}}

getvs

Returns all values, []string, where key matches its argument. Returns an error if key is not found.

{{range getvs "/*"}}
    value: {{.}}
{{end}}

getenv

Wrapper for os.Getenv. Retrieves the value of the environment variable named by the key. It returns the value, which will be empty if the variable is not present. Optionally, you can give a default value that will be returned if the key is not present.

export HOSTNAME=`hostname`
hostname: {{getenv "HOSTNAME"}}

With a default value

ipaddr: {{getenv "HOST_IP" "127.0.0.1"}}

datetime

Alias for time.Now

# Generated by confd {{datetime}}

Outputs:

# Generated by confd 2015-01-23 13:34:56.093250283 -0800 PST
# Generated by confd {{datetime.Format "Jan 2, 2006 at 3:04pm (MST)"}}

Outputs:

# Generated by confd Jan 23, 2015 at 1:34pm (EST)

See the time package for more usage: http://golang.org/pkg/time/

split

Wrapper for strings.Split. Splits the input string on the separating string and returns a slice of substrings.

{{ $url := split (getv "/deis/service") ":" }}
    host: {{index $url 0}}
    port: {{index $url 1}}

toUpper

Alias for strings.ToUpper Returns uppercased string.

key: {{toUpper "value"}}

toLower

Alias for strings.ToLower. Returns lowercased string.

key: {{toLower "Value"}}

json

Returns an map[string]interface{} of the json value.

lookupSRV

Wrapper for net.LookupSRV. The wrapper also sorts the SRV records alphabetically by combining all the fields of the net.SRV struct to reduce unnecessary config reloads.

{{range lookupSRV "mail" "tcp" "example.com"}}
  target: {{.Target}}
  port: {{.Port}}
  priority: {{.Priority}}
  weight: {{.Weight}}
{{end}}

base64Encode

Returns a base64 encoded string of the value.

key: {{base64Encode "Value"}}

base64Decode

Returns the string representing the decoded base64 value.

key: {{base64Decode "VmFsdWU="}}

Add keys to etcd

etcdctl set /services/zookeeper/host1 '{"Id":"host1", "IP":"192.168.10.11"}'
etcdctl set /services/zookeeper/host2 '{"Id":"host2", "IP":"192.168.10.12"}'

Create the template resource

[template]
src = "services.conf.tmpl"
dest = "/tmp/services.conf"
keys = [
  "/services/zookeeper/"
]

Create the template

{{range gets "/services/zookeeper/*"}}
{{$data := json .Value}}
  id: {{$data.Id}}
  ip: {{$data.IP}}
{{end}}

Advanced Map Traversals

Once you have parsed the JSON, it is possible to traverse it with normal Go template functions such as index.

A more advanced structure, like this:

{
  "animals": [
    {"type": "dog", "name": "Fido"},
    {"type": "cat", "name": "Misse"}
  ]
}

It can be traversed like this:

{{$data := json (getv "/test/data/")}}
type: {{ (index $data.animals 1).type }}
name: {{ (index $data.animals 1).name }}
{{range $data.animals}}
{{.name}}
{{end}}

jsonArray

Returns a []interface{} from a json array such as ["a", "b", "c"].

{{range jsonArray (getv "/services/data/")}}
val: {{.}}
{{end}}

ls

Returns all subkeys, []string, where path matches its argument. Returns an empty list if path is not found.

{{range ls "/deis/services"}}
   value: {{.}}
{{end}}

lsdir

Returns all subkeys, []string, where path matches its argument. It only returns subkeys that also have subkeys. Returns an empty list if path is not found.

{{range lsdir "/deis/services"}}
   value: {{.}}
{{end}}

dir

Returns the parent directory of a given key.

{{with dir "/services/data/url"}}
dir: {{.}}
{{end}}

join

Alias for the strings.Join function.

{{$services := getvs "/services/elasticsearch/*"}}
services: {{join $services ","}}

replace

Alias for the strings.Replace function.

{{$backend := getv "/services/backend/nginx"}}
backend = {{replace $backend "-" "_" -1}}

lookupIP

Wrapper for net.LookupIP function. The wrapper also sorts (alphabeticaly) the IP addresses. This is crucial since in dynamic environments DNS servers typically shuffle the addresses linked to domain name. And that would cause unnecessary config reloads.

{{range lookupIP "some.host.local"}}
    server {{.}};
{{end}}

atoi

Alias for the strconv.Atoi function.

{{seq 1 (atoi (getv "/count"))}}

hostname

Wrapper for os.Hostname. Retrieves the value of the host name reported by the kernel.

hostname: {{ hostname }}

contains

Alias for strings.Contains. Reports whether a substring is within a string.

{{if contains (getv "/app/features") "debug"}}
debug_mode = true
{{end}}

trimSuffix

Alias for strings.TrimSuffix. Removes the suffix from the end of a string.

# Remove .example.com suffix from hostname
server_name: {{trimSuffix (getv "/app/hostname") ".example.com"}}

lookupIPV4

Similar to lookupIP but returns only IPv4 addresses. Results are sorted alphabetically.

{{range lookupIPV4 "some.host.local"}}
    server {{.}};
{{end}}

lookupIPV6

Similar to lookupIP but returns only IPv6 addresses. Results are sorted alphabetically.

{{range lookupIPV6 "some.host.local"}}
    server [{{.}}];
{{end}}

lookupIfaceIPV4

Returns the IPv4 address of a network interface by name.

bind_address: {{lookupIfaceIPV4 "eth0"}}

lookupIfaceIPV6

Returns the IPv6 address of a network interface by name.

bind_address_v6: {{lookupIfaceIPV6 "eth0"}}

fileExists

Checks if a file exists at the given path. Returns true if the file exists.

{{if fileExists "/etc/myapp/custom.conf"}}
include /etc/myapp/custom.conf
{{end}}

parseBool

Alias for strconv.ParseBool. Parses a string into a boolean value. Accepts: 1, t, T, TRUE, true, True, 0, f, F, FALSE, false, False.

{{if parseBool (getv "/app/debug" "false")}}
log_level = debug
{{end}}

reverse

Reverses the order of elements in an array. Works with []string and []KVPair.

# Reverse order of servers
{{range reverse (getvs "/servers/*")}}
    server {{.}};
{{end}}

# Reverse order of KVPairs
{{range reverse (gets "/config/*")}}
    {{.Key}} = {{.Value}}
{{end}}

sortByLength

Sorts a string array by the length of each string (shortest first).

{{range sortByLength (getvs "/paths/*")}}
    location {{.}} { }
{{end}}

sortKVByLength

Sorts a KVPair array by the length of each key (shortest first). Useful for ordering paths or prefixes.

{{range sortKVByLength (gets "/routes/*")}}
    # Key: {{.Key}} (length: {{len .Key}})
    route {{.Key}} {{.Value}}
{{end}}

seq

Creates a sequence of integers, similar to GNU seq. Takes a start and end value (inclusive).

# Generate sequence from 1 to 5: [1, 2, 3, 4, 5]
{{range seq 1 5}}
worker_{{.}}:
    enabled: true
{{end}}

# Dynamic range based on config value
{{range seq 1 (atoi (getv "/app/workers" "3"))}}
    - worker{{.}}
{{end}}

Math Functions

Basic arithmetic operations on integers.

add

Adds two integers.

port: {{add (atoi (getv "/base_port")) 1}}

sub

Subtracts the second integer from the first.

max_connections: {{sub (atoi (getv "/total_connections")) 10}}

mul

Multiplies two integers.

buffer_size: {{mul (atoi (getv "/buffer_kb")) 1024}}

div

Divides the first integer by the second.

workers_per_core: {{div (atoi (getv "/total_workers")) 4}}

mod

Returns the remainder of dividing the first integer by the second.

{{if eq (mod $index 2) 0}}
# Even index
{{end}}

include

Includes another template file. The included template has access to all the same functions and can also use include for nested includes (up to 10 levels deep). Cycle detection prevents infinite loops.

{{ include "header.tmpl" }}

With data context

You can pass data to the included template:

{{ include "server.tmpl" . }}
{{ include "upstream.tmpl" $data }}

Include from subdirectory

Templates can be organized in subdirectories:

{{ include "partials/footer.tmpl" }}
{{ include "nginx/upstream.tmpl" . }}

Example: Reusable server block

/etc/confd/templates/partials/server-block.tmpl:

server {
    listen {{ .Port }};
    server_name {{ .ServerName }};
    location / {
        proxy_pass http://{{ .Upstream }};
    }
}

/etc/confd/templates/nginx.conf.tmpl:

{{range gets "/services/*"}}
{{$data := json .Value}}
{{ include "partials/server-block.tmpl" $data }}
{{end}}

Note: Included template paths are relative to the templates directory (default: /etc/confd/templates/). Directory traversal outside the templates directory is not allowed for security.

Utility Functions

Defaults & Conditionals

default

Returns the value if non-empty, otherwise returns the default value. "Empty" means nil, false, 0, "", or an empty slice/map.

database_host: {{default "localhost" (getv "/db/host")}}

ternary

Returns the first value if the condition is true, otherwise the second. The condition is the last argument for pipeline use.

mode: {{ternary "debug" "release" (parseBool (getv "/app/debug" "false"))}}

coalesce

Returns the first non-empty value from its arguments.

host: {{coalesce (getv "/app/host" "") (getenv "HOST") "localhost"}}

empty

Returns true if the value is empty (nil, false, 0, "", empty slice/map).

{{if not (empty (getv "/app/name" ""))}}
app_name: {{getv "/app/name"}}
{{end}}

JSON

toJson

Marshals a value to a compact JSON string.

config: {{toJson (dict "host" (getv "/db/host") "port" (atoi (getv "/db/port")))}}

fromJson

Unmarshals a JSON string into a value. Works with objects, arrays, strings, and numbers.

{{$config := fromJson (getv "/app/config")}}
host: {{index $config "host"}}

toPrettyJson

Marshals a value to a pretty-printed JSON string with 2-space indentation.

{{toPrettyJson (dict "host" (getv "/db/host") "port" (atoi (getv "/db/port")))}}

Formatting

indent

Indents every line of a string by the specified number of spaces.

upstream_config: |
{{indent 4 (getv "/nginx/upstream_block")}}

nindent

Same as indent but prepends a newline. Useful in YAML where you need the content to start on the next line.

config:{{nindent 2 (getv "/app/yaml_block")}}

quote

Wraps a string in double quotes.

name: {{quote (getv "/app/name")}}

squote

Wraps a string in single quotes.

name: {{squote (getv "/app/name")}}

Regex

regexMatch

Returns true if the string matches the regular expression pattern.

{{if regexMatch "^prod" (getv "/env/name")}}
log_level: warn
{{end}}

regexFind

Returns the first match of the pattern in the string, or an empty string if no match.

version: {{regexFind "[0-9]+\\.[0-9]+" (getv "/app/version_string")}}

regexReplaceAll

Replaces all matches of the pattern with the replacement string.

sanitized: {{regexReplaceAll "[^a-zA-Z0-9]" (getv "/app/name") "_"}}

Strings

trimPrefix

Alias for strings.TrimPrefix. Removes the prefix from the beginning of a string.

name: {{trimPrefix (getv "/service/fqdn") "service-"}}

repeat

Repeats a string the specified number of times.

separator: {{repeat 80 "-"}}

nospace

Removes all whitespace from a string.

id: {{nospace (getv "/app/display_name")}}

snakecase

Converts a string to snake_case.

env_var: {{toUpper (snakecase (getv "/app/settingName"))}}

camelcase

Converts a string to camelCase.

property: {{camelcase (getv "/app/setting_name")}}

kebabcase

Converts a string to kebab-case.

slug: {{kebabcase (getv "/app/ServiceName")}}

Collections

dict

Creates a key-value map from pairs of arguments (alias for map). Returns an error if given an odd number of arguments.

{{$server := dict "host" (getv "/server/host") "port" (getv "/server/port")}}
server: {{index $server "host"}}:{{index $server "port"}}

list

Creates a list from its arguments.

{{range list "web" "api" "worker"}}
  service: {{.}}
{{end}}

hasKey

Returns true if a map contains the given key.

{{$config := fromJson (getv "/app/config")}}
{{if hasKey $config "debug"}}
debug: {{index $config "debug"}}
{{end}}

keys

Returns the sorted keys of a map.

{{$config := fromJson (getv "/app/config")}}
{{range keys $config}}
  {{.}}: {{index $config .}}
{{end}}

values

Returns the values of a map, sorted by key for deterministic output.

{{range values (fromJson (getv "/app/ports"))}}
  - {{.}}
{{end}}

append

Appends a value to a list and returns the new list.

{{$servers := list "web1" "web2"}}
{{$servers = append $servers "web3"}}

pluck

Extracts a key from a list of maps. Skips maps that don't contain the key.

{{$services := list (dict "name" "web" "port" 80) (dict "name" "api" "port" 8080)}}
ports: {{toJson (pluck "port" (index $services 0) (index $services 1))}}

Hashing

sha256sum

Returns the hex-encoded SHA-256 hash of a string.

config_hash: {{sha256sum (getv "/app/config")}}

Example Usage

etcdctl set /nginx/domain 'example.com'
etcdctl set /nginx/root '/var/www/example_dotcom'
etcdctl set /nginx/worker_processes '2'
etcdctl set /app/upstream/app1 "10.0.1.100:80"
etcdctl set /app/upstream/app2 "10.0.1.101:80"

/etc/confd/templates/nginx.conf.tmpl

worker_processes {{getv "/nginx/worker_processes"}};

upstream app {
{{range getvs "/app/upstream/*"}}
    server {{.}};
{{end}}
}

server {
    listen 80;
    server_name www.{{getv "/nginx/domain"}};
    access_log /var/log/nginx/{{getv "/nginx/domain"}}.access.log;
    error_log /var/log/nginx/{{getv "/nginx/domain"}}.log;

    location / {
        root              {{getv "/nginx/root"}};
        index             index.html index.htm;
        proxy_pass        http://app;
        proxy_redirect    off;
        proxy_set_header  Host             $host;
        proxy_set_header  X-Real-IP        $remote_addr;
        proxy_set_header  X-Forwarded-For  $proxy_add_x_forwarded_for;
    }
}

Output: /etc/nginx/nginx.conf

worker_processes 2;

upstream app {
    server 10.0.1.100:80;
    server 10.0.1.101:80;
}

server {
    listen 80;
    server_name www.example.com;
    access_log /var/log/nginx/example.com.access.log;
    error_log /var/log/nginx/example.com.error.log;

    location / {
        root              /var/www/example_dotcom;
        index             index.html index.htm;
        proxy_pass        http://app;
        proxy_redirect    off;
        proxy_set_header  Host             $host;
        proxy_set_header  X-Real-IP        $remote_addr;
        proxy_set_header  X-Forwarded-For  $proxy_add_x_forwarded_for;
    }
}

Complex example

This examples show how to use a combination of the templates functions to do nested iteration.

Add keys to etcd

etcdctl mkdir /services/web/cust1/
etcdctl mkdir /services/web/cust2/
etcdctl set /services/web/cust1/2 '{"IP": "10.0.0.2"}'
etcdctl set /services/web/cust2/2 '{"IP": "10.0.0.4"}'
etcdctl set /services/web/cust2/1 '{"IP": "10.0.0.3"}'
etcdctl set /services/web/cust1/1 '{"IP": "10.0.0.1"}'

Create the template resource

[template]
src = "services.conf.tmpl"
dest = "/tmp/services.conf"
keys = [
  "/services/web"
]

Create the template

{{range $dir := lsdir "/services/web"}}
upstream {{base $dir}} {
    {{$custdir := printf "/services/web/%s/*" $dir}}{{range gets $custdir}}
    server {{$data := json .Value}}{{$data.IP}}:80;
    {{end}}
}

server {
    server_name {{base $dir}}.example.com;
    location / {
        proxy_pass {{base $dir}};
    }
}
{{end}}

Output:/tmp/services.conf

upstream cust1 {
    server 10.0.0.1:80;
    server 10.0.0.2:80;
}

server {
    server_name cust1.example.com;
    location / {
        proxy_pass cust1;
    }
}

upstream cust2 {
    server 10.0.0.3:80;
    server 10.0.0.4:80;
}

server {
    server_name cust2.example.com;
    location / {
        proxy_pass cust2;
    }
}

Go's text/template package is very powerful. For more details on it's capabilities see its documentation.