How would you use Hashicorp's Nomad 'template stanza' to generate an nginx config file through the Nomad job file?

Asked 23/3, 2017 at 0:16 Answered 15/6, 2022 at 23:36

Solved nginx docker consul consul-template nomad

With the assumption that Consul and Nomad has been configured to run on a pool of resource. How would you rendered a template file for the sole purpose of generating e.g. an Nginx 'default.conf' file.

Using the template stanza configuration below, as an example; Nomad fails to generate a default.conf 'file'; instead a default.conf 'directory' is created.

template {
    source        = "/path/to/tmp.ctmpl"
    destination   = "folder/default.conf"
    change_mode   = "restart"
    change_signal = "SIGINT"
}

I'm either missing a trick, or have misunderstood the functionalities of the 'template stanza'.

One of the issue with the template generating a directory rather than a file is, you cannot mount a directory to a config file path. So running a task that uses the Nomad docker driver with the exemplar 'docker' volume config results in an error.

volumes = ["/path/to/job/folder/default.conf:/etc/nginx/conf.d/default.conf" ]

Or is it impossible to have the template stanza generate a config file?

*P.s. Using Nomad build 0.5.5**

Maretz answered 23/3, 2017 at 0:16 Comment(0)

I just put together a little Nomad job showing this working, so you may have a slight configuration error. To allow you to run the job yourself I have made it available as a gist here. In the same gist I have a nginx.conf that has nginx listen on whatever port is in the Nomad job file.

Here is the Nomad job:

job "nginx" {
  datacenters = ["dc1"]
  type = "service"
  group "cache" {
    count = 1
    task "redis" {
      driver = "docker"
      config {
        image = "nginx:1.11.10"
        volumes = ["new/default.conf:/etc/nginx/conf.d/default.conf" ]
        network_mode = "host"
      }

      artifact {
        source = "https://gist.githubusercontent.com/dadgar/2dcf68ab5c49f7a36dcfe74171ca7936/raw/c287c16dbc9ddc16b18fa5c65a37ff25d2e0e667/nginx.conf"
      }

      template {
        source        = "local/nginx.conf"
        destination   = "new/default.conf"
        change_mode   = "restart"
      }

      resources {
        network {
          mbits = 10
          port "nginx" {
            static = 8080
          }
        }
      }
    }
  }
}

I can then query that address and see that nginx is bound to that port, thus the template being mounted is working properly.

$ curl http://127.0.0.1:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

If you take a look at the gist, I show the file being rendered and mounted properly as well.

Hope this helps you! Also be sure to check out the community page for getting help. We have both a Gitter room and a mailing list.

Semiweekly answered 24/3, 2017 at 6:6 Comment(1)

Thanks for the example, it works as expected. Also, good point, i'll check out the gitter room for future enquires. – Maretz 24/3, 2017 at 21:9

It is much easier to just put the configuration generated in either local or secrets folders in the working directory of the job (alloc directory). These folders will be available in the container as /secrets and /local. No volume mounts required.

Schnauzer answered 16/3, 2018 at 1:16 Comment(1)

How does one do that though? Putting the configuration inside local? I have the configuration file alongside my .nomad file. – Peluso 1/7, 2021 at 8:39

Ended up following the above solution based on the following error:

"Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type"

The following snippet it is how I made work nginx as a reverse proxy for my jenkins instance ( all nomad based ). This template is part of a multitasks job, made of a jenkins and an nginx one.

Hopefully it will be of some use to someone else in the same situation I was, for the last couple of days.

task "nginx" {

  driver = "docker"

  resources {
    cpu    = 75 
    memory = 75
  }   

  service {
    name = "jenkins-nginx"
    tags = ["urlprefix-/jenkins-nginx"]
    port = "http" 
    check {
      name     = "nginx port alive"
      type     = "http"
      path     = "/login"
      interval = "10s"
      timeout  = "2s"
    }   
  }   

      template {
        change_mode     = "restart"
        destination     = "local/default.conf"
        data = <<EOH
    upstream jenkins {
      server {{ env "NOMAD_ADDR_jenkins" }}; 
    }   
    server {
        listen {{ env "NOMAD_PORT_http" }}; 

        location / { 
               proxy_redirect              off;
               proxy_pass_header           Server;
               proxy_set_header            X-Real-IP $remote_addr;
               proxy_set_header            X-Forwarded-For $proxy_add_x_forwarded_for;
               proxy_set_header            Host $http_host;
               proxy_set_header            X-NginX-Proxy true;
               proxy_set_header            X-Accel-Buffering no; 
               proxy_connect_timeout       5;  
               proxy_http_version          1.1;
               proxy_read_timeout          240;
               proxy_intercept_errors      on; 
               keepalive_timeout           3600;
               proxy_set_header Connection ''; 
               chunked_transfer_encoding   off;
               proxy_buffering             off;
               proxy_cache                 off;
               proxy_pass                  http://jenkins;
        }   
    }   
EOH
      }   

      config {
        image = "nginx"

        network_mode = "host"
        ports = ["http","https"]

        volumes = [ "local/default.conf:/etc/nginx/conf.d/default.conf" ]
      }   
    }

Blotter answered 2/7, 2021 at 9:54 Comment(0)

You simply use this job :

job "nginx" {
  datacenters = ["dc1"]
  type = "service"
  group "cache" {
    count = 1
    task "redis" {
      template {
        change_mode = "noop"
        destination = "local/default.conf"
        data = <<EOH
server {
    listen       {{ env "NOMAD_PORT_nginx" }};
    server_name  localhost;

    #charset koi8-r;
    #access_log  /var/log/nginx/log/host.access.log  main;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

}
EOH
      }
      driver = "docker"
      config {
        image = "nginx:1.11.10"
        volumes = ["local/default.conf:/etc/nginx/conf.d/default.conf" ]
        network_mode = "host"
      }

      resources {
        network {
          mbits = 10
          port "nginx" {
            static = 8080
          }
        }
      }
    }
  }
}

Sinfonietta answered 15/6, 2022 at 23:36 Comment(0)

Recommended topics

Hot tags