How to resolve pod hostnames from other pods?

Question

• I have 2 pods running on 2 nodes, each pod runs in different node.
• These nodes are on the same subnet and can TCP/UDP/ICMP themselves.

These pods got some hostnames, ie:

• drill-staging-75cddd789-kbzsq
• drill-staging-75cddd789-amsrj

From pod drill-staging-75cddd789-kbzsq I cannot resolve host name for drill-staging-75cddd789-amsrj and vice versa. Resolving self pod's name works.

I tried setting various dnsPolicies:

• ClusterFirst: no luck
• Default: no luck
• ClusterFirstWithHostNet: no luck and it event couldn't resolve hostname of it's own node
• None: not tried (I don't think it's a good way)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "app.name" . }}
  namespace: {{ .Values.global.namespace }}
spec:
  replicas: 2
  selector:
    matchLabels:
      app: {{ include "app.name" . }}
  template:
    metadata:
      labels:
        app: {{ include "app.name" . }}
    spec:
      containers:
      - name: {{ include "app.name" . }}
        image: ...
        resources:
          ...
        ports:
          ...
        imagePullPolicy: Always
      restartPolicy: Always

Leukemia

Does this answer your question? how to use Kubernetes DNS for pods? – Tisatisane

I think you need to explicitly use podname.namespace.pod.cluster.local. In most cases going through a Service is better practice. – Tisatisane

@DavidMaze Well, maybe I put the question wrong. What I'm trying to accomplish is to create a cluster for Apache Drill. Each drill pod registers to the zookeeper by it's own hostname. The hostname inside the pod is not a FQDN so this way they register with unresolvable address. But they need to resolve and talk to each other ... Basically it would be enough if the hostname could be set to the FQDN in the yaml. Unfortunately, Drill itself does not have any configuration for this .. Is there any other solution for this cases? – Leukemia

kubernetes.io/docs/concepts/workloads/controllers/statefulset – Stochmal

Answer 1

Normally, only Services get DNS names, not Pods. So, by default, you can't refer to another Pod directly by a domain name, only by its IP address.

Pods get DNS names only under certain condidtions that include a headless Service, as explained in the documentation. In particular, the conditions are:

• The Pods have a hostname field
• The Pods have a subdomain field
• There is a headless Service (in the same namespace) that selects the Pods
• The name of the headless Service equals the subdomain field of the Pods

In this case, each Pod gets a fully-qualified domain name of the following form:

my-hostname.my-subdomain.default.svc.cluster.local

Where my-hostname is the hostname field of the Pod and my-subdomain is the subdomain field of the Pod.

Note: the DNS name is created for the "hostname" of the Pod and not the "name" of the Pod.

You can test this with the following setup:

apiVersion: v1
kind: Service
metadata:
  name: my-subdomain
spec:
  selector:
    name: my-test
  clusterIP: None
---
apiVersion: v1
kind: Pod
metadata:
  name: my-pod-1
  labels:
    name: my-test
spec:
  hostname: my-hostname-1
  subdomain: my-subdomain
  containers:
  - image: weibeld/ubuntu-networking
    command: [sleep, "3600"]
    name: ubuntu-networking
---
apiVersion: v1
kind: Pod
metadata:
  name: my-pod-2
  labels:
    name: my-test
spec:
  hostname: my-hostname-2
  subdomain: my-subdomain
  containers:
  - image: weibeld/ubuntu-networking
    command: [sleep, "3600"]
    name: ubuntu-networking

After applying this, you can exec into one of the Pods:

kubectl exec -ti my-pod-1 bash

And you should be able to resolve the fully-qualifed domain names of the two Pods:

host my-hostname-1.my-subdomain.default.svc.cluster.local
host my-hostname-2.my-subdomain.default.svc.cluster.local

Since you're making the requests from the same namespace as the target Pods, you can abbreviate the domain name to:

host my-hostname-1.my-subdomain
host my-hostname-2.my-subdomain

Hypostyle

Super useful answer and sample spec. Doc isn't at all so clear about it. – Superbomb

This specifically means, that the pods can not come from a deployment, as deployment does not specify pod's hostname ( So either a stateful set should be used, or there's an option to use pod's IP address instead of hostname address part – Kovar