In a ROOTLESS podman setup, how to communicate between containers in different pods

Question

I read all I could find, but documentation on this scenario is scant or unclear for podman. I have the following (contrived) ROOTLESS podman setup:

• pod-1 name: pod1

  Container names in pod1:

  • p1c1 -- This is also it's assigned hostname within pod1
  • p1c2 -- This is also it's assigned hostname within pod1
  • p1c3 -- This is also it's assigned hostname within pod1

• pod-2 name: pod2

  Container names in pod2:

  • p2c1 -- This is also it's assigned hostname within pod2
  • p2c2 -- This is also it's assigned hostname within pod2
  • p2c3 -- This is also it's assigned hostname within pod2

I keep certain containers in different pods specifically to avoid port conflict, and to manage containers as groups.

QUESTION:

Give the above topology, how do I communicate between, say, p1c1 and p2c1? In other words, step-by-step, what podman(1) commands do I issue to collect the necessary addressing information for pod1:p1c1 and pod2:p2c1, and then use that information to configure applications in them so they can communicate with one another?

Thank you in advance!

EDIT: For searchers, additional information can be found here.

Minutely

The same question can be found here: github.com/containers/podman/issues/10054 – Daliadalila

Indeed. I cross-posted this question on GitHub, too. – Minutely

Answer 1

Podman doesn't have anything like the "services" concept in Swarm or Kubernetes to provide for service discovery between pods. Your options boil down to:

1. Run both pods in the same network namespace, or
2. Expose the services by publishing them on host ports, and then access them via the host

For the first solution, we'd start by creating a network:

podman network create shared

And then creating both pods attached to the shared network:

podman pod create --name pod1 --network shared
podman pod create --name pod2 --network shared

With both pods running on the same network, containers can refer to the other pod by name. E.g, if you were running a web service in p1c1 on port 80, in p2c1 you could curl http://pod1.

For the second option, you would do something like:

podman pod create --name pod1 -p 1234:1234 ...
podman pod create --name pod2 ...

Now if p1c1 has a service listening on port 1234, you can access that from p2c1 at <some_host_address>:1234.

---

If I'm interpreting option 1 correctly, if the applications in p1c1 and p2c1 both use, say, port 8080; then there won't be any conflict anywhere (either within the pods and the outer host) IF I publish using something like this: 8080:8080 for app in p1c1 and 8081:8080 for app in p2c1? Is this interpretation correct?

That's correct. Each pod runs with its own network namespace (effectively, it's own ip address), so services in different pods can listen on the same port.

Can the network (not ports) of a pod be reassigned once running? REASON: I'm using podman-compose(1), which creates things for you in a pod, but I may need to change things (like the network assignment) after the fact. Can this be done?

In general you cannot change the configuration of a pod or a container; you can only delete it and create a new one. Assuming that podman-compose has relatively complete support for the docker-compose.yaml format, you should be able to set up the network correctly in your docker-compose.yaml file (you would create the network manually, and then reference it as an external network in your compose file).

Here is a link to the relevant Docker documentation. I haven't tried this myself with podman.

Bulldog

If I'm interpreting option 1 correctly, if the applications in p1c1 and p2c1 both use, say, port 8080; then there won't be any conflict anywhere (either within the pods and the outer host) IF I publish using something like this: 8080:8080 for app in p1c1 and 8081:8080 for app in p2c1? Is this interpretation correct? – Minutely

This is a separate question to the above comment question (so I have two follow-ups now). Can the network (not ports) of a pod be reassigned once running? REASON: I'm using podman-compose(1), which creates things for you in a pod, but I may need to change things (like the network assignment) after the fact. Can this be done? – Minutely

I've updated the question with answers to your comments. – Bulldog

Indeed, I'm hoping that podman-compose supports external networks too, because that's what I do with docker-compose. Thank you for the follow-ups. This Q&A session will be useful to others I think. – Minutely

Answer 2

Accepted answer from @larsks will only work for rootful containers. In other words, run every podman commands with sudo prefix. (For instance when you connect postgres container from spring boot application container, you will get SocketTimeout exception)

If two containers will work on the same host, then get the ip address of the host, then <ipOfHost>:<port>. Example: 192.168.1.22:5432

For more information you can read this blog => https://www.redhat.com/sysadmin/container-networking-podman

Throaty

Answer 3

Note: The above solution of creating networks, only works in rootful mode. You cannot do podman network create as a rootless user.

Encourage

That's incorrect. I run jdoe$ podman network create mynetwork as a non-root user (i.e. as myself) all the time. In fact, I never use rootful podman. To avoid being down-voted, I recommend correcting this. Also, this should be a comment not an answer, because it doesn't answer the question (it's a side comment). – Minutely

I literally get an error that says, "network create is not supported in rootless mode" – Encourage

You probably have an old version of podman(1) or have not configured your podman(1) environment correctly. Upgrade podman(1) to the latest, then search for combinations of rootless and ~/.config/containers/storage.conf, ~/.config/containers/containers.conf and /etc/subuid and /etc/subgid for examples. – Minutely

I'm stuck using version 1.6.4 on CentOS 7 version 1160. I work for a government contractor and I don't have the luxury of upgrading because that's the version that has been approved for us to use. It would be months before I'm allowed to upgrade to even version 1.9 – Encourage

Bummer. But your answer is still incorrect and you should remove it. – Minutely