How to use a different dns name for OpenShift 3.11 routes than the default wildcard dns name?

Asked 22/10, 2019 at 12:33 Answered 4/2, 2021 at 13:15

Solved kubernetes dns ibm-cloud openshift

I'm not able to get a custom domain record working with an openshift cluster. I've read tons of articles, StackOverflow posts, and this youtube video https://www.youtube.com/watch?v=Y7syr9d5yrg. All seem to "almost" be usefull for me, but there is always something missing and I'm not able to get this working by myself.

The scenario is as follows. I've got an openshift cluster deployed on an IBM Cloud account. I've registered myinnovx.com. I want to use it with an openshift app. Cluster details:

oc v3.11.0+0cbc58b
kubernetes v1.11.0+d4cacc0
openshift v3.11.146
kubernetes v1.11.0+d4cacc0

I've got an application deployed with a blue/green strategy. In the following screenshot, you can see the routes I've available.

mobile-blue: I created this one manually pointing to my custom domain mobileoffice.myinnovx.com

mobile-office: Created with oc expose service mobile-office --name=mobile-blue to use external access.

mobile-green: Openshift automatically generated a route for the green app version. (Source2Image deployment)

mobile-blue: Openshift automatically generated a route for the blue app version. (Source2Image deployment)

I've set up a two CNAME record on my DNS edit page as follows:

In several blogs/articles, I've found that I'm supposed to point my wildcard record to the router route canonical name. But I don't have any route canonical name in my cluster. I don't even have an Ingress route configured.

I'm at a loss here as to what I'm missing. Any help is greatly appreciated. This is the response I get testing my DNS:

This is a current export of my DNS:

$ORIGIN myinnovx.com.
$TTL 86400
@ IN SOA ns1.softlayer.com. msalimbe.ar.ibm.com. (
                       2019102317        ; Serial
                       7200              ; Refresh
                       600               ; Retry
                       1728000           ; Expire
                       3600)             ; Minimum

@                      86400    IN NS    ns1.softlayer.com.
@                      86400    IN NS    ns2.softlayer.com.


*.myinnovx.com         900      IN CNAME .mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud.
mobileoffice           900      IN CNAME mobile-office-mobile-office.mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud
mobile-test.myinnovx.com 900      IN A     169.63.244.76

Chromosome answered 22/10, 2019 at 12:33 Comment(0)

I think you almost got it, Matias.

The FQDN - mobile-office-mobile-office.mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud - resolves for me to an IP that is part of SOFTLAYER-RIPE-4-30-31 and is accessible from the Internet. So, it should be possible to configure what you want.

That snapshot in your question of the DNS records isn't displaying the entries in full but what might be missing is a dot . at the end of both the "Host/Service" and "Value/Target". Something like this:

mobileoffice.myinnovx.com. CNAME 900 (15min)  mobile-office-mobile-office.mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud.

Shadwell answered 23/10, 2019 at 0:5 Comment(12)

At the moment I got two CNAME records: Host: *.myinnovx.com to value .mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud. Host: mobileoffice.myinnovx.com to value mobile-office-mobile-office.mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud. If I try to add a dot to mobileoffice.myinnovx.com I get this message: The host value "mobileoffice.myinnovx.com." must be a relative value or the @ character to represent your domain name. – Chromosome 23/10, 2019 at 13:26

What I saw few minutes ago with dig is that the OpenShift route is a CNAME itself mapped to mycluster-342148-2656....appdomain.cloud, which in turn is an A record pointing to 169.63.244.74. Make mobile-office-mobile-office.mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud. an A record pointing to 169.63.244.74 and the rest as in my answer. – Shadwell 23/10, 2019 at 14:35

I'm not able to create records with a dot at then for the host value. For example I get the message The host value "mobile-office-mobile-office.mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud." must be a relative value or the @ character to represent your domain name. – Chromosome 23/10, 2019 at 14:53

Judging by the error message...could you edit the CNAME record for mobileoffice.myinnovx.com to have just mobileoffice as the host? That would become a host name relative to myinnovx.com. As different providers require the info be entered differently in their DNS consoles, including whether they require a dot/period at the end...try first with a dot at the end and next without, keeping in mind that it may take awhile for DNS changes to get propagated. Enter mobile-office-mobile-office.mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud for target. – Shadwell 23/10, 2019 at 17:6

I am sorry - don't add a dot at the end. I Couldnt edit the comment as it was more than 5 minutes after it was posted. – Shadwell 23/10, 2019 at 17:13

I now see mobileoffice.myinnovx.com CNAME record! Let's fix the target by removing the dot at the end so it is an absolute domain and not relative to myinnovx.com. – Shadwell 23/10, 2019 at 17:24

Not sure how you checked for the CNAME, but just removed the dot from the target value. Fingers crossed. (Thank you) – Chromosome 23/10, 2019 at 17:36

Just checked with dig and found it: ;; ANSWER SECTION: mobileoffice.myinnovx.com. 900 IN CNAME mobile-office-mobile-office.mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud.myinnovx.com. But it has added .myinnovx.com. at the end !?! – Chromosome 23/10, 2019 at 17:40

At some point it was actually correct and without .myinnovx.com at the end. I was actually able to go to mobileoffice.myinnovx.com with a browser and saw the login form. Could you retrace the changes you made? Like, removing dot from the end of the target? Also, keep in mind that it takes awhile for DNS changes to be propagated, literally, around the world. – Shadwell 23/10, 2019 at 17:54

Yeah. I had removed the dot from the wildcard hostname CNAME record target value. I got anxious. Anyway, I've readded the dot, I'll wait for a while and test. And then test without the dot. – Chromosome 23/10, 2019 at 18:3

Right now I getting an AA with the domain added at the end: mobileoffice.myinnovx.com. 894 IN CNAME mobile-office-mobile-office.mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud.myinnovx.com. – Chromosome 23/10, 2019 at 18:5

Finally Its working. I'm getting an SSL error because of the different domain. I'll need to deal with that now, but at least I know what was the problem. In the end, I deleted the wildcard record, and left only the one for mobileoffice. Thanks a lot apisim!! – Chromosome 23/10, 2019 at 21:20

Most of what I'm about to say only applies to OpenShift 3.x. In OpenShift 4.x things are sufficiently different that most of the below doesn't quite apply.

By default OpenShift 3.11 exposes applications via Red Hat's custom HAProxy Ingress Controller (colloquially known as the "Router"). The typical design in a OpenShft 3.x cluster is to designate particular cluster hosts for running cluster infrastructure workloads like the HAProxy router and the internal OpenShift registry (usually using the node-role.kubernetes.io/infra=true node labels).

For convenience purposes so admins don't have to manually create a DNS record for each exposed OpenShift application, there is a wildcard DNS entry that points to the load balancer associated with the HAProxy Router. The DNS name of this is configured in the openshift_master_default_subdomain of the ansible inventory file used to do your cluster installation.

The structure of this record is generally something like *.apps.<cluster name>.<dns subdomain>, but it can be anything you like.

If you want to have a prettier DNS name for your applications you can do a couple things.

The first is to create a DNS entry myapp.example.com pointing to your load balancer and have your load balancer configured to forward those requests to the cluster hosts where the HAProxy Router is running on port 80/443. You can then configure your application's Route object to use hostname myapp.example.com instead of the default <app name>-<project name>.apps.<cluster name>.<dns subdomain>.

Another method would be to do what your suggesting and let the application use the default wildcard route name, but create a DNS CNAME pointing to the original wildcard route name. For example if my openshift_master_default_subdomain is apps.openshift-dev.example.com and my application route is myapp-myproject.apps.openshift-dev.example.com then I could create a CNAME DNS record myapp.example.com pointing to myapp-myproject.apps.openshift-dev.example.com.

The key thing that makes either of the above work is that the HAProxy router doesn't care what the hostname of the request is. All its going to do is match the Host header (SNI must be set in the case of TLS requests and the HAProxy router configured for pass through) of the incoming request against all of Route objects in the cluster and see if any of them match. So if your DNS/Load Balancer configuration is setup to bring requests to the HAProxy Router and the Host header matches a Route, that request will get forwarded to the appropriate OpenShift service.

In your case I don't think you have the CNAME pointed at the right place. You need to point your CNAME at the wildcard hostname your application Route is using.

Walkover answered 22/10, 2019 at 14:16 Comment(3)

Thanks for your reply. If my openshift generated url is mobile-office-mobile-office.mycluster-342148-26562a7d6831df3dfa…. Is it safe to assume that my wildcard hostname apllication Route is *.mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud? – Chromosome 22/10, 2019 at 14:49

I keep trying different ways to create the CNAME. I don't think I've got access to my Load Balancer. Since the cluster is created on the Cloud, I didn't install it myself, I don't have access to config files an such. – Chromosome 22/10, 2019 at 15:34

I'm not that familiar with IBM Cloud, but the above URL looks right. You'll have to make sure there is a Route configured to match the hostname for that CNAME and not just for the wildcard. – Walkover 22/10, 2019 at 15:37

I think you almost got it, Matias.

mobileoffice.myinnovx.com. CNAME 900 (15min)  mobile-office-mobile-office.mycluster-342148-26562a7d6831df3dfa02975385757d2d-0001.us-south.containers.appdomain.cloud.

Shadwell answered 23/10, 2019 at 0:5 Comment(12)

I am sorry - don't add a dot at the end. I Couldnt edit the comment as it was more than 5 minutes after it was posted. – Shadwell 23/10, 2019 at 17:13

Not sure how you checked for the CNAME, but just removed the dot from the target value. Fingers crossed. (Thank you) – Chromosome 23/10, 2019 at 17:36

Also, please note the instructions for custom DNS setup for a route on OpenShift v4 are a bit different and are not correctly displayed in the web console:

apps.<clustername>.<clusterid>.<shard>.openshiftapps.com will not resolve to anything. *.apps.<clustername>.<clusterid>.<shard>.openshiftapps.com is the wildcard entry, so you need something prepending that.

To align with the way it was on v3 we usually chose the arbitrary string elb, e.g. - elb.apps.<clustername>.<clusterid>.<shard>.openshiftapps.com. That will hit the routers.

Here is the related BZ - https://bugzilla.redhat.com/show_bug.cgi?id=1925132

Rochellrochella answered 4/2, 2021 at 13:15 Comment(0)

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags