Facebook scraper uses incorrect DNS data > my site is not gettng scraped

Question

I recently moved one of my sites (gezondbenjij.nl) to a new hosting account. This resulted in a new IP address.

Unfortunately, since the move, the Facebook scraper cannot find my site on the new IP address. It still uses the old IP. All DNS settings are correct, and every browser/client/tool finds the correct site at 178.22.57.204 (gezondbenjij.nl). Except for Facebook.. The facebook scraper lands in my old hosting account. So I guess their hostfile or DNS cache still holds the old data. Even after a couple of weeks.

I have set a domain alias at gezondbenjij.com --> gezondbenjij.nl, and redirected the old hosting account to this .com domain. In this way, facebook is redirected to the correct site via a bypass, but still is not able to scrape the site. So the URLs I try to share on facebook will not generate snippets at this moment.

Is there any way to force (or kindly ask) Facebook to update their DNS cache? I used the fb URL linter, but it will not solve the issue. It will only reset facebook's html cache, not the DNS cache..

Berga

what page are you sharing and what is the debug tool showing? are you sure the DNS is the error here and that it's not due to your tags having a circular redirect or something like that? – Twelfthtide

Yeah, I'm positively sure it's a DNS thing. I was trying any page from gezondbenjij.nl in the debug tool and I either got a 206 response or a 404 response (for posts created after the move). Ever since I redirected the htaccess in my old hosting account (so old ip) to a newly created domain which serves as alias for the existing domain, I get 200 responses, but with crawling errors. I'm still quite stuck on this :( – Berga

Have you solved this? The same issue happened to me. The page is on new hosting for one month and there is still a problem with scraper. – Kasi

Answer 1

Check your IPv4 and IPv6 IP addresses in your DNS configuration. I had similar issue and the problem was that I've updated only IPv4 DNS record, while keeping old IPv6 (Facebook uses IPv6 in it's Tools). My hosting provider corrected it and problem was solved.

Kasi

OMG thank you SO MUCH. I would never have been able to solve this problem without your answer. – Aloke