Pinterest rich pin not working & validator fails?

Question

We've been struggling with this problem for several months yet no avail. Rich pins on Pinterest used to work for our site but suddenly they stopped working and one after another all the rich pins (clickable title and price) disappeared for the pins of our site.

For example, with this product, we clearly have the open graph information up front in the head section of the HTML code:

<meta property="og:type" content="product" />
<meta property="og:title" content="Backless Gray Lace Tulle Flower Girl Dress with Big Bow" />
<meta property="og:description" content="This dress is made of high quality lace and tulle fabric; Unique Cross back with straight neckline in the front; Big ... Shop now use NYC2018 for 9% off TODAY!" />
<meta property="og:url" content="http://www.princessly.com/backless-gray-lace-tulle-flower-girl-dress-with-big-bow.html" />
<meta property="og:site_name" content="Princessly.com" />
<meta property="og:price:amount" content="49.62" />
<meta property="og:price:currency" content="USD" />
<meta property="og:availability" content="instock" />

As per their rich pin requirements here. It's all working good when we first implemented it on our site around half a year ago. Unfortunately for some reason it stopped working 1 month ago. See the pin page. It shows none of these rich pin information no more.

When we try the product page with the rich pin validator (https://developers.pinterest.com/tools/url-debugger/), it's still NOT working and keeps giving this error for every URL we test:

We were unable to retrieve any data from your URL.

Like this: http://www.princessly.com/pinterest_error.png

It's not a very useful error message. We are not sure if they can't access the page (http code? blank page or whatever?) or just don't find any open graph information there.

We tried everything asking around webmaster forums and seeking Pinterest official support which basically was not helpful other than giving us the docs we already knew and blamed our server for blocking access of their bot but after confirming with our hosting team several times, there were no errors at all regarding Pinterest.com accessing our site:

54.85.173.173 - - [11/Oct/2016:21:22:00 -0500] "GET /u-back-ivory-cotton-ruffle-neckline-flower-girl-dress.html HTTP/1.1" 200 22477 "-" "Pinterest/0.2 (+http://www.pinterest.com/)"
54.227.228.14 - - [11/Oct/2016:21:27:20 -0500] "GET /u-back-ivory-cotton-ruffle-neckline-flower-girl-dress.html HTTP/1.1" 200 22477 "-" "Pinterest/0.2 (+http://www.pinterest.com/)"
52.87.185.68 - - [11/Oct/2016:21:28:06 -0500] "GET /u-back-ivory-cotton-ruffle-neckline-flower-girl-dress.html HTTP/1.1" 200 22477 "-" "Pinterest/0.2 (+http://www.pinterest.com/)" 

Since we are a women products merchant, Pinterest means quite a sum of the sales. We have seen considerable drop of revenue since rich pin disappearance. We believe this issue is costing real money as we speak.

We are desperate and think we should try our luck here on SO. Not sure if this fits in programming questions but it sure involves code of HTML and Open Graph and one of the major social network who offers developer API.

As we can't get our hands on Pinterest bot and how it sees our page, there seems very limited ways for us to debug this.

Any idea how this happens and how to debug & fix it? Any Pinterest staff developer here could offer some insight? Thank you!

UPDATE 1

Here are the HTTP USER AGENTS I recorded via index.php of Princessly.com for the last 24 hours that contain 'Pinterest': http://www.princessly.com/Pinterest.txt

My host WiredTree was kind enough to check all this again for me. Seems Pinterest bots were indeed able to access my site no problem.

---

On checking the access logs for the site "princessly.com", I can see successful accesses from "Pinterest" to the site "Princessly.com". Please see the snippet from logs.

======

194.103.189.40 - - [15/Nov/2016:03:32:31 -0600] "GET /skin/frontend/default/forest_fashion/images/bkg_btn_sprite.png HTTP/1.1" 200 2503 "http://www.princessly.com/skin/frontend/default/forest_fashion/css/styles.css" "Pinterest/0.2 +http://pinterest.com/" 194.103.189.40 - - [15/Nov/2016:03:32:31 -0600] "GET /skin/frontend/default/forest_fashion/images/tabs_border.gif HTTP/1.1" 200 43 "http://www.princessly.com/skin/frontend/default/forest_fashion/css/styles.css" "Pinterest/0.2 +http://pinterest.com/" 194.103.189.40 - - [15/Nov/2016:03:32:31 -0600] "GET /skin/frontend/default/forest_fashion/images/pager_bg.gif HTTP/1.1" 200 102 "http://www.princessly.com/skin/frontend/default/forest_fashion/css/styles.css" "Pinterest/0.2 +http://pinterest.com/"

======

Is there any way I can find the name of the Pinterest bot That I can detect as HTTP USER AGENT?

From the logs I can see that the the Pinterest bot tag is "Pinterest/0.2 +http://pinterest.com".

1. Can we know if the requests made by Pinterest bot timed out?

I have checked the Apache logs for the IP addresses from which the Pinterest access were noted and I could not find any time out errors.

2. How do we know if we block AWS requests?

On checking I could not see any block rules setup for the site "Princessly.com". I have checked the .htaccess file and robots.txt file for the site. I have now whitelisted the AWS IP address range as per the link you have provided.

Can you please check the details now and let me know if there is any user panel for Pinterest or other details so that we can dig deeper.

---

UPDATE 2

Following advice from @Mackan, I created a few identical bare minimal pages exactly the same with each other and with correct open graph information, and uploaded them to several different sites. I'm recording all the PHP USER AGENT that accesses the page:

1. Sites on the same server as Princessly.com

Site IP: 173.199.151.128

http://www.princessly.com/ivory-lace-tulle-wedding-flower-girl-dress-with-sheer-neck-TEST123.php

http://www.princessly.com/Pinterest_agents.txt

-- FAIL

Site IP: 173.199.151.128

http://www.usabledatabases.com/ivory-lace-tulle-wedding-flower-girl-dress-with-sheer-neck-TEST123.php

http://www.usabledatabases.com/Pinterest_agents.txt

-- FAIL

Site IP: 173.199.151.130

http://www.bintley.com/ivory-lace-tulle-wedding-flower-girl-dress-with-sheer-neck-TEST123.php

http://www.bintley.com/Pinterest_agents.txt

-- FAIL

2. Different server than Princessly.com

https://www.thedataplanet.com/ivory-lace-tulle-wedding-flower-girl-dress-with-sheer-neck-TEST123.php

https://www.thedataplanet.com/Pinterest_agents.txt

-- SUCCESS

3. Another different server than Princessly.com

http://www.formkid.com/ivory-lace-tulle-wedding-flower-girl-dress-with-sheer-neck-TEST123.php

http://www.formkid.com/Pinterest_agents.txt

-- SUCCESS

After testing with the rich pin validator, it seems it's a server issue. The request never reaches the PHP file for all sites on the Princessly.com server.

WiredTree support was very kind to help me in finding the culprit but after shifting through over 5 support staff, nothing promising thus far. We have gone through .htaccess, Apache conf, modSecurity, firewall, whitelisting Pinterest IPs, AWS IPs, etc. Nothing worked.

This is really really weird.

UPDATE 3

Even weirder is, if you look at http://www.princessly.com/Pinterest_agents.txt you can see there are INDEED entries of "Pinterest/0.2 (+http://www.pinterest.com/)" there.

However if you give it a manual test, it FAILS and DOESN'T record the access at all.

It seems to be breaking intermittently, maybe.

This is pretty much in the quantum physics arena and I don't know what to do now, except the last resort that is to order and provision a new server and migrate Princessly.com there see how it goes.

Your idea?

Bosch

Have you tried to revalidate your pin? – Waistline

@claudios, how does one do that? – Bosch

developers.pinterest.com/tools/url-debugger – Sexagesima

the link by @Sexagesima – Waistline

@claudios, thanks but already tried it for as many urls as we could and it fails every time. – Bosch

If it fails, what's the error? You stated there is no way for you to debug, but the validator seems like a good place to start. – Sexagesima

What error you got? – Waistline

@Mackan, I've updated the error details in the question body. Basically, I got the error "We were unable to retrieve any data from your URL." for every page I test. – Bosch

Info on the error: If you're seeing We weren't able to retrieve any data from your URL, then Pinterest may be timing out before we can create a Pin on your site. Unfortunately, this isn't something we can fix on our end, and usually happens when a site is slow. This error also appears if your site blocks Amazon Web Services (AWS) requests. You'll have to remove this block before applying for Rich Pins. – Sexagesima

You're not using any kind of redirects for bots (or clients that can't handle javascript)? – Sexagesima

@Mackan, I checked these with my host before but I'm checking again with them. Will let you know. No, our site has no redirects for (specific) bots nor gives different content to clients without javascript capabilities. I just hand coded a PHP cURL script to fetch my pages and they all seem all right with the correct open graph information. – Bosch

Yes, I fetched the page using Pinterestbot user-agent and it looked ok (although 4.5Mb of data in over 250 requests). Perhaps your host can make use of these origins for Pinterestbot: udger.com/resources/ua-list/bot-detail?bot=Pinterest – Sexagesima

@Mackan, reply from my host team was updated in my question. Seems Pinterest had no issue accessing my site. This is really frustrating. – Bosch

Annoying for sure. I would create a minimal "product page", only containing the required meta's and html and see if that would pass the validator. If it works, it's a matter of adding functionality (javascripts etc.) until you find the possible culprit. Out of curiosity: did you see any pinterestbot requests on product pages (the html request), or only on the stylesheets/resources? – Sexagesima

@Mackan, that's a great idea! I will try it out and let you know! – Bosch

@Mackan, question updated. Thanks for bearing with me. – Bosch

Looking at your access.log it says the server returned "22477" bytes, comparing with: curl -so /dev/null "http://www.princessly.com/u-back-ivory-cotton-ruffle-neckline-flower-girl-dress.html" -w '%{size_download} - %{size_header}\n' 112716 - 628 - This request definitely failed somewhere, look at the sent response from the apache server to get more information, maybe you can use mod_dumpio or simply sniff using tcpdump. Once you know the actual error you can hopefully find out more. – Inwrought

@dav1d, I will give it a try. But I thought the 22477 size was the gzip-ed size of the page so it's significantly smaller? – Bosch

@kavoir.com unfortunately you might be right here, I haven thought of that: 22740 - 647 with gzip header. Maybe still worth a shot and the size is just a coincidence. – Inwrought

Hopefully this will get noticed from an apache-guru or similar. I'm thinking the "manual check" (validation) might be sent from another host than the bot. Looking at the request it sends when validating it originates from 54.85.94.5:443 (i.e. developers.pinterest.com I guess). If that's part of your whitelist I'm out of ideas :( – Sexagesima

@Mackan, can you please try the url debugger again with some of our pages? It seems to be working now as we just migrated the site to another server. It's really a hassle and let's hope this would do it. Can you please write an answer to this and I'll grant you the bounty. – Bosch

Getting " 503: Service Temporarily Unavailable" on all request to your site right now. Perhaps just bad timing. I'll try again in a bit. Edit: This renders in browser but is not validated in pinterest: princessly.com/… (173.199.151.128) – Sexagesima

Seems to be working fine now! :) I left an answer as per request with more details. – Sexagesima

Answer 1

After debugging the issue, using https://developers.pinterest.com/tools/url-debugger/, the problem seems to stem from the server-side.

If you're seeing We weren't able to retrieve any data from your URL, then Pinterest may be timing out before we can create a Pin on your site. Unfortunately, this isn't something we can fix on our end, and usually happens when a site is slow. This error also appears if your site blocks Amazon Web Services (AWS) requests. You'll have to remove this block before applying for Rich Pins.

Creating some "minimal but working" pages and running them through the validator made the issue clearer, but unfortunately didn't resolve it. Either this is a problem with the local server (Apache) or possibly an issue with the pinterest bot.

The Pinterest bot seems to hit the server OK, but the validator can't parse or reach the endpoint. Since the minimal examples returns in ~less than a second, and contains a bare minimum to satisfy Pinterests requirements, it's clear that the issue is not within your code.

Migrating the solution to another host seems to have fixed the problem.

Edit

The new host seems to be working better (using one of the minimal examples):

Sexagesima

Hi Mackan, I m stuck with the same issue about rich pins. So, did you manage to solve the issue by moving to new host? Did you manage to get any more details about the issue? In my case it s not that simple to just move to new host. You may have a look at my question here. #44572755. Your input is much appreciated! Thank you!! – Sis

Answer 2

We have the same issue. In our case this is being caused because the server has multiple SSL certificates and uses SNI so the client can get the certificate. However Pinterest is using curl on the back end which is NOT SNI aware. We have logged this with Pinterest

Gratulant

Answer 3

For me the problem was a missing intermediate SSL certificate. By reinstalling my SSL certificate and this time including the intermediate SSL certificate, I was able to successfully validate with the Pinterest Rich Pins Validator.

Weasel

Answer 4

For me the problem was I was using "http://example.com". The validator wouldn't recognize my site. Not AT ALL. Not even the site name or the favicon. Then I switched to "http://www.example.com" and it worked perfectly.

Adelbert