How can I retrieve og/meta attributes of a resource?

Question

I'm making an application that retrieve tweets on Twitter of a user.

Those feeds contains links to external resources, such as Articles, Webpage or YouTube video.

I get trought the Twitter API the JSON of these feeds, but there arent included the og: attributes of the content. And I'd like to catch them and show to my website.

Such as this question of StackOverflow:

<meta name="og:type" content="website" />
<meta name="og:image" content="http://cdn.sstatic.net/stackoverflow/img/[email protected]?v=fde65a5a78c6"/>
<meta name="og:title" content="How can I check classes that ends with?" />
<meta name="og:description" content="I have some elements such as:
    &amp;lt;div class=&quot;button 17-facebook-dashboard-check&quot;&amp;gt;Elem1&amp;lt;div&amp;gt;
    &amp;lt;div class=&quot;button 18-google-dashboard-check&quot;&amp;gt;Elem2&amp;lt;div&amp;gt;
    &amp;lt;div class=&quot;button " />
<meta name="og:url" content="https://mcmap.net/q/748757/-how-can-i-check-classes-that-ends-with"/>

I'd like to catch those informations for each shared resource on each tweet. So I think I'll, for each tweet (which for me is a box) do an ajax request client side, download the html and parse it, retrieving og:title, og:description, og:type and og:image.

Is this the best approch? What's about parse this data in Javascript/Jquery?

Pole

Answer 1

These og: attributes are Open Graph Protocol attributes, there are many way to get these datas : you should check the codes of Open Graph Protocol parser which may be very usefull for you, and this PHP and jQuery Facebook link parser.

You can also check this StackOverflow Question about PHP parsing and this Opengraph PHP parser and dynamically use them with ajax calls.

Finally, this StackOverflow question about JQuery and pure JavaScript parsing is very interesting and could really help you.

Hope you'll find what you need ! ;)

Whitacre

Answer 2

DISCLAIMER: OpenGraph.io is a commercial product I work on and support.

As you mentioned, often times there are no OG tags to work with. There are all sorts of scenarios you can come across (e.g. encoding, misusing HTML tags, etc). If you want to handle the edge cases I'd recommend http://www.opengraph.io/

One of its major benefits is that it will infer information like the title or description (if you end up needing it) from the content on the page if OpenGraph tags don't exist.

To get information about a site use (link should be URL encoded):

$.ajax('http://opengraph.io/api/1.0/site/http%3A%2F%2Fwww.washingtontimes.com%2F')
  .done(function(data){
    console.log(data);
  });

Which will return something like:

{
  "hybridGraph": {
    "title": "Washington Times - Politics, Breaking News, US and World News",
    "description": "The Washington Times delivers breaking news and commentary on the issues that affect the future of our nation.",
    "image": "http://twt-assets.washtimes.com/v4/images/logo-twt.4b20fb5d7b29.svg",
    "url": "http://www.washingtontimes.com/",
    "type": "site",
    "site_name": "Washington Times "
  },
  "openGraph": {...},
  "htmlInferred": {...},
  "requestInfo": {...}
}

Saraann

Answer 3

You can use open-graph-scraper for this:

const ogs = require('open-graph-scraper');
const options = { url: 'http://ogp.me/' };
ogs(options)
  .then((data) => {
    const { error, result, response } = data;
    console.log('error:', error);  // This returns true or false. True if there was an error. The error itself is inside the results object.
    console.log('result:', result); // This contains all of the Open Graph results
    console.log('response:', response); // This contains the HTML of page
  })

Note: on the client side you'll hit into CORS issues, so this is best done on the server side if possible.

In my instance I'm using Next.js, and I added a file in /pages/api/metadata/[...url].ts which I could then query, that would do the ogs lookup and then return the result, which I can then use in my front-end.

Guitarfish

Answer 4

Anyone finding this question who is looking for a way to grab OG (open graph) metadata values using the browser console (Chrome or other) can do it using ES6 JavaScript.

Example:

To grab the "description" tag, (which will also return the site byline for WordPress website) use this one-liner code snippet I wrote to do just that:

document.querySelectorAll('meta[property="og:description"]')[0]

This does not address grabbing stuff remotely off a server with Ajax, this is simply a browser-based solution.

Here is another quick example. Let's say you want to grab all the metadata properties and store them in an object that can be passed. This is most easily tested on a good WordPress website, but should work wherever there are open graph meta tags.

/*

10/01/18

Eric Hepperle

Grab all OG Meta Tags values on a webpage

Total time spent to create and test: 1 hr.

*/

console.clear();

// Store all our properties in one object
var ogWebsite = {};

//var metas = document.querySelectorAll('meta[property="og:description"]')[0]
var metaTags = document.querySelectorAll('meta');

var propTagCount = 0;

[...metaTags].forEach(function(tag, i) {
    
    // console.log(tag);
    
    if (tag.hasAttribute('property')) {
        
        var propName = tag.getAttribute('property');
        // console.log("%c\t%s", "background: orange; color: black", propName);
        console.log(propName);

        // Get the value of the OG property attribute
        var ogMetaValue = document.querySelectorAll("meta[property='" + propName +"']")[0].content;
        
        console.log("%cogMetaValue: %s","background: purple; color: white;", ogMetaValue);
        
        // Add property to ogWebsite object. We can do this because
        //  ES6 (2015) allows varible keys with object literals.
        //  To work, you must use bracket "[]" notation instead of dots.
        ogWebsite[propName] = ogMetaValue;
        
        ++propTagCount;        
    }
    
    
});

console.log("%cTotal meta tags: %s", "background: bisque; color: brown; font-weight: bold;", metaTags.length);
console.log("%cTotal meta tags with 'property' attribute: %s", "background: cadetblue; color: white; font-weight: bold;", propTagCount);

// Display the final object:
console.log(ogWebsite);

Kizzee