get image url of rss with rome library

Question

I having a rss file in following :

<?xml version="1.0" encoding="UTF-8" ?>
  <rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/">
   <channel>
     <title> سایپا نیوز </title>
     <link>http://www.saipanews.com/</link>
     <description></description>
     <language>fa</language>

       <item>
        <author></author>
        <pretitle></pretitle>
        <title>پیام تبریک دکتر جمالی به مناسبت فرارسیدن سالروز ولادت حضرت علی(ع) و روز پدر</title>
        <link>http://www.saipanews.com/view-6751.html</link>
        <pubdate>2016-04-20 10:58:00</pubdate>
        <description>سایپا نیوز: مدیرعامل گروه خودروسازی سایپا همزمان با فرارسیدن سالروز میلاد باسعادت حضرت علی(ع) و روز پدر، طی پیامی به تمامی پدران متعهد و پرتلاش ایران زمین تبریک گفت.</description>
        <secid>0</secid>
        <typid>8</typid>
        <image>http://www.saipanews.com/media/image/jamali/jmali.JPG</image>
    </item>

    <item>
        <author></author>
        <pretitle></pretitle>
        <title>فرهنگ رانندگی بین خطوط در معابر شهری در حال گسترش است </title>
        <link>http://www.saipanews.com/view-6748.html</link>
        <pubdate>2016-04-19 11:27:00</pubdate>
        <description>سایپا نیوز: به گزارش سایپا نیوز و به نقل از فرارو، از آنجایی که فرهنگ رانندگی مجموعه ای از رفتارهای درست رانندگی و آداب زندگی اجتماعی بهنگام تردد در شهرها و جاده ها است، رانندگی در بین خطوط معابر شهری یکی از نمادهای فرهنگ رانندگی در کشورهای درحال توسعه و توسعه یافته می باشد.</description>
        <secid>0</secid>
        <typid>8</typid>
        <image>http://www.saipanews.com/media/image/farhang%20ranandegi/252887_331.jpg</image>

    </item>
  </channel>
 </rss>

I want to get image's urls.
I use Rome library but not found any solution.
how to get image's url in item with Rome library ?

Ellsworthellwood

Post the code you tried. – Monahan

Answer 1

I for that get image tag , build new rss parser on the following:

public class NewRssParser extends RSS094Parser implements WireFeedParser {

public NewRssParser() {
    this("rss_2.0");
}

protected NewRssParser(String type) {
    super(type);
}

protected String getRSSVersion() {
    return "2.0";
}

protected boolean isHourFormat24(Element rssRoot) {
    return false;
}

protected Description parseItemDescription(Element rssRoot, Element eDesc) {
    Description desc = super.parseItemDescription(rssRoot, eDesc);
    desc.setType("text/html"); // change as per
                                // https://rome.dev.java.net/issues/show_bug.cgi?id=26
    return desc;
}

public boolean isMyType(Document document) {
    boolean ok;
    Element rssRoot = document.getRootElement();
    ok = rssRoot.getName().equals("rss");
    if (ok) {
        ok = false;
        Attribute version = rssRoot.getAttribute("version");
        if (version != null) {
            // At this point, as far ROME is concerned RSS 2.0, 2.00 and
            // 2.0.X are all the same, so let's use startsWith for leniency.
            ok = version.getValue().startsWith(getRSSVersion());
        }
    }
    return ok;
}

@Override
public Item parseItem(Element arg0, Element arg1) {
    Item item = super.parseItem(arg0, arg1);

    Element imageElement = arg1.getChild("image", getRSSNamespace());
    if (imageElement != null) {
        String imageUrl = imageElement.getText();

        Element urlElement = imageElement.getChild("url");
        imageUrl = urlElement != null ? urlElement.getText() : imageUrl;

        Enclosure enc = new Enclosure();
        enc.setType("image");
        enc.setUrl(imageUrl);

        item.getEnclosures().add(enc);
    }

    return item;
}

}

in the class override parseItem method and add code for get image element and add image's url to Enclosures.

then add following line to rome.properties file :

WireFeedParser.classes=[packge name].NewRssParser

Example :

WireFeedParser.classes=ir.armansoft.newscommunity.newsgathering.parser.impl.NewRssParser

Ellsworthellwood

Answer 2

Rome wont provide the <image> tag because it does not belong to the namespace it is in. So the feed isn't valid:

line 18, column 3: Undefined item element: image (29 occurrences) [help]
            <image>http://www.saipanews.com/media/image/%D8%AA%D9%88%D9%84%D9%8A%D8%A ...

If the image tag would be in a different namespace, like this:

<image:image>http://www.saipanews.com/media/image/%D8%AA%D9%88%D9%84%D9%8A%D8%AF/2.jpg</image:image>

You could get foreing markup in this way:

for(SyndEntry entry : feed.getEntries()) {
    for (Element element : entry.getForeignMarkup()) {
        System.out.println("element: " + element.toString());
    }
}

And the result would be

element: [Element: <image:image [Namespace: http://purl.org/rss/1.0/modules/image/]/>]

Unless the feed is fixed, It seems that there isn't a way to get the image url with Rome library at the moment.

Middleton

Answer 3

The Answer is so simple. First get the syndContent using the Roam API. Find the code for the reading images and all content from RSS

<%@ page import="com.rometools.rome.feed.synd.SyndFeed"%>
<%@ page import="com.rometools.rome.feed.synd.SyndEntry"%>
<%@ page import="com.rometools.rome.feed.synd.SyndContent"%>
<%@ page import="com.rometools.modules.mediarss.MediaEntryModule"%>
<%@ page import="com.rometools.rome.feed.module.Module"%>
<%@ page import="com.rometools.modules.mediarss.types.Thumbnail"%>
<%@ page import="java.util.Iterator"%>
<%@ page import="java.util.List"%>
<html>
<head>
<title>website</title>
<link href="/css/style.css" rel="stylesheet" type="text/css" />
</head>
<body>
    <h1>Home</h1>
    <%
        HttpSession session1=request.getSession(false);

        SyndFeed syndFeed11= (SyndFeed) session1.getAttribute("syndFeed");

    %>
    <h2><%=syndFeed11.getTitle()%></h2>
    <ul>
        <% 
           Iterator it = syndFeed11.getEntries().iterator();
           while (it.hasNext())
           {
              SyndEntry entry = (SyndEntry) it.next();
         %>
        <li><a href="<%=entry.getLink()%>"><%=entry.getTitle()%></a> <%

                List<SyndContent> syndContents=entry.getContents();
        System.out.println(syndContents.size());
                for(SyndContent syndContent:syndContents)
                {
                    System.out.println(syndContent.getMode());
                    System.out.println("This is content"+syndContent.getValue());
                    %>
                    //This is The STRING WHICH CONTAINS the link to the image apply regex expression to get SAMPLE_LINK out of "<img src"LINK">"
                    <%=syndContent.getValue() %>>
                    <%
                }
                //SyndContent syndContent=syndContents.get(0);



                for (Module module : entry.getModules()) {
            if (module instanceof MediaEntryModule) {
                MediaEntryModule media = (MediaEntryModule)module;
                for (Thumbnail thumb : media.getMetadata().getThumbnail()) {
                    %><img src="<%=thumb.getUrl() %>" />
            <%
                }
            }
        }
            %></li>
        <%  } %>
    </ul>
</body>
</html>

Bellow is the Servlet Class:-

package website.web;
import java.io.IOException;
import java.io.PrintWriter;
import java.net.URL;
import javax.servlet.RequestDispatcher;
import javax.servlet.ServletConfig;
import javax.servlet.ServletContext;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpSession;

import org.apache.log4j.Logger;

import com.rometools.rome.feed.synd.SyndFeed;
import com.rometools.rome.io.FeedException;
import com.rometools.rome.io.SyndFeedInput;
import com.rometools.rome.io.XmlReader;

public class HomeServlet extends HttpServlet {

/**
 * 
 */
private static final long serialVersionUID = 1L;
private Logger logger = Logger.getLogger(this.getClass());

@Override
public void init(ServletConfig config) throws ServletException {
    super.init(config);
}

@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
        throws ServletException, IOException {
    String rssUrl=(String)req.getAttribute("rss");
    logger.debug("Retrieving yahoo news feed");
    URL url = new URL("https://www.reddit.com/.rss");
    SyndFeedInput syndFeedInput = new SyndFeedInput();
    HttpSession session=req.getSession();
    SyndFeed syndFeed = null;
    XmlReader xmlReader = new XmlReader(url);
    try {
        syndFeed = syndFeedInput.build(xmlReader);
        System.out.println("Donr");

    } catch (IllegalArgumentException e) {
        logger.error("", e);
    } catch (FeedException e) {
        logger.error("", e);
    }
    logger.debug("Forwarding to home.jsp");
    req.setAttribute("syndFeed11", syndFeed);
    PrintWriter out = resp.getWriter();
    out.println("<h1>");
    out.println();
    session.setAttribute("syndFeed", syndFeed);
    out.println("</h1>");
    ServletContext context = getServletContext();
    RequestDispatcher dispatcher = context.getRequestDispatcher("/WEB-INF/jsp/home.jsp");
    dispatcher.forward(req,resp);

}
}

Juliettajuliette

Answer 4

I solved this problem by parsing the feed with Rome and then parsing it again to get the raw jdom Document. Then I can get the item elements from the feed and look for images. Bit hacky but it easier than extending the RSS parsers and so on.

byte[] data = ... bytes for the feed ...
SyndFeedInput input = new SyndFeedInput()
input.allowDoctypes = true
SyndFeed sf = input.build(new XmlReader(new ByteArrayInputStream(data)))

Document doc = new MyWireFeedInput().getDocument(new XmlReader(new ByteArrayInputStream(data)))
Element channel = doc.rootElement.getChild("channel")
List<Element> items = channel ? channel.getChildren("item") : null

List<SyndEntry> entries = sf.entries
for (int i = 0; i < entries.size(); i++) {
    SyndEntry entry = entries[i]
    Element item = items ? items[i] : null
    if (item) {
        Element image = item.getChild("image")
        ... add it to enclosures or whatever ...
    }
}

Here is the class that gets the jdom Document:

/**
 * This is a hack to get at the protected {@link WireFeedInput#createSAXBuilder()} method so we can get the
 * raw jdom document for the feed to extract elements (e.g. 'image') not parsed by the built in feed parsers.
 */
public class MyWireFeedInput extends WireFeedInput {

    Document getDocument(Reader reader) {
        final SAXBuilder saxBuilder = createSAXBuilder();
        try {
            if (xmlHealerOn) reader = new XmlFixerReader(reader)
            return saxBuilder.build(reader);
        } catch (final JDOMParseException ex) {
            throw new ParsingFeedException("Invalid XML: " + ex.getMessage(), ex);
        } catch (final IllegalArgumentException ex) {
            throw ex;
        } catch (final Exception ex) {
            throw new ParsingFeedException("Invalid XML", ex);
        }
    }
}

Lilt