Home

Parsing RSS with Perl

Wed, 08/22/2007 - 00:45 — onaje

This is based on a script provided in the 'Add RSS feeds to your Web site with Perl XML::RSS' from
http://articles.techrepublic.com.com/5100-6228_11-5487340.html

In the original script, it was assumed that the rss news feed would be located on your server. To get
around this limitation, use LWP to get the contents of a remote file, save it to a file on your server
then parse the file.

#!/usr/bin/perl -w
#use strict;

use XML::RSS;
use LWP::Simple;
#use Data::Dumper;

my $r = new XML::RSS;

$r->parse( get 'http://onaje.com/rss.xml' );
http://newsrss.bbc.co.uk/rss/newsonline_world_edition/business/rss.xml

# finally, save the rss file.
$r->save('source1.xml');

#print Dumper( $r );

# print HTML header and page
print "Content-Type: text/html\n\n";
print "<html><head><basefont face=Arial size=8></head><body>";
print "<table width=100% border=1 cellspacing=0 cellpadding=5>";
print "<tr>";

# set data sources
@sources = ('source1.xml');

# iterate over source list
# process each source
foreach $s (@sources)
{
        # parse RSS feed
        $r->parsefile($s);

        # print channel name
        print "<td valign=top>";
        print $r->channel('title') . "<p />";
        print "<ul>";
        # print titles and URLs of news items
        foreach $item (@{$r->{'items'}})
        {
                $title = $item->{'title'};
                $url = $item->{'link'};
                print "<li><a href=\"$url\">$title</a><br \>";
        }
        print "</ul>";
        print "</td>";
}

# print footers
print "</tr>";
print "</table>";
print "</body></html>";

# and quit.
exit;

This script can be improved by adding things such as a 'conditional get for the remote file'.
A conditional get first asks whether a file has changed or not, and only download it if it has.
“If this document has changed since I last looked at it, give me the new version. If it hasn't just
tell me it hasn't changed and give me nothing.” This mechanism is called “Conditional GET.”

See Conditional GET with LWP & Perl
See HTTP Conditional Get for RSS Hackers

      Subscribe in a reader