logo
Free, unlimited AI code reviews that run on commit
git-lrc git-lrc GitHub Install Now We'd appreciate a star git-lrc - Free, unlimited AI code reviews that run on commit | Product Hunt git-lrc - Free, unlimited AI code reviews that run on commit | Product Hunt

MediaWiki::DumpFile::FastPages - Fastest way to parse a page dump file

About

       This is a subclass of MediaWiki::DumpFile::Pages that configures it to run in fast mode and uses a custom
       iterator that dispenses with the duck-typed MediaWiki::DumpFile::Pages::Page object that fast mode uses
       giving a slight processing speed boost.

       See the MediaWiki::DumpFile::Pages documentation for information about fast mode.

Author

       Tyler Riddle, "<triddle at gmail.com>"

Bugs

       Please see MediaWiki::DumpFile for information on how to report bugs in this software.

History

       This package originally started life as a very limited hack using only XML::LibXML::Reader and seeking to
       text and title nodes in the document.  Implementing a parser for the full document was a daunting task
       and this package sat in the hopes that other people might find it useful.

       Because XML::TreePuller can expose the underlying XML::LibXML::Reader object and sync itself back up
       after the cursor was moved out from underneath it, I was able to integrate the logic from this package
       into the main ::Pages parser.

Methods

       All of the methods of MediaWiki::DumpFile::Pages are also available on this subclass.

   new
       This is the constructor for this package. It is called with a single parameter: the location of a
       MediaWiki pages dump file or a reference to an already open file handle.

   next
       Returns a two element list where the first element is the article title and the second element is the
       article text. Returns an empty list when there are no more pages available.

Name

       MediaWiki::DumpFile::FastPages - Fastest way to parse a page dump file

Synopsis

         use MediaWiki::DumpFile::FastPages;

         $pages = MediaWiki::DumpFile::FastPages->new($file);
         $pages = MediaWiki::DumpFile::FastPages->new(\*FH);

         while(($title, $text) = $pages->next) {
           print "Title: $title\n";
           print "Text: $text\n";
         }

See Also