LWP::UserAgent::Determined - a virtual browser that retries errors

       Originally created by Sean M. Burke, "sburke@cpan.org"

       Currently maintained by Jesse Vincent "jesse@fsck.com"

perl v5.36.0                                       2022-10-13                    LWP::UserAgent::Determined(3pm)

       Copyright  2004, Sean M. Burke, all rights reserved.  This program is free software; you can redistribute
       it and/or modify it under the same terms as Perl itself.

       This program is distributed in the hope that it will be useful, but without any  warranty;  without  even
       the implied warranty of merchantability or fitness for a particular purpose.

       This class works just like LWP::UserAgent (and is based on it, by being a subclass of it), except that
       when you use it to get a web page but run into a possibly-temporary error (like a DNS lookup timeout),
       it'll wait a few seconds and retry a few times.

       It also adds some methods for controlling exactly what errors are considered retry-worthy and how many
       times to wait and for how many seconds, but normally you needn't bother about these, as the default
       settings are relatively sane.

       This class works by overriding LWP::UserAgent's "simple_request" method with its own  around-method  that
       just loops.  See the source of this module; it's straightforward.  Relatively.

       This module inherits all of LWP::UserAgent's methods, and adds the following.

       $timing_string = $browser->timing();
       $browser->timing( "10,30,90" )
           The  "timing"  method  gets  or sets the string that controls how many times it should retry, and how
           long the pauses should be.

           If you specify empty-string, this means not to retry at all.

           If you specify a string consisting of a single number, like  "10",  that  means  that  if  the  first
           request  doesn't  succeed,  then  "$browser->get(...)"  (or  any  other  method based on "request" or
           "simple_request") should wait 10 seconds and try again (and if that fails, then it's final).

           If you specify a string with several numbers in it (like "10,30,90"), then that  means  $browser  can
           retry  as  that many times (i.e., one initial try, plus a maximum of the three retries, because three
           numbers  there),  and  that  it  should  wait  first  those  numbers  of  seconds  each   time.    So
           "$browser->timing( "10,30,90" )" basically means:

             try the request; return it unless it's a temporary-looking error;
             sleep 10;
             retry the request; return it unless it's a temporary-looking error;
             sleep 30;
             retry the request; return it unless it's a temporary-looking error;
             sleep 90  the request;
             return it;

           The default value is "1,3,15".

       $http_codes_hr = $browser->codes_to_determinate();
           This  returns the hash that is the set of HTTP codes that merit a retry (like 500 and 408, but unlike
           404 or 200).  You can delete or add entries like so;

             $http_codes_hr = $browser->codes_to_determinate();
             delete $http_codes_hr->{408};
             $http_codes_hr->{567} = 1;

           (You can actually set a whole new hashset with "$browser->codes_to_determinate($new_hr)", but there's
           usually no benefit to that as opposed to the above.)

           The current default is 408 (Timeout) plus some 5xx codes.

       $browser->before_determined_callback()
       $browser->before_determined_callback( \&some_routine );
       $browser->after_determined_callback()
       $browser->after_determined_callback( \&some_routine );
           These read (first two) or set (second two) callbacks that are called before the  actual  HTTP/FTP/etc
           request is made.  By default, these are set to undef, meaning nothing special is called.  If you want
           to  alter  try requests, or inspect responses before any retrying is considered, you can set up these
           callbacks.

           The arguments passed to these routines are:

           0: the current $browser object
           1: an arrayref to the list of timing pauses (based on $browser->timing)
           2: the duration of the number of seconds we'll pause if this request fails this time, or undef if
           this is the last chance.
           3: the value of $browser->codes_to_determinate
           4: an arrayref of the arguments we pass to LWP::UserAgent::simple_request (the first of which is the
           request object)
           (5): And, only for after_determined_callback, the response we just got.

           Example use:

             $browser->before_determined_callback( sub {
               print "Trying ", $_[4][0]->uri, " ...\n";
             });

       LWP::UserAgent::Determined - a virtual browser that retries errors

       LWP, LWP::UserAgent

         use strict;
         use LWP::UserAgent::Determined;
         my $browser = LWP::UserAgent::Determined->new;
         my $response = $browser->get($url, headers... );