Boulder::Store - Simple persistent storage for Stone tag/value objects

Author

       Lincoln D. Stein <lstein@cshl.org>, Cold Spring Harbor Laboratory, Cold Spring Harbor, NY.   This  module
       can be used and distributed on the same terms as Perl itself.

Caveats

       Boulder::Store makes heavy use of the flock() call in order to avoid corruption of DB_File databases when
       multiple  processes  try  to  write  simultaneously.   flock()  may not work correctly across NFS mounts,
       particularly on Linux machines that are not running the  rpc.lockd  daemon.   Please  confirm  that  your
       flock()  works  across  NFS  before  attempting  to use Boulder::Store.  If the store.t test hangs during
       testing, this is the likely culprit.

Class Methods

       $store = Boulder::Store->new("database/path",$writable)
           The  new()  method  creates  a  new  Boulder::Store  object  and associates it with the database file
           provided in the first parameter (undef is a valid pathname, in which case all methods  work  but  the
           data isn't stored).  The second parameter should be a true value if you want to open the database for
           writing.  Otherwise it's opened read only.

           Because  the  underlying  storage  implementation  is  not  multi-user, only one process can have the
           database for writing at a time.  A fcntl()-based locking mechanism is used to give a process that has
           the database opened for writing exclusive access to the database.  This also  prevents  the  database
           from  being  opened  for  reading  while  another  process  is  writing to it (this is a good thing).
           Multiple simultaneous processes can open the database read only.

           Physically the data is stored in a human-readable file with the extension ".data".

Description

       Boulder::Store provides persistent storage for Boulder objects using a simple DB_File implementation.  To
       use it, you need to have Berkeley db installed (also known as libdb), and the Perl DB_File module.  See
       the DB_File package for more details on obtaining Berkeley db if you do not already have it.

       Boulder::Store provides an unsophisticated query mechanism which takes advantage of indexes that you
       specify.  Despite its lack of sophistication, the query system is often very helpful.

Name

       Boulder::Store - Simple persistent storage for Stone tag/value objects

Object Methods

       $stone = $store->read_record(@taglist)
           The semantics of this call are exactly the same  as  in  Boulder::Stream.   Stones  are  returned  in
           sequential  order,  starting  with  the first record.  In addition to their built-in tags, each stone
           returned from this call has an additional tag called "record_no".   This  is  the  zero-based  record
           number of the stone in the database.  Use the reset() method to begin iterating from the beginning of
           the database.

           If  called  in  an  array  context,  read_record()  returns a list of all stones in the database that
           contains one or more of the provided tags.

       $stone = $store->write_record($stone [,$index])
           This has the same semantics as Boulder::Stream.  A stone is appended to the end of the database.   If
           successful,  this  call  returns the record number of the new entry.  By providing an optional second
           parameter, you can control where the stone is entered.  A positive numeric index will write the stone
           into the database at that position.  A value of -1 will use the Stone's internal  record  number  (if
           present) to determine where to place it.

       $stone = $store->get($record_no)
           This  is  random access to the database.  Provide a record number and this call will return the stone
           stored at that position.

       $record_number = $store->put($stone,$record_no)
           This is a random write to the database.  Provide a record number and this call stores  the  stone  at
           the indicated position, replacing whatever was there before.

           If  no  record  number  is provided, this call will look for the presence of a 'record_no' tag in the
           stone itself and put it back in that position.  This allows you to pull a stone out of the  database,
           modify  it,  and then put it back in without worrying about its record number.  If no record is found
           in the stone, then the effect is identical to write_record().

           The record number of the inserted stone is returned from this call, or -1 if an error occurred.

       $store->delete($stone),Boulder::Store::delete($record_no)
           These method calls delete a stone from the database.  You can provide either the record number  or  a
           stone  containing  the  'record_no'  tag.  Warning: if the database is heavily indexed deletes can be
           time-consuming as it requires the index to be brought back into synch.

       $record_count = $store->length()
           This returns the length of the database, in records.

       $store->reset()
           This resets the database, nullifying any queries  in  effect,  and  causing  read_record()  to  begin
           fetching stones from the first record.

       $store->query(%query_array)
           This  creates  a  query  on the database used for selecting stones in read_record().  The query is an
           associative array.  Three types of keys/value pairs are allowed:

           (1) $index=>$value
               This instructs Boulder::Store to look for stones containing the specified tags in which the tag's
               value (determined by the Stone index() method) exactly matches the provided value.  Example:

                       $db->query('STS.left_primer.length'=>30);

               Only the non-bracketed forms of the index string are allowed (this is probably a bug...)

               If the tag path was declared  to  be  an  index,  then  this  search  will  be  fast.   Otherwise
               Boulder::Store must iterate over every record in the database.

           (2) EVAL=>'expression'
               This  instructs  Boulder::Store  to look for stones in which the provided expression evaluates to
               true.  When the expression is evaluated, the variable $s will be  set  to  the  current  record's
               stone.  As a shortcut, you can use "<index.string>" as shorthand for "$s->index('index.string')".

           (3) EVAL=>['expression1','expression2','expression3'...]
               This   lets   you   provide   a  whole  bunch  of  expressions,  and  is  exactly  equivalent  to
               EVAL=>'(expression1) && (expression2) && (expression3)'.

           You can mix query types in the parameter provided to query().  For example, here's how to look up all
           stones in which the sex is male and the age is greater than 30:

                   $db->query('sex'=>'M',EVAL=>'<age> > 30');

           When a query is in effect, read_record() returns only Stones that satisfy the  query.   In  an  array
           context,  read_record()  returns  a  list  of  all  Stones  that  satisfy  the  query.   When no more
           satisfactory Stones are found, read_record() returns undef until a new query is entered or reset() is
           called.

       $store->add_index(@indices)
           Declare one or more tag paths to be a part of a fast index.  read_record()  will  take  advantage  of
           this record when processing queries.  For example:

                   $db->add_index('age','sex','person.pets');

           You can add indexes any time you like, when the database is first created or later.  There is a trade
           off:   write_record(),  put(),  and other data-modifying calls will become slower as more indexes are
           added.

           The index is stored in an external file with the extension ".index".  An index file is  created  even
           if you haven't indexed any tags.

       $store->reindex_all()
           Call this if the index gets screwed up (or lost).  It rebuilds it from scratch.

Synopsis

        Boulder:Store;

        my $store=new Boulder::Store('test.db',1);
        my $s = new Stone (Name=>'george',
                           Age=>23,
                           Sex=>M,
                           Address=>{
                                  Street=>'29 Rockland drive',
                                  Town=>'Fort Washington',
                                  ZIP=>'77777'
                                  }
                              );
        $store->put($s);
        $store->put(new Stone(Name=>'fred',
                              Age=>30,
                              Sex=>M,
                              Address=>{
                                          Street=>'19 Gravel Path',
                                          Town=>'Bedrock',
                                          ZIP=>'12345'},
                              Phone=>{
                                        Day=>'111-1111',
                                        Eve=>'222-2222'
                                        }
                                    ));
        $store->put(new Stone(Name=>'andrew',
                              Age=>18,
                              Sex=>M));

        $store->add_index('Name');

        my $stone = $store->get(0);
        print "name = ",$stone->Name;