Description of algorithms used in 2008-4-MayLoad.lisp "Live data" refers to data residing within the Lisp environment Live data can be either a first-class citizen stored in a standard place, or a set of uniform properties attached to labels which are uninterned symbols. Each control point is a symbol with one or both of these properties: - :DATE = The timestamp (Universal Time) for which this data is correct - :DATA = The actual data (only in the case where it's a first-class citizen) The initial condition of each control point is with neither property set, whereupon :DATE defaults to 0 (which is older than any actual timestamp). The timestamp of any disk file is the date last written. Requesting a particular item of live data to be "current" traces backwards recursively through the dataflow to make sure all inputs are "current", per the following algorithm at each control point: * Case: live data which is loaded directly from raw input file: The live data's timestamp must be at least as recent as the file's timestamp. Otherwise the file is loaded into memory, the resultant value is stored on the :DATA property, and the :DATE property is set to exactly equal the file's timestamp. * Case: live data which is quickly/cheaply computed from precursor data: The live data's timestamp must be at least as recent as the latest of the timestamps on all precursor data. Otherwise the live data is computed from the percursor data, the result is stored on the :DATA property, and the :DATE property is set to exactly equal the most recent of the precursor timestamps. * Case: live data which requires time-consuming computation from earlier data: A disk file is used to back up such data to avoid unnecessary recomputation. The live data's timestamp must be at least as recent as the latest of the timestamps on all precursor data and also the backup file. Otherwise the live data is computed from the percursor data or loaded from the backup file, whichever has the most recent timestamp, then for data that contains reference to uninterned symbols all such symbols are converted to standard symbols known interally, then the result is stored on the :DATA property, and the :DATE property is set to be exactly the most recent of the precursor timestamps. After that is checked/updated, next the backup file's timestamp must be at least as recent as the live data (which is provably true if the previous step has loaded the data from the file). Otherwise (if the file is out of date), the live data is written to the file, using the roll-files utility to assure failsafe operation if the program or system crashes in the middle of a write operation, then the timestamp on the live data is changed to exactly equal the timestamp of that newly written file, thus provably assuring neither re-load nor re-save will be needed again until and unless some precursor value later changes.