‘fast-stats’, omega & alpha

I pulled this morning’s stats on the EOD (end-of-day, the omega) and first update for the day. the alpha.

Green Highlight, the EOD elapsed time was 2:19.
EOD processing, of course has the most accumulated donor updates. In addition to the typical hourly updates, several files are transferred on the data server and EOD files are rsynced with the remote web host.

Purple Highlight, the first update of the day, was 0:51
This includes the basic update processes:

  • Donor file download, 2.5 million records
  • Extraction (decompress), Validating the file
  • Verifying the timestamp update
  • Processing the differential update data for 5 million records (two data sets)
  • Extracting team and group data
  • Generate web pages from the extracted data
  • Transfer web pages to host server