Condor Accounting Setup
Tags:
- Obtain a copy of the Condor accounting scripts here.
- Do all of the following steps as the condor user on your Central Manager. This example assumes an install in /scratch.
- Make a directory in /scratch named accounting, then make a dir /scratch/accounting/usage and chmod 775 usage.
- Unpack accounting.tar.gz in the accounting dir.
- Install the Perl Mail::Mailer module (perl – MCPAN -e ‘install Mail::Mailer’)
- Install the Perl DBI module (perl – MCPAN -e ‘install DBI’)
- Install the Perl Config::General module (perl – MCPAN -e ‘install Config::General’)
used http://search.cpan.org/~tlinden/Config-General-2.31/ since newer versions break compatibility with procedural style of use —griswold
- Install the Perl DBD::SQLite module (perl – MCPAN -e ‘install DBD::SQLite’)
- Build DBD-SQLite-1.13 in /scratch/DBD-SQLite-1.13 (find tarball in cfengine packages dir or download from http://search.cpan.org/dist/DBD-SQLite/)
- Create a config file for site-specific settings — mine is called .cndracct — in /home/condor/. Note that you can put the config in another location and refer to it when running the cron jobs as “—config=/scratch/accounting/.cndracct” It should look something like:
pool = nmi-condor.cs.wisc.edu # value of CONDOR_HOST from your condor_config file
accountingdb_name = dbi:SQLite:dbname=/scratch/accounting/accounting
mappingdb_name = dbi:SQLite:dbname=/scratch/accounting/accounting
usagepath = /scratch/accounting/usage
gunzipcmd = /bin/gunzip # path to gunzip on local machine
gzipcmd = /bin/gzip # path to gzip on local machine
condorhome=/usr/local/condor # path to Condor binaries on local machine
condorconfig = /home/condor/condor_config # path to condor_config file on local machine
- Add all of your users to the mapping database. Currently this is a manual step. Run the following command for all users, substituting for PROJECT, USER and YOURDOMAIN.
- /scratch/accounting/addmapping —project=PROJECT —reporting-user=USER@YOURDOMAIN —condor-user=USER@YOURDOMAIN
- Add the following to your crontab:
# Enable Condor usage accounting. Sample every 15 minutes,
# and import this data into the database every half hour.
0,15,30,45 * * * * /scratch/accounting/sample
25,55 * * * * /scratch/accounting/import
# Send daily and weekly emails of the accounting information
0 1 * * * /scratch/accounting/report —daily —email “” —subject “Daily Condor usage #D”
15 1 * * 1 /scratch/accounting/report —weekly —email “” —subject “Weekly Condor usage #D”
Getting “raw”/“real” numbers.
Copy ‘report’ to ‘report.raw’ and change two things: ‘active_users’ to ‘active_condor_users’ and ‘user_activity’ to ‘condor_user_activity’. Then duplicate the crontab entries above, changing 0/15 to 1/16. This will produce a duplicate report which actually includes the CPU used by all Condor jobs, not just those whose users have a “mapping” set up. The output from ‘report.raw’ is less-pretty but more accurate.
»
- Printer-friendly version
- Login or register to post comments
