From owner-nwchem-users@emsl.pnl.gov Tue Aug 31 10:34:35 2004 Received: from odyssey.emsl.pnl.gov (localhost [127.0.0.1]) by odyssey.emsl.pnl.gov (8.12.10/8.12.10) with ESMTP id i7VHYYFd012455 for ; Tue, 31 Aug 2004 10:34:34 -0700 (PDT) Received: (from majordom@localhost) by odyssey.emsl.pnl.gov (8.12.10/8.12.10/Submit) id i7VHYYZt012454 for nwchem-users-outgoing; Tue, 31 Aug 2004 10:34:34 -0700 (PDT) Date: Tue, 31 Aug 2004 12:39:00 -0500 From: Charles Edwin Webster Subject: Re: system wide scratch directory setting Cc: Glenn Johnson , nwchem-users@emsl.pnl.gov Message-id: <4134B7B4.FFC72144@mail.chem.tamu.edu> MIME-version: 1.0 X-Mailer: Mozilla 4.76C-SGI [en] (X11; I; IRIX 6.5 IP32) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7bit X-Accept-Language: en X-Chemistry_Department-MailScanner-Information: Chemistry_Helpdesk X-Chemistry_Department-MailScanner: Found to be clean X-MailScanner-From: webster@mail.chem.tamu.edu References: Sender: owner-nwchem-users@emsl.pnl.gov Precedence: bulk glenn, i see that you're using PBS, so the other way to avoid any directory problems is to place the env var settings in your PBS prologue script. ##--begin-- ## edit for path of commands #!/bin/sh /sbin/mkdir /usr/scratch/$1 /sbin/chmod 700 /usr/scratch/$1 /sbin/chown $2 /usr/scratch/$1 SCRATCH_DIR=/usr/scratch/$1; export SCRATCH_DIR ##--end-- and clean it up with the epilogue: ##--begin-- ## edit for path of commands #!/bin/sh # edit for path of commands /sbin/rm -rf /usr/scratch/$1 ##--end-- i hope that this helps. charles "Jeffrey L. Tilson" wrote: > Hello Glenn, > > I see Edo also answered your question. I wanted to make one point. If, you > permit multiple nwchem jobs to run on the same node and if the scratch > directory is shared, you could potentially have a name clash. I know I'm > not very imaginative with naming. If this is not a problem, then go ahead > and put the following in the queue scripts. > > setenv SCRATCH_DIR /usr/scratch/$USER > > or if you want (but not my preference) put a > > setenv SCRATCH_DIR /usr/scratch > > in the default login environment. > --jeff > > ########################################################################### > Jeffrey L. Tilson, Ph.D. > Computational Scientist > Center for Computational Research > 9 Norton Hall, Buffalo NY 14260-1800 > > (716) 645-6500 (o) > (716) 645-6505 (f) > jtilson@ccr.buffalo.edu > www.ccr.buffalo.edu > > "Thus, the task is, not so much to see what no one has seen, but to > think what nobody has yet thought, about that which everybody sees" > -Schroedinger. > > ########################################################################### > > On Tue, 31 Aug 2004, Glenn Johnson wrote: > > > I realized I was not exactly clear in my question. A little description of my system is probably in order. > > > > I have a cluster set up where the home directory is NFS exported from the master node to all of the computational nodes. I have a queuing system that dispatches jobs to the nodes. Each node also has a local disk that I wish to use for scratch space. I do not want the current working directory to be the scratch directory because of the load it would put on the NFS server. I also would like to avoid having the users remember to properly set up the scratch_dir in their input files. Each of the computational nodes has a scratch directory called "/usr/scratch" that is local to the respective machine. > > > > What I would like to do is set a default scratch directory to: > > "/usr/scratch" or "/usr/scratch/$USER" > > > > Thus, the user would not have to set scratch_dir in the input and the scratch directory on the disk local to the node assigned to run the job by the queuing system would be used. I hope that makes sense. I could set this up by writing a wrapper script that uses the $TMPDIR defined by the queuing system but I thought it might be possible to set it up in NWChem directly because I saw the SCRATCH_DEF_DIR variable in config/makefile.h. > > > > Thanks. > > > > -- > > Glenn Johnson > > > > >>> "Jeffrey L. Tilson" 08/31/04 10:55AM >>> > > > > My machines support many users, so I think it best for each to have their > > own scratch directory. I also use a batch system on my machines, so we > > usually use the following at runtime > > > > setenv SCRATCH_DIR $PBS_O_WORKDIR > > (tcsh of course) > > > > --jeff > > ########################################################################### > > Jeffrey L. Tilson, Ph.D. > > Computational Scientist > > Center for Computational Research > > 9 Norton Hall, Buffalo NY 14260-1800 > > > > (716) 645-6500 (o) > > (716) 645-6505 (f) > > jtilson@ccr.buffalo.edu > > www.ccr.buffalo.edu > > > > "Thus, the task is, not so much to see what no one has seen, but to > > think what nobody has yet thought, about that which everybody sees" > > -Schroedinger. > > > > ########################################################################### > > > > > > > > On Tue, 31 Aug 2004, Glenn Johnson wrote: > > > > > Is there a way to set a default system wide scratch directory for NWChem-4.6? I thought that setting SCRATCH_DEF_DIR would do it but it does not seem to be used anywhere in the code, only set in makefile.h. > > > > > > Thanks. > > > > > > -- > > > Glenn Johnson > >