From owner-nwchem-users@emsl.pnl.gov Tue Mar 13 16:06:13 2007 Received: from odyssey.emsl.pnl.gov (localhost [127.0.0.1]) by odyssey.emsl.pnl.gov (8.13.8/8.13.8) with ESMTP id l2DN6CJZ009612 for ; Tue, 13 Mar 2007 16:06:13 -0700 (PDT) Received: (from majordom@localhost) by odyssey.emsl.pnl.gov (8.13.8/8.13.8/Submit) id l2DN6CVQ009611 for nwchem-users-outgoing-0915; Tue, 13 Mar 2007 16:06:12 -0700 (PDT) X-Authentication-Warning: odyssey.emsl.pnl.gov: majordom set sender to owner-nwchem-users@emsl.pnl.gov using -f X-Possible-Spoof: True X-IronPort-AV: i="4.14,281,1170662400"; d="scan'208"; a="15304841:sNHT60128831" X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Subject: RE: [NWCHEM] TCE and memory Date: Tue, 13 Mar 2007 16:06:09 -0700 Message-ID: In-Reply-To: <20070313223819.0315E11B724@ws4-1.us4.outblaze.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [NWCHEM] TCE and memory Thread-Index: AcdlwotiEMVdPrGQTNWWJ4v+Qc07tAAAGsjA References: <20070313223819.0315E11B724@ws4-1.us4.outblaze.com> From: "Kowalski, Karol" To: "Cheri McFerrin" , Cc: , X-OriginalArrivalTime: 13 Mar 2007 23:06:10.0478 (UTC) FILETIME=[307C0CE0:01C765C4] Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by odyssey.emsl.pnl.gov id l2DN6BRK009608 Sender: owner-nwchem-users@emsl.pnl.gov Precedence: bulk Hi Cheri, From what you have sent to me it follows that you are experiencing problems with 4-index transformation. You have two options: (1) either to use the replicated files: tce io replicated qcisd scf end (2) or, only if you are using the ROHF reference, to store 2-electron integrals in orbital form: scf rohf doublet end tce io ga qcisd scf end Best, Karol -----Original Message----- From: owner-nwchem-developers@emsl.pnl.gov [mailto:owner-nwchem-developers@emsl.pnl.gov] On Behalf Of Cheri McFerrin Sent: Tuesday, March 13, 2007 3:38 PM To: nwchem-users@emsl.pnl.gov Cc: rhall@lsu.edu; honggao@lsu.edu Subject: [NWCHEM] TCE and memory NWChem Users, I am trying to use the TCE (tensor contraction module) to optimize a structure with QCISD and the aug-cc-pVDZ basis. When a memory statement is NOT included in the input file, I receive an error message stating the job needs 5000 mb global memory. When I include a memory statement (memory global 5000 mb) in the inp file, I receive an error message stating the job needs 911 bytes of local memory. The output also states only ~1500 mb of memory was obtained. In other words, I am only accessing the memory in the cpu. I need more. The computer I am using is casper at LSU. I am using the Power5 Parallel Execution nodes there. http://appl003.lsu.edu/ocsweb/hpchome.nsf/$Content/hardware+listing?Open Document Each Power 5 node on casper has 8 cpus. The available memory per cpu is approximately 1600 mb. The total available memory per node then becomes 12800 mb, sufficient for my job. My jobs are dying at input line 38 or at "task tce optimize". My script file (qcisd_phenoxyl.cmd), included here, requests 1 node and 1 cpu. How can a create an input file which will allow me to use 1 cpu and access more memory than 1600 mb? My input (qcisd_phenoxyl.nw), my script (qcisd_phenoxyl.cmd) and one of my failed outputs (qcisd_phenoxyl.output) are attached here. thank you sooo much for your help!!! cheri mcferrin *****qcisd_phenoxyl.cmd*********************************** #! /bin/ksh #@ job_type = parallel #@ input = /dev/null #@ output = /home/c5/cmcfer1/NWCHEM/qcisd_phenoxyl.output #@ error = /home/c5/cmcfer1/NWCHEM/qcisd_phenoxyl.err #@ initialdir = /home/c5/cmcfer1/NWCHEM #@ notify_user = cmcfer1@lsu.edu #@ class = MP5L #@ node = 1 #@ total_tasks =8 #@ notification = error #@ requirements = (Arch == "Power5") #@ queue /usr/local/nwchem/bin/nwchem qcisd_phenoxyl.nw ******qcisd_phenoxyl.nw********************************* echo scratch_dir /work/active/cmcfer1 permanent_dir /home/c5/cmcfer1/NWCHEM start qcisd_phenoxyl memory ?????? geometry units angstroms C 0.000000 0.000000 0.000000 C 0.000000 0.000000 1.439791 C 1.276571 0.000000 2.105628 C 2.440785 0.000000 1.383802 C 2.402413 0.000000 -0.016761 C 1.178382 0.000000 -0.698487 O -1.067258 0.000000 2.086821 H -0.957978 0.000000 -0.491708 H 1.170862 0.000000 -1.776070 H 3.323720 0.000000 -0.575277 H 3.392756 0.000000 1.888773 H 1.269608 0.000000 3.182404 end basis C library aug-cc-pVDZ H library aug-cc-pVDZ O library aug-cc-pVDZ end scf doublet end tce scf qcisd end task tce optimize *****qcisd_phenoxyl.output*********************** Integral file = /work/active/cmcfer1/qcisd_phenoxyl.aoints.0 Record size in doubles = 65536 No. of integs per rec = 43688 Max. records in memory = 2959 Max. records in file = 119047 No. of bits per label = 8 No. of bits per value = 64 int2e_file: Failed to allocate cache ... halving #quartets = 7.504D+06 #integrals = 1.495D+08 #direct = 0.0% #cached =100.0% Fock matrix recomputed 1-e file size = 28614 1-e file name = /work/active/cmcfer1 Cpu & wall time / sec 76.9 79.2 2-e (intermediate) file size = 4788841200 2-e (intermediate) file name = /work/active/cmcfer1 available GA memory 791133988 bytes ------------------------------------------------------------------------ createfile: failed ga_create size=********* ------------------------------------------------------------------------ ------------------------------------------------------------------------ current input line : 38: task tce optimize ------------------------------------------------------------------------ ------------------------------------------------------------------------ You need to assign at least 4788841200 bytes more global memory ------------------------------------------------------------------------ For more information see the NWChem manual at http://www.emsl.pnl.gov/docs/nwchem/nwchem.html For further details see manual section: NWChem -> Top Level Directives -> Memory 0:0:createfile: failed ga_create size=:: 493873904 0: ARMCI aborting 493873904 (0x1d6feaf0).