From owner-nwchem-users@emsl.pnl.gov Thu Jul 17 15:32:58 2008 Received: from odyssey.emsl.pnl.gov (localhost.pnl.gov [127.0.0.1]) by odyssey.emsl.pnl.gov (8.14.1/8.14.1) with ESMTP id m6HMWvMp016326 for ; Thu, 17 Jul 2008 15:32:58 -0700 (PDT) Received: (from majordom@localhost) by odyssey.emsl.pnl.gov (8.14.1/8.14.1/Submit) id m6HMWvqj016325 for nwchem-users-outgoing-0915; Thu, 17 Jul 2008 15:32:57 -0700 (PDT) X-Authentication-Warning: odyssey.emsl.pnl.gov: majordom set sender to owner-nwchem-users@emsl.pnl.gov using -f X-Ironport-SG: OK_Domains X-Ironport-SBRS: 2.4 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Aq4AAMdkf0hIDtyYfGdsb2JhbACRekMBAQsDBAYHEQOXQ4ZJ X-IronPort-AV: E=Sophos;i="4.31,205,1215414000"; d="scan'208";a="55501005" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=YRwZqvxpk0e8kuyIrisHZHE61duOX7buRDxs/qYg+ZY=; b=TCX32BYl7QR02Prw9ysDQFRvCZNT8ZnUD9EJGuzReiCax9jbvtODEJg8WEOQO6pZT1 3bVa2CQQowgFLGtZb5u2JwVYDBtWrRaBDuGDWyMxqVw1y6tVTGTiPGPFrAV6I6Op2MeC VZ8jMLiT9SMfjtP8TMqZ+yFXnx2em4J76mIxA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=iBD+qImYA9SMeKxx6hRiicv46e7GRETFYds7j0Km2t9ADH2d1vpA90NbVGyn4Iowo+ WdTIYdOlpm0tCnRLtE7c7mM1H5kx+4kVKdTK5Ip42U/yUDxVbpLTkEPuKcpE0IaNmnUD DZC1sFtP05mgAXqRhtw20tUiY8cBFxVw8tkkM= Message-ID: <96f4bb620807171532j15f8ac81t31a54ae58985d7d2@mail.gmail.com> Date: Thu, 17 Jul 2008 17:32:53 -0500 From: "Jeff Hammond" To: "Jeff Nucciarone" Subject: Re: [NWCHEM] new to NWChem, disk file size for dft calculation Cc: nwchem-users@emsl.pnl.gov In-Reply-To: <487FB8BC.6050000@pnl.gov> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <487FAF01.4020202@psu.edu> <487FB8BC.6050000@pnl.gov> Sender: owner-nwchem-users@emsl.pnl.gov Precedence: bulk If you want to store integrals in-core and make use of your large memory, try adding this to your input file: scf semidirect memsize 2000000000 filesize 0 end which will store 2 gwords (16 gbytes) of integrals in memory and none on disk. You can try more by increasing that number. The job does not crash your job if the integral cache cannot be allocated in the memory available to NWChem, so there is no harm in setting this number to a large value, as long as it does not roll over the maximum integer size on your machine (shouldn't be an issue for a 64-bit machine). This will work in a DFT calculation as well an SCF calculation. I tested to confirm that this is true. I tried setting "memory stack 100 mb heap 100 mb global 100 mb" and "semidirect memsize 50000000 filesize 0" and it ran just fine. The "memsize 50000000" is 400 mb, by the way. Changing all three "100 mb" to "500 mb" eliminated the "integral cache cannot be allocated" warnings. If you want to nearly max out your 128 gb, you can try "memory stack 60 gb heap 60 gb global 2 gb" (if running with only one processor). You probably can't set more than 2 gb of global memory because the default Linux settings don't allow it, but it doesn't matter since the integral cache is not allocated from global memory as best I can tell from running tests on the memory options. Both sets of options semidirect options I gave above are faster than "direct" by a factor of ~5 for H2O with cc-pVQZ. You might want to recompile with LARGE_FILES=T since you're clearing using a 64-bit machine, not that it matters much for in-core calculations anyways. Best, Jeff On Thu, Jul 17, 2008 at 4:25 PM, Dunyou Wang wrote: > Jeff, > > For that purpose, you can add the 'direct' directive in your input. You can > also find the detail information about this in the manual. > > Hope this helps > Dunyou > > Jeff Nucciarone wrote: >> >> Hello. >> >> I am new to using NWChem and have a question regarding disk usage for DFT. >> >> Whether I run in serial or parallel, I seem to get a set of large aoints >> files: >> >>> -rw-r--r-- 1 nucci nucci 2.0G Jul 17 16:20 cell_tyr.aoints.0 >>> -rw-r--r-- 1 nucci nucci 2.0G Jul 17 16:21 cell_tyr.aoints.0x001 >>> -rw-r--r-- 1 nucci nucci 2.0G Jul 17 16:22 cell_tyr.aoints.0x002 >>> -rw-r--r-- 1 nucci nucci 2.0G Jul 17 16:23 cell_tyr.aoints.0x003 >>> -rw-r--r-- 1 nucci nucci 2.0G Jul 17 16:24 cell_tyr.aoints.0x004 >>> -rw-r--r-- 1 nucci nucci 2.0G Jul 17 16:25 cell_tyr.aoints.0x005 >>> -rw-r--r-- 1 nucci nucci 2.0G Jul 17 16:26 cell_tyr.aoints.0x006 >>> -rw-r--r-- 1 nucci nucci 2.0G Jul 17 16:27 cell_tyr.aoints.0x007 >>> -rw-r--r-- 1 nucci nucci 71M Jul 17 16:27 cell_tyr.aoints.0x008 >> >> if I am running on a machine with slow disk this can greatly impact my >> performance. Is there a way to have all this computed in core rather than >> on disk? What option or combination of options would I use? I tried >> experimenting 'memory' but that seems to have no effect on the size or >> creation of these large files. I have fat machines with up to 128 GB of >> memory so I can cram a lot into core. >> >> My input file is (I was playing with the mem setting so in this case it is >> small): >> >> # DFT energy >> # test for cell_tyr >> # July 8, 2008 >> >> start cell_tyr >> >> title "cell_tyr single point B3LYP/6-31G**" >> >> scratch_dir /tmp >> memory 1024 mb >> echo >> >> geometry units angstrom >> C 0.000000 0.000000 0.000000 >> H 0.000000 0.000000 1.100449 >> H 1.037093 0.000000 -0.367374 >> H -0.370439 -1.001656 -0.261605 >> C -1.502439 0.610756 -1.813506 >> O -0.845968 1.001797 -0.598408 >> C -2.902722 1.214058 -1.885818 >> O -3.743114 0.599033 -0.903898 >> C -3.512606 1.010866 -3.269015 >> O -4.714812 1.765263 -3.359461 >> C -2.539695 1.433394 -4.369667 >> C -1.179544 0.796033 -4.167840 >> O -0.696635 1.128265 -2.864818 >> C -0.182994 1.252014 -5.235461 >> O 0.280287 2.577439 -5.003614 >> H -1.573775 -0.484124 -1.878744 >> H -2.829243 2.293904 -1.690118 >> H -3.545665 -0.481839 -0.876660 >> H -3.734699 -0.058419 -3.396241 >> H -5.568944 1.079602 -3.467854 >> H -2.427986 2.526618 -4.339577 >> H -1.293499 -0.295025 -4.248923 >> H -0.655830 1.241845 -6.228267 >> H 0.698807 0.595683 -5.241518 >> H -0.502569 3.294711 -5.292721 >> O -3.027536 1.025403 -5.660275 >> H -2.405374 1.297967 -6.338199 >> N -0.003480 -5.757259 -1.411673 >> H 0.746663 -5.003095 -1.135180 >> H -0.470181 -6.161777 -0.501107 >> C -0.980212 -5.284349 -2.422955 >> C -1.046533 -6.229278 -3.613617 >> O -0.030399 -6.608133 -4.162592 >> C -0.504094 -3.923642 -2.942575 >> C -1.481718 -3.401574 -3.993500 >> C -2.818263 -3.194023 -3.705312 >> C -1.033898 -3.127153 -5.245395 >> C -3.668525 -2.724957 -4.684322 >> C -1.874191 -2.678038 -6.204218 >> C -3.199346 -2.466892 -5.942324 >> O -4.044251 -1.992971 -6.925423 >> H -1.977388 -5.179878 -1.970949 >> H 0.491484 -4.033308 -3.395260 >> H -0.451004 -3.210259 -2.107976 >> H -3.202699 -3.405064 -2.696905 >> H 0.031544 -3.275640 -5.472891 >> H -4.731757 -2.556205 -4.453592 >> H -1.496388 -2.476601 -7.217800 >> H -3.670518 -1.026377 -7.295129 >> H 0.678218 -6.592256 -1.629887 >> O -2.145867 -6.568657 -3.956188 >> end >> >> basis >> * library 6-31G** >> end >> dft >> print low >> xc b3lyp >> incore >> end >> task dft >> >> Thanks, >> >> --Jeff >> > > -- Jeff Hammond The University of Chicago http://home.uchicago.edu/~jhammond/