From owner-nwchem-users@emsl.pnl.gov Wed Oct 1 10:19:25 2008 Received: from odyssey.emsl.pnl.gov (localhost.pnl.gov [127.0.0.1]) by odyssey.emsl.pnl.gov (8.14.1/8.14.1) with ESMTP id m91HJOoa028187 for ; Wed, 1 Oct 2008 10:19:25 -0700 (PDT) Received: (from majordom@localhost) by odyssey.emsl.pnl.gov (8.14.1/8.14.1/Submit) id m91HJOff028186 for nwchem-users-outgoing-0915; Wed, 1 Oct 2008 10:19:24 -0700 (PDT) X-Authentication-Warning: odyssey.emsl.pnl.gov: majordom set sender to owner-nwchem-users@emsl.pnl.gov using -f X-IronPort-AV: E=Sophos;i="4.33,344,1220252400"; d="scan'208";a="84035516" Message-ID: <48E3B119.5020803@pnl.gov> Date: Wed, 01 Oct 2008 10:19:21 -0700 From: Dunyou Wang User-Agent: Thunderbird 2.0.0.17 (X11/20080914) MIME-Version: 1.0 To: Yinhe Cheng CC: nwchem-users@emsl.pnl.gov, owner-nwchem-users@emsl.pnl.gov Subject: Re: [NWCHEM] nwchem 5.1 on AIX References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 01 Oct 2008 17:19:20.0972 (UTC) FILETIME=[D7AF6CC0:01C923E9] Sender: owner-nwchem-users@emsl.pnl.gov Precedence: bulk Yinhe, On the IBM power 5 AIX 5.3 with LAPI64 target, we have different setting than yours: LAPI_USE_SHM=yes with MP_SHARED_MEMORY=yes MP_EUIDEVICE=sn_all MP_EUILIB=us MP_ADAPTER_USE=dedicated MP_RMPOOL=1 MP_USE_BULK_XFER=yes no MALLOCTYPE specified. It works fine for us with such setting. Best wishes Dunyou Yinhe Cheng wrote: > > Thanks for your help. After following your instructions and "export > XLF11=y" nwchem seems to build ok. > > > However when I tried to run it, got the following error > rtdb_seq_put: put failed for "" in ./cluster8e.db > ------------------------------------------------------------------------ > util_set_rtdb_state: rtdb_put failed 911 > ------------------------------------------------------------------------ > ------------------------------------------------------------------------ > current input line : > 0: task mp2 gradient > ------------------------------------------------------------------------ > ------------------------------------------------------------------------ > An error occured in the Runtime Database > ------------------------------------------------------------------------ > For more information see the NWChem manual at > http://www.emsl.pnl.gov/docs/nwch > em/nwchem.html > > For further details see manual section: > > > > util_set_rtdb_state: rtdb_put failed 911 > ------------------------------------------------------------------------ > ------------------------------------------------------------------------ > current input line : > 0: > ------------------------------------------------------------------------ > ------------------------------------------------------------------------ > An error occured in the Runtime Database > ------------------------------------------------------------------------ > For more information see the NWChem manual at > http://www.emsl.pnl.gov/docs/nwch > em/nwchem.html > .... .... > > The only thing in cluster8e.db is "hdbm v1.0" > > > Following is my ENV setup for the run: > # LAPI setup for SMP > > export LAPI_USE_SHM=only > export MP_EUILIB=ip > > # PE setup: affinity does not help this application > > export MP_MSG_API=MPI_LAPI > export MP_PROCS=4 > export MP_SHARED_MEMORY=yes > > # AIX setup: affinity does not help this application > > export RT_GRQ=ON > export AIXTHREAD_SCOPE=S > export MALLOCTYPE=buckets > > # Specify executable and input case > > export NWCHEM_EXECUTABLE=./nwchem > export CASE=h2o7 > > ldedit -blpdata $NWCHEM_EXECUTABLE > $NWCHEM_EXECUTABLE $CASE.nw > $CASE.out 2>&1 > > > > ------------------------------- > Yinhe Cheng, PhD > Life Sciences Solutions Enablement > IBM Systems Group, Austin, TX > Phone:512-838-6803 T/L:678-6803 > > > > *Dunyou Wang * > Sent by: owner-nwchem-users@emsl.pnl.gov > > 09/29/08 12:15 PM > > > To > Yinhe Cheng/Watson/IBM@IBMUS > cc > nwchem-users@emsl.pnl.gov > Subject > Re: [NWCHEM] nwchem 5.1 on AIX > > > > > > > > > Hi Yinhe, > > Sorry to reply your message this late. I just came back to my office today > from my vacation. > > The compilation issue you saw here is due to a newer xlf compiler version > 11.1. Please copy this attached GNUmakefile into your > $NWCHEM_TOP/src/NWints/texas to replace the old one, which will solve the > compilation issue. Please don't change anything else in the makefile.h. > > You definitely need to use MPI on either LAPI64 or IBM64, so you have to set > up USE_MPI=y. Here is the example to set up the environmental variables > on LAPI64: > > > setenv NWCHEM_TOP /scratch/scratchdirs/dunyou/nwchem-5.1.1 > setenv NWCHEM_MODULES all > setenv NWCHEM_TARGET LAPI64 > setenv USE_MPI y > setenv LARGE_FILES TRUE > setenv NOAIO y > setenv LIBMPI "-binitfi\ni:poe_remote_main -lmpi_r -lvtd_r > -I/usr/lpp/ssp/css/include -llapi_r -lpthreads > -L/usr/lpp/ppe.poe/lib/threads > -L/usr/lpp/ppe.poe/lib -L/usr/lpp/ppe.poe/lib/ip " > setenv MPI_INCLUDE /usr/lpp/ppe.poe/include > setenv MPI_LIB /usr/lpp/ppe.poe/lib > > also set up : > setenv MP_MSG_API MPI_LAPI > setenv RT_GRQ ON > > for running on LAPI. > > Everything would be fine for you to compile and run on LAPI64 and IBM64 > platform (change the TARGET according to IBM64). > > Hope this helps and let me know if you still have problem. > > Dunyou > > > > > Yinhe Cheng wrote: >> >> I am trying to build NWChem 5.1 binaries of target IBM64 and LAPI64 on >> IBM POWER5 systems with >> OS: AIX5.3 >> Compilers: IBM XL Fortran Enterprise Edition for AIX, V11.1and IBM XL >> C/C++ Enterprise Edition for AIX, V9.0. >> >> I got the following error when using the default compiler options: >> 1586-346 (U) An error occurred during code generation. The code >> generation return code was 40. >> 1501-511 Compilation failed for file trobsa.f. >> >> After I get rid of -pipa -qhot from the FOPTIMIZE and change -O4 to -O3. >> It finally compiled ok. >> But at runtime I got the following error: >> rtdb_seq_put: put failed for "" in ./cluster8e.db >> ------------------------------------------------------------------------ >> util_set_rtdb_state: rtdb_put failed 911 >> ------------------------------------------------------------------------ >> ------------------------------------------------------------------------ >> current input line : >> 0: task mp2 gradient >> ------------------------------------------------------------------------ >> ------------------------------------------------------------------------ >> An error occured in the Runtime Database >> ------------------------------------------------------------------------ >> >> And it seems to be some error related to hdbm >> status = hdbm_replace(db, key, value); in rtdb/rtdb_seq.c caused the >> problem. >> >> Same problem for both IBM64 and LAPI64. >> >> From >> > http://www.emsl.pnl.gov/docs/nwchem/nwchem-support/2008/05/0045.RE:_NWCHEM_Parallel_DIIS_SCF_on_IBM_serv >> >> It seems it is possible to build target IBM64 and LAPI64 successfully. I >> am wondering what compilers are used, on what AIX version, and is there >> anything else need to be done besides set the following ENVs: >> export NWCHEM_TOP=`pwd` >> export LARGE_FILES=TRUE >> export NWCHEM_MODULES=all >> export NWCHEM_TARGET=LAPI64 >> unset USE_MPI >> unset LIBMPI >> unset MPI_LIB >> unset MPI_INCLUDE >> >> Best, >> Yinhe Cheng > > # $Id: GNUmakefile,v 1.52 2008/05/06 16:53:51 d3p307 Exp $ > > include ../../config/makefile.h > > USE_TEXAS = YEP > > ifeq ($(USE_TEXAS),YEP) > > ifeq ($(TARGET),SOLARIS) > ifeq ($(FC), frt) > FOPTIMIZE += -xamtfer,tfer_i > else > FOPTIMIZE += -inline=amtfer,tfer_i > endif > endif > ifeq ($(TARGET),SP) > FOPTIMIZE += -Q+amtfer:tfer_i > endif > ifeq ($(TARGET),LAPI) > FOPTIMIZE += -Q+amtfer:tfer_i > endif > > LIB_DEFINES = -DUSE_TEXAS > OBJ = abcd.o > OBJ_OPTIMIZE = texas_face.o \ > texas_hf.o \ > big_service.o \ > getmem.o \ > amshift.o \ > assemblx.o \ > spec_block.o \ > cshneg.o \ > d_datlog.o \ > spec_calcint.o \ > spec_destiny.o \ > memoha.o \ > precalc2.o \ > prepint.o \ > lab_reorder.o \ > rt123.o \ > shells.o \ > time.o \ > zeroint.o \ > missing.o \ > texas_stubs.o \ > uniq_pairs.o \ > transfor.o \ > gencon.o \ > derivat.o \ > convert.o \ > test_calc2.o \ > screened_op.o \ > case_impl.o > ifdef XLF11 > OBJ += trobsa.o > else > OBJ_OPTIMIZE += trobsa.o > endif > > else > OBJ = texas_stubs.o > OBJ_OPTIMIZE = > endif > > USES_BLAS = amshift.f \ > big_service.F \ > trobsa.F getmem.F > > LIBRARY = libnwints.a > HEADERS = screened_op.fh \ > case.fh > > include ../../config/makelib.h > ifeq ($(_CPU),ia64) > ifeq ($(FC),ifort) > FDEBUG= > FOPTIONS+= $(FOPTIMIZE) > -mP3OPT_ecg_fp_control_safety=TRUE -g -save > endif > endif > > txs: > @touch texas_stubs.F > $(MAKE) "USE_TEXAS=YEP" > > removeobj: > ar d $(LIBRARY_PATH) $(OBJ) $(OBJ_OPTIMIZE) > >