From owner-nwchem-users@emsl.pnl.gov Wed Oct 24 09:09:38 2007 Received: from odyssey.emsl.pnl.gov (localhost [127.0.0.1]) by odyssey.emsl.pnl.gov (8.14.1/8.14.1) with ESMTP id l9OG9b6l029732 for ; Wed, 24 Oct 2007 09:09:38 -0700 (PDT) Received: (from majordom@localhost) by odyssey.emsl.pnl.gov (8.14.1/8.14.1/Submit) id l9OG9bBM029731 for nwchem-users-outgoing-0915; Wed, 24 Oct 2007 09:09:37 -0700 (PDT) X-Authentication-Warning: odyssey.emsl.pnl.gov: majordom set sender to owner-nwchem-users@emsl.pnl.gov using -f X-IronPort-AV: E=Sophos;i="4.21,325,1188802800"; d="scan'208";a="35322775" Message-ID: <471F6E3D.8000905@pnl.gov> Date: Wed, 24 Oct 2007 09:09:33 -0700 From: Dunyou Wang User-Agent: Thunderbird 2.0.0.6 (X11/20070728) MIME-Version: 1.0 To: marc@klingon.uab.es CC: nwchem-users@emsl.pnl.gov Subject: Re: [NWCHEM] NWCHEM OFED/IFC compilation, progress, I guess References: <310ccfc80710150311jb65ea8cw74ed63f7d23c081b@mail.gmail.com> <"47 1 49AB6.6090104"@klingon.uab.es> <4714DE25.5090301@pnl.gov> <47186788.9060009@klingon.uab.es> <4718D4D6.1060209@pnl.gov> <471F252F.3080806@klingon.uab.es> In-Reply-To: <471F252F.3080806@klingon.uab.es> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 24 Oct 2007 16:09:33.0202 (UTC) FILETIME=[43E42F20:01C81658] Sender: owner-nwchem-users@emsl.pnl.gov Precedence: bulk I think it's got to do with the MVAPICH2 installation over the OFED, (rdma errors). Have a look at this discussion: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/2007-February.txt Hope this helps Dunyou Marc Noguera wrote: > Hi, > I'd say I have made some progress, thank you very much for your help. > I have compiled again MVAPICH2 making sure to use Portland compiler, > everything was correct. After that I have compiled NWchem using > MVAPICH2/PGI. > Everything went smoothly and environment was setup as mentioned before > except for the LIBMPI var which was LIBMPI="-lmpich -libumad > -libcommon -lrdmacm". These libraries are found under the > /usr/local/ofed/lib64 directory. Did I link with the correct > libraries? Anyhow, I was able to obtain a nwchem binary after this > modification. > Excuse me if my questions are too simple, but I am facing some run > time problems now which I think arise from the infiniband installation > not from nwchem compilation but I am not totally sure because i can > run a simple sudoku solver program using the same mvapich2+infiniband. > The error is the following: > > Fatal error in MPI_Init: > Other MPI error, error stack: > MPIR_Init_thread(259)...........: Initialization failed > MPID_Init(102)..................: channel initialization failed > MPIDI_CH3_Init(178).............: > MPIDI_CH3I_RMDA_init(115).......: rdma_get_control_parameters > rdma_get_control_parameters(432): > rdma_open_hca(376)..............: Failed to open HCA number 0 > Fatal error in MPI_Init: > Other MPI error, error stack: > MPIR_Init_thread(259)...........: Initialization failed > MPID_Init(102)..................: channel initialization failed > MPIDI_CH3_Init(178).............: > MPIDI_CH3I_RMDA_init(115).......: rdma_get_control_parameters > rdma_get_control_parameters(432): > rdma_open_hca(376)..............: Failed to open HCA number 0 > rank 2 in job 9 borg71.uab.es_33096 caused collective abort of all > ranks > exit status of rank 2: return code 1 > rank 1 in job 9 borg71.uab.es_33096 caused collective abort of all > ranks > exit status of rank 1: return code 1 > > Can this error be related to NWchem compilation? > Thank you very much in advance > Marc > > En/na Dunyou Wang ha escrit: >> Is your MVAPHCH2 compiled using the same compiler as you compile >> NWChem? Try to use the same compiler for both. --Dunyou >> >> Marc Noguera wrote: >> >>> Dear all, >>> MVAPICH2_dir variable is defined but I forgot to mention it. I have >>> copied the Makefile you provided but compilation seems to produce the >>> same error message at the linking stage. >>> You can find complete make.log at >>> http://klingon.uab.es/marc/make.log.17_10.gz >>> Nevertheless I have managed to succesfully compile NWchem with >>> mvapich-0.9.7 in the same environment and compiler without a hassle, >>> under the same nwchem tree and same modifications. So I am not sure if >>> problems com from the MVAPICH2 compilation. >>> >>> Any idea? >>> Thank you >>> Marc >>> En/na Dunyou Wang ha escrit: >>> >>>> Mark, >>>> >>>> First, I saw that you've not defined your $MVAPICH2_dir in your >>>> environmental variable, so the compiler can't find the MPI routines. >>>> Second, please replace the GNUmakefile under your $NWCHEM_TOP/src/blas >>>> with the attached one. This will get rid of the cgemm link error. >>>> >>>> Hope this helps >>>> Dunyou >>>> >>>> Marc Noguera wrote: >>>> >>>> >>>>> Dear list, >>>>> I am trying to compile NWchem 5.0 on a opteron system + infiniband >>>>> I am using OFED v1.1 software stack and MVAPICH2-1.0. I am also using >>>>> Portland fortran compiler 7.0-4 for x86_64. >>>>> I have compiled MVAPICH2 using the same compiler with no trouble. >>>>> However when compiling the I obtain the following error (at the >>>>> end of >>>>> the mail) at the final stage of the compilation( >>>>> http://klingon.uab.es/marc/make.log.gz for the complete log). >>>>> Although I have searched for OFED compilation on the listh and >>>>> applied >>>>> comments that I have found compilation won't finish correctly. Any >>>>> idea why compilation is failing? What am I missing? >>>>> Any help will be appreciated >>>>> Thanks in advance >>>>> >>>>> Marc >>>>> >>>>> -----------Changes----------------- >>>>> Following previous comments in this list I am using global arrays >>>>> 4.1b(or 4.07) instead of the version that is included in the >>>>> nwchem-5.0 tarball. Moreover, I have made some changes to the >>>>> original >>>>> code as specified previously in this list: >>>>> >>>>> I have changed $NWCHEM_TOP/src/tools/armci/src/openib.c from >>>>> armci_max_num_sg_ent=30; >>>>> to >>>>> armci_max_num_sg_ent=29; >>>>> >>>>> I have also changed $NWCHEM_TOP/src/util/util_ga_test.F at line 1419. >>>>> >>>>> from >>>>> call ga_locate_region(g_a, 1, n, 1,n, map,np) >>>>> to >>>>> status=ga_locate_region(g_a, 1, n, 1,n, map,np) >>>>> >>>>> and $NWCHEM_TOP/src/blas/GNUmakefile at line 75. >>>>> >>>>> from >>>>> xerbla.o cgeru.o csasum.o >>>>> to >>>>> xerbla.o cgeru.o scasum.o >>>>> >>>>> >>>>> ------------Environment------------------ >>>>> My compile envirnoment is as follows: >>>>> export LARGE_FILES=true >>>>> export TCGRSH=/usr/bin/ssh >>>>> export NWCHEM_TOP=/users/sysuser/soft/nwchem-5.0 >>>>> export NWCHEM_TARGET=LINUX64 >>>>> export NWCHEM_MODULES=all >>>>> export USE_MPI=y >>>>> export ARMCI_NETWORK=OPENIB >>>>> export IB_HOME=/usr/local/ofed >>>>> export IB_LIB=/usr/local/ofed/lib64 >>>>> export IB_INCLUDE=/usr/local/ofed/include >>>>> export MPI_LOC=$MVAPICH2_dir >>>>> export MPI_LIB=$MPI_LOC/lib >>>>> export MPI_INCLUDE=$MPI_LOC/include >>>>> export LIBMPI=-lmpich >>>>> >>>>> and compile instructions are: >>>>> >>>>> make FC=pgf90 nwchem_config >>>>> make FC=pgf90 >>>>> >>>>> ---make output--------- >>>>> make nwchem.o stubs.o >>>>> make[1]: warning: -jN forced in submake: disabling jobserver mode. >>>>> pgf90 -i8 -Mdalign -Mllalign -Kieee -tp k8-64 -g -O2 -I. >>>>> -I/users/sysuser/soft/nwchem-5.0/src/include >>>>> -I/users/sysuser/soft/nwchem-5.0/src/tools/include -DEXT_INT -DLINUX >>>>> -DLINUX64 -DCHKUNDFLW -DPGLINUX -DPARALLEL_DIAG >>>>> -DCOMPILATION_DATE="'`date`'" >>>>> -DCOMPILATION_DIR="'/users/sysuser/soft/nwchem-5.0'" >>>>> -DNWCHEM_BRANCH="'5.0'" -c -o nwchem.o nwchem.F >>>>> pgf90 -i8 -Mdalign -Mllalign -Kieee -tp k8-64 -g -O2 -I. >>>>> -I/users/sysuser/soft/nwchem-5.0/src/include >>>>> -I/users/sysuser/soft/nwchem-5.0/src/tools/include -DEXT_INT -DLINUX >>>>> -DLINUX64 -DCHKUNDFLW -DPGLINUX -DPARALLEL_DIAG >>>>> -DCOMPILATION_DATE="'`date`'" >>>>> -DCOMPILATION_DIR="'/users/sysuser/soft/nwchem-5.0'" >>>>> -DNWCHEM_BRANCH="'5.0'" -c -o stubs.o stubs.F >>>>> pgf90 -i8 -Mdalign -Mllalign -Kieee -tp k8-64 -g -O2 >>>>> -L/users/sysuser/soft/nwchem-5.0/lib/LINUX64 >>>>> -L/users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64 -o >>>>> /users/sysuser/soft/nwchem-5.0/bin/LINUX64/nwchem nwchem.o stubs.o >>>>> -lnwctask -lccsd -lmcscf -lselci -lmp2 -lmoints -lstepper -ldriver >>>>> -ldftgrad -lnwdft -lgradients -lcphf -lesp -lddscf -lguess -lhessian >>>>> -lvib -lnwcutil -lrimp2 -lproperty -lnwints -ldplot -lvscf -letrans >>>>> -ltddft -ltce -lbq -lcons -lnwcutil -lpario -lglobal -lma -lpeigs >>>>> -lperfm -lcons -lbq -lnwcutil -llapack -lblas -llapack -lblas >>>>> -larmci -L/QFsoft/MVAPICH2/mvapich2-1.0_PGI7.0_64//lib -ltcgmsg-mpi >>>>> -lmpich -L/usr/local/ofed/lib64 -libverbs >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libglobal.a(global.util.o)(.text+0x33c1): >>>>> >>>>> >>>>> In function `ga_wtime_': >>>>> : undefined reference to `MPI_Wtime' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libglobal.a(collect.o)(.text+0x17a): >>>>> >>>>> >>>>> In function `ga_msg_sync_': >>>>> : undefined reference to `MPI_Barrier' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libglobal.a(collect.o)(.text+0x1a9): >>>>> >>>>> >>>>> In function `ga_msg_pgroup_sync_': >>>>> : undefined reference to `MPI_Barrier' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libglobal.a(matmul.o)(.text+0xd6d): >>>>> >>>>> >>>>> In function `GAI_DGEMM': >>>>> : undefined reference to `cgemm_' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libglobal.a(matmul.o)(.text+0x5074): >>>>> >>>>> >>>>> In function `ga_matmul_mirrored': >>>>> : undefined reference to `cgemm_' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libglobal.a(matmul.o)(.text+0x6d58): >>>>> >>>>> >>>>> In function `nga_matmul_patch': >>>>> : undefined reference to `cgemm_' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(fence.o)(.text+0x254): >>>>> >>>>> >>>>> In function `ARMCI_Barrier': >>>>> : undefined reference to `MPI_Barrier' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x54): >>>>> >>>>> >>>>> In function `get_group_clus_id': >>>>> : undefined reference to `MPI_Comm_group' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x6e): >>>>> >>>>> >>>>> In function `get_group_clus_id': >>>>> : undefined reference to `MPI_Group_translate_ranks' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x352): >>>>> >>>>> >>>>> In function `ARMCI_Bcast_': >>>>> : undefined reference to `MPI_Comm_compare' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x37d): >>>>> >>>>> >>>>> In function `ARMCI_Bcast_': >>>>> : undefined reference to `MPI_Bcast' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x3f8): >>>>> >>>>> >>>>> In function `ARMCI_Group_create': >>>>> : undefined reference to `MPI_Group_incl' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x41d): >>>>> >>>>> >>>>> In function `ARMCI_Group_create': >>>>> : undefined reference to `MPI_Comm_create' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x440): >>>>> >>>>> >>>>> In function `ARMCI_Group_create': >>>>> : undefined reference to `MPI_Group_rank' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x46b): >>>>> >>>>> >>>>> In function `ARMCI_Group_rank': >>>>> : undefined reference to `MPI_Group_rank' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x47c): >>>>> >>>>> >>>>> In function `ARMCI_Group_size': >>>>> : undefined reference to `MPI_Group_size' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x4a0): >>>>> >>>>> >>>>> In function `ARMCI_Absolute_id': >>>>> : undefined reference to `MPI_Comm_group' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x4bb): >>>>> >>>>> >>>>> In function `ARMCI_Absolute_id': >>>>> : undefined reference to `MPI_Group_translate_ranks' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x56c): >>>>> >>>>> >>>>> In function `armci_group_init': >>>>> : undefined reference to `MPI_Comm_group' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(groups.o)(.text+0x57c): >>>>> >>>>> >>>>> In function `armci_group_init': >>>>> : undefined reference to `MPI_Group_rank' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x795): >>>>> >>>>> >>>>> In function `armci_msg_barrier': >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:398: >>>>> undefined reference to `MPI_Barrier' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x7ad): >>>>> >>>>> >>>>> In function `armci_msg_me': >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:425: >>>>> undefined reference to `MPI_Comm_rank' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x7c8): >>>>> >>>>> >>>>> In function `armci_msg_nproc': >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:441: >>>>> undefined reference to `MPI_Comm_size' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x7d6): >>>>> >>>>> >>>>> In function `armci_timer': >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:461: >>>>> undefined reference to `MPI_Wtime' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x7f0): >>>>> >>>>> >>>>> In function `armci_msg_abort': >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:475: >>>>> undefined reference to `MPI_Abort' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x15cd): >>>>> >>>>> >>>>> In function `armci_msg_brdcst': >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:728: >>>>> undefined reference to `MPI_Bcast' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x1604): >>>>> >>>>> >>>>> In function `armci_msg_snd': >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:743: >>>>> undefined reference to `MPI_Send' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x1648): >>>>> >>>>> >>>>> In function `armci_msg_rcv': >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:767: >>>>> undefined reference to `MPI_Recv' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x1661):/users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:768: >>>>> >>>>> >>>>> undefined reference to `MPI_Get_count' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x16a3): >>>>> >>>>> >>>>> In function `armci_msg_rcvany': >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:790: >>>>> undefined reference to `MPI_Recv' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x16d2):/users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:794: >>>>> >>>>> >>>>> undefined reference to `MPI_Get_count' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x4f6b): >>>>> >>>>> >>>>> In function `armci_msg_group_barrier': >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:1844: >>>>> undefined reference to `MPI_Barrier' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x4fed): >>>>> >>>>> >>>>> In function `armci_grp_clus_brdcst': >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:1859: >>>>> undefined reference to `MPI_Comm_group' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x5000):/users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:1860: >>>>> >>>>> >>>>> undefined reference to `MPI_Group_incl' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x5012):/users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:1862: >>>>> >>>>> >>>>> undefined reference to `MPI_Comm_create' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x5036):/users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:1869: >>>>> >>>>> >>>>> undefined reference to `MPI_Comm_free' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libarmci.a(message.o)(.text+0x503f):/users/sysuser/soft/nwchem-5.0/src/tools/armci/src/message.c:1870: >>>>> >>>>> >>>>> undefined reference to `MPI_Group_free' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(misc.o)(.text+0x1f): >>>>> >>>>> >>>>> In function `NNODES_': >>>>> : undefined reference to `MPI_Comm_size' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(misc.o)(.text+0x3f): >>>>> >>>>> >>>>> In function `NODEID_': >>>>> : undefined reference to `MPI_Comm_rank' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(misc.o)(.text+0x12a): >>>>> >>>>> >>>>> In function `ALT_PBEGIN_': >>>>> : undefined reference to `MPI_Initialized' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(misc.o)(.text+0x141): >>>>> >>>>> >>>>> In function `ALT_PBEGIN_': >>>>> : undefined reference to `MPI_Comm_size' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(misc.o)(.text+0x150): >>>>> >>>>> >>>>> In function `ALT_PBEGIN_': >>>>> : undefined reference to `MPI_Comm_rank' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(misc.o)(.text+0x171): >>>>> >>>>> >>>>> In function `ALT_PBEGIN_': >>>>> : undefined reference to `MPI_Barrier' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(misc.o)(.text+0x192): >>>>> >>>>> >>>>> In function `ALT_PBEGIN_': >>>>> : undefined reference to `MPI_Init' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(misc.o)(.text+0x1a1): >>>>> >>>>> >>>>> In function `ALT_PBEGIN_': >>>>> : undefined reference to `MPI_Errhandler_set' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(misc.o)(.text+0x1fc): >>>>> >>>>> >>>>> In function `pend_': >>>>> : undefined reference to `MPI_Finalize' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(misc.o)(.text+0x21f): >>>>> >>>>> >>>>> In function `tcgtime_': >>>>> : undefined reference to `MPI_Wtime' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(misc.o)(.text+0x267): >>>>> >>>>> >>>>> In function `tcgtime_': >>>>> : undefined reference to `MPI_Wtime' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(misc.o)(.text+0xd9): >>>>> >>>>> >>>>> In function `Error': >>>>> : undefined reference to `MPI_Abort' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x79): >>>>> >>>>> >>>>> In function `SND_': >>>>> : undefined reference to `MPI_Isend' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0xf0): >>>>> >>>>> >>>>> In function `SND_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x138): >>>>> >>>>> >>>>> In function `SND_': >>>>> : undefined reference to `MPI_Send' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x169): >>>>> >>>>> >>>>> In function `SND_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x24d): >>>>> >>>>> >>>>> In function `RCV_': >>>>> : undefined reference to `MPI_Irecv' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x2ef): >>>>> >>>>> >>>>> In function `RCV_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x33b): >>>>> >>>>> >>>>> In function `RCV_': >>>>> : undefined reference to `MPI_Recv' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x353): >>>>> >>>>> >>>>> In function `RCV_': >>>>> : undefined reference to `MPI_Get_count' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x39f): >>>>> >>>>> >>>>> In function `RCV_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x3d6): >>>>> >>>>> >>>>> In function `RCV_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x467): >>>>> >>>>> >>>>> In function `WAITCOM_': >>>>> : undefined reference to `MPI_Wait' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x4b1): >>>>> >>>>> >>>>> In function `WAITCOM_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x527): >>>>> >>>>> >>>>> In function `PROBE_': >>>>> : undefined reference to `MPI_Iprobe' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(p2p.o)(.text+0x568): >>>>> >>>>> >>>>> In function `PROBE_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(nxtval-armci.o)(.text+0x79): >>>>> >>>>> >>>>> In function `NXTVAL_': >>>>> : undefined reference to `MPI_Barrier' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(nxtval-armci.o)(.text+0x96): >>>>> >>>>> >>>>> In function `NXTVAL_': >>>>> : undefined reference to `MPI_Barrier' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(nxtval-armci.o)(.text+0x1b7): >>>>> >>>>> >>>>> In function `install_nxtval': >>>>> : undefined reference to `MPI_Barrier' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(wrap.o)(.text+0x495): >>>>> >>>>> >>>>> In function `igop_': >>>>> : undefined reference to `MPI_Reduce' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(wrap.o)(.text+0x4ba): >>>>> >>>>> >>>>> In function `igop_': >>>>> : undefined reference to `MPI_Bcast' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(wrap.o)(.text+0x5b5): >>>>> >>>>> >>>>> In function `igop_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(wrap.o)(.text+0x5ef): >>>>> >>>>> >>>>> In function `igop_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(collect.o)(.text+0x1d4): >>>>> >>>>> >>>>> In function `IGOP_': >>>>> : undefined reference to `MPI_Reduce' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(collect.o)(.text+0x1f9): >>>>> >>>>> >>>>> In function `IGOP_': >>>>> : undefined reference to `MPI_Bcast' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(collect.o)(.text+0x2f5): >>>>> >>>>> >>>>> In function `IGOP_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(collect.o)(.text+0x32f): >>>>> >>>>> >>>>> In function `IGOP_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(collect.o)(.text+0x644): >>>>> >>>>> >>>>> In function `DGOP_': >>>>> : undefined reference to `MPI_Reduce' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(collect.o)(.text+0x669): >>>>> >>>>> >>>>> In function `DGOP_': >>>>> : undefined reference to `MPI_Bcast' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(collect.o)(.text+0x76f): >>>>> >>>>> >>>>> In function `DGOP_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(collect.o)(.text+0x7a9): >>>>> >>>>> >>>>> In function `DGOP_': >>>>> : undefined reference to `MPI_Error_string' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(collect.o)(.text+0x895): >>>>> >>>>> >>>>> In function `SYNCH_': >>>>> : undefined reference to `MPI_Barrier' >>>>> /users/sysuser/soft/nwchem-5.0/src/tools/lib/LINUX64/libtcgmsg-mpi.a(collect.o)(.text+0x8b4): >>>>> >>>>> >>>>> In function `BRDCST_': >>>>> : undefined reference to `MPI_Bcast' >>>>> make: *** [all] Error 2 >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> #$Id: GNUmakefile,v 1.40 2007/05/23 16:43:50 d3p307 Exp $ >>>> ifdef USE_64TO32 >>>> USE_I4FLAGS=y >>>> endif >>>> include ../config/makefile.h >>>> >>>> LIBRARY = libblas.a >>>> >>>> # >>>> # USE both double and single now (short term solution for added >>>> single precision >>>> # data in GA). Still keep the original layout, in case later need to >>>> be changed >>>> # again. >>>> VPATH = double single >>>> >>>> # PRECISION = double single >>>> # #ifeq ($(TARGET),KSR) >>>> # PRECISION = single #endif >>>> #ifeq ($(TARGET),CRAY-T3D) >>>> # PRECISION = single >>>> #endif >>>> #ifeq ($(TARGET),CRAY-T3E) >>>> # PRECISION = single >>>> #endif >>>> # >>>> # >>>> #ifndef PRECISION >>>> #error: >>>> # @echo BLAS makefile does not have PRECISION defined >>>> # @exit 1 >>>> #endif >>>> >>>> >>>> # >>>> # Use full vectorization compiler options >>>> # >>>> >>>> ifdef FVECTORIZE >>>> FOPTIMIZE := $(FVECTORIZE) >>>> endif >>>> >>>> # xerbla and lsame are taken out from single >>>> #ifeq ($(PRECISION),double) >>>> ####################################################################### >>>> >>>> # ######## ######## ######## ######## >>>> OBJ_OPTIMIZE = dasum.o daxpy.o dcopy.o >>>> ddot.o \ >>>> dgemm.o dgemv.o dger.o >>>> dnrm2.o \ >>>> drot.o dscal.o dspmv.o >>>> dspr.o \ >>>> dswap.o dsymm.o dsymv.o >>>> dsyr2.o \ >>>> dsyr2k.o dsyrk.o dtrmm.o >>>> dtrmv.o \ >>>> dtrsm.o dtrsv.o dznrm2.o >>>> idamax.o \ >>>> izamax.o \ >>>> lsame.o zaxpy.o dcabs1.o >>>> zcopy.o \ >>>> zdotc.o zdscal.o zgemm.o >>>> zgemv.o \ >>>> zgerc.o zhemm.o zhemv.o >>>> zher2.o \ >>>> zher2k.o zherk.o zscal.o >>>> zswap.o \ >>>> ztrmm.o ztrmv.o ztrsm.o >>>> ztrsv.o \ >>>> xerbla.o zgeru.o dzasum.o dsyr.o #else >>>> ####################################################################### >>>> >>>> # ######## ######## ######## ######## >>>> OBJ_OPTIMIZE +=sasum.o saxpy.o scopy.o sdot.o \ >>>> sgemm.o sgemv.o sger.o >>>> snrm2.o \ >>>> srot.o sscal.o sspmv.o >>>> sspr.o \ >>>> sswap.o ssymm.o ssymv.o >>>> ssyr2.o \ >>>> ssyr2k.o ssyrk.o strmm.o >>>> strmv.o \ >>>> strsm.o strsv.o scnrm2.o >>>> isamax.o \ >>>> caxpy.o ccopy.o \ >>>> cdotc.o csscal.o cgemm.o >>>> cgemv.o \ >>>> cgerc.o chemm.o chemv.o >>>> cher2.o \ >>>> cher2k.o cherk.o cscal.o >>>> cswap.o \ >>>> ctrmm.o ctrmv.o ctrsm.o >>>> ctrsv.o \ >>>> cgeru.o scasum.o #endif >>>> ifeq ($(TARGET),CRAY-T3D) >>>> OBJ_OPTIMIZE = idamax.o >>>> endif >>>> ifeq ($(TARGET),CRAY-T3E) >>>> OBJ_OPTIMIZE = idamax.o >>>> endif >>>> >>>> ifeq ($(TARGET),MACX) >>>> ifdef USE_VECLIB >>>> OBJ_OPTIMIZE = lsame.o endif >>>> endif >>>> ifeq ($(TARGET),IBM) >>>> ifdef USE_ESSL >>>> OBJ_OPTIMIZE = lsame.o xerbla.o izamax.o dcabs1.o >>>> endif >>>> endif >>>> ifeq ($(TARGET),IBM64) >>>> ifdef USE_INTEGER4 >>>> ifdef USE_ESSL >>>> OBJ_OPTIMIZE = lsame.o xerbla.o izamax.o dcabs1.o >>>> endif >>>> endif >>>> endif >>>> ifeq ($(TARGET),SP) >>>> ifdef USE_ESSL >>>> OBJ_OPTIMIZE = lsame.o xerbla.o izamax.o dcabs1.o >>>> endif >>>> endif >>>> ifeq ($(TARGET),LAPI) >>>> ifdef USE_ESSL >>>> OBJ_OPTIMIZE = lsame.o xerbla.o dcabs1.o izamax.o >>>> endif >>>> endif >>>> >>>> #caching for dgemm >>>> ifeq ($(TARGET),LINUX64) >>>> ifeq ($(_CPU),x86_64) >>>> LIB_DEFINES += -DCACHE1M >>>> endif >>>> ifeq ($(_CPU),ia64) >>>> LIB_DEFINES += -DCACHE6M >>>> endif >>>> endif >>>> >>>> include ../config/makelib.h >>>> >>>> >>>> >> >> >> >