MPI VERSION : CRAY MPICH2 XT version 5.1.3 (ANL base 1.2.1p1) BUILD INFO : Built Tue Dec 7 18:06:42 2010 (svn rev 8190) PE 0: MPICH environment settings: PE 0: MPICH_ENV_DISPLAY = 1 PE 0: MPICH_VERSION_DISPLAY = 1 PE 0: MPICH_ABORT_ON_ERROR = 1 PE 0: MPICH_CPUMASK_DISPLAY = 0 PE 0: MPICH_RANK_REORDER_METHOD = 1 PE 0: MPICH_RANK_REORDER_DISPLAY = 0 PE 0: MPICH_MEMCPY_MEM_CHECK = 0 PE 0: MPICH_MAX_THREAD_SAFETY = single PE 0: MPICH_COLL_SYNC = 0 PE 0: MPICH_MSG_QUEUE_DBG = 0 PE 0: MPICH_CPU_YIELD = 0 PE 0: MPICH_MSGS_PER_PROC = 160000 PE 0: MPICH/SMP environment settings: PE 0: MPICH_SMP_OFF = 0 PE 0: MPICH_SMPDEV_BUFS_PER_PROC = 32 PE 0: MPICH_SMP_SINGLE_COPY_SIZE = 131072 PE 0: MPICH_SMP_SINGLE_COPY_OFF = 0 PE 0: MPICH/PORTALS environment settings: PE 0: MPICH_MAX_SHORT_MSG_SIZE = 960000 PE 0: MPICH_UNEX_BUFFER_SIZE = 629145600 PE 0: MPICH_PTL_UNEX_EVENTS = 90000 PE 0: MPICH_PTL_OTHER_EVENTS = 2048 PE 0: MPICH_PTL_MEMD_LIMIT = 2048 PE 0: MPICH_VSHORT_OFF = 0 PE 0: MPICH_MAX_VSHORT_MSG_SIZE = 1024 PE 0: MPICH_VSHORT_BUFFERS = 32 PE 0: MPICH_PTL_EAGER_LONG = 0 PE 0: MPICH_PTL_MATCH_OFF = 0 PE 0: MPICH_PTL_SEND_CREDITS = 703 (-1) PE 0: MPICH/COLLECTIVE environment settings: PE 0: MPICH_FAST_MEMCPY = 0 PE 0: MPICH_COLL_OPT_OFF = 0 PE 0: MPICH_BCAST_ONLY_TREE = 1 PE 0: MPICH_ALLTOALL_SHORT_MSG = 1024 PE 0: MPICH_REDUCE_SHORT_MSG = 65536 PE 0: MPICH_REDUCE_LARGE_MSG = 131072 PE 0: MPICH_ALLREDUCE_LARGE_MSG = 262144 PE 0: MPICH_ALLGATHER_VSHORT_MSG = 2048 PE 0: MPICH_ALLGATHERV_VSHORT_MSG= 2048 PE 0: MPICH_ALLTOALLVW_FCSIZE = 32 PE 0: MPICH_ALLTOALLVW_SENDWIN = 20 PE 0: MPICH_ALLTOALLVW_RECVWIN = 20 PE 0: MPICH_SCATTERV_MAX_PROCS = 512 PE 0: MPICH/MPIIO environment settings: PE 0: MPICH_MPIIO_HINTS_DISPLAY = 0 PE 0: MPICH_MPIIO_CB_ALIGN = 2 PE 0: MPICH_MPIIO_META_CHUNK_SIZE = 500 PE 0: MPICH_MPIIO_SEND_CHUNK_SIZE = 100000 PE 0: MPICH_MPIIO_HINTS = NULL (seq_comm_setcomm) initialize ID ( 7 GLOBAL ) pelist = 0 63 1 ( npes = 64) ( nthreads = 1) (seq_comm_setcomm) initialize ID ( 2 ATM ) pelist = 0 63 1 ( npes = 64) ( nthreads = 1) (seq_comm_setcomm) initialize ID ( 1 LND ) pelist = 0 63 1 ( npes = 64) ( nthreads = 1) (seq_comm_setcomm) initialize ID ( 4 ICE ) pelist = 0 63 1 ( npes = 64) ( nthreads = 1) (seq_comm_setcomm) initialize ID ( 5 GLC ) pelist = 0 63 1 ( npes = 64) ( nthreads = 1) (seq_comm_setcomm) initialize ID ( 3 OCN ) pelist = 0 63 1 ( npes = 64) ( nthreads = 1) (seq_comm_setcomm) initialize ID ( 6 CPL ) pelist = 0 63 1 ( npes = 64) ( nthreads = 1) (seq_comm_joincomm) initialize ID ( 8 CPLATM ) join IDs = 6 2 ( npes = 64) ( nthreads = 1) (seq_comm_joincomm) initialize ID ( 9 CPLLND ) join IDs = 6 1 ( npes = 64) ( nthreads = 1) (seq_comm_joincomm) initialize ID ( 10 CPLICE ) join IDs = 6 4 ( npes = 64) ( nthreads = 1) (seq_comm_joincomm) initialize ID ( 11 CPLOCN ) join IDs = 6 3 ( npes = 64) ( nthreads = 1) (seq_comm_joincomm) initialize ID ( 12 CPLGLC ) join IDs = 6 5 ( npes = 64) ( nthreads = 1) (seq_comm_printcomms) ID layout : global pes vs local pe for each ID gpe LND ATM OCN ICE GLC CPL GLOBAL CPLATM CPLLND CPLICE CPLOCN CPLGLC nthrds --- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ 0 : 0 0 0 0 0 0 0 0 0 0 0 0 1 1 : 1 1 1 1 1 1 1 1 1 1 1 1 1 2 : 2 2 2 2 2 2 2 2 2 2 2 2 1 3 : 3 3 3 3 3 3 3 3 3 3 3 3 1 4 : 4 4 4 4 4 4 4 4 4 4 4 4 1 12 : 12 12 12 12 12 12 12 12 12 12 12 12 1 20 : 20 20 20 20 20 20 20 20 20 20 20 20 1 28 : 28 28 28 28 28 28 28 28 28 28 28 28 1 36 : 36 36 36 36 36 36 36 36 36 36 36 36 1 44 : 44 44 44 44 44 44 44 44 44 44 44 44 1 52 : 52 52 52 52 52 52 52 52 52 52 52 52 1 60 : 60 60 60 60 60 60 60 60 60 60 60 60 1 61 : 61 61 61 61 61 61 61 61 61 61 61 61 1 53 : 53 53 53 53 53 53 53 53 53 53 53 53 1 45 : 45 45 45 45 45 45 45 45 45 45 45 45 1 37 : 37 37 37 37 37 37 37 37 37 37 37 37 1 29 : 29 29 29 29 29 29 29 29 29 29 29 29 1 21 : 21 21 21 21 21 21 21 21 21 21 21 21 1 5 : 5 5 5 5 5 5 5 5 5 5 5 5 1 62 : 62 62 62 62 62 62 62 62 62 62 62 62 1 54 : 54 54 54 54 54 54 54 54 54 54 54 54 1 46 : 46 46 46 46 46 46 46 46 46 46 46 46 1 38 : 38 38 38 38 38 38 38 38 38 38 38 38 1 30 : 30 30 30 30 30 30 30 30 30 30 30 30 1 22 : 22 22 22 22 22 22 22 22 22 22 22 22 1 6 : 6 6 6 6 6 6 6 6 6 6 6 6 1 63 : 63 63 63 63 63 63 63 63 63 63 63 63 1 55 : 55 55 55 55 55 55 55 55 55 55 55 55 1 47 : 47 47 47 47 47 47 47 47 47 47 47 47 1 39 : 39 39 39 39 39 39 39 39 39 39 39 39 1 31 : 31 31 31 31 31 31 31 31 31 31 31 31 1 23 : 23 23 23 23 23 23 23 23 23 23 23 23 1 7 : 7 7 7 7 7 7 7 7 7 7 7 7 1 56 : 56 56 56 56 56 56 56 56 56 56 56 56 1 48 : 48 48 48 48 48 48 48 48 48 48 48 48 1 40 : 40 40 40 40 40 40 40 40 40 40 40 40 1 32 : 32 32 32 32 32 32 32 32 32 32 32 32 1 24 : 24 24 24 24 24 24 24 24 24 24 24 24 1 8 : 8 8 8 8 8 8 8 8 8 8 8 8 1 (t_initf) Read in prof_inparm namelist from: drv_in 13 : 13 13 13 13 13 13 13 13 13 13 13 13 1 14 : 14 14 14 14 14 14 14 14 14 14 14 14 1 15 : 15 15 15 15 15 15 15 15 15 15 15 15 1 16 : 16 16 16 16 16 16 16 16 16 16 16 16 1 17 : 17 17 17 17 17 17 17 17 17 17 17 17 1 18 : 18 18 18 18 18 18 18 18 18 18 18 18 1 19 : 19 19 19 19 19 19 19 19 19 19 19 19 1 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4090.01 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 Memory block size conversion in bytes is 4088.02 9 : 9 9 9 9 9 9 9 9 9 9 9 9 1 10 : 10 10 10 10 10 10 10 10 10 10 10 10 1 11 : 11 11 11 11 11 11 11 11 11 11 11 11 1 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 Memory block size conversion in bytes is 4090.01 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4090.01 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 25 : 25 25 25 25 25 25 25 25 25 25 25 25 1 26 : 26 26 26 26 26 26 26 26 26 26 26 26 1 27 : 27 27 27 27 27 27 27 27 27 27 27 27 1 33 : 33 33 33 33 33 33 33 33 33 33 33 33 1 34 : 34 34 34 34 34 34 34 34 34 34 34 34 1 35 : 35 35 35 35 35 35 35 35 35 35 35 35 1 41 : 41 41 41 41 41 41 41 41 41 41 41 41 1 42 : 42 42 42 42 42 42 42 42 42 42 42 42 1 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 43 : 43 43 43 43 43 43 43 43 43 43 43 43 1 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4090.01 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 Memory block size conversion in bytes is 4088.02 Memory block size conversion in bytes is 4088.02 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4090.01 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 Memory block size conversion in bytes is 4090.01 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4090.01 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 Memory block size conversion in bytes is 4090.01 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4090.01 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 Memory block size conversion in bytes is 4088.02 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 49 : 49 49 49 49 49 49 49 49 49 49 49 49 1 57 : 57 57 57 57 57 57 57 57 57 57 57 57 1 58 : 58 58 58 58 58 58 58 58 58 58 58 58 1 59 : 59 59 59 59 59 59 59 59 59 59 59 59 1 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 50 : 50 50 50 50 50 50 50 50 50 50 50 50 1 Memory block size conversion in bytes is 4090.01 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 51 : 51 51 51 51 51 51 51 51 51 51 51 51 1 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4090.01 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4090.01 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4090.01 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 Memory block size conversion in bytes is 4088.02 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4090.01 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 Memory block size conversion in bytes is 4088.02 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4090.01 8 MB memory alloc in MB is 8.00 8 MB memory dealloc in MB is 0.00 Memory block size conversion in bytes is 4088.02 _pmii_daemon(SIGCHLD): [NID 00468] [c3-0c2s5n0] [Mon Apr 25 00:25:33 2011] PE 0 exit signal Segmentation fault [NID 00468] 2011-04-25 00:25:09 Apid 5134430: initiated application termination Application 5134430 exit codes: 139 Application 5134430 exit signals: Killed Application 5134430 resources: utime 0, stime 0