climateprediction.net home page
Task 16662133

Task 16662133

Name hadam3pm2_e94w_1991_10_008714883_2
Workunit 8861357
Created 16 Jun 2014, 15:00:25 UTC
Sent 16 Jun 2014, 15:24:40 UTC
Report deadline 29 May 2015, 20:44:40 UTC
Received 25 Jun 2014, 15:59:28 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 4 (0x00000004) Unknown error code
Computer ID 1298068
Run time 6 days 5 hours 38 min 53 sec
CPU time 6 days 5 hours 38 min 53 sec
Validate state Invalid
Credit 1,914.64
Device peak FLOPS 3.05 GFLOPS
Application version UK Met Office HadAM3P (global only) with MOSES II landsurface scheme v7.03
i686-apple-darwin
Stderr
<core_client_version>7.0.65</core_client_version>
<![CDATA[
<message>
process exited with code 4 (0x4, -252)
</message>
<stderr_txt>
6:43 (11332): Can't acquire lockfile (-154) - waiting 35s
23:37:18 (11332): Can't acquire lockfile (-154) - exiting
23:47:20 (11599): Can't acquire lockfile (-154) - waiting 35s
23:47:55 (11599): Can't acquire lockfile (-154) - exiting
23:53:20 (11731): Can't acquire lockfile (-154) - waiting 35s
23:53:55 (11731): Can't acquire lockfile (-154) - exiting
00:03:55 (12005): Can't acquire lockfile (-154) - waiting 35s
00:04:30 (12005): Can't acquire lockfile (-154) - exiting
00:14:40 (12324): Can't acquire lockfile (-154) - waiting 35s
00:15:15 (12324): Can't acquire lockfile (-154) - exiting
00:25:16 (12635): Can't acquire lockfile (-154) - waiting 35s
00:25:51 (12635): Can't acquire lockfile (-154) - exiting
00:35:54 (12941): Can't acquire lockfile (-154) - waiting 35s
00:36:29 (12941): Can't acquire lockfile (-154) - exiting
00:46:31 (13265): Can't acquire lockfile (-154) - waiting 35s
00:47:06 (13265): Can't acquire lockfile (-154) - exiting
00:57:08 (13579): Can't acquire lockfile (-154) - waiting 35s
00:57:43 (13579): Can't acquire lockfile (-154) - exiting
01:07:53 (13849): Can't acquire lockfile (-154) - waiting 35s
01:08:28 (13849): Can't acquire lockfile (-154) - exiting
01:18:39 (14110): Can't acquire lockfile (-154) - waiting 35s
01:19:14 (14110): Can't acquire lockfile (-154) - exiting
01:29:17 (14371): Can't acquire lockfile (-154) - waiting 35s
01:29:52 (14371): Can't acquire lockfile (-154) - exiting
01:39:56 (14625): Can't acquire lockfile (-154) - waiting 35s
01:40:31 (14625): Can't acquire lockfile (-154) - exiting
01:50:35 (14937): Can't acquire lockfile (-154) - waiting 35s
01:51:10 (14937): Can't acquire lockfile (-154) - exiting
02:01:19 (15579): Can't acquire lockfile (-154) - waiting 35s
02:01:54 (15579): Can't acquire lockfile (-154) - exiting
02:12:01 (16213): Can't acquire lockfile (-154) - waiting 35s
02:12:36 (16213): Can't acquire lockfile (-154) - exiting
02:22:38 (16837): Can't acquire lockfile (-154) - waiting 35s
02:23:13 (16837): Can't acquire lockfile (-154) - exiting
02:33:14 (17460): Can't acquire lockfile (-154) - waiting 35s
02:33:49 (17460): Can't acquire lockfile (-154) - exiting
03:08:17 (19948): Can't acquire lockfile (-154) - waiting 35s
03:08:52 (19948): Can't acquire lockfile (-154) - exiting
03:18:57 (20282): Can't acquire lockfile (-154) - waiting 35s
03:19:32 (20282): Can't acquire lockfile (-154) - exiting
03:29:41 (20537): Can't acquire lockfile (-154) - waiting 35s
03:30:16 (20537): Can't acquire lockfile (-154) - exiting
03:40:23 (20796): Can't acquire lockfile (-154) - waiting 35s
03:40:58 (20796): Can't acquire lockfile (-154) - exiting
03:51:03 (21055): Can't acquire lockfile (-154) - waiting 35s
03:51:39 (21055): Can't acquire lockfile (-154) - exiting
04:01:47 (21309): Can't acquire lockfile (-154) - waiting 35s
04:02:22 (21309): Can't acquire lockfile (-154) - exiting
04:12:33 (21568): Can't acquire lockfile (-154) - waiting 35s
04:13:08 (21568): Can't acquire lockfile (-154) - exiting
04:23:12 (21841): Can't acquire lockfile (-154) - waiting 35s
04:23:47 (21841): Can't acquire lockfile (-154) - exiting
04:33:56 (22285): Can't acquire lockfile (-154) - waiting 35s
04:34:31 (22285): Can't acquire lockfile (-154) - exiting
04:44:33 (22726): Can't acquire lockfile (-154) - waiting 35s
04:45:08 (22726): Can't acquire lockfile (-154) - exiting
04:55:09 (23175): Can't acquire lockfile (-154) - waiting 35s
04:55:44 (23175): Can't acquire lockfile (-154) - exiting
05:05:46 (23641): Can't acquire lockfile (-154) - waiting 35s
05:06:21 (23641): Can't acquire lockfile (-154) - exiting
05:09:19 (23775): Can't acquire lockfile (-154) - waiting 35s
05:09:54 (23775): Can't acquire lockfile (-154) - exiting
05:20:01 (24085): Can't acquire lockfile (-154) - waiting 35s
05:20:36 (24085): Can't acquire lockfile (-154) - exiting
05:30:38 (24537): Can't acquire lockfile (-154) - waiting 35s
05:31:13 (24537): Can't acquire lockfile (-154) - exiting
05:41:21 (25027): Can't acquire lockfile (-154) - waiting 35s
05:41:56 (25027): Can't acquire lockfile (-154) - exiting
05:52:02 (25531): Can't acquire lockfile (-154) - waiting 35s
05:52:37 (25531): Can't acquire lockfile (-154) - exiting
06:02:45 (26003): Can't acquire lockfile (-154) - waiting 35s
06:03:20 (26003): Can't acquire lockfile (-154) - exiting
06:07:31 (26169): Can't acquire lockfile (-154) - waiting 35s
06:08:06 (26169): Can't acquire lockfile (-154) - exiting
06:18:12 (26239): Can't acquire lockfile (-154) - waiting 35s
06:18:47 (26239): Can't acquire lockfile (-154) - exiting
06:28:53 (26305): Can't acquire lockfile (-154) - waiting 35s
06:29:28 (26305): Can't acquire lockfile (-154) - exiting
06:40:25 (26377): Can't acquire lockfile (-154) - waiting 35s
06:41:00 (26377): Can't acquire lockfile (-154) - exiting
06:51:10 (26519): Can't acquire lockfile (-154) - waiting 35s
06:51:46 (26519): Can't acquire lockfile (-154) - exiting
07:01:53 (26592): Can't acquire lockfile (-154) - waiting 35s
07:02:28 (26592): Can't acquire lockfile (-154) - exiting
07:12:33 (26660): Can't acquire lockfile (-154) - waiting 35s
07:13:08 (26660): Can't acquire lockfile (-154) - exiting
07:23:15 (26727): Can't acquire lockfile (-154) - waiting 35s
07:23:51 (26727): Can't acquire lockfile (-154) - exiting
07:33:58 (26794): Can't acquire lockfile (-154) - waiting 35s
07:34:33 (26794): Can't acquire lockfile (-154) - exiting
07:44:50 (26873): Can't acquire lockfile (-154) - waiting 35s
07:45:25 (26873): Can't acquire lockfile (-154) - exiting
07:56:12 (26951): Can't acquire lockfile (-154) - waiting 35s
07:56:47 (26951): Can't acquire lockfile (-154) - exiting
08:07:40 (27032): Can't acquire lockfile (-154) - waiting 35s
08:08:15 (27032): Can't acquire lockfile (-154) - exiting
08:18:18 (27100): Can't acquire lockfile (-154) - waiting 35s
08:18:53 (27100): Can't acquire lockfile (-154) - exiting
08:29:00 (27166): Can't acquire lockfile (-154) - waiting 35s
08:29:35 (27166): Can't acquire lockfile (-154) - exiting
08:39:44 (27233): Can't acquire lockfile (-154) - waiting 35s
08:40:19 (27233): Can't acquire lockfile (-154) - exiting
08:50:27 (27312): Can't acquire lockfile (-154) - waiting 35s
08:51:02 (27312): Can't acquire lockfile (-154) - exiting
09:01:45 (27387): Can't acquire lockfile (-154) - waiting 35s
09:02:20 (27387): Can't acquire lockfile (-154) - exiting
09:12:24 (27454): Can't acquire lockfile (-154) - waiting 35s
09:12:59 (27454): Can't acquire lockfile (-154) - exiting
09:23:03 (27520): Can't acquire lockfile (-154) - waiting 35s
09:23:38 (27520): Can't acquire lockfile (-154) - exiting
09:33:44 (27587): Can't acquire lockfile (-154) - waiting 35s
09:34:19 (27587): Can't acquire lockfile (-154) - exiting
09:44:33 (27660): Can't acquire lockfile (-154) - waiting 35s
09:45:08 (27660): Can't acquire lockfile (-154) - exiting
09:55:27 (27735): Can't acquire lockfile (-154) - waiting 35s
09:56:02 (27735): Can't acquire lockfile (-154) - exiting
10:06:18 (27900): Can't acquire lockfile (-154) - waiting 35s
10:06:53 (27900): Can't acquire lockfile (-154) - exiting
10:16:59 (28023): Can't acquire lockfile (-154) - waiting 35s
10:17:34 (28023): Can't acquire lockfile (-154) - exiting
10:27:40 (28143): Can't acquire lockfile (-154) - waiting 35s
10:28:15 (28143): Can't acquire lockfile (-154) - exiting
10:38:22 (28265): Can't acquire lockfile (-154) - waiting 35s
10:38:57 (28265): Can't acquire lockfile (-154) - exiting
10:49:06 (28410): Can't acquire lockfile (-154) - waiting 35s
10:49:41 (28410): Can't acquire lockfile (-154) - exiting
11:00:03 (28535): Can't acquire lockfile (-154) - waiting 35s
11:00:38 (28535): Can't acquire lockfile (-154) - exiting
11:03:46 (28573): Can't acquire lockfile (-154) - waiting 35s
11:04:21 (28573): Can't acquire lockfile (-154) - exiting
11:14:27 (28643): Can't acquire lockfile (-154) - waiting 35s
11:15:02 (28643): Can't acquire lockfile (-154) - exiting
11:25:13 (28711): Can't acquire lockfile (-154) - waiting 35s
11:25:48 (28711): Can't acquire lockfile (-154) - exiting
11:36:02 (28779): Can't acquire lockfile (-154) - waiting 35s
11:36:37 (28779): Can't acquire lockfile (-154) - exiting
11:46:49 (28860): Can't acquire lockfile (-154) - waiting 35s
11:47:24 (28860): Can't acquire lockfile (-154) - exiting

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: 
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  00399B82  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0039864B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0035FE0D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  003161B5  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00315917  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00352762  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0034FDFC  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002E34AC  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002722A3  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002BF9C0  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002EA01F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0000285B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00002789  Unknown               Unknown  Unknown
Unknown            00000008  Unknown               Unknown  Unknown
Model crash detected, will try to restart...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  00399B82  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0039864B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0035FE0D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  003161B5  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00315917  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00352762  Unknown    Model crash detected, will try to restart...
11:57:30 (28930): Can't acquire lockfile (-154) - waiting 35s
11:58:05 (28930): Can't acquire lockfile (-154) - exiting
12:09:00 (29009): Can't acquire lockfile (-154) - waiting 35s
12:09:35 (29009): Can't acquire lockfile (-154) - exiting
12:19:39 (29078): Can't acquire lockfile (-154) - waiting 35s
12:20:14 (29078): Can't acquire lockfile (-154) - exiting
12:30:20 (29147): Can't acquire lockfile (-154) - waiting 35s
12:30:55 (29147): Can't acquire lockfile (-154) - exiting
12:41:04 (29213): Can't acquire lockfile (-154) - waiting 35s
12:41:39 (29213): Can't acquire lockfile (-154) - exiting
12:51:46 (29295): Can't acquire lockfile (-154) - waiting 35s
12:52:21 (29295): Can't acquire lockfile (-154) - exiting
13:02:21 (29379): Can't acquire lockfile (-154) - waiting 35s
13:02:56 (29379): Can't acquire lockfile (-154) - exiting
13:13:09 (29513): Can't acquire lockfile (-154) - waiting 35s
13:13:44 (29513): Can't acquire lockfile (-154) - exiting
13:23:51 (29579): Can't acquire lockfile (-154) - waiting 35s
13:24:26 (29579): Can't acquire lockfile (-154) - exiting
13:34:32 (29646): Can't acquire lockfile (-154) - waiting 35s
13:35:07 (29646): Can't acquire lockfile (-154) - exiting
13:45:15 (29719): Can't acquire lockfile (-154) - waiting 35s
13:45:50 (29719): Can't acquire lockfile (-154) - exiting
13:55:58 (29795): Can't acquire lockfile (-154) - waiting 35s
13:56:33 (29795): Can't acquire lockfile (-154) - exiting
14:07:33 (29872): Can't acquire lockfile (-154) - waiting 35s
14:08:08 (29872): Can't acquire lockfile (-154) - exiting
14:13:25 (29920): Can't acquire lockfile (-154) - waiting 35s
14:14:00 (29920): Can't acquire lockfile (-154) - exiting
14:24:07 (29990): Can't acquire lockfile (-154) - waiting 35s
14:24:42 (29990): Can't acquire lockfile (-154) - exiting
14:34:49 (30057): Can't acquire lockfile (-154) - waiting 35s
14:35:24 (30057): Can't acquire lockfile (-154) - exiting
14:45:37 (30137): Can't acquire lockfile (-154) - waiting 35s
14:46:12 (30137): Can't acquire lockfile (-154) - exiting
14:56:18 (30211): Can't acquire lockfile (-154) - waiting 35s
14:56:53 (30211): Can't acquire lockfile (-154) - exiting
error:  cannot create hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.namelists
        No space left on device
error:  cannot create hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.stashc
        No space left on device
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.namelists after 11 attempts
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.namelists after 11 attempts
forrtl: severe (24): end-of-file during read, unit 4, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.stashc
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  00399B82  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0039864B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0035FE0D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  003161B5  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00315917  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0033A1FE  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00114047  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002BF85A  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002EA01F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0000285B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00002789  Unknown               Unknown  Unknown
Unknown            00000008  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30211, iMonCtr=1
Model crash detected, will try to restart...
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.namelists after 11 attempts
forrtl: severe (24): end-of-file during read, unit 4, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.stashc
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  00399B82  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0039864B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0035FE0D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  003161B5  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00315917  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0033A1FE  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00114047  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002BF85A  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002EA01F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0000285B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00002789  Unknown               Unknown  Unknown
Unknown            00000008  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30211, iMonCtr=1
Model crash detected, will try to restart...
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.namelists after 11 attempts
forrtl: severe (24): end-of-file during read, unit 4, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.stashc
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  00399B82  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0039864B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0035FE0D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  003161B5  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00315917  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0033A1FE  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00114047  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002BF85A  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002EA01F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0000285B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00002789  Unknown               Unknown  Unknown
Unknown            00000008  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30211, iMonCtr=1
Model crash detected, will try to restart...
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.namelists after 11 attempts
forrtl: severe (24): end-of-file during read, unit 4, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.stashc
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  00399B82  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0039864B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0035FE0D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  003161B5  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00315917  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0033A1FE  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00114047  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002BF85A  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002EA01F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0000285B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00002789  Unknown               Unknown  Unknown
Unknown            00000008  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30211, iMonCtr=1
Model crash detected, will try to restart...
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.namelists after 11 attempts
forrtl: severe (24): end-of-file during read, unit 4, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.stashc
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  00399B82  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0039864B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0035FE0D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  003161B5  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00315917  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0033A1FE  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00114047  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002BF85A  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002EA01F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0000285B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00002789  Unknown               Unknown  Unknown
Unknown            00000008  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30211, iMonCtr=1
Model crash detected, will try to restart...
cpdnmonitor: cannot open input file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.namelists after 11 attempts
forrtl: severe (24): end-of-file during read, unit 4, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/jobs/xhjlo.stashc
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  00399B82  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0039864B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0035FE0D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  003161B5  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00315917  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0033A1FE  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00114047  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002BF85A  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002EA01F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0000285B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00002789  Unknown               Unknown  Unknown
Unknown            00000008  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30211, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
error:  zipfile probably corrupt (segmentation violation)
hadam3pm2_7.03_i686-apple-darwin(30211,0xac7522c0) malloc: *** error for object 0x500000: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Signal 3 received, exiting...
16:59:24 (33419): Can't acquire lockfile (-154) - waiting 35s
16:59:59 (33419): Can't acquire lockfile (-154) - exiting
17:10:04 (33675): Can't acquire lockfile (-154) - waiting 35s
17:10:39 (33675): Can't acquire lockfile (-154) - exiting
17:20:42 (33932): Can't acquire lockfile (-154) - waiting 35s
17:21:17 (33932): Can't acquire lockfile (-154) - exiting
17:31:24 (34187): Can't acquire lockfile (-154) - waiting 35s
17:31:59 (34187): Can't acquire lockfile (-154) - exiting
17:42:05 (34442): Can't acquire lockfile (-154) - waiting 35s
17:42:40 (34442): Can't acquire lockfile (-154) - exiting
17:52:47 (34707): Can't acquire lockfile (-154) - waiting 35s
17:53:22 (34707): Can't acquire lockfile (-154) - exiting
18:03:27 (34968): Can't acquire lockfile (-154) - waiting 35s
18:04:02 (34968): Can't acquire lockfile (-154) - exiting
18:14:05 (35227): Can't acquire lockfile (-154) - waiting 35s
18:14:40 (35227): Can't acquire lockfile (-154) - exiting
18:24:43 (35480): Can't acquire lockfile (-154) - waiting 35s
18:25:18 (35480): Can't acquire lockfile (-154) - exiting
19:05:35 (37042): Can't acquire lockfile (-154) - waiting 35s
19:06:11 (37042): Can't acquire lockfile (-154) - exiting
19:06:54 (37063): Can't acquire lockfile (-154) - waiting 35s
19:07:29 (37063): Can't acquire lockfile (-154) - exiting
19:19:24 (37398): Can't acquire lockfile (-154) - waiting 35s
19:19:59 (37398): Can't acquire lockfile (-154) - exiting
20:07:19 (38590): Can't acquire lockfile (-154) - waiting 35s
20:07:54 (38590): Can't acquire lockfile (-154) - exiting
20:18:00 (38902): Can't acquire lockfile (-154) - waiting 35s
20:18:35 (38902): Can't acquire lockfile (-154) - exiting
20:21:59 (38997): Can't acquire lockfile (-154) - waiting 35s
20:22:34 (38997): Can't acquire lockfile (-154) - exiting
20:32:40 (39306): Can't acquire lockfile (-154) - waiting 35s
20:33:15 (39306): Can't acquire lockfile (-154) - exiting
20:43:21 (39617): Can't acquire lockfile (-154) - waiting 35s
20:43:56 (39617): Can't acquire lockfile (-154) - exiting
21:22:32 (41778): Can't acquire lockfile (-154) - waiting 35s
21:23:07 (41778): Can't acquire lockfile (-154) - exiting
21:45:13 (43104): Can't acquire lockfile (-154) - waiting 35s
21:45:48 (43104): Can't acquire lockfile (-154) - exiting
21:55:50 (43907): Can't acquire lockfile (-154) - waiting 35s
21:56:25 (43907): Can't acquire lockfile (-154) - exiting
22:26:50 (46251): Can't acquire lockfile (-154) - waiting 35s
22:27:25 (46251): Can't acquire lockfile (-154) - exiting
22:37:28 (47061): Can't acquire lockfile (-154) - waiting 35s
22:38:03 (47061): Can't acquire lockfile (-154) - exiting
22:48:08 (47884): Can't acquire lockfile (-154) - waiting 35s
22:48:43 (47884): Can't acquire lockfile (-154) - exiting
22:58:50 (48696): Can't acquire lockfile (-154) - waiting 35s
22:59:25 (48696): Can't acquire lockfile (-154) - exiting
23:08:30 (49352): Can't acquire lockfile (-154) - waiting 35s
23:09:05 (49352): Can't acquire lockfile (-154) - exiting
23:19:10 (49973): Can't acquire lockfile (-154) - waiting 35s
23:19:45 (49973): Can't acquire lockfile (-154) - exiting
23:29:52 (50598): Can't acquire lockfile (-154) - waiting 35s
23:30:27 (50598): Can't acquire lockfile (-154) - exiting
23:40:35 (51221): Can't acquire lockfile (-154) - waiting 35s
23:41:10 (51221): Can't acquire lockfile (-154) - exiting
23:51:18 (51856): Can't acquire lockfile (-154) - waiting 35s
23:51:53 (51856): Can't acquire lockfile (-154) - exiting
00:02:00 (52488): Can't acquire lockfile (-154) - waiting 35s
00:02:35 (52488): Can't acquire lockfile (-154) - exiting
00:06:18 (52713): Can't acquire lockfile (-154) - waiting 35s
00:06:53 (52713): Can't acquire lockfile (-154) - exiting
00:16:58 (53023): Can't acquire lockfile (-154) - waiting 35s
00:17:33 (53023): Can't acquire lockfile (-154) - exiting
00:27:38 (53494): Can't acquire lockfile (-154) - waiting 35s
00:28:13 (53494): Can't acquire lockfile (-154) - exiting
00:38:20 (53988): Can't acquire lockfile (-154) - waiting 35s
00:38:55 (53988): Can't acquire lockfile (-154) - exiting
00:48:56 (54488): Can't acquire lockfile (-154) - waiting 35s
00:49:31 (54488): Can't acquire lockfile (-154) - exiting
01:03:25 (55141): Can't acquire lockfile (-154) - waiting 35s
01:04:00 (55141): Can't acquire lockfile (-154) - exiting
02:03:45 (56607): Can't acquire lockfile (-154) - waiting 35s
02:04:20 (56607): Can't acquire lockfile (-154) - exiting
02:14:20 (56883): Can't acquire lockfile (-154) - waiting 35s
02:14:55 (56883): Can't acquire lockfile (-154) - exiting

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: 
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  00399B82  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0039864B  Unknown            Model crash detected, will try to restart...
02:21:23 (57040): Can't acquire lockfile (-154) - waiting 35s
02:21:58 (57040): Can't acquire lockfile (-154) - exiting
02:32:05 (57108): Can't acquire lockfile (-154) - waiting 35s
02:32:40 (57108): Can't acquire lockfile (-154) - exiting
02:42:53 (57176): Can't acquire lockfile (-154) - waiting 35s
02:43:28 (57176): Can't acquire lockfile (-154) - exiting
02:53:34 (57258): Can't acquire lockfile (-154) - waiting 35s
02:54:09 (57258): Can't acquire lockfile (-154) - exiting
03:04:49 (57337): Can't acquire lockfile (-154) - waiting 35s
03:05:24 (57337): Can't acquire lockfile (-154) - exiting
03:15:31 (57471): Can't acquire lockfile (-154) - waiting 35s
03:16:06 (57471): Can't acquire lockfile (-154) - exiting
03:27:03 (57544): Can't acquire lockfile (-154) - waiting 35s
03:27:38 (57544): Can't acquire lockfile (-154) - exiting
03:38:16 (57616): Can't acquire lockfile (-154) - waiting 35s
03:38:51 (57616): Can't acquire lockfile (-154) - exiting
03:48:56 (57701): Can't acquire lockfile (-154) - waiting 35s
03:49:31 (57701): Can't acquire lockfile (-154) - exiting
03:59:51 (57781): Can't acquire lockfile (-154) - waiting 35s
04:00:26 (57781): Can't acquire lockfile (-154) - exiting
04:10:29 (57849): Can't acquire lockfile (-154) - waiting 35s
04:11:04 (57849): Can't acquire lockfile (-154) - exiting
04:21:07 (57914): Can't acquire lockfile (-154) - waiting 35s
04:21:42 (57914): Can't acquire lockfile (-154) - exiting
04:32:37 (57989): Can't acquire lockfile (-154) - waiting 35s
04:33:12 (57989): Can't acquire lockfile (-154) - exiting
04:43:19 (58056): Can't acquire lockfile (-154) - waiting 35s
04:43:54 (58056): Can't acquire lockfile (-154) - exiting
04:54:03 (58140): Can't acquire lockfile (-154) - waiting 35s
04:54:38 (58140): Can't acquire lockfile (-154) - exiting
05:04:41 (58239): Can't acquire lockfile (-154) - waiting 35s
05:05:16 (58239): Can't acquire lockfile (-154) - exiting
05:15:26 (58362): Can't acquire lockfile (-154) - waiting 35s
05:16:01 (58362): Can't acquire lockfile (-154) - exiting
05:26:09 (58482): Can't acquire lockfile (-154) - waiting 35s
05:26:44 (58482): Can't acquire lockfile (-154) - exiting
05:36:50 (58607): Can't acquire lockfile (-154) - waiting 35s
05:37:25 (58607): Can't acquire lockfile (-154) - exiting
05:47:34 (58925): Can't acquire lockfile (-154) - waiting 35s
05:48:09 (58925): Can't acquire lockfile (-154) - exiting
05:57:54 (59209): Can't acquire lockfile (-154) - waiting 35s
05:58:29 (59209): Can't acquire lockfile (-154) - exiting
06:08:32 (59476): Can't acquire lockfile (-154) - waiting 35s
06:09:07 (59476): Can't acquire lockfile (-154) - exiting
06:19:11 (59814): Can't acquire lockfile (-154) - waiting 35s
06:19:46 (59814): Can't acquire lockfile (-154) - exiting
06:33:10 (60576): Can't acquire lockfile (-154) - waiting 35s
06:33:45 (60576): Can't acquire lockfile (-154) - exiting
07:00:45 (62199): Can't acquire lockfile (-154) - waiting 35s
07:01:20 (62199): Can't acquire lockfile (-154) - exiting
07:06:41 (62490): Can't acquire lockfile (-154) - waiting 35s
07:07:16 (62490): Can't acquire lockfile (-154) - exiting
07:17:20 (62756): Can't acquire lockfile (-154) - waiting 35s
07:17:55 (62756): Can't acquire lockfile (-154) - exiting
07:28:00 (63012): Can't acquire lockfile (-154) - waiting 35s
07:28:35 (63012): Can't acquire lockfile (-154) - exiting
07:38:42 (63269): Can't acquire lockfile (-154) - waiting 35s
07:39:17 (63269): Can't acquire lockfile (-154) - exiting
07:49:23 (63541): Can't acquire lockfile (-154) - waiting 35s
07:49:58 (63541): Can't acquire lockfile (-154) - exiting
07:57:53 (63731): Can't acquire lockfile (-154) - waiting 35s
07:58:28 (63731): Can't acquire lockfile (-154) - exiting
08:08:31 (63984): Can't acquire lockfile (-154) - waiting 35s
08:09:06 (63984): Can't acquire lockfile (-154) - exiting
08:19:11 (64239): Can't acquire lockfile (-154) - waiting 35s
08:19:46 (64239): Can't acquire lockfile (-154) - exiting
08:29:52 (64493): Can't acquire lockfile (-154) - waiting 35s
08:30:27 (64493): Can't acquire lockfile (-154) - exiting
08:40:35 (64748): Can't acquire lockfile (-154) - waiting 35s
08:41:10 (64748): Can't acquire lockfile (-154) - exiting
08:51:14 (65023): Can't acquire lockfile (-154) - waiting 35s
08:51:49 (65023): Can't acquire lockfile (-154) - exiting
08:59:13 (65198): Can't acquire lockfile (-154) - waiting 35s
08:59:48 (65198): Can't acquire lockfile (-154) - exiting
09:09:54 (65453): Can't acquire lockfile (-154) - waiting 35s
09:10:29 (65453): Can't acquire lockfile (-154) - exiting
09:20:33 (65708): Can't acquire lockfile (-154) - waiting 35s
09:21:08 (65708): Can't acquire lockfile (-154) - exiting
09:31:13 (65967): Can't acquire lockfile (-154) - waiting 35s
09:31:48 (65967): Can't acquire lockfile (-154) - exiting
09:41:57 (66225): Can't acquire lockfile (-154) - waiting 35s
09:42:32 (66225): Can't acquire lockfile (-154) - exiting
09:52:37 (66488): Can't acquire lockfile (-154) - waiting 35s
09:53:12 (66488): Can't acquire lockfile (-154) - exiting
09:59:13 (66633): Can't acquire lockfile (-154) - waiting 35s
09:59:48 (66633): Can't acquire lockfile (-154) - exiting
10:09:51 (67109): Can't acquire lockfile (-154) - waiting 35s
10:10:26 (67109): Can't acquire lockfile (-154) - exiting
11:01:47 (69554): Can't acquire lockfile (-154) - waiting 35s
11:02:22 (69554): Can't acquire lockfile (-154) - exiting
11:12:55 (70034): Can't acquire lockfile (-154) - waiting 35s
11:13:30 (70034): Can't acquire lockfile (-154) - exiting
11:23:32 (70497): Can't acquire lockfile (-154) - waiting 35s
11:24:07 (70497): Can't acquire lockfile (-154) - exiting
11:34:10 (70929): Can't acquire lockfile (-154) - waiting 35s
11:34:45 (70929): Can't acquire lockfile (-154) - exiting
11:44:55 (71370): Can't acquire lockfile (-154) - waiting 35s
11:45:30 (71370): Can't acquire lockfile (-154) - exiting
11:55:31 (71806): Can't acquire lockfile (-154) - waiting 35s
11:56:06 (71806): Can't acquire lockfile (-154) - exiting
12:06:16 (72243): Can't acquire lockfile (-154) - waiting 35s
12:06:51 (72243): Can't acquire lockfile (-154) - exiting
12:17:00 (72684): Can't acquire lockfile (-154) - waiting 35s
12:17:35 (72684): Can't acquire lockfile (-154) - exiting
12:27:44 (73119): Can't acquire lockfile (-154) - waiting 35s
12:28:19 (73119): Can't acquire lockfile (-154) - exiting
12:38:28 (73555): Can't acquire lockfile (-154) - waiting 35s
12:39:03 (73555): Can't acquire lockfile (-154) - exiting
12:49:05 (73996): Can't acquire lockfile (-154) - waiting 35s
12:49:40 (73996): Can't acquire lockfile (-154) - exiting
12:58:55 (74354): Can't acquire lockfile (-154) - waiting 35s
12:59:30 (74354): Can't acquire lockfile (-154) - exiting
13:09:35 (74610): Can't acquire lockfile (-154) - waiting 35s
13:10:10 (74610): Can't acquire lockfile (-154) - exiting
13:20:16 (74869): Can't acquire lockfile (-154) - waiting 35s
13:20:51 (74869): Can't acquire lockfile (-154) - exiting
13:30:59 (75124): Can't acquire lockfile (-154) - waiting 35s
13:31:34 (75124): Can't acquire lockfile (-154) - exiting
13:41:38 (75379): Can't acquire lockfile (-154) - waiting 35s
13:42:13 (75379): Can't acquire lockfile (-154) - exiting
13:52:17 (75642): Can't acquire lockfile (-154) - waiting 35s
13:52:52 (75642): Can't acquire lockfile (-154) - exiting
13:58:37 (75780): Can't acquire lockfile (-154) - waiting 35s
13:59:12 (75780): Can't acquire lockfile (-154) - exiting
14:09:17 (76038): Can't acquire lockfile (-154) - waiting 35s
14:09:52 (76038): Can't acquire lockfile (-154) - exiting
14:19:55 (76296): Can't acquire lockfile (-154) - waiting 35s
14:20:30 (76296): Can't acquire lockfile (-154) - exiting
14:30:38 (76552): Can't acquire lockfile (-154) - waiting 35s
14:31:13 (76552): Can't acquire lockfile (-154) - exiting
14:41:19 (76807): Can't acquire lockfile (-154) - waiting 35s
14:41:54 (76807): Can't acquire lockfile (-154) - exiting
14:52:00 (77077): Can't acquire lockfile (-154) - waiting 35s
14:52:35 (77077): Can't acquire lockfile (-154) - exiting
14:57:06 (77189): Can't acquire lockfile (-154) - waiting 35s
14:57:41 (77189): Can't acquire lockfile (-154) - exiting
15:07:46 (77484): Can't acquire lockfile (-154) - waiting 35s
15:08:21 (77484): Can't acquire lockfile (-154) - exiting
15:18:23 (77799): Can't acquire lockfile (-154) - waiting 35s
15:18:58 (77799): Can't acquire lockfile (-154) - exiting
15:29:02 (78109): Can't acquire lockfile (-154) - waiting 35s
15:29:37 (78109): Can't acquire lockfile (-154) - exiting
15:39:42 (78414): Can't acquire lockfile (-154) - waiting 35s
15:40:17 (78414): Can't acquire lockfile (-154) - exiting
15:50:24 (78734): Can't acquire lockfile (-154) - waiting 35s
15:50:59 (78734): Can't acquire lockfile (-154) - exiting

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: 
15:57:46 (78937): Can't acquire lockfile (-154) - waiting 35s
15:58:21 (78937): Can't acquire lockfile (-154) - exiting
16:08:26 (79200): Can't acquire lockfile (-154) - waiting 35s
16:09:01 (79200): Can't acquire lockfile (-154) - exiting
16:19:04 (79455): Can't acquire lockfile (-154) - waiting 35s
16:19:39 (79455): Can't acquire lockfile (-154) - exiting
16:29:46 (79706): Can't acquire lockfile (-154) - waiting 35s
16:30:21 (79706): Can't acquire lockfile (-154) - exiting
16:40:32 (79967): Can't acquire lockfile (-154) - waiting 35s
16:41:07 (79967): Can't acquire lockfile (-154) - exiting
16:51:11 (80237): Can't acquire lockfile (-154) - waiting 35s
16:51:46 (80237): Can't acquire lockfile (-154) - exiting
17:01:50 (80544): Can't acquire lockfile (-154) - waiting 35s
17:02:26 (80544): Can't acquire lockfile (-154) - exiting
17:12:32 (80990): Can't acquire lockfile (-154) - waiting 35s
17:13:07 (80990): Can't acquire lockfile (-154) - exiting
17:23:15 (81432): Can't acquire lockfile (-154) - waiting 35s
17:23:51 (81432): Can't acquire lockfile (-154) - exiting
17:33:52 (81869): Can't acquire lockfile (-154) - waiting 35s
17:34:27 (81869): Can't acquire lockfile (-154) - exiting
17:44:29 (82442): Can't acquire lockfile (-154) - waiting 35s
17:45:04 (82442): Can't acquire lockfile (-154) - exiting
17:55:08 (83072): Can't acquire lockfile (-154) - waiting 35s
17:55:44 (83072): Can't acquire lockfile (-154) - exiting
18:01:09 (83369): Can't acquire lockfile (-154) - waiting 35s
18:01:44 (83369): Can't acquire lockfile (-154) - exiting
18:11:49 (83625): Can't acquire lockfile (-154) - waiting 35s
18:12:24 (83625): Can't acquire lockfile (-154) - exiting
18:22:27 (83879): Can't acquire lockfile (-154) - waiting 35s
18:23:02 (83879): Can't acquire lockfile (-154) - exiting
18:33:08 (84134): Can't acquire lockfile (-154) - waiting 35s
18:33:43 (84134): Can't acquire lockfile (-154) - exiting
18:43:49 (84390): Can't acquire lockfile (-154) - waiting 35s
18:44:24 (84390): Can't acquire lockfile (-154) - exiting
18:54:30 (84657): Can't acquire lockfile (-154) - waiting 35s
18:55:06 (84657): Can't acquire lockfile (-154) - exiting
19:05:14 (85068): Can't acquire lockfile (-154) - waiting 35s
19:05:49 (85068): Can't acquire lockfile (-154) - exiting
19:15:52 (85498): Can't acquire lockfile (-154) - waiting 35s
19:16:28 (85498): Can't acquire lockfile (-154) - exiting
19:26:36 (85934): Can't acquire lockfile (-154) - waiting 35s
19:27:11 (85934): Can't acquire lockfile (-154) - exiting
19:37:15 (86511): Can't acquire lockfile (-154) - waiting 35s
19:37:50 (86511): Can't acquire lockfile (-154) - exiting
19:47:51 (87132): Can't acquire lockfile (-154) - waiting 35s
19:48:26 (87132): Can't acquire lockfile (-154) - exiting
19:58:32 (87743): Can't acquire lockfile (-154) - waiting 35s
19:59:07 (87743): Can't acquire lockfile (-154) - exiting
20:09:02 (88343): Can't acquire lockfile (-154) - waiting 35s
20:09:37 (88343): Can't acquire lockfile (-154) - exiting
20:19:43 (88651): Can't acquire lockfile (-154) - waiting 35s
20:20:18 (88651): Can't acquire lockfile (-154) - exiting
20:30:24 (88964): Can't acquire lockfile (-154) - waiting 35s
20:30:59 (88964): Can't acquire lockfile (-154) - exiting
20:41:05 (89323): Can't acquire lockfile (-154) - waiting 35s
20:41:40 (89323): Can't acquire lockfile (-154) - exiting
20:51:48 (89676): Can't acquire lockfile (-154) - waiting 35s
20:52:23 (89676): Can't acquire lockfile (-154) - exiting
21:03:50 (89987): Can't acquire lockfile (-154) - waiting 35s
21:04:25 (89987): Can't acquire lockfile (-154) - exiting
21:14:29 (90251): Can't acquire lockfile (-154) - waiting 35s
21:15:04 (90251): Can't acquire lockfile (-154) - exiting
21:25:05 (90682): Can't acquire lockfile (-154) - waiting 35s
21:25:40 (90682): Can't acquire lockfile (-154) - exiting
21:35:50 (91117): Can't acquire lockfile (-154) - waiting 35s
21:36:25 (91117): Can't acquire lockfile (-154) - exiting
21:46:34 (91735): Can't acquire lockfile (-154) - waiting 35s
21:47:09 (91735): Can't acquire lockfile (-154) - exiting
21:57:16 (92351): Can't acquire lockfile (-154) - waiting 35s
21:57:51 (92351): Can't acquire lockfile (-154) - exiting

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: 
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  00399B82  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0039864B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0035FE0D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  003161B5  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00315917  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00352762  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0034FDFC  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002E34AC  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002722A3  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002BF9C0  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002EA01F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0000285B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00002789  Unknown               Unknown  Unknown
Unknown            00000008  Unknown               Unknown  Unknown
Model crash detected, will try to restart...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  00399B82  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0039864B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0035FE0D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  003161B5  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00315917  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00352762  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0034FDFC  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002E34AC  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002722A3  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002BF9C0  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  002EA01F  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0000285B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00002789  Unknown               Unknown  Unknown
Unknown            00000008  Unknown               Unknown  Unknown
Model crash detected, will try to restart...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3pm2_e94w_1991_10_008714883/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadam3pm2_um_7.03  00399B82  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0039864B  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  0035FE0D  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  003161B5  Unknown               Unknown  Unknown
hadam3pm2_um_7.03  00315917  error:  zipfile probably corrupt (segmentation violation)
hadam3pm2_7.03_i686-apple-darwin(31914,0xac7522c0) malloc: *** error for object 0x400000: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
22:01:42 (92565): Can't acquire lockfile (-154) - waiting 35s
22:02:17 (92565): Can't acquire lockfile (-154) - exiting
22:12:21 (92632): Can't acquire lockfile (-154) - waiting 35s
22:12:56 (92632): Can't acquire lockfile (-154) - exiting
22:23:08 (92697): Can't acquire lockfile (-154) - waiting 35s
22:23:43 (92697): Can't acquire lockfile (-154) - exiting
22:33:54 (92765): Can't acquire lockfile (-154) - waiting 35s
22:34:29 (92765): Can't acquire lockfile (-154) - exiting
22:44:35 (92836): Can't acquire lockfile (-154) - waiting 35s
22:45:10 (92836): Can't acquire lockfile (-154) - exiting
22:55:21 (93047): Can't acquire lockfile (-154) - waiting 35s
22:55:56 (93047): Can't acquire lockfile (-154) - exiting
23:06:01 (93306): Can't acquire lockfile (-154) - waiting 35s
23:06:36 (93306): Can't acquire lockfile (-154) - exiting
23:16:38 (93559): Can't acquire lockfile (-154) - waiting 35s
23:17:13 (93559): Can't acquire lockfile (-154) - exiting
23:27:19 (93816): Can't acquire lockfile (-154) - waiting 35s
23:27:54 (93816): Can't acquire lockfile (-154) - exiting
23:38:03 (94158): Can't acquire lockfile (-154) - waiting 35s
23:38:38 (94158): Can't acquire lockfile (-154) - exiting
23:49:05 (94757): Can't acquire lockfile (-154) - waiting 35s
23:49:40 (94757): Can't acquire lockfile (-154) - exiting
00:07:21 (95873): Can't acquire lockfile (-154) - waiting 35s
00:07:56 (95873): Can't acquire lockfile (-154) - exiting
00:25:15 (97020): Can't acquire lockfile (-154) - waiting 35s
00:25:50 (97020): Can't acquire lockfile (-154) - exiting
00:35:53 (97696): Can't acquire lockfile (-154) - waiting 35s
00:36:28 (97696): Can't acquire lockfile (-154) - exiting
00:46:33 (98389): Can't acquire lockfile (-154) - waiting 35s
00:47:08 (98389): Can't acquire lockfile (-154) - exiting
00:57:12 (99071): Can't acquire lockfile (-154) - waiting 35s
00:57:47 (99071): Can't acquire lockfile (-154) - exiting
01:03:20 (99389): Can't acquire lockfile (-154) - waiting 35s
01:03:55 (99389): Can't acquire lockfile (-154) - exiting
01:15:20 (99717): Can't acquire lockfile (-154) - waiting 35s
01:15:55 (99717): Can't acquire lockfile (-154) - exiting
01:26:06 (99972): Can't acquire lockfile (-154) - waiting 35s
01:26:41 (99972): Can't acquire lockfile (-154) - exiting
01:36:47 (368): Can't acquire lockfile (-154) - waiting 35s
01:37:22 (368): Can't acquire lockfile (-154) - exiting
01:47:29 (632): Can't acquire lockfile (-154) - waiting 35s
01:48:04 (632): Can't acquire lockfile (-154) - exiting
01:58:12 (889): Can't acquire lockfile (-154) - waiting 35s
01:58:47 (889): Can't acquire lockfile (-154) - exiting
02:08:48 (1144): Can't acquire lockfile (-154) - waiting 35s
02:09:23 (1144): Can't acquire lockfile (-154) - exiting
02:19:25 (1413): Can't acquire lockfile (-154) - waiting 35s
02:20:00 (1413): Can't acquire lockfile (-154) - exiting
02:30:08 (1668): Can't acquire lockfile (-154) - waiting 35s
02:30:43 (1668): Can't acquire lockfile (-154) - exiting
02:40:49 (1940): Can't acquire lockfile (-154) - waiting 35s
02:41:24 (1940): Can't acquire lockfile (-154) - exiting
02:51:25 (2498): Can't acquire lockfile (-154) - waiting 35s
02:52:00 (2498): Can't acquire lockfile (-154) - exiting
03:02:02 (3120): Can't acquire lockfile (-154) - waiting 35s
03:02:37 (3120): Can't acquire lockfile (-154) - exiting
03:12:44 (3744): Can't acquire lockfile (-154) - waiting 35s
03:13:19 (3744): Can't acquire lockfile (-154) - exiting
03:23:21 (4423): Can't acquire lockfile (-154) - waiting 35s
03:23:56 (4423): Can't acquire lockfile (-154) - exiting
03:34:01 (5052): Can't acquire lockfile (-154) - waiting 35s
03:34:36 (5052): Can't acquire lockfile (-154) - exiting
03:44:43 (5680): Can't acquire lockfile (-154) - waiting 35s
03:45:18 (5680): Can't acquire lockfile (-154) - exiting
03:55:22 (6310): Can't acquire lockfile (-154) - waiting 35s
03:55:57 (6310): Can't acquire lockfile (-154) - exiting
04:05:20 (6843): Can't acquire lockfile (-154) - waiting 35s
04:05:55 (6843): Can't acquire lockfile (-154) - exiting
04:15:59 (7097): Can't acquire lockfile (-154) - waiting 35s
04:16:34 (7097): Can't acquire lockfile (-154) - exiting
04:26:38 (7518): Can't acquire lockfile (-154) - waiting 35s
04:27:13 (7518): Can't acquire lockfile (-154) - exiting
04:37:15 (7958): Can't acquire lockfile (-154) - waiting 35s
04:37:50 (7958): Can't acquire lockfile (-154) - exiting
04:48:00 (8419): Can't acquire lockfile (-154) - waiting 35s
04:48:35 (8419): Can't acquire lockfile (-154) - exiting
04:58:40 (8856): Can't acquire lockfile (-154) - waiting 35s
04:59:15 (8856): Can't acquire lockfile (-154) - exiting
05:02:29 (8998): Can't acquire lockfile (-154) - waiting 35s
05:03:04 (8998): Can't acquire lockfile (-154) - exiting
05:13:08 (9119): Can't acquire lockfile (-154) - waiting 35s
05:13:43 (9119): Can't acquire lockfile (-154) - exiting
05:23:49 (9235): Can't acquire lockfile (-154) - waiting 35s
05:24:24 (9235): Can't acquire lockfile (-154) - exiting
05:34:31 (9361): Can't acquire lockfile (-154) - waiting 35s
05:35:06 (9361): Can't acquire lockfile (-154) - exiting
05:45:14 (9488): Can't acquire lockfile (-154) - waiting 35s
05:45:49 (9488): Can't acquire lockfile (-154) - exiting
05:55:55 (9609): Can't acquire lockfile (-154) - waiting 35s
05:56:30 (9609): Can't acquire lockfile (-154) - exiting
06:06:42 (9704): Can't acquire lockfile (-154) - waiting 35s
06:07:17 (9704): Can't acquire lockfile (-154) - exiting
06:17:22 (9957): Can't acquire lockfile (-154) - waiting 35s
06:17:57 (9957): Can't acquire lockfile (-154) - exiting
06:28:03 (10210): Can't acquire lockfile (-154) - waiting 35s
06:28:38 (10210): Can't acquire lockfile (-154) - exiting
06:56:19 (11334): Can't acquire lockfile (-154) - waiting 35s
06:56:54 (11334): Can't acquire lockfile (-154) - exiting
07:36:08 (13624): Can't acquire lockfile (-154) - waiting 35s
07:36:43 (13624): Can't acquire lockfile (-154) - exiting
07:57:48 (14935): Can't acquire lockfile (-154) - waiting 35s
07:58:23 (14935): Can't acquire lockfile (-154) - exiting
08:07:16 (15426): Can't acquire lockfile (-154) - waiting 35s
08:07:51 (15426): Can't acquire lockfile (-154) - exiting
08:17:56 (15683): Can't acquire lockfile (-154) - waiting 35s
08:18:31 (15683): Can't acquire lockfile (-154) - exiting
08:28:35 (15936): Can't acquire lockfile (-154) - waiting 35s
08:29:10 (15936): Can't acquire lockfile (-154) - exiting
08:39:13 (16258): Can't acquire lockfile (-154) - waiting 35s
08:39:48 (16258): Can't acquire lockfile (-154) - exiting
08:49:55 (16531): Can't acquire lockfile (-154) - waiting 35s
08:50:30 (16531): Can't acquire lockfile (-154) - exiting
08:59:27 (16747): Can't acquire lockfile (-154) - waiting 35s
09:00:02 (16747): Can't acquire lockfile (-154) - exiting
09:10:07 (17028): Can't acquire lockfile (-154) - waiting 35s
09:10:42 (17028): Can't acquire lockfile (-154) - exiting
09:20:48 (17345): Can't acquire lockfile (-154) - waiting 35s
09:21:23 (17345): Can't acquire lockfile (-154) - exiting
09:31:24 (17792): Can't acquire lockfile (-154) - waiting 35s
09:31:59 (17792): Can't acquire lockfile (-154) - exiting
09:42:01 (18233): Can't acquire lockfile (-154) - waiting 35s
09:42:36 (18233): Can't acquire lockfile (-154) - exiting
09:52:43 (18686): Can't acquire lockfile (-154) - waiting 35s
09:53:18 (18686): Can't acquire lockfile (-154) - exiting
10:03:24 (19152): Can't acquire lockfile (-154) - waiting 35s
10:03:59 (19152): Can't acquire lockfile (-154) - exiting
10:14:05 (19674): Can't acquire lockfile (-154) - waiting 35s
10:14:40 (19674): Can't acquire lockfile (-154) - exiting
10:24:43 (20175): Can't acquire lockfile (-154) - waiting 35s
10:25:18 (20175): Can't acquire lockfile (-154) - exiting
10:35:25 (20685): Can't acquire lockfile (-154) - waiting 35s
10:36:00 (20685): Can't acquire lockfile (-154) - exiting
10:46:09 (21748): Can't acquire lockfile (-154) - waiting 35s
10:46:44 (21748): Can't acquire lockfile (-154) - exiting
10:56:50 (22898): Can't acquire lockfile (-154) - waiting 35s
10:57:25 (22898): Can't acquire lockfile (-154) - exiting
11:07:32 (23376): Can't acquire lockfile (-154) - waiting 35s
11:08:07 (23376): Can't acquire lockfile (-154) - exiting
11:13:49 (23601): Can't acquire lockfile (-154) - waiting 35s
11:14:24 (23601): Can't acquire lockfile (-154) - exiting
11:24:28 (24064): Can't acquire lockfile (-154) - waiting 35s
11:25:03 (24064): Can't acquire lockfile (-154) - exiting
11:35:09 (24529): Can't acquire lockfile (-154) - waiting 35s
11:35:44 (24529): Can't acquire lockfile (-154) - exiting
11:45:47 (24978): Can't acquire lockfile (-154) - waiting 35s
11:46:22 (24978): Can't acquire lockfile (-154) - exiting
11:56:23 (25419): Can't acquire lockfile (-154) - waiting 35s
11:56:58 (25419): Can't acquire lockfile (-154) - exiting
12:08:30 (25883): Can't acquire lockfile (-154) - waiting 35s
12:09:05 (25883): Can't acquire lockfile (-154) - exiting
12:19:08 (26163): Can't acquire lockfile (-154) - waiting 35s
12:19:43 (26163): Can't acquire lockfile (-154) - exiting
12:29:50 (26427): Can't acquire lockfile (-154) - waiting 35s
12:30:26 (26427): Can't acquire lockfile (-154) - exiting
12:40:31 (26683): Can't acquire lockfile (-154) - waiting 35s
12:41:06 (26683): Can't acquire lockfile (-154) - exiting
12:51:07 (27040): Can't acquire lockfile (-154) - waiting 35s
12:51:42 (27040): Can't acquire lockfile (-154) - exiting
13:01:44 (27301): Can't acquire lockfile (-154) - waiting 35s
13:02:19 (27301): Can't acquire lockfile (-154) - exiting
13:12:28 (27554): Can't acquire lockfile (-154) - waiting 35s
13:13:03 (27554): Can't acquire lockfile (-154) - exiting
13:17:59 (27674): Can't acquire lockfile (-154) - waiting 35s
13:18:34 (27674): Can't acquire lockfile (-154) - exiting
13:28:41 (27926): Can't acquire lockfile (-154) - waiting 35s
13:29:16 (27926): Can't acquire lockfile (-154) - exiting
13:39:18 (28183): Can't acquire lockfile (-154) - waiting 35s
13:39:53 (28183): Can't acquire lockfile (-154) - exiting
13:50:00 (28622): Can't acquire lockfile (-154) - waiting 35s
13:50:35 (28622): Can't acquire lockfile (-154) - exiting
14:00:42 (28880): Can't acquire lockfile (-154) - waiting 35s
14:01:17 (28880): Can't acquire lockfile (-154) - exiting
14:11:25 (29138): Can't acquire lockfile (-154) - waiting 35s
14:12:00 (29138): Can't acquire lockfile (-154) - exiting
14:22:05 (29408): Can't acquire lockfile (-154) - waiting 35s
14:22:40 (29408): Can't acquire lockfile (-154) - exiting
14:32:41 (29858): Can't acquire lockfile (-154) - waiting 35s
14:33:17 (29858): Can't acquire lockfile (-154) - exiting
14:43:26 (30302): Can't acquire lockfile (-154) - waiting 35s
14:44:01 (30302): Can't acquire lockfile (-154) - exiting
14:54:07 (30765): Can't acquire lockfile (-154) - waiting 35s
14:54:42 (30765): Can't acquire lockfile (-154) - exiting
15:04:49 (31220): Can't acquire lockfile (-154) - waiting 35s
15:05:24 (31220): Can't acquire lockfile (-154) - exiting
15:15:34 (31709): Can't acquire lockfile (-154) - waiting 35s
15:16:09 (31709): Can't acquire lockfile (-154) - exiting
15:26:13 (32231): Can't acquire lockfile (-154) - waiting 35s
15:26:48 (32231): Can't acquire lockfile (-154) - exiting
15:36:55 (32776): Can't acquire lockfile (-154) - waiting 35s
15:37:30 (32776): Can't acquire lockfile (-154) - exiting
15:47:36 (33334): Can't acquire lockfile (-154) - waiting 35s
15:48:11 (33334): Can't acquire lockfile (-154) - exiting
15:58:21 (33938): Can't acquire lockfile (-154) - waiting 35s
15:58:56 (33938): Can't acquire lockfile (-154) - exiting
16:09:02 (34468): Can't acquire lockfile (-154) - waiting 35s
16:09:37 (34468): Can't acquire lockfile (-154) - exiting
16:13:29 (34663): Can't acquire lockfile (-154) - waiting 35s
16:14:04 (34663): Can't acquire lockfile (-154) - exiting
16:24:07 (34985): Can't acquire lockfile (-154) - waiting 35s
16:24:42 (34985): Can't acquire lockfile (-154) - exiting
16:34:48 (35266): Can't acquire lockfile (-154) - waiting 35s
16:35:23 (35266): Can't acquire lockfile (-154) - exiting
16:45:30 (35585): Can't acquire lockfile (-154) - waiting 35s
16:46:05 (35585): Can't acquire lockfile (-154) - exiting
16:56:12 (35960): Can't acquire lockfile (-154) - waiting 35s
16:56:47 (35960): Can't acquire lockfile (-154) - exiting
17:06:55 (36332): Can't acquire lockfile (-154) - waiting 35s
17:07:30 (36332): Can't acquire lockfile (-154) - exiting
17:17:31 (36624): Can't acquire lockfile (-154) - waiting 35s
17:18:06 (36624): Can't acquire lockfile (-154) - exiting
17:28:09 (36877): Can't acquire lockfile (-154) - waiting 35s
17:28:44 (36877): Can't acquire lockfile (-154) - exiting
17:38:53 (37145): Can't acquire lockfile (-154) - waiting 35s
17:39:28 (37145): Can't acquire lockfile (-154) - exiting
17:49:30 (37586): Can't acquire lockfile (-154) - waiting 35s
17:50:05 (37586): Can't acquire lockfile (-154) - exiting
17:56:27 (37841): Can't acquire lockfile (-154) - waiting 35s
17:57:02 (37841): Can't acquire lockfile (-154) - exiting
18:07:05 (38098): Can't acquire lockfile (-154) - waiting 35s
18:07:40 (38098): Can't acquire lockfile (-154) - exiting
18:17:51 (38361): Can't acquire lockfile (-154) - waiting 35s
18:18:26 (38361): Can't acquire lockfile (-154) - exiting
18:28:28 (38616): Can't acquire lockfile (-154) - waiting 35s
18:29:03 (38616): Can't acquire lockfile (-154) - exiting
18:39:08 (38993): Can't acquire lockfile (-154) - waiting 35s
18:39:43 (38993): Can't acquire lockfile (-154) - exiting
18:49:50 (39327): Can't acquire lockfile (-154) - waiting 35s
18:50:25 (39327): Can't acquire lockfile (-154) - exiting
19:00:33 (39776): Can't acquire lockfile (-154) - waiting 35s
19:01:08 (39776): Can't acquire lockfile (-154) - exiting
19:11:13 (40221): Can't acquire lockfile (-154) - waiting 35s
19:11:48 (40221): Can't acquire lockfile (-154) - exiting
19:21:57 (40726): Can't acquire lockfile (-154) - waiting 35s
19:22:32 (40726): Can't acquire lockfile (-154) - exiting
19:32:36 (41352): Can't acquire lockfile (-154) - waiting 35s
19:33:11 (41352): Can't acquire lockfile (-154) - exiting
19:43:13 (41976): Can't acquire lockfile (-154) - waiting 35s
19:43:48 (41976): Can't acquire lockfile (-154) - exiting
19:53:53 (42611): Can't acquire lockfile (-154) - waiting 35s
19:54:28 (42611): Can't acquire lockfile (-154) - exiting
19:56:36 (42724): Can't acquire lockfile (-154) - waiting 35s
19:57:11 (42724): Can't acquire lockfile (-154) - exiting
20:07:17 (43160): Can't acquire lockfile (-154) - waiting 35s
20:07:52 (43160): Can't acquire lockfile (-154) - exiting
20:17:57 (43656): Can't acquire lockfile (-154) - waiting 35s
20:18:32 (43656): Can't acquire lockfile (-154) - exiting
20:28:42 (44154): Can't acquire lockfile (-154) - waiting 35s
20:29:17 (44154): Can't acquire lockfile (-154) - exiting
20:39:26 (44656): Can't acquire lockfile (-154) - waiting 35s
20:40:01 (44656): Can't acquire lockfile (-154) - exiting
20:50:11 (45170): Can't acquire lockfile (-154) - waiting 35s
20:50:46 (45170): Can't acquire lockfile (-154) - exiting
21:00:54 (45664): Can't acquire lockfile (-154) - waiting 35s
21:01:29 (45664): Can't acquire lockfile (-154) - exiting
21:11:34 (46109): Can't acquire lockfile (-154) - waiting 35s
21:12:09 (46109): Can't acquire lockfile (-154) - exiting
21:22:16 (46553): Can't acquire lockfile (-154) - waiting 35s
21:22:51 (46553): Can't acquire lockfile (-154) - exiting
21:32:53 (47078): Can't acquire lockfile (-154) - waiting 35s
21:33:28 (47078): Can't acquire lockfile (-154) - exiting
21:43:30 (47702): Can't acquire lockfile (-154) - waiting 35s
21:44:05 (47702): Can't acquire lockfile (-154) - exiting
21:54:08 (48338): Can't acquire lockfile (-154) - waiting 35s
21:54:43 (48338): Can't acquire lockfile (-154) - exiting
22:04:44 (48971): Can't acquire lockfile (-154) - waiting 35s
22:05:19 (48971): Can't acquire lockfile (-154) - exiting
22:58:48 (52113): Can't acquire lockfile (-154) - waiting 35s
22:59:23 (52113): Can't acquire lockfile (-154) - exiting
23:09:26 (52378): Can't acquire lockfile (-154) - waiting 35s
23:10:01 (52378): Can't acquire lockfile (-154) - exiting
23:20:02 (52634): Can't acquire lockfile (-154) - waiting 35s
23:20:37 (52634): Can't acquire lockfile (-154) - exiting
23:30:42 (52889): Can't acquire lockfile (-154) - waiting 35s
23:31:17 (52889): Can't acquire lockfile (-154) - exiting
23:41:20 (53148): Can't acquire lockfile (-154) - waiting 35s
23:41:55 (53148): Can't acquire lockfile (-154) - exiting
23:52:01 (53411): Can't acquire lockfile (-154) - waiting 35s
23:52:36 (53411): Can't acquire lockfile (-154) - exiting
00:02:37 (53690): Can't acquire lockfile (-154) - waiting 35s
00:03:12 (53690): Can't acquire lockfile (-154) - exiting
00:13:14 (54000): Can't acquire lockfile (-154) - waiting 35s
00:13:49 (54000): Can't acquire lockfile (-154) - exiting
00:23:51 (54305): Can't acquire lockfile (-154) - waiting 35s
00:24:26 (54305): Can't acquire lockfile (-154) - exiting
00:34:30 (54617): Can't acquire lockfile (-154) - waiting 35s
00:35:05 (54617): Can't acquire lockfile (-154) - exiting
00:56:06 (55308): Can't acquire lockfile (-154) - waiting 35s
00:56:41 (55308): Can't acquire lockfile (-154) - exiting
01:36:14 (56309): Can't acquire lockfile (-154) - waiting 35s
01:36:49 (56309): Can't acquire lockfile (-154) - exiting
01:46:57 (56593): Can't acquire lockfile (-154) - waiting 35s
01:47:32 (56593): Can't acquire lockfile (-154) - exiting
01:57:37 (56852): Can't acquire lockfile (-154) - waiting 35s
01:58:12 (56852): Can't acquire lockfile (-154) - exiting
02:08:15 (57115): Can't acquire lockfile (-154) - waiting 35s
02:08:50 (57115): Can't acquire lockfile (-154) - exiting
02:18:51 (57370): Can't acquire lockfile (-154) - waiting 35s
02:19:26 (57370): Can't acquire lockfile (-154) - exiting
02:29:30 (57623): Can't acquire lockfile (-154) - waiting 35s
02:30:05 (57623): Can't acquire lockfile (-154) - exiting
02:40:09 (57882): Can't acquire lockfile (-154) - waiting 35s
02:40:44 (57882): Can't acquire lockfile (-154) - exiting
02:50:52 (58267): Can't acquire lockfile (-154) - waiting 35s
02:51:27 (58267): Can't acquire lockfile (-154) - exiting
02:56:14 (58458): Can't acquire lockfile (-154) - waiting 35s
02:56:49 (58458): Can't acquire lockfile (-154) - exiting
03:06:52 (58720): Can't acquire lockfile (-154) - waiting 35s
03:07:27 (58720): Can't acquire lockfile (-154) - exiting
03:17:28 (59040): Can't acquire lockfile (-154) - waiting 35s
03:18:03 (59040): Can't acquire lockfile (-154) - exiting
03:28:08 (59294): Can't acquire lockfile (-154) - waiting 35s
03:28:43 (59294): Can't acquire lockfile (-154) - exiting
03:38:48 (59556): Can't acquire lockfile (-154) - waiting 35s
03:39:23 (59556): Can't acquire lockfile (-154) - exiting
03:49:27 (59817): Can't acquire lockfile (-154) - waiting 35s
03:50:02 (59817): Can't acquire lockfile (-154) - exiting
04:00:05 (60079): Can't acquire lockfile (-154) - waiting 35s
04:00:40 (60079): Can't acquire lockfile (-154) - exiting
04:10:43 (60336): Can't acquire lockfile (-154) - waiting 35s
04:11:18 (60336): Can't acquire lockfile (-154) - exiting
04:21:20 (60589): Can't acquire lockfile (-154) - waiting 35s
04:21:55 (60589): Can't acquire lockfile (-154) - exiting
04:31:58 (60845): Can't acquire lockfile (-154) - waiting 35s
04:32:33 (60845): Can't acquire lockfile (-154) - exiting
04:42:42 (61176): Can't acquire lockfile (-154) - waiting 35s
04:43:18 (61176): Can't acquire lockfile (-154) - exiting
04:53:27 (61639): Can't acquire lockfile (-154) - waiting 35s
04:54:03 (61639): Can't acquire lockfile (-154) - exiting
04:56:09 (61718): Can't acquire lockfile (-154) - waiting 35s
04:56:44 (61718): Can't acquire lockfile (-154) - exiting
05:06:48 (61831): Can't acquire lockfile (-154) - waiting 35s
05:07:23 (61831): Can't acquire lockfile (-154) - exiting
05:17:32 (61955): Can't acquire lockfile (-154) - waiting 35s
05:18:07 (61955): Can't acquire lockfile (-154) - exiting
05:28:19 (62078): Can't acquire lockfile (-154) - waiting 35s
05:28:54 (62078): Can't acquire lockfile (-154) - exiting
05:39:06 (62205): Can't acquire lockfile (-154) - waiting 35s
05:39:41 (62205): Can't acquire lockfile (-154) - exiting
05:49:46 (62334): Can't acquire lockfile (-154) - waiting 35s
05:50:21 (62334): Can't acquire lockfile (-154) - exiting
06:00:26 (62459): Can't acquire lockfile (-154) - waiting 35s
06:01:01 (62459): Can't acquire lockfile (-154) - exiting
06:11:40 (62531): Can't acquire lockfile (-154) - waiting 35s
06:12:15 (62531): Can't acquire lockfile (-154) - exiting
06:22:24 (62605): Can't acquire lockfile (-154) - waiting 35s
06:22:59 (62605): Can't acquire lockfile (-154) - exiting
06:33:05 (62673): Can't acquire lockfile (-154) - waiting 35s
06:33:40 (62673): Can't acquire lockfile (-154) - exiting
06:43:47 (62742): Can't acquire lockfile (-154) - waiting 35s
06:44:22 (62742): Can't acquire lockfile (-154) - exiting
06:54:37 (62826): Can't acquire lockfile (-154) - waiting 35s
06:55:12 (62826): Can't acquire lockfile (-154) - exiting
Model exiting, can't make directory hadam3pm2_e94w_1991_10_008714883
06:55:12 (62826): called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
19 Jun 2014 23:39:41 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 66,308 253,957 3.8300
19 Jun 2014 20:08:50 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 63,428 242,970 3.8306
19 Jun 2014 16:38:04 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 60,548 231,909 3.8302
19 Jun 2014 13:12:23 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 57,668 220,419 3.8222
19 Jun 2014 09:01:27 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 54,788 208,867 3.8123
19 Jun 2014 05:20:47 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 51,908 197,306 3.8011
19 Jun 2014 02:00:10 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 49,028 186,723 3.8085
18 Jun 2014 22:34:30 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 46,148 176,029 3.8144
18 Jun 2014 19:13:51 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 43,268 165,432 3.8234
18 Jun 2014 15:48:07 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 40,388 154,449 3.8241
18 Jun 2014 12:01:08 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 37,508 143,119 3.8157
17 Jun 2014 14:26:49 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 20,228 77,111 3.8121
17 Jun 2014 11:06:17 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 17,348 66,556 3.8365
17 Jun 2014 07:50:51 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 14,468 55,702 3.8500
17 Jun 2014 04:25:20 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 11,588 43,986 3.7958
17 Jun 2014 01:04:57 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 8,708 32,585 3.7420
16 Jun 2014 22:04:40 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 5,828 21,974 3.7704
16 Jun 2014 18:59:22 1298068 16662133 hadam3pm2_e94w_1991_10_008714883_2 2,948 11,428 3.8765


©2024 climateprediction.net