Task 17909649

Name	hadcm3n_8cp7_1980_40_008725878_4
Workunit	8871856
Created	8 Feb 2015, 5:25:25 UTC
Sent	8 Feb 2015, 5:26:32 UTC
Report deadline	10 May 2015, 12:53:43 UTC
Received	10 Feb 2015, 15:21:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1340074
Run time	2 days 6 hours 54 min 55 sec
CPU time	1 days 10 hours 1 min 21 sec
Validate state	Invalid
Credit	1,244.16
Device peak FLOPS	3.33 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.4.36</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... OPEN: File Creation Failed: No space left on device OPEN: Unable to Open File dataout/8cp7ka.dai3bc0 for Read/Write Model crashed: DUMPCTL : Fail to open output dump - may already exist tmp/pipe_dummy 2048 12:06:52 (39143): No heartbeat from core client for 30 sec - exiting 12:06:53 (39143): No heartbeat from core client for 30 sec - exiting 12:06:54 (39143): No heartbeat from core client for 30 sec - exiting 12:06:55 (39143): No heartbeat from core client for 30 sec - exiting 12:06:56 (39143): No heartbeat from core client for 30 sec - exiting 12:06:57 (39143): No heartbeat from core client for 30 sec - exiting 12:06:58 (39143): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... OPEN: File Creation Failed: No space left on device OPEN: Unable to Open File dataout/8cp7ko.dai41e0 for Read/Write Model crashed: DUMPCTL : Fail to open output dump - may already exist tmp/pipe_dummy 2048 forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_8cp7_1980_40_008725878/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... OPEN: File Creation Failed: No space left on device OPEN: Unable to Open File dataout/8cp7ka.dai4540 for Read/Write Model crashed: DUMPCTL : Fail to open output dump - may already exist tmp/pipe_dummy 2048 19:04:44 (56680): No heartbeat from core client for 30 sec - exiting 19:04:45 (56680): No heartbeat from core client for 30 sec - exiting 19:04:46 (56680): No heartbeat from core client for 30 sec - exiting 19:04:47 (56680): No heartbeat from core client for 30 sec - exiting 19:04:48 (56680): No heartbeat from core client for 30 sec - exiting 19:04:49 (56680): No heartbeat from core client for 30 sec - exiting 19:04:50 (56680): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:26:03 (66460): Can't acquire lockfile (-154) - waiting 35s Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:26:38 (66460): Can't acquire lockfile (-154) - exiting 06:26:45 (66473): Can't acquire lockfile (-154) - waiting 35s Suspended CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: Write Failed: Result too large BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: Write Failed: Result too large BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: Write Failed: Result too large BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: Write Failed: Result too large BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: Write Failed: Result too large BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Feb 2015 07:41:38	1340074	17909649	hadcm3n_8cp7_1980_40_008725878_4	103,680	110,428	1.0651
09 Feb 2015 18:29:47	1340074	17909649	hadcm3n_8cp7_1980_40_008725878_4	77,760	77,314	0.9943
09 Feb 2015 05:17:13	1340074	17909649	hadcm3n_8cp7_1980_40_008725878_4	51,840	38,633	0.7452
08 Feb 2015 17:31:00	1340074	17909649	hadcm3n_8cp7_1980_40_008725878_4	25,920	879	0.0339