Task 15773201

Name	hadcm3n_3inx_1980_40_008363904_0
Workunit	8514763
Created	10 May 2013, 18:29:53 UTC
Sent	10 May 2013, 18:46:21 UTC
Report deadline	10 Aug 2013, 2:13:32 UTC
Received	22 Jun 2013, 8:55:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1236877
Run time	9 days 6 hours 54 min 27 sec
CPU time	8 days 13 hours 55 min 3 sec
Validate state	Invalid
Credit	8,709.12
Device peak FLOPS	3.77 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> Das Gerät erkennt den Befehl nicht. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 21:04:21 (6160): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5880, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5880, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6668, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6668, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6668, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6668, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6100, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6100, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6100, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Jun 2013 18:53:25	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	725,760	772,658	1.0646
17 Jun 2013 18:42:53	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	699,840	745,144	1.0647
16 Jun 2013 14:03:05	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	673,920	717,530	1.0647
15 Jun 2013 19:01:58	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	648,000	689,540	1.0641
15 Jun 2013 10:55:38	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	622,080	661,628	1.0636
12 Jun 2013 19:15:30	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	596,160	633,955	1.0634
10 Jun 2013 17:21:57	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	570,240	606,231	1.0631
09 Jun 2013 14:47:00	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	544,320	578,600	1.0630
08 Jun 2013 18:41:04	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	518,400	550,805	1.0625
08 Jun 2013 10:59:33	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	492,480	523,126	1.0622
06 Jun 2013 17:19:18	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	466,560	495,460	1.0619
03 Jun 2013 18:43:20	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	440,640	467,859	1.0618
02 Jun 2013 14:20:43	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	414,720	440,165	1.0614
01 Jun 2013 20:13:26	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	388,800	412,163	1.0601
01 Jun 2013 12:30:48	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	362,880	384,363	1.0592
30 May 2013 18:16:26	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	336,960	356,687	1.0585
27 May 2013 19:57:19	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	311,040	328,910	1.0575
26 May 2013 17:43:03	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	285,120	301,176	1.0563
26 May 2013 10:20:37	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	259,200	273,565	1.0554
25 May 2013 15:39:21	1236877	15773201	hadcm3n_3inx_1980_40_008363904_0	233,280	245,852	1.0539