Task 19115863

Name	hadam3p_pnw_xgxr_200512_12_010212095_0
Workunit	10212095
Created	1 Dec 2015, 9:57:29 UTC
Sent	1 Dec 2015, 18:06:41 UTC
Report deadline	12 Nov 2016, 23:26:41 UTC
Received	3 Jan 2016, 4:27:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1085470
Run time	4 days 8 hours 9 min 46 sec
CPU time	3 days 23 hours 26 min 57 sec
Validate state	Invalid
Credit	2,503.40
Device peak FLOPS	2.83 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v7.27 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... 20:28:30 (3980): No heartbeat from client for 30 sec - exiting 20:28:31 (3980): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:28:32 (3980): No heartbeat from client for 30 sec - exiting 20:28:32 (3980): timer handler: client dead, exiting 20:28:33 (3980): No heartbeat from client for 30 sec - exiting 20:28:33 (3980): timer handler: client dead, exiting 20:28:34 (3980): No heartbeat from client for 30 sec - exiting 20:28:34 (3980): timer handler: client dead, exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5992, selfPID=5992, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5260, selfPID=5260, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6760, selfPID=6760, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6052, selfPID=6052, iMonCtr=2 21:42:10 (7832): No heartbeat from client for 30 sec - exiting 21:42:12 (7832): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:42:13 (7832): No heartbeat from client for 30 sec - exiting 21:42:13 (7832): timer handler: client dead, exiting 21:42:14 (7832): No heartbeat from client for 30 sec - exiting 21:42:14 (7832): timer handler: client dead, exiting 21:42:16 (7832): No heartbeat from client for 30 sec - exiting 21:42:16 (7832): timer handler: client dead, exiting 21:42:17 (7832): No heartbeat from client for 30 sec - exiting 21:42:17 (7832): timer handler: client dead, exiting 21:42:18 (7832): No heartbeat from client for 30 sec - exiting 21:42:18 (7832): timer handler: client dead, exiting 22:04:33 (5284): No heartbeat from client for 30 sec - exiting 22:04:34 (5284): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:35 (5284): No heartbeat from client for 30 sec - exiting 22:04:36 (5284): timer handler: client dead, exiting Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5812, selfPID=5812, iMonCtr=2 22:04:37 (5284): No heartbeat from client for 30 sec - exiting 22:04:37 (5284): timer handler: client dead, exiting 22:04:38 (5284): No heartbeat from client for 30 sec - exiting 22:04:38 (5284): timer handler: client dead, exiting 22:04:39 (5284): No heartbeat from client for 30 sec - exiting 22:04:39 (5284): timer handler: client dead, exiting 22:04:40 (5284): No heartbeat from client for 30 sec - exiting 22:04:40 (5284): timer handler: client dead, exiting 22:04:41 (5284): No heartbeat from client for 30 sec - exiting 22:04:41 (5284): timer handler: client dead, exiting 22:04:42 (5284): No heartbeat from client for 30 sec - exiting 22:04:42 (5284): timer handler: client dead, exiting 22:29:32 (5476): No heartbeat from client for 30 sec - exiting 22:29:32 (5476): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:30:32 (7800): No heartbeat from client for 30 sec - exiting 22:30:32 (7800): timer handler: client dead, exiting 22:30:33 (7800): No heartbeat from client for 30 sec - exiting 22:30:33 (7800): timer handler: client dead, exiting 22:30:34 (7800): No heartbeat from client for 30 sec - exiting 22:30:34 (7800): timer handler: client dead, exiting 22:30:35 (7800): No heartbeat from client for 30 sec - exiting 22:30:35 (7800): timer handler: client dead, exiting 22:30:36 (7800): No heartbeat from client for 30 sec - exiting 22:30:36 (7800): timer handler: client dead, exiting 22:30:37 (7800): No heartbeat from client for 30 sec - exiting 22:30:37 (7800): timer handler: client dead, exiting 22:30:38 (7800): No heartbeat from client for 30 sec - exiting 22:30:38 (7800): timer handler: client dead, exiting 22:30:40 (7800): No heartbeat from client for 30 sec - exiting 22:30:40 (7800): timer handler: client dead, exiting 22:30:41 (7800): No heartbeat from client for 30 sec - exiting 22:30:41 (7800): timer handler: client dead, exiting 22:30:42 (7800): No heartbeat from client for 30 sec - exiting 22:30:42 (7800): timer handler: client dead, exiting 22:30:43 (7800): No heartbeat from client for 30 sec - exiting 22:30:43 (7800): timer handler: client dead, exiting 22:30:44 (7800): No heartbeat from client for 30 sec - exiting 22:30:44 (7800): timer handler: client dead, exiting 22:30:45 (7800): No heartbeat from client for 30 sec - exiting 22:30:45 (7800): timer handler: client dead, exiting 22:30:46 (7800): No heartbeat from client for 30 sec - exiting 22:30:46 (7800): timer handler: client dead, exiting 22:30:47 (7800): No heartbeat from client for 30 sec - exiting 22:30:47 (7800): timer handler: client dead, exiting 22:30:48 (7800): No heartbeat from client for 30 sec - exiting 22:30:48 (7800): timer handler: client dead, exiting 22:30:49 (7800): No heartbeat from client for 30 sec - exiting 22:30:49 (7800): timer handler: client dead, exiting 22:30:50 (7800): No heartbeat from client for 30 sec - exiting 22:30:50 (7800): timer handler: client dead, exiting 22:30:52 (7800): No heartbeat from client for 30 sec - exiting 22:30:52 (7800): timer handler: client dead, exiting 22:30:53 (7800): No heartbeat from client for 30 sec - exiting 22:30:53 (7800): timer handler: client dead, exiting 22:30:54 (7800): No heartbeat from client for 30 sec - exiting 22:30:54 (7800): timer handler: client dead, exiting 22:30:55 (7800): No heartbeat from client for 30 sec - exiting 22:30:55 (7800): timer handler: client dead, exiting 22:30:56 (7800): No heartbeat from client for 30 sec - exiting 22:30:56 (7800): timer handler: client dead, exiting 22:30:57 (7800): No heartbeat from client for 30 sec - exiting 22:30:57 (7800): timer handler: client dead, exiting 22:30:58 (7800): No heartbeat from client for 30 sec - exiting 22:30:58 (7800): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:11:03 (3080): No heartbeat from client for 30 sec - exiting 02:11:03 (3080): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 05:32:33 (7892): No heartbeat from client for 30 sec - exiting 05:32:33 (7892): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:46:41 (4156): No heartbeat from client for 30 sec - exiting 06:46:42 (4156): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:47:27 (2900): No heartbeat from client for 30 sec - exiting 07:47:27 (2900): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:47:28 (2900): No heartbeat from client for 30 sec - exiting 07:47:28 (2900): timer handler: client dead, exiting 07:47:29 (2900): No heartbeat from client for 30 sec - exiting 07:47:29 (2900): timer handler: client dead, exiting 07:47:30 (2900): No heartbeat from client for 30 sec - exiting 07:47:30 (2900): timer handler: client dead, exiting 08:50:04 (8004): No heartbeat from client for 30 sec - exiting 08:50:05 (8004): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:50:06 (8004): No heartbeat from client for 30 sec - exiting 08:50:06 (8004): timer handler: client dead, exiting 09:24:13 (2280): No heartbeat from client for 30 sec - exiting 09:24:13 (2280): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:26:38 (7124): No heartbeat from client for 30 sec - exiting 10:26:38 (7124): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:26:39 (7124): No heartbeat from client for 30 sec - exiting 10:26:39 (7124): timer handler: client dead, exiting 10:26:41 (7124): No heartbeat from client for 30 sec - exiting 10:26:41 (7124): timer handler: client dead, exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:13:28 (5240): No heartbeat from client for 30 sec - exiting 19:13:29 (5240): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:24:57 (5148): No heartbeat from client for 30 sec - exiting 21:33:41 (5148): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:33:42 (5148): No heartbeat from client for 30 sec - exiting 21:33:42 (5148): timer handler: client dead, exiting 21:33:43 (5148): No heartbeat from client for 30 sec - exiting 21:33:43 (5148): timer handler: client dead, exiting 21:33:44 (5148): No heartbeat from client for 30 sec - exiting 21:33:44 (5148): timer handler: client dead, exiting 21:33:45 (5148): No heartbeat from client for 30 sec - exiting 21:33:45 (5148): timer handler: client dead, exiting 21:33:46 (5148): No heartbeat from client for 30 sec - exiting 21:33:46 (5148): timer handler: client dead, exiting 21:33:47 (5148): No heartbeat from client for 30 sec - exiting 21:33:47 (5148): timer handler: client dead, exiting 21:33:48 (5148): No heartbeat from client for 30 sec - exiting 21:33:48 (5148): timer handler: client dead, exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:04:28 (2272): No heartbeat from client for 30 sec - exiting 04:04:28 (2272): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:04:29 (2272): No heartbeat from client for 30 sec - exiting 04:04:29 (2272): timer handler: client dead, exiting 05:07:58 (5288): No heartbeat from client for 30 sec - exiting 05:36:48 (5288): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:36:50 (5288): No heartbeat from client for 30 sec - exiting 05:36:50 (5288): timer handler: client dead, exiting 05:36:51 (5288): No heartbeat from client for 30 sec - exiting 05:36:51 (5288): timer handler: client dead, exiting 05:37:47 (2484): No heartbeat from client for 30 sec - exiting 05:37:47 (2484): timer handler: client dead, exiting 05:37:48 (2484): No heartbeat from client for 30 sec - exiting 05:37:48 (2484): timer handler: client dead, exiting 05:37:50 (2484): No heartbeat from client for 30 sec - exiting 05:37:50 (2484): timer handler: client dead, exiting 05:37:51 (2484): No heartbeat from client for 30 sec - exiting 05:37:51 (2484): timer handler: client dead, exiting 05:37:52 (2484): No heartbeat from client for 30 sec - exiting 05:37:52 (2484): timer handler: client dead, exiting 05:37:53 (2484): No heartbeat from client for 30 sec - exiting 05:37:53 (2484): timer handler: client dead, exiting 05:37:54 (2484): No heartbeat from client for 30 sec - exiting 05:37:54 (2484): timer handler: client dead, exiting 05:37:55 (2484): No heartbeat from client for 30 sec - exiting 05:37:55 (2484): timer handler: client dead, exiting 05:37:57 (2484): No heartbeat from client for 30 sec - exiting 05:37:57 (2484): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6404, selfPID=6404, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 07:33:10 (7668): No heartbeat from client for 30 sec - exiting 07:33:10 (7668): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:18:27 (6800): No heartbeat from client for 30 sec - exiting 11:21:50 (6800): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:21:52 (6800): No heartbeat from client for 30 sec - exiting 11:21:52 (6800): timer handler: client dead, exiting 11:33:29 (7748): No heartbeat from client for 30 sec - exiting 11:40:45 (7748): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:40:46 (7748): No heartbeat from client for 30 sec - exiting 11:40:46 (7748): timer handler: client dead, exiting 12:12:08 (7804): No heartbeat from client for 30 sec - exiting 12:12:08 (7804): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:26:44 (4844): No heartbeat from client for 30 sec - exiting 13:26:44 (4844): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/hadam3p_pnw_xgxr_200512_12_010212095/dataout\xgxrma.pck6aug 14:27:48 (2736): No heartbeat from client for 30 sec - exiting 14:27:48 (2736): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:44:43 (7632): No heartbeat from client for 30 sec - exiting 14:46:06 (7632): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:10:14 (6900): No heartbeat from client for 30 sec - exiting 16:10:14 (6900): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:10:15 (6900): No heartbeat from client for 30 sec - exiting 16:10:15 (6900): timer handler: client dead, exiting 16:10:16 (6900): No heartbeat from client for 30 sec - exiting 16:10:16 (6900): timer handler: client dead, exiting 16:10:17 (6900): No heartbeat from client for 30 sec - exiting 16:10:17 (6900): timer handler: client dead, exiting 16:10:18 (6900): No heartbeat from client for 30 sec - exiting 16:10:18 (6900): timer handler: client dead, exiting 16:10:19 (6900): No heartbeat from client for 30 sec - exiting 16:10:19 (6900): timer handler: client dead, exiting 16:10:21 (6900): No heartbeat from client for 30 sec - exiting 16:10:21 (6900): timer handler: client dead, exiting 16:10:22 (6900): No heartbeat from client for 30 sec - exiting 16:10:22 (6900): timer handler: client dead, exiting 16:10:23 (6900): No heartbeat from client for 30 sec - exiting 16:10:23 (6900): timer handler: client dead, exiting 16:10:24 (6900): No heartbeat from client for 30 sec - exiting 16:10:24 (6900): timer handler: client dead, exiting 16:10:25 (6900): No heartbeat from client for 30 sec - exiting 16:10:25 (6900): timer handler: client dead, exiting 17:23:12 (5952): No heartbeat from client for 30 sec - exiting 17:23:12 (5952): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:04:45 (5032): No heartbeat from client for 30 sec - exiting 22:04:45 (5032): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:56:16 (4744): No heartbeat from client for 30 sec - exiting 23:56:16 (4744): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1796, selfPID=1796, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Error converting file to netcdf: C:\ProgramData\BOINC/projects/climateprediction.net/hadam3p_pnw_xgxr_200512_12_010212095/dataout\xgxrma.pck6sep CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3008, selfPID=3008, iMonCtr=2 09:21:56 (5116): No heartbeat from client for 30 sec - exiting 09:21:56 (5116): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7836, selfPID=7836, iMonCtr=2 09:36:06 (6544): No heartbeat from client for 30 sec - exiting 09:36:06 (6544): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:37:01 (7416): No heartbeat from client for 30 sec - exiting 10:37:01 (7416): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:40:47 (3328): No heartbeat from client for 30 sec - exiting 16:40:47 (3328): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:47:06 (6852): No heartbeat from client for 30 sec - exiting 16:47:06 (6852): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:03:42 (1672): No heartbeat from client for 30 sec - exiting 20:03:42 (1672): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:25:56 (6516): No heartbeat from client for 30 sec - exiting 21:50:15 (6516): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:50:16 (6516): No heartbeat from client for 30 sec - exiting 21:50:16 (6516): timer handler: client dead, exiting 21:50:17 (6516): No heartbeat from client for 30 sec - exiting 21:50:17 (6516): timer handler: client dead, exiting 21:50:18 (6516): No heartbeat from client for 30 sec - exiting 21:50:18 (6516): timer handler: client dead, exiting Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakg.pipe_dummy 2048 Leaving CPDN_Main::Monitor... 21:51:09 (1232): No heartbeat from client for 30 sec - exiting 21:51:09 (1232): timer handler: client dead, exiting 21:51:10 (1232): No heartbeat from client for 30 sec - exiting 21:51:10 (1232): timer handler: client dead, exiting 21:51:11 (1232): No heartbeat from client for 30 sec - exiting 21:51:11 (1232): timer handler: client dead, exiting 21:51:12 (1232): No heartbeat from client for 30 sec - exiting 21:51:12 (1232): timer handler: client dead, exiting 21:51:14 (1232): No heartbeat from client for 30 sec - exiting 21:51:14 (1232): timer handler: client dead, exiting 21:51:15 (1232): No heartbeat from client for 30 sec - exiting 21:51:15 (1232): timer handler: client dead, exiting 21:51:16 (1232): No heartbeat from client for 30 sec - exiting 21:51:16 (1232): timer handler: client dead, exiting 21:51:16 (1232): called boinc_finish(0) </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_pnw_xgxr_200512_12_010212095_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_xgxr_200512_12_010212095_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Jan 2016 10:42:35	1085470	19115863	hadam3p_pnw_xgxr_200512_12_010212095_0	115,211	316,363	2.7459
01 Jan 2016 19:00:38	1085470	19115863	hadam3p_pnw_xgxr_200512_12_010212095_0	103,787	285,691	2.7527
01 Jan 2016 01:47:36	1085470	19115863	hadam3p_pnw_xgxr_200512_12_010212095_0	92,459	257,391	2.7838
31 Dec 2015 10:30:53	1085470	19115863	hadam3p_pnw_xgxr_200512_12_010212095_0	80,939	225,860	2.7905
30 Dec 2015 18:13:19	1085470	19115863	hadam3p_pnw_xgxr_200512_12_010212095_0	69,419	193,849	2.7924
30 Dec 2015 02:54:04	1085470	19115863	hadam3p_pnw_xgxr_200512_12_010212095_0	57,899	161,882	2.7959
29 Dec 2015 12:00:21	1085470	19115863	hadam3p_pnw_xgxr_200512_12_010212095_0	46,379	129,838	2.7995
28 Dec 2015 21:01:12	1085470	19115863	hadam3p_pnw_xgxr_200512_12_010212095_0	34,859	97,638	2.8009
28 Dec 2015 06:13:13	1085470	19115863	hadam3p_pnw_xgxr_200512_12_010212095_0	23,339	65,296	2.7977
27 Dec 2015 17:07:27	1085470	19115863	hadam3p_pnw_xgxr_200512_12_010212095_0	11,819	33,243	2.8127