Task 14804223

Name	hadam3p_pnw_bc4d_1989_1_008005498_0
Workunit	8160612
Created	20 Jun 2012, 15:30:16 UTC
Sent	20 Jun 2012, 15:30:33 UTC
Report deadline	2 Jun 2013, 20:50:33 UTC
Received	3 Jul 2012, 14:18:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741523 (0xC000012D) Unknown error code
Computer ID	694991
Run time	2 days 12 hours 42 min 46 sec
CPU time	57 min 18 sec
Validate state	Invalid
Credit	1,253.67
Device peak FLOPS	2.34 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> - exit code -1073741523 (0xc000012d) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error ferror - Unit 61 - Return code = 32 Model crashed: STWORK : I/O error - PP fixed length header tmp/xaakm.pipe_dummy 2048 Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 5 Called boinc_finish 22:17:24 (788): No heartbeat from core client for 30 sec - exiting 22:17:25 (788): No heartbeat from core client for 30 sec - exiting 22:18:26 (5692): Can't acquire lockfile (1450) - waiting 35s 22:19:01 (5692): Can't acquire lockfile (1450) - exiting 22:19:01 (5692): Error: Insufficient system resources exist to complete the requested service. (0x5aa) error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01001218 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01002207 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003236 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003237 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003254 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003255 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003274 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003275 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003276 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003277 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003278 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003279 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003280 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003281 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01003286 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01005207 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01005208 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01005222 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01005223 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/stasets/X01010206 error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/STASHmaster/STASHmaster_A error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/STASHmaster/STASHmaster_O error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/STASHmaster/STASHmaster_S error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/ancil/ctldata/STASHmaster/STASHmaster_W error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/datain/varlevs.xml checkdir error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/dataout unable to process dataout/. checkdir error: cannot create hadam3p_pnw_bc4d_1989_1_008005498/tmp unable to process tmp/. hadam3p_pnw_bc4d_1989_1_008005498/jobs/spec3a_lw_3_asol2c_hadcm3: write error (disk full?). Continue? (y/n/^C) warning: hadam3p_pnw_bc4d_1989_1_008005498/jobs/spec3a_lw_3_asol2c_hadcm3 is probably truncated SetEntriesInAcl Error 1114 Could not create shared memory region hadam3p_pnw_bc4d_1989_1_008005498, 24665344 Error in creating shared memory region! Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... 16:08:49 (3628): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Jun 2012 23:08:38	694991	14804223	hadam3p_pnw_bc4d_1989_1_008005498_0	57,696	170,356	2.9526
24 Jun 2012 15:57:32	694991	14804223	hadam3p_pnw_bc4d_1989_1_008005498_0	46,176	136,545	2.9571
23 Jun 2012 17:46:45	694991	14804223	hadam3p_pnw_bc4d_1989_1_008005498_0	34,656	102,026	2.9440
23 Jun 2012 04:08:14	694991	14804223	hadam3p_pnw_bc4d_1989_1_008005498_0	23,136	68,032	2.9405
21 Jun 2012 11:29:01	694991	14804223	hadam3p_pnw_bc4d_1989_1_008005498_0	11,616	34,331	2.9555