Task 16715788

Name	hadam3p_eu_n2zt_2013_1_008798377_0
Workunit	8944355
Created	7 Jul 2014, 15:07:38 UTC
Sent	4 Aug 2014, 13:53:10 UTC
Report deadline	17 Jul 2015, 19:13:10 UTC
Received	7 Aug 2014, 15:10:05 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	25 (0x00000019) Unknown error code
Computer ID	1298068
Run time	22 hours 30 min 2 sec
CPU time
Validate state	Invalid
Credit	399.11
Device peak FLOPS	3.03 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 25 (0x19, -231) </message> <stderr_txt> 21:50:07 (90708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:41:29 (5478): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:10:01 (8926): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:54:20 (32816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... zip I/O error: No space left on device zip error: Output file write failure (write error on zip file) 07:01:25 (37611): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:58:18 (40347): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... OPEN: File Creation Failed: No space left on device OPEN: Unable to Open File dataout/n2ztga.dal3ca0 for Read/Write forrtl: No space left on device forrtl: severe (38): error during write, unit 0, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadam3p_eu_n2zt_2013_1_008798377/dataout/xaakg.err Image PC Routine Line Source hadrm3p_eu_um_6.0 00365662 Unknown Unknown Unknown hadrm3p_eu_um_6.0 0036412B Unknown Unknown Unknown hadrm3p_eu_um_6.0 003289AD Unknown Unknown Unknown hadrm3p_eu_um_6.0 002DA7E5 Unknown Unknown Unknown hadrm3p_eu_um_6.0 002D9F47 Unknown Unknown Unknown hadrm3p_eu_um_6.0 0031F669 Unknown Unknown Unknown hadrm3p_eu_um_6.0 0031C1CF Unknown Unknown Unknown hadrm3p_eu_um_6.0 000212FB Unknown Unknown Unknown hadrm3p_eu_um_6.0 002B9D2C Unknown Unknown Unknown hadrm3p_eu_um_6.0 0000192B Unknown Unknown Unknown hadrm3p_eu_um_6.0 00001859 Unknown Unknown Unknown Unknown 00000003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=42667, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish SIGSEGV: segmentation violation Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=55636, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=56033, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=57812, iMonCtr=2 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=61196, selfPID=61197, iMonCtr=1 Signal 3 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=63272, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=63566, iMonCtr=2 Signal 3 received, exiting... Called boinc_finish Signal 3 received, exiting... Called boinc_finish Signal 3 received, exiting... Called boinc_finish Signal 3 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=67598, selfPID=67593, iMonCtr=1 Signal 3 received, exiting... Signal 3 received, exiting... Called boinc_finish Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=68294, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=69291, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=69698, iMonCtr=2 Signal 3 received, exiting... Called boinc_finish Signal 3 received, exiting... Called boinc_finish Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=74862, selfPID=74863, iMonCtr=1 Signal 3 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=75074, iMonCtr=2 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=76490, selfPID=76491, iMonCtr=1 Signal 3 received, exiting... Signal 3 received, exiting... Called boinc_finish Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=76758, selfPID=76753, iMonCtr=1 Signal 3 received, exiting... Called boinc_finish Regional Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Signal 3 received, exiting... Signal 3 received, exiting... Called boinc_finish Called boinc_finish Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=79829, selfPID=79830, iMonCtr=1 Signal 3 received, exiting... Signal 3 received, exiting... Called boinc_finish Called boinc_finish Signal 3 received, exiting... Signal 3 received, exiting... Called boinc_finish Called boinc_finish Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=82494, selfPID=82495, iMonCtr=1 Signal 3 received, exiting... Called boinc_finish Signal 3 received, exiting... Called boinc_finish Signal 3 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=83479, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=83831, iMonCtr=2 Signal 3 received, exiting... Signal 3 received, exiting... Called boinc_finish Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=85465, selfPID=85461, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=89111, selfPID=89106, iMonCtr=1 Model crash detected, will try to restart... Signal 3 received, exiting... Called boinc_finish Signal 3 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=91413, iMonCtr=2 Signal 3 received, exiting... Called boinc_finish Signal 3 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=92450, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=92526, selfPID=92522, iMonCtr=1 Signal 3 received, exiting... Signal 3 received, exiting... Called boinc_finish Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=93539, iMonCtr=2 Signal 3 received, exiting... Signal 3 received, exiting... Called boinc_finish Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=149, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4528, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4996, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5483, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6612, selfPID=6608, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9402, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10117, selfPID=10109, iMonCtr=1 Model crash detected, will try to restart... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10152, selfPID=10153, iMonCtr=1 Signal 3 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=15553, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=17419, selfPID=17414, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21365, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=22141, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28968, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30630, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34464, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=35682, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=37019, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=38901, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=39045, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=39513, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=39603, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=39783, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=41029, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=43654, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=44669, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=48400, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=50662, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=52310, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=53725, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=63055, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=63275, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=63853, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=64570, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=70653, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=73542, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=77298, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=84829, iMonCtr=2 Signal 3 received, exiting... Signal 3 received, exiting... Called boinc_finish Called boinc_finish Signal 3 received, exiting... Signal 3 received, exiting... Called boinc_finish Called boinc_finish Signal 3 received, exiting... Signal 3 received, exiting... Called boinc_finish Called boinc_finish Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Aug 2014 11:51:38	1298068	16715788	hadam3p_eu_n2zt_2013_1_008798377_0	23,136	61,367	2.6524
05 Aug 2014 02:11:01	1298068	16715788	hadam3p_eu_n2zt_2013_1_008798377_0	11,616	31,424	2.7052