climateprediction.net home page
Task 16016341

Task 16016341

Name hadcm3n_o1si_1980_40_008384091_3
Workunit 8534950
Created 14 Sep 2013, 13:52:43 UTC
Sent 14 Sep 2013, 14:00:22 UTC
Report deadline 14 Dec 2013, 21:27:33 UTC
Received 13 Nov 2013, 19:03:08 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1213041
Run time 16 days 3 hours 35 min 47 sec
CPU time 13 days 2 hours 58 min 29 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 2.41 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr 
<core_client_version>7.0.64</core_client_version>
<![CDATA[
<message>
The device does not recognize the command.
 (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5188, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6528, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5040, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:42:15 (5260): No heartbeat from core client for 30 sec - exiting
16:42:16 (5260): No heartbeat from core client for 30 sec - exiting
16:42:17 (5260): No heartbeat from core client for 30 sec - exiting
16:42:18 (5260): No heartbeat from core client for 30 sec - exiting
16:42:19 (5260): No heartbeat from core client for 30 sec - exiting
16:42:20 (5260): No heartbeat from core client for 30 sec - exiting
16:42:21 (5260): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:42:23 (5260): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5832, iMonCtr=1
Model crash detected, will try to restart...
16:52:43 (5924): No heartbeat from core client for 30 sec - exiting
16:52:44 (5924): No heartbeat from core client for 30 sec - exiting
16:52:45 (5924): No heartbeat from core client for 30 sec - exiting
16:52:46 (5924): No heartbeat from core client for 30 sec - exiting
16:52:47 (5924): No heartbeat from core client for 30 sec - exiting
16:52:48 (5924): No heartbeat from core client for 30 sec - exiting
16:52:49 (5924): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:52:50 (5924): No heartbeat from core client for 30 sec - exiting
16:26:16 (5148): No heartbeat from core client for 30 sec - exiting
16:26:17 (5148): No heartbeat from core client for 30 sec - exiting
16:26:18 (5148): No heartbeat from core client for 30 sec - exiting
16:26:19 (5148): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:35:27 (5540): No heartbeat from core client for 30 sec - exiting
16:35:28 (5540): No heartbeat from core client for 30 sec - exiting
16:35:29 (5540): No heartbeat from core client for 30 sec - exiting
16:35:30 (5540): No heartbeat from core client for 30 sec - exiting
16:35:31 (5540): No heartbeat from core client for 30 sec - exiting
16:35:32 (5540): No heartbeat from core client for 30 sec - exiting
16:35:33 (5540): No heartbeat from core client for 30 sec - exiting
16:35:34 (5540): No heartbeat from core client for 30 sec - exiting
16:35:35 (5540): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4148, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
09:53:01 (5548): No heartbeat from core client for 30 sec - exiting
09:53:02 (5548): No heartbeat from core client for 30 sec - exiting
09:53:03 (5548): No heartbeat from core client for 30 sec - exiting
09:53:04 (5548): No heartbeat from core client for 30 sec - exiting
09:53:05 (5548): No heartbeat from core client for 30 sec - exiting
09:53:06 (5548): No heartbeat from core client for 30 sec - exiting
09:53:07 (5548): No heartbeat from core client for 30 sec - exiting
09:53:09 (5548): No heartbeat from core client for 30 sec - exiting
09:53:10 (5548): No heartbeat from core client for 30 sec - exiting
09:53:11 (5548): No heartbeat from core client for 30 sec - exiting
09:53:12 (5548): No heartbeat from core client for 30 sec - exiting
09:53:13 (5548): No heartbeat from core client for 30 sec - exiting
09:53:14 (5548): No heartbeat from core client for 30 sec - exiting
09:53:15 (5548): No heartbeat from core client for 30 sec - exiting
09:53:16 (5548): No heartbeat from core client for 30 sec - exiting
09:53:17 (5548): No heartbeat from core client for 30 sec - exiting
09:53:18 (5548): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:55:16 (6020): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5624, iMonCtr=1
Model crash detected, will try to restart...
17:59:57 (5328): No heartbeat from core client for 30 sec - exiting
17:59:58 (5328): No heartbeat from core client for 30 sec - exiting
17:59:59 (5328): No heartbeat from core client for 30 sec - exiting
18:00:00 (5328): No heartbeat from core client for 30 sec - exiting
18:00:01 (5328): No heartbeat from core client for 30 sec - exiting
18:00:02 (5328): No heartbeat from core client for 30 sec - exiting
18:00:03 (5328): No heartbeat from core client for 30 sec - exiting
18:00:04 (5328): No heartbeat from core client for 30 sec - exiting
18:00:05 (5328): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:09:30 (5160): No heartbeat from core client for 30 sec - exiting
18:09:31 (5160): No heartbeat from core client for 30 sec - exiting
18:09:32 (5160): No heartbeat from core client for 30 sec - exiting
18:09:33 (5160): No heartbeat from core client for 30 sec - exiting
18:09:34 (5160): No heartbeat from core client for 30 sec - exiting
18:09:35 (5160): No heartbeat from core client for 30 sec - exiting
18:09:36 (5160): No heartbeat from core client for 30 sec - exiting
18:09:37 (5160): No heartbeat from core client for 30 sec - exiting
18:09:38 (5160): No heartbeat from core client for 30 sec - exiting
18:09:40 (5160): No heartbeat from core client for 30 sec - exiting
18:09:41 (5160): No heartbeat from core client for 30 sec - exiting
18:09:42 (5160): No heartbeat from core client for 30 sec - exiting
18:09:43 (5160): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4056, iMonCtr=1
Model crash detected, will try to restart...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4056, iMonCtr=1
Model crash detected, will try to restart...
12:38:21 (5336): No heartbeat from core client for 30 sec - exiting
12:38:22 (5336): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:43:58 (5292): No heartbeat from core client for 30 sec - exiting
18:43:59 (5292): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:48:41 (6124): No heartbeat from core client for 30 sec - exiting
17:48:42 (6124): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:48:43 (6124): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
18:44:40 (5304): No heartbeat from core client for 30 sec - exiting
18:44:42 (5304): No heartbeat from core client for 30 sec - exiting
18:44:43 (5304): No heartbeat from core client for 30 sec - exiting
18:44:44 (5304): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1644, iMonCtr=1
Model crash detected, will try to restart...
16:37:02 (1660): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:21:13 (6492): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:19:05 (3676): No heartbeat from core client for 30 sec - exiting
15:19:07 (3676): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:43:40 (5820): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:20:18 (2040): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
16:44:18 (5904): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...

zip error: Could not create output file (was replacing the original zip file)
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/atmos_restart.day after 11 attempts
17:16:06 (6712): No heartbeat from core client for 30 sec - exiting
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
17:16:07 (6712): No heartbeat from core client for 30 sec - exiting
17:16:08 (6712): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o1si_1980_40_008384091/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
13 Nov 2013 17:13:27 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 518,400 1,134,008 2.1875
10 Nov 2013 17:01:33 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 492,480 1,078,941 2.1908
08 Nov 2013 16:50:31 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 466,560 1,023,579 2.1939
06 Nov 2013 18:21:05 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 440,640 968,965 2.1990
04 Nov 2013 18:47:46 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 414,720 913,689 2.2031
02 Nov 2013 19:33:06 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 388,800 858,147 2.2072
27 Oct 2013 16:39:46 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 362,880 801,006 2.2074
24 Oct 2013 20:36:52 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 336,960 743,652 2.2069
20 Oct 2013 20:33:53 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 311,040 686,611 2.2075
18 Oct 2013 19:45:26 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 285,120 629,041 2.2062
14 Oct 2013 18:52:33 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 259,200 572,218 2.2076
12 Oct 2013 19:18:28 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 233,280 515,142 2.2083
07 Oct 2013 16:32:49 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 207,360 458,286 2.2101
05 Oct 2013 16:30:32 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 181,440 401,001 2.2101
03 Oct 2013 16:21:09 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 155,520 343,575 2.2092
28 Sep 2013 22:10:29 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 129,600 286,342 2.2094
23 Sep 2013 17:41:39 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 103,680 228,546 2.2043
21 Sep 2013 19:10:48 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 77,760 170,829 2.1969
20 Sep 2013 12:15:31 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 51,840 113,634 2.1920
15 Sep 2013 22:12:52 1213041 16016341 hadcm3n_o1si_1980_40_008384091_3 25,920 57,054 2.2012


©2024 climateprediction.net