Task 16573686

Name	hadam3pm2_e990_1991_10_008715031_0
Workunit	8861505
Created	19 Apr 2014, 9:32:01 UTC
Sent	19 Apr 2014, 10:22:53 UTC
Report deadline	1 Apr 2015, 15:42:53 UTC
Received	20 Jun 2014, 23:46:20 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1325086
Run time	19 days 16 hours 16 min 32 sec
CPU time	11 days 20 hours 37 min 15 sec
Validate state	Invalid
Credit	8,900.08
Device peak FLOPS	3.09 GFLOPS
Application version	UK Met Office HadAM3P (global only) with MOSES II landsurface scheme v7.03 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.27</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|2dec.nc oa.pc\|3jan.nc oa.pc\|3feb.nc oa.pc\|3mar.nc oa.pc\|3apr.nc oa.pc\|3may.nc oa.pc\|3jun.nc oa.pc\|3jul.nc oa.pc\|3aug.nc oa.pc\|3sep.nc oa.pc\|3oct.nc oa.pc\|3nov.nc oa.pe\|2dec.nc oa.pe\|3jan.nc oa.pe\|3feb.nc oa.pe\|3mar.nc oa.pe\|3apr.nc oa.pe\|3may.nc oa.pe\|3jun.nc oa.pe\|3jul.nc oa.pe\|3aug.nc oa.pe\|3sep.nc oa.pe\|3oct.nc oa.pe\|3nov.nc oa.pc\|3dec.nc oa.pc\|4jan.nc oa.pc\|4feb.nc oa.pc\|4mar.nc oa.pc\|4apr.nc oa.pc\|4may.nc oa.pc\|4jun.nc oa.pc\|4jul.nc oa.pc\|4aug.nc oa.pc\|4sep.nc oa.pc\|4oct.nc oa.pc\|4nov.nc oa.pe\|3dec.nc oa.pe\|4jan.nc oa.pe\|4feb.nc oa.pe\|4mar.nc oa.pe\|4apr.nc oa.pe\|4may.nc oa.pe\|4jun.nc oa.pe\|4jul.nc oa.pe\|4aug.nc oa.pe\|4sep.nc oa.pe\|4oct.nc oa.pe\|4nov.nc Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|6dec.nc oa.pc\|7jan.nc oa.pc\|7feb.nc oa.pc\|7mar.nc oa.pc\|7apr.nc oa.pc\|7may.nc oa.pc\|7jun.nc oa.pc\|7jul.nc oa.pc\|7aug.nc oa.pc\|7sep.nc oa.pc\|7oct.nc oa.pc\|7nov.nc oa.pe\|6dec.nc oa.pe\|7jan.nc oa.pe\|7feb.nc oa.pe\|7mar.nc oa.pe\|7apr.nc oa.pe\|7may.nc oa.pe\|7jun.nc oa.pe\|7jul.nc oa.pe\|7aug.nc oa.pe\|7sep.nc oa.pe\|7oct.nc oa.pe\|7nov.nc Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:29:44 (3583): No heartbeat from client for 30 sec - exiting 23:29:44 (3583): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:35:02 (15619): No heartbeat from client for 30 sec - exiting 00:35:02 (15619): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:42:54 (15697): No heartbeat from client for 30 sec - exiting 00:42:54 (15697): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:50:01 (15731): No heartbeat from client for 30 sec - exiting 00:50:01 (15731): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:52:58 (15758): No heartbeat from client for 30 sec - exiting 00:52:58 (15758): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:56:19 (15785): No heartbeat from client for 30 sec - exiting 00:56:19 (15785): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:59:02 (15812): No heartbeat from client for 30 sec - exiting 00:59:02 (15812): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:01:37 (15838): No heartbeat from client for 30 sec - exiting 01:01:37 (15838): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:15:15 (15868): No heartbeat from client for 30 sec - exiting 01:15:15 (15868): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:17:56 (15899): No heartbeat from client for 30 sec - exiting 01:17:56 (15899): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:20:22 (15925): No heartbeat from client for 30 sec - exiting 01:20:22 (15925): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:22:36 (15948): No heartbeat from client for 30 sec - exiting 01:22:36 (15948): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:27:21 (15972): No heartbeat from client for 30 sec - exiting 01:27:21 (15972): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:29:29 (15996): No heartbeat from client for 30 sec - exiting 01:29:29 (15996): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:34:14 (16019): No heartbeat from client for 30 sec - exiting 01:34:14 (16019): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:35:52 (16046): No heartbeat from client for 30 sec - exiting 01:35:52 (16046): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:39:14 (16076): No heartbeat from client for 30 sec - exiting 01:39:14 (16076): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:41:45 (16099): No heartbeat from client for 30 sec - exiting 01:41:45 (16099): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:52:14 (16123): No heartbeat from client for 30 sec - exiting 01:52:14 (16123): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:52:55 (16149): No heartbeat from client for 30 sec - exiting 01:52:55 (16149): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:54:21 (16172): No heartbeat from client for 30 sec - exiting 01:54:21 (16172): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:05:08 (16191): No heartbeat from client for 30 sec - exiting 02:05:08 (16191): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:07:00 (16224): No heartbeat from client for 30 sec - exiting 02:07:00 (16224): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:09:14 (16244): No heartbeat from client for 30 sec - exiting 02:09:14 (16244): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:16:53 (16267): No heartbeat from client for 30 sec - exiting 02:16:53 (16267): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:29:07 (16295): No heartbeat from client for 30 sec - exiting 02:29:07 (16295): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:33:29 (16321): No heartbeat from client for 30 sec - exiting 02:33:29 (16321): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:42:05 (16348): No heartbeat from client for 30 sec - exiting 02:42:05 (16348): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:50:09 (16383): No heartbeat from client for 30 sec - exiting 02:50:09 (16383): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:57:34 (16404): No heartbeat from client for 30 sec - exiting 02:57:34 (16404): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:00:09 (16428): No heartbeat from client for 30 sec - exiting 03:00:09 (16428): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:02:12 (16456): No heartbeat from client for 30 sec - exiting 03:02:12 (16456): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:05:05 (16478): No heartbeat from client for 30 sec - exiting 03:05:05 (16478): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:12:13 (16506): No heartbeat from client for 30 sec - exiting 03:12:13 (16506): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:14:54 (16530): No heartbeat from client for 30 sec - exiting 03:14:54 (16530): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:17:29 (16553): No heartbeat from client for 30 sec - exiting 03:17:29 (16553): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:20:51 (16580): No heartbeat from client for 30 sec - exiting 03:20:51 (16580): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:24:57 (16604): No heartbeat from client for 30 sec - exiting 03:24:57 (16604): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:27:32 (16627): No heartbeat from client for 30 sec - exiting 03:27:32 (16627): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:34:06 (16651): No heartbeat from client for 30 sec - exiting 03:34:06 (16651): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:36:44 (16678): No heartbeat from client for 30 sec - exiting 03:36:44 (16678): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:38:56 (16714): No heartbeat from client for 30 sec - exiting 03:38:56 (16714): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:41:58 (16737): No heartbeat from client for 30 sec - exiting 03:41:58 (16737): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:44:30 (16761): No heartbeat from client for 30 sec - exiting 03:44:30 (16761): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:47:06 (16784): No heartbeat from client for 30 sec - exiting 03:47:06 (16784): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:49:35 (16808): No heartbeat from client for 30 sec - exiting 03:49:35 (16808): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:01:27 (16831): No heartbeat from client for 30 sec - exiting 04:01:27 (16831): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:02:08 (16861): No heartbeat from client for 30 sec - exiting 04:02:08 (16861): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:05:19 (16881): No heartbeat from client for 30 sec - exiting 04:05:19 (16881): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:08:44 (16902): No heartbeat from client for 30 sec - exiting 04:08:44 (16902): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:09:45 (16922): No heartbeat from client for 30 sec - exiting 04:09:45 (16922): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:11:22 (16942): No heartbeat from client for 30 sec - exiting 04:11:22 (16942): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|8dec.nc oa.pc\|9jan.nc oa.pc\|9feb.nc oa.pc\|9mar.nc oa.pc\|9apr.nc oa.pc\|9may.nc oa.pc\|9jun.nc oa.pc\|9jul.nc oa.pc\|9aug.nc oa.pc\|9sep.nc oa.pc\|9oct.nc oa.pc\|9nov.nc oa.pe\|8dec.nc oa.pe\|9jan.nc oa.pe\|9feb.nc oa.pe\|9mar.nc oa.pe\|9apr.nc oa.pe\|9may.nc oa.pe\|9jun.nc oa.pe\|9jul.nc oa.pe\|9aug.nc oa.pe\|9sep.nc oa.pe\|9oct.nc oa.pe\|9nov.nc Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|9dec.nc oa.pc\|0jan.nc oa.pc\|0feb.nc oa.pc\|0mar.nc oa.pc\|0apr.nc oa.pc\|0may.nc oa.pc\|0jun.nc oa.pc\|0jul.nc oa.pc\|0aug.nc oa.pc\|0sep.nc oa.pc\|0oct.nc oa.pc\|0nov.nc oa.pe\|9dec.nc oa.pe\|0jan.nc oa.pe\|0feb.nc oa.pe\|0mar.nc oa.pe\|0apr.nc oa.pe\|0may.nc oa.pe\|0jun.nc oa.pe\|0jul.nc oa.pe\|0aug.nc oa.pe\|0sep.nc oa.pe\|0oct.nc oa.pe\|0nov.nc Model crashed: æM Model crashed: æM Model crashed: æM Model crashed: æM Model crashed: æM Model crashed: æM Sorry, too many model crashes! :-( 01:42:27 (3548): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Jun 2014 16:58:02	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	308,228	1,665,306	5.4028
19 Jun 2014 21:24:04	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	305,348	1,645,749	5.3897
19 Jun 2014 06:00:54	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	302,468	1,625,710	5.3748
18 Jun 2014 04:58:43	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	299,588	1,605,833	5.3601
17 Jun 2014 12:11:23	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	296,708	1,586,521	5.3471
16 Jun 2014 20:59:34	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	293,828	1,567,255	5.3339
16 Jun 2014 06:03:07	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	290,948	1,547,220	5.3179
15 Jun 2014 18:47:07	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	288,068	1,527,729	5.3034
15 Jun 2014 07:06:07	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	285,188	1,507,719	5.2868
14 Jun 2014 14:04:40	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	282,308	1,488,240	5.2717
14 Jun 2014 02:48:35	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	279,428	1,468,086	5.2539
13 Jun 2014 07:21:44	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	273,668	1,426,607	5.2129
12 Jun 2014 10:59:37	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	270,788	1,403,984	5.1848
11 Jun 2014 21:33:22	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	267,908	1,381,377	5.1562
11 Jun 2014 09:37:03	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	265,028	1,357,454	5.1219
10 Jun 2014 20:45:49	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	262,148	1,333,976	5.0886
10 Jun 2014 12:09:55	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	259,268	1,309,104	5.0492
10 Jun 2014 09:04:13	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	256,388	1,285,801	5.0151
10 Jun 2014 09:03:34	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	253,508	1,262,369	4.9796
10 Jun 2014 09:02:56	1325086	16573686	hadam3pm2_e990_1991_10_008715031_0	250,628	1,238,961	4.9434