climateprediction.net home page
Corrupt data files being generated

Corrupt data files being generated

Message boards : Number crunching : Corrupt data files being generated
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile old_user36084
Avatar

Send message
Joined: 15 Jan 05
Posts: 31
Credit: 1,249,348
RAC: 0
Message 8135 - Posted: 30 Jan 2005, 13:52:32 UTC
Last modified: 30 Jan 2005, 13:53:48 UTC

I’m running BOINC_cli as a service so the only way I know of looking at the data files is using CPview2. I look at only the backup data set files and avoid the current files encase of conflict.

When I look at the *.pg files for one of my models it appears to have got the data in a muddle. The data titled Daily Minimum Surface (1.5m) Temperature shows precipitation rate data plus the Wind After Timestep contains lots of ticks across the screen. This could be due to a bug in CPview2 and not that the data sets are corrupt.

Furthermore the program has got in a muddle with calculating the cpu time see <a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/trickle.php?resultid=487942">results</a>. For the last tickle it calculated a CPU time of approximately 3 hours, but it actual took 10 hours like the previous tickles.

The other model that is running on this dual system appears to be ok. I have restored this boinc with a backup, replacing the complete boinc program file as discussed in <a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/forum_thread.php?id=1672">thread</a>. Also I’ve upgraded from 4.13 to 4.19.

Perhaps I should try restoring boinc to a previous backup and see if the problem occurs again.
ID: 8135 · Report as offensive     Reply Quote
Profile old_user36084
Avatar

Send message
Joined: 15 Jan 05
Posts: 31
Credit: 1,249,348
RAC: 0
Message 8161 - Posted: 30 Jan 2005, 19:01:43 UTC
Last modified: 30 Jan 2005, 19:03:13 UTC

I must add that this CPU time error occurred during the transition from phase 1 to 2; is this known to cause errors in cpu time calculation?

I've just discovered how to insert images into threads, so here are the CPview2 shots of what I think is muddled data:

<img src="http://mysite.wanadoo-members.co.uk/terahertzgap/images/1-picture1.jpg">

<img src="http://mysite.wanadoo-members.co.uk/terahertzgap/images/1-picture2.jpg">

<img src="http://mysite.wanadoo-members.co.uk/terahertzgap/images/1-picture3.jpg">

Do you think this data set looks corrupted?


Goto this <a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/forum_thread.php?id=140">thread</a> for the usage of html tags and embedded images in threads.
ID: 8161 · Report as offensive     Reply Quote
Profile Honza
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 390
Credit: 2,475,242
RAC: 0
Message 8164 - Posted: 30 Jan 2005, 19:21:22 UTC - in response to Message 8161.  

Hi juicedry,
for me, it's not easy to assess for those pictures. I'm used to look at near surface temperatures. Also, a scale needs to be present in order to assess extreme etc. So, with some limitation, screenshot looks good (e.g. not corrupted).

&gt; I must add that this CPU time error occurred during the transition from phase 1 to 2; is this known to cause errors in cpu time calculation?
&gt;
&gt; I've just discovered how to insert images into threads, so here are the CPview2 shots of what I think is muddled data:
&gt;
&gt; Do you think this data set looks corrupted?
ID: 8164 · Report as offensive     Reply Quote
Profile old_user36084
Avatar

Send message
Joined: 15 Jan 05
Posts: 31
Credit: 1,249,348
RAC: 0
Message 8227 - Posted: 31 Jan 2005, 10:04:40 UTC
Last modified: 1 Feb 2005, 14:37:18 UTC

I've found the near surface temperature plots and they appear ok, here is an example.

<img src="http://mysite.wanadoo-members.co.uk/terahertzgap/images/2-picture1.jpg">

<img src="http://mysite.wanadoo-members.co.uk/terahertzgap/images/2-picture2.jpg">

Perhaps the data set is fine and it's me getting anxious about wasting CPU cycles if it all goes tits up. It's me first model!

Even so I'm still bothered about the mis-calculation of the CPU time during the transition from phase 1 to 2. Is this a known bug? Should I be concerned?
ID: 8227 · Report as offensive     Reply Quote
Profile Honza
Volunteer moderator

Send message
Joined: 5 Aug 04
Posts: 390
Credit: 2,475,242
RAC: 0
Message 8234 - Posted: 31 Jan 2005, 11:32:09 UTC - in response to Message 8227.  

I think your model is going just fine.
If you are concerned about something wrong may happen, a regular backup of BOINC folder may be a good solution.
Some participants find it rewarding watching their models progress etc. That a positive feature of CPDN - one can watch meaningful and even itntermediate results of their effort.

&gt; I've found the near surface temperature plots and they appear ok, here is an example.
&gt; Perhaps the data set is fine and it's me getting anxious about wasting CPU cycles if it all goes tits up. It's me first model!
&gt; Even so I'm still bothered about the mis-calculation of the CPU time during the transition from phase 1 to 2. Is this a known bug? Should I be concerned?

ID: 8234 · Report as offensive     Reply Quote
old_user23880
Volunteer tester

Send message
Joined: 10 Oct 04
Posts: 223
Credit: 4,664
RAC: 0
Message 8303 - Posted: 1 Feb 2005, 2:48:11 UTC

Hi Juicedry

I've been looking at your graphics - can't compare them with mine because my computer is a boinc refusenik and I'm back on classic.

The first 2 maps - max and min temps in May 1813; do these show the average max/min temp ranges for the month, or the average actual max/min temps? If the latter, how does Trinidad have a lower max temp than the south pole?

Comparing the figures under the first 2 pics; how can the figures for the range (bottom figure, left-hand side under each map) be so very different for the max and minimum maps? The first of these figure says from zero to something, so why doesn't the second also say from zero to whatever else?

Looking at your 3rd pic - May 1813 total precipitation rate; how can the range go from -6.5? This is measured in kg per square metre, which must mean litres of water. How can the mid-Atlantic have a negative amount of precipitation?

The temps for 1810 look good, but I think other people should also look at your 1813 maps. My own failed boinc models had all gone seriously wrong by 1812. Maybe I just haven't understood what your first 2 maps with their figures are showing.
__________________________________________________

ID: 8303 · Report as offensive     Reply Quote
Profile old_user36084
Avatar

Send message
Joined: 15 Jan 05
Posts: 31
Credit: 1,249,348
RAC: 0
Message 8329 - Posted: 1 Feb 2005, 9:10:36 UTC

I've updated the plots in my previous posts with scales, and showing the same time period, May/Summer 1813, which was randomly chosen. When I look at the .pg data files from my other running models they appear to make sense. In this model the .pe data files are ok, but the .pg as shown are definitely not realisable, with temperatures shown as practical absolute zero, can't even do that in me 4 K cryostat!

I think that this may be a bug in CPview2 in reading the data set files. I believe the first picture (Daily Minimum Surface Temp) is actual showing Total Precipitation Rate, the numbers would then make sense along with the global location of the max and mins. Whereas the second plot is actual U Component of Wind After Timestep and not Maximum Surface Temp. Likewise with the third plot which I have deduced to be actual showing V Component of Wind After Timestep and not the CPview2 stated Total Precipitation Rate.

Perhaps the model is running fine and CPView2 is just displaying the data in a muddled fashion. I can zip these data set files and post them to anyone with a good eye at reading the data set files and see if they are ok.

Ian

PS Avoiding his PhD; much more exciting looking at CP plots of my running models.
ID: 8329 · Report as offensive     Reply Quote
old_user23880
Volunteer tester

Send message
Joined: 10 Oct 04
Posts: 223
Credit: 4,664
RAC: 0
Message 8396 - Posted: 2 Feb 2005, 3:10:07 UTC

Hi again Juicedry

Yes, what you say makes sense. The graph is the important thing to see whether it's working, and it should be more or less flat in phase 1, as yours is.

Have a look at the Ping Geophi thread further up this same messageboard. PeteB and Geophi are experts in the graphics.

What is your PhD about, if you don't mind me asking?
__________________________________________________

ID: 8396 · Report as offensive     Reply Quote
Profile old_user36084
Avatar

Send message
Joined: 15 Jan 05
Posts: 31
Credit: 1,249,348
RAC: 0
Message 8416 - Posted: 2 Feb 2005, 9:15:25 UTC

For brief PhD info check out my <a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/view_profile.php?userid=36084">profile</a>.
ID: 8416 · Report as offensive     Reply Quote
Profile old_user36084
Avatar

Send message
Joined: 15 Jan 05
Posts: 31
Credit: 1,249,348
RAC: 0
Message 8435 - Posted: 2 Feb 2005, 11:36:14 UTC

If Martin Sykes, creator of CPview2, is checking this forum please could I get a reply about the muddled data sets being shown by the program.

This has now started to occur in my other models on a different PC. Perhaps it is a known bug with CPview2, but as we can't access the other forum I can't find info.

Your help will be much appreciated.
ID: 8435 · Report as offensive     Reply Quote
old_user23880
Volunteer tester

Send message
Joined: 10 Oct 04
Posts: 223
Credit: 4,664
RAC: 0
Message 8506 - Posted: 3 Feb 2005, 1:46:58 UTC
Last modified: 3 Feb 2005, 2:51:53 UTC

As soon (or as late) as the classic forum is up again, you can send him, or them, a PM about the graphics. I can't see any way of sending PMs through the boinc board.

Or go to the cpdn home page, click on user of the week, find Martin at the bottom of the archive because he was the first UOTW, and send him an email.
__________________________________________________

ID: 8506 · Report as offensive     Reply Quote

Message boards : Number crunching : Corrupt data files being generated

©2024 climateprediction.net