climateprediction.net home page
Posts by Dana Jacobsen

Posts by Dana Jacobsen

1) Questions and Answers : Unix/Linux : Backlog with project backoff (Message 41074)
Posted 18 Nov 2010 by Dana Jacobsen
Post:
The message was "Project file upload handler is missing".

As Thyme indicated, I had 'hnndler' instead of handler in 12 places in client_state.xml. Stopping boinc, editing the file, then restarting has resulted in uploads moving.

What's funny is I just read one of the threads about 'handlrr' yesterday, and had the same issue (with the same fix) a year ago.

Thanks again!
2) Questions and Answers : Unix/Linux : Backlog with project backoff (Message 41065)
Posted 17 Nov 2010 by Dana Jacobsen
Post:
I've got one machine that's building up a backlog of finished models that it seems unable to upload. The Transfers tab is full of 'Upload pending (project backoff: xx:xx:xx)'. It just got upgraded to Fedora 13 a week ago.

My other machines don't seem to be having any issues. Any ideas?
3) Questions and Answers : Unix/Linux : Graphics in Linux HadSM3 BOINC 64 bit Client enabled version (Message 39588)
Posted 19 Apr 2010 by Dana Jacobsen
Post:
On my 64-bit Fedora 12 machine I had to do \'yum install libXt.i686 libXmu.i686\' to get them to work.

The process I used on my two x64 Fedora machines was:

cd /var/lib/boinc/projects/climateprediction.net
ls *graph*
# pick one of them, I chose the one below
ldd hadsm3mh_graphics_6.03_i686-pc-linux-gnu
# see what libraries are missing -- they say \'not found\'
# my Fedora 11 machine was missing libXmu
yum search libXmu
# For this Fedora 11 machine it is libXmu.i586
yum install libXmu.i586

Hooray, I have graphics on the Linux machines now.


I used a similar process to see why CPDN tasks failed on my Fedora 12 i920 machine. They seem to use quite a few 32-bit libraries. For instance I was happily running the HADSM3 models, but when the HADAM3P models came out they\'d instantly fail on this machine. Turns out I had to install libstdc++-4.4.2-20.fc12.i686 for them to work, which was not required for the other models.
4) Message boards : Number crunching : Wishlist: Add CUDA support (Message 39555)
Posted 15 Apr 2010 by Dana Jacobsen
Post:
Applications that aren\'t embarassingly parallel can still be sped up quite nicely by CUDA. For example, 3D incompressible Navier-Stokes flow solvers regularly achieve on the order of 10x improvement over high end CPUs, and they have multiple dependent steps. Even on GPU clusters where there is internode communication between each step, one can get impressive results (>100x speedup over threaded CPU code).

I suspect a much larger issue is the sheer code size. A simple flow solver might be 4-10 thousand lines with maybe 2k lines of CUDA. There are quite a few reference benchmarks to test their quality. In contrast, the CPDN code is 1 million lines of Fortran (Christensen, Aina, Stainforth 2005) and the results have gone through extensive and very lengthy testing. I doubt all 1M lines are core code, but it\'s pretty clear there is a lot going on.

One of the things we found, and is pretty well supported in the literature for flow solvers, is that porting individual kernels to the GPU gets you a relatively small overall speedup -- nothing in the 10x range. Besides the obvious Amdahl\'s Law bit (making 1% of your code infinitely fast results in a very small overall application improvement), the main problem is copying the data back and forth to the GPU. It turns out that\'s a major time sink. So while yes, one could take the small and compute intensive pressure Poisson solver and make it run at 100+ GFLOPS which is much faster than the CPU, it ends up not speeding up your entire app much. A few more kernels and you might hit 2-3x depending on the hot spots in the application. The big gain comes when you get all the data on the GPU and can stop the memory traffic. That means putting pretty much all your core application in CUDA.

Assume that\'s all possible -- that the core computation is more in the realm of 10-100k lines of code and is all ported over (getting CUDA to run isn\'t too hard, but getting it to run fast is a fair bit of work). Now there is a huge validation task. It doesn\'t matter how fast it is if it isn\'t correct, after all. Unfortunately this is a pretty daunting job.


Anyway, I\'d love to see CPDN in CUDA also, but there are some good reasons why it\'s not happening. I suspect we\'ll have more luck waiting for the next generation of models to come out of a research group, with data-parallel operation written from the start, and some grad students doing the grunt work of validating them. Caveats being that I do CFD in CUDA but don\'t do climate modeling nor do I have any additional insight into CPDN other than the public papers and posts.
5) Questions and Answers : Unix/Linux : Not able to upload result (Message 38832)
Posted 2 Feb 2010 by Dana Jacobsen
Post:
For followup, the tasks were still stuck and some more tasks started jamming up. Upon examining client_state.xml, I noticed the url\'s for some tasks in client_state.xml had \'.../file_upload_hadller\' instead of \'.../file_upload_handler\'.

I stopped boinc-client, backed up, did a search/replace in client_state.xml, restarted boinc-client, and watched everything upload successfully.
6) Questions and Answers : Unix/Linux : Not able to upload result (Message 38796)
Posted 30 Jan 2010 by Dana Jacobsen
Post:
Did you ever find a solution for this? I\'ve got 5 results that keep giving the same \"Project file upload handler is missing\" error, and have refused to upload for a few days now. Other hadsmfub results have successfully gone through in the meantime, so generic networking doesn\'t seem to be the problem.




©2024 climateprediction.net