Re: the clustering search

one promising one

• Simple, Thread-safe Approximate Nearest Neighbor (STANN) C++
  

the clustering search

some libraries for clustering and (in general) for machine learning are listed below

• SGI MLC++ http://www.sgi.com/tech/mlc/
• OpenCV MLL
• The Open Source C Clustering Library
• The Wikipedia-Clustering project
• a specialized K-means library from The University of Mariland

these and some others to be looked up.

MTP discussions

one optimization technique springs to mind:

we can break up the initial 2d array into lots of small 2d arrays and optimise each each separately. then we can stitch them back. this can probably give better results.

MTP discussions

in section 4.1 of the paper, the authors say they used a sparse variant of the levenberg marquardt algo. the levmar site (http://www.ics.forth.gr/~lourakis/levmar/) you found earlier is exactly that. i don't think gsl wala has sparse matrix support. so will switch to levmar.

para3dhwt

Its been a snail's pace thanks to arbit errors and the debugging involved. We have adopted the modular approach making separate files for all the functions involved, keeping the related ones in the same file. What it has cost us at a speed of around 0.25x for just 3dhwt is a time of around 15 seconds for a 100 frame chunk. Thats at least better than the 38 minutes it once took! I seriously wish this gives a good compression over the usual stuff for all the time its taking... and this was just for order 1 wavelets! Well, we do have some optimizations already in mind; leaving apart the init 2 without X and gdm that we have come down to.

We have a few major hurdles to overcome though - one is the time factor, this time meaning the deadline for the BTP after which it is as good as dead; a second staring us in the face is the disappearance of nodes from the cluster owing to the upgradation of the lab that's going on... No we still won't be given the new ThinkSmarts to work on and no we (=>at least I) don't intend to appeal either for it'll mean wasting another weekend over installation of the components.

Installing OpenCV with ffmpeg

Making it work took some time and effort and repetition owing to the 4 nodes being individually separate entities. Here's what we and the ffmpeg and opencv tarballs had to go through

1. untar ffmpeg
2. ./configure --enable-shared --enable-swscale --enable-gpl
3. make
4. sudo make install
5. untar opencv
6. sudo apt-get install patch ;if not already installed
   
7. patch otherlibs/highgui/cvcap_ffmpeg.cpp ../nfs/opencv-1.0.0-cvcapffmpegundefinedsymbols.patch
8. #4 from the page http://www.rainsoft.de/projects/ffmpeg_opencv.html
9. su
   cd /usr/local/include/
   mkdir ffmpeg
   cp libavcodec/* ffmpeg/
   cp libavdevice/* ffmpeg/
   cp libavformat/* ffmpeg/
   cp libavutil/* ffmpeg/
   cp libswscale/* ffmpeg/
   exit
10. change FFMPEGLIBS="-lavcodec -lavformat" to FFMPEGLIBS="-lavcodec -lavformat -lswscale" in configure
    
11. ./configure --enable-shared
12. make
13. sudo make install
14. sudo ldconfig

Sandy instead made a patch file for both step #8 and #10 so if anyone ever reads this and needs them, can contact him.

From the Horse's mouth: a copy of the MPI reference manual

I'm diving head first into MPI and related PPing since:
1. its fun and I always wanted to do it!
2. I need to do it for my summer project at IISc which will otherwise take months on the single/dual processor machines.
I was surprised when Hollow told me that you did not have the full manual for MPI (MIT press).
I am a bit aware of some of the problems you ran into while setting up the Beowulf cluster. I will try to get solutions for these since there are people here who are proficient at this sort of stuff.
In any case, I will be adding some notes which I feel are important from the reference manual under this tag. Also, I'll post some bioinformatics problems that can be done when the cluster is up and running on all its feet.