broadcastCommand, a.k.a visualizing Kittens on a Cluster.

-
Picture1

When running your omegalib scripts in a cluster environment, you may need to execute some operation only on a specific node. That’s what the isMaster() and  getHostname() functions are for. Sometimes, after doing some node-specific work (especially on the master node) you want to notify all the other nodes you are done. A typical scenario is accessing some asset on the network and creating a copy on the local cache in the shared filesystem. As an example asset we’ll consider the canonical cute kitty picture.  You want the master node to perform the actual transfer, and notify all the other nodes of transfer completion, without blocking them. The nodes can then spam the displays with thousands of kitty-textured cubes (or spheres or cylinders. imagination is your limit).
The omegalib C++ API supported this for a while. In omegalib 3.7 the function is now available to python applications as well, through the broadcastCommand() call. BroadcastCommand accepts a python statement, and queues it for execution on all nodes of the cluster. Its typical use is at the end of an if(isMaster()) block, to notify nodes of work completion:
import urllib
        # On the master node, fetch an image somewhere on the internet
        if(isMaster())
                urllib.urlretrieve("http://www.digimouth.com/news/media/2011/09/google-logo.jpg", "local-image.jpg")
                # Call this when you are done
                broadcastCommand('onImageLoaded()')
        
        # This will be called on all nodes once the image is loaded. You can open the image here.
        # (Assuming use of a shared filesystem in your cluster installation)
        def onImageLoaded():
                img = loadImage("local-image.jpg")



For more information about the omegalib application control python API check this page!

Comments

Post a Comment

Popular posts from this blog

Parallel Gaussian Elimination Using MPI

-
Image
In this project, we had to implement a parallel solver for linear equation systems, using a technique known as Gaussian elimination (GE). As with many other algorithms for solving linear equation systems, GE is performed on the matrix representation of the system, Ax = b, where A is the coefficient matrix and b is the vector of known values. GE works by applying a set of elementary row operations (swapping rows, multiplying a row by a non-zero number, adding a multiple of one row to another) in order to turn the coefficient matrix into upper triangular form. The algorithm has been implemented in C, using the Message Passing Interface (MPI) API. The parallel GE program measured execution times for its four main components: data distribution, Forward Elimination , Pivot row identification, Back substitution. Read the full report here
Read more

Parallel Beam Tracing and Visualization of 200 Million Sonar Points

-
Image
For the final project in the UIC Spring 2013 Parallel Processing Class, I worked with a classmate to optimize our current implementation of a sonar beam tracer used for  The NASA ENDURANCE Project . The beam tracer is used to process the raw data collected by the ENDURANCE AUV in Lake Bonney, Antarctica, and correct sound beam paths using information about the water chemistry. The data is clustered and noise filtered to generate a final 3D Point cloud that can be visualized in our CAVE2 system. The ENDURANCE Sonar data in CAVE2. Photo(c) Lance Long. A lot of corrections and adjustments need to be applied to this data before the final results are acceptable. Some of the corrections derive from the sound speed estimation through the water column, AUV navigation corrections, water level changes and noise filtering thresholds. All this parameters influence the final result, and ideally researchers can tweak them and see the result of their actions in real time. Given the size of
Read more

Flickering 2D transform elements in Chrome: a simple fix

-
This quick post is to point out a fix to an issue I had to deal with a few weeks ago and was caused by a Chrome update. Components with absolute positioning and a 2D CSS transform started flickering when placed inside a scrollable component. A brief search online revealed several people with similar problems, but the suggested fixes were all very different. I tested several of them (including obvious changes to the z order) before finding the one that actually did the trick. What eventually worked for me was adding an empty 3D transform to the same elements with 2D transforms: transform:translate3d(0,0,0); I guess the reason why this works is that it forces elements to be drawn via a different rendering / compositing path than standard 2D elements.
Read more