I've been doing a lot of MPI development recently, and had a few MPI programs that crashed. After this happened, I kept receiving the following error on when using mpi-lam:
The selected RPI failed to initialize during MPI_INIT.
It turns out the problem is that the system is running out of inter-process communication resources, as they don't properly get freed unless MPI_Finalize() is called. You can see the current resources with the ipcs command, and then free them using ipcrm. Since removing each one is a pain, I made a quick perl script that generates a list of rm commands to free everything of a specific type that your user has allocated.
Here it is:
#!/usr/bin/perl
$ex = shift;
if( !$ex ) {
print "Usage: ipcrmAll.pl [-s -m -q]\n";
exit 1;
}
foreach(`ipcs $ex ` ) {
@data = split / /,$_;
if( $data[1] =~ m/\d\d*/ ) {
print "ipcrm $ex $data[1]\n";
}
}
To use it, just run ./ipcrmAll.pl -s to generate a free list for all semaphores.
- Louis's blog
- Login to post comments
