When you are running multiple scenarios with the same input files (often with the same mesh), you would like to run multiple scenarios job-parallel. To avoid race conditions, each run has to be in its own directory. However, this means you have to re-run the mesh partitioner for each scenario, which eats up time if your mesh is large. Another way around this is to run the partitioner once, and then copy the localized (PE*) directories to each scenario directory. However, this may eat a lot of memory, and in the case of large meshes, this may take a long time as well.
The best workaround is to use symbolic links, which is a trick that creates pseudo-files that are just points to the files without having to actually copy them. This is employed in Benjamin Pachev's ensemble framework, but this is in case you want to do it manually yourself.
Step-by-step guide
- Separate the common files and scenario files, by putting the common files into one folder and the scenario-specific files into their own individual folders.
- Use adcprep to partition the input files in the shared folder. It may ask for certain input files that are not shared, just use a dummy file in that case.
- Symbolically link the global files (executables, fort.14, etc ...) into each scenario folder.
- cd to the shared folder and use this fancy bash one-liner to link the PE folders into your scenario folder. This line uses the find utility's powerful exec feature to
- Exclude files (in this case fort.15) that will be written to from being linked. Modify this as needed. If hotstarting exclude the hotstart files.
- Find localized files and their corresponding PE folder
- Make an empty copy of the corresponding PE folder and link the localized file.
find . ! -name "fort.15" -type f -exec sh -c 'mkdir -p "/path-to-scenario/$(dirname "{}")" && ln -s "$(readlink -f "{}")" "/path-to-scenario/$(dirname "{}")/$(basename "{}")"' \;
Related articles