Troubleshooting

Error loading shared libraries

Most things in Salvus are statically linked, except MPI, thus dynamic MPI libraries must be available. If these are not present, errors like the following happen:

Copy
error while loading shared libraries: libmpi.so.12: cannot
open shared object file: No such file or directory

On sites without central job submission system (thus usually single nodes), just let the Mondaic downloader also download MPI. It will but the MPI libraries in the lib/ folder and SalvusFlow will set the LD_LIBRARY_PATH accordingly when running Salvus.

For other sites, please load a suitable MPI module (see here for more details). In some cases the module system might not set the $LD_LIBRARY_PATH - then it has to be done manually in the SalvusFlow config file for the corresponding site.

Jupyter kernel dies

If you see the Jupyter notebook kernel dying when importing parts of Salvus make sure that you have a valid license and an active internet connection. If one of these is the issue, there should be an error message in the terminal that started Jupyter notebook. For technical reasons we cannot show this message in the Jupyter notebook.

Certificate errors while running the downloader

On old systems (or just having an old version of curl which might for example also happen when inside a python environment) the certificate might not be valid, so errors like this one might occur when attempting to run Mondaic's downloader:

Copy
curl: (77) error setting certificate verify locations:
  CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none

In these cases you could attempt to fix your curl installation or just manually do what the get.mondaic.com script is doing:

1. Download either https://get.mondaic.com/salvus_downloader_linux or https://get.mondaic.com/salvus_downloader_osx (depending on your operating system) by whatever means possible (wget, browser, copy on other machine and scp it over, ...).

2. Give it execution rights and run it:

   Copy
   $ chmod u+x salvus_downloader_*
   $ ./salvus_downloader_*

3. The rest is business as usual.

Illegal instructions

An error like

Copy
Illegal instruction

occurs if attempting to run a binary that has instructions that the current CPU cannot interpret. If you see this when running Salvus you likely downloaded the wrong binary for your system. Please see our documentation about instruction sets for more details.

Our downloader should auto-detect your CPU and choose the correct binaries. If that detection is faulty, please let us know.

Error during import of salvus_flow

The arrow package, which is used by SalvusFlow, currently undergoes a major update. This may lead to the following error message when trying to import salvus_flow:

Copy
AttributeError: module 'arrow.factory' has no attribute 'ArrowParseWarning'

Some older Salvus releases require a specific version of arrow. To check which version you have installed, run the following line in a terminal

Copy
python -c "import arrow; print(arrow.__version__)"

To upgrade your local installation of arrow, run

Copy
pip install --upgrade arrow==0.14.7

Salvus version 0.10.3 and above will also work with the arrow release (0.15.0).

Error in salvus-flow-config.toml

Copy
ValueError: '/some/path/salvus-flow-config.toml' does not appear to be a valid TOML file. Parsing it failed with: Unbalanced quotes (line XX column XX char XXX)

When manually editing the salvus-flow-config.toml the font encoding of your editor might give you ” instead of ", which leads to the error message above. This is especially likely to happen when you copy&paste information into the editor.

Make sure to use the correct quotes everywhere, and the error message will disappear.

MPI gethostbyname failed

Salvus fails with an error like the following:

Copy
Fatal error in MPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(474)..............:
MPID_Init(190).....................: channel initialization failed
MPIDI_CH3_Init(89).................:
MPID_nem_init(320).................:
MPID_nem_tcp_init(173).............:
MPID_nem_tcp_get_business_card(420):
MPID_nem_tcp_init(379).............: gethostbyname failed, localhost (errno 1)

Please follow the steps described here: https://stackoverflow.com/questions/23112515/mpich2-gethostbyname-failed