Remote TACC Execution Fails When Importing openpyxl (PermissionError in SimCenter Python Environment)

[PooriaMZ94]

Remote quoFEM/qWHALE runs on TACC fail with a PermissionError when a Python model imports the openpyxl package. The failure occurs before the model executes and appears to be caused by the restricted, read-only SimCenter Python environment used for backend execution. This prevents users from using any Python packages that depend on PIL (Pillow), typing_extensions, cached files, or modules requiring read/write access inside the environment.

[fmckenna]

The SimCenter tapis apps utilize their own Python installation for a number of reasons. If you need additional Python modules loaded when your script runs you will have to create your own such environment (the default python modules at TACC fail when mixing OpenSeesPy and dakota (or any MPI application) due to way OpenSeesPy was built. Your best solution is to create a tapis app that uses your own and point quoFEM to use that app in Preferences.

ps. While I could provde some option when the remote run box pops up, it would only be temporary as I need to revise how Python is handled when running on the HPC machines. Too many nodes running Python scripts with lots of mopule loads kills the file system performance for all and TACC would like an alternative approach in which the python env is stored in a single compressed file, copied to all compute nodes and uncompressed when a job runs.

[fmckenna]

ps. if needed I can help you with the tapis app.
https://simcenter.designsafe-ci.org/collaborate/office-hours/

[PooriaMZ94]

Thank you very much for your reply. I've requested the office hours.