集中力を欠くなぁ。
学生さんと1on1ミーティング。進捗いい感じ。
BGM代わりにYouTube開いているんだけど、なぜかおすすめされたつまようじ造構造物耐震コンテスト2010（崇城大学）が面白かった。こういうコンテスト系、やってみたいなぁ。
研究室サーバの調子が悪い。
$ mpirun -np 4 ./a.out
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 1178
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 1313
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 2331
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 3148
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 3180
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file server/pmix_server.c at line 2170
[name.of.server:202006] PMIX ERROR: OUT-OF-RESOURCE in file client/pmix_client.c at line 223
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 1178
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 1313
[name.of.server:202006] OPAL ERROR: Error in file pmix2x_client.c at line 112
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 2331
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 3148
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 3180
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file server/pmix_server.c at line 2170
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 1178
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 1313
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 2331
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 3148
[name.of.server:202008] PMIX ERROR: OUT-OF-RESOURCE in file client/pmix_client.c at line 223
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 3180
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file server/pmix_server.c at line 2170
[name.of.server:202008] OPAL ERROR: Error in file pmix2x_client.c at line 112
[name.of.server:202007] PMIX ERROR: OUT-OF-RESOURCE in file client/pmix_client.c at line 223
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[name.of.server:202006] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
[name.of.server:202007] OPAL ERROR: Error in file pmix2x_client.c at line 112
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[name.of.server:202008] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 1178
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 1313
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 2331
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 3148
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 3180
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[name.of.server:202001] PMIX ERROR: OUT-OF-RESOURCE in file server/pmix_server.c at line 2170
[name.of.server:202007] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
[name.of.server:202009] PMIX ERROR: OUT-OF-RESOURCE in file client/pmix_client.c at line 223
[name.of.server:202009] OPAL ERROR: Error in file pmix2x_client.c at line 112
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[name.of.server:202009] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
--------------------------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:
Process name: [[46247,1],0]
Exit code: 1
--------------------------------------------------------------------------
[watanabe@watanabe-login ~]$ mpirun -np 2 ./a.out
[name.of.server:202103] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 1178
[name.of.server:202103] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 1313
[name.of.server:202103] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 2331
[name.of.server:202103] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 3148
[name.of.server:202103] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 3180
[name.of.server:202103] PMIX ERROR: OUT-OF-RESOURCE in file server/pmix_server.c at line 2170
[name.of.server:202103] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 1178
[name.of.server:202103] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 1313
[name.of.server:202103] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 2331
[name.of.server:202103] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 3148
[name.of.server:202103] PMIX ERROR: OUT-OF-RESOURCE in file gds_dstore.c at line 3180
[name.of.server:202108] PMIX ERROR: OUT-OF-RESOURCE in file client/pmix_client.c at line 223
[name.of.server:202108] OPAL ERROR: Error in file pmix2x_client.c at line 112
[name.of.server:202103] PMIX ERROR: OUT-OF-RESOURCE in file server/pmix_server.c at line 2170
[name.of.server:202109] PMIX ERROR: OUT-OF-RESOURCE in file client/pmix_client.c at line 223
まず、エラーを出しているのがPMIXで、その実装はおそらくOpenPMIx、これはProcess Management Interfaceらしい。
ftraceを入れてみる。
sudo yum install -y trace-cmd
ディスク容量が足りずに失敗し、それでディスクが一杯になっていることに気が付く。これが原因か。
うーん、logwatchを入れるべきか。