L3 and L4 data only Submitted, not Running

Hello there. I couldn’t get L2A data before, but when I did what wrote in the topic I sent the link below, finally I started to get L2A data.

However, as seen in the screenshot above, I only got L2A data. Currently, L3A, L3B, L4A data all appear as “Submitted”. It was added as an extra jobs because I edited the “first run time” part from the “dashboard” tab after it remained as “Submitted” for 1 week. And also I added manually job from the “custom jobs” tab, but still all of them are waiting as “Submitted” as in the screenshot below.

Also, it doesn’t work even if I issue the “SLURM” start command. Maybe the problem is status of SLURM:

How do I get them to “Running”?
Dear @cudroiu and @lnicola , I would be grateful if you or anyone else could help. Thanks.

Hello,

Indeed, the issue seems to be with SLURM.
You could try re-installing SLURM using the script below.
To do that you could:

  • Go to “System overview” tab and pause all the jobs that you already submitted. This is to resume them after that otherwise you might be forced to launch them again.
  • Unzip and copy the scrip in the archive attached in the “install_scripts” directory from your Sen2Agri installation package.
    install_slurm_only.zip (3.1 KB)
  • Run :

sudo ./install_slurm_only.sh

  • Check if you notice any issues during reinstalling SLURM and if is the case, please copy and paste them here.
  • You can check also if slurm is running OK by doing:

sudo su -l sen2agri-service
srun ls -al

  • If everything is OK, you can go into the “System overview” tab and Resume all the jobs you previously paused.

Hope this helps.

Best regards,
Cosmin

Hello @cudroiu, thanks for reply.
I did exactly what you said, but as a result of the commands I wrote, I encountered some problems.
I am sending the results.

sudo ./install_slurm_only.sh

srun ls -al

[root@localhost ~]# sudo su -l sen2agri-service
[sen2agri-service@localhost ~]$ srun ls -al
srun: error: s_p_parse_file: unable to read “/etc/slurm/slurm.conf”: Permission denied
srun: fatal: Unable to process configuration file
[sen2agri-service@localhost ~]$

Screenshot_17

How can I resolve this issue with installing and using SLURM? Thanks.

Could you provide the results of ?

sudo ls -l /etc/slurm/
sudo cat /etc/slurm/slurm.conf

You can also try :

sudo chmod -R a+r /etc/slurm/

And then check if slurm starts OK (you can try a sudo systemctl restart slurm).

Cosmin

1 Like

Thanks Mr. @cudroiu
Result of sudo ls -l /etc/slurm/ & sudo cat /etc/slurm/slurm.conf :

sudo chmod -R a+r /etc/slurm/ → and after entering this command, SLURM is activated.
Screenshot_18

When I “Resume” jobs from the “system overview” tab, they became “Running”. I think it works now.

Good to hear that.
Seemed that your slurm.conf and slurmdbd.conf had somehow the wrong read permissions (please note the commands in the system are ran under the sen2agri-service user):

-rwxr-x— 1 root root 1752 Jul 23 10:45 slurm.conf
-rwxr-x—. 1 root root 738 Jul 23 10:45 slurmdbd.conf

Best Regards,
Cosmin