Hello, I have a problem with L2A processor. I’ve installed Sen2Agri 2.0 from the beginning but MAJA does not process Sentinel2 properly. I have such a message in the logs “Unknown format found in output directory”. What can cause this problem? Attached, I am sending logs.
demmaccs.log (1.3 KB)
demmaccs_34UDE.log (4.5 KB)
Hello,
Could you try running the command :
/usr/share/sen2agri/sen2agri-demmaccs/demmaccs.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd --processes-number-dem 1 --processes-number-maccs 1 --gipp-dir /mnt/archive/gipp_maja --working-dir /mnt/archive/demmaccs_tmp/ --maccs-launcher /opt/maja/3.2.2/bin/maja --delete-temp True /mnt/archive/dwn_def/s2/default/ketrzynski_2019/S2A_MSIL1C_20190105T094401_N0207_R036_T34UDE_20190105T101012.SAFE /mnt/archive/maccs_def/ketrzynski_2019/l2a/S2A_MSIL2A_20190105T094401_N0207_R036_T34UDE_20190105T101012.SAFE/ --tiles-to-process 34UDE
under the user sen2agri-service (sudo su -l sen2agri-service) and check the error messages that are given by the demmaccs?
Please let me know.
Best regards,
Cosmin
Could you please do the following operations (as the previous test execution might have been “polluted” by other executions):
- Stop the sen2agri-demmaccs services executing:
sudo systemctl stop sen2agri-demmaccs sen2agri-demmaccs.timer
- Execute the command I specified above
- Extract the log and send it again to us
- Restart the sen2agri-demmaccs using:
sudo systemctl start sen2agri-demmaccs sen2agri-demmaccs.timer
Cosmin
Hello,
I did executed the command first as sen2agri user and then as root user. Under the root user processing have been completed but under sen2agri user there was an error. Maybe I should add someware additional permissions to the sen2agri user? Attached I’m sending the logs.
sen2agri_user_log.txt (52.1 KB)
root_user_log.txt (666.9 KB)
Hello,
As the MAJA is processing OK the products under the user root, it seems that you have some issues with the rights to the /mnt/archive.
In this case, you should:
- stop the sen2agri-demmaccs:
sudo systemctl stop sen2agri-demmaccs sen2agri-demmaccs.timer
- change recursively the access rights to the /mnt/archive to 777
sudo chmod -R 777 /mnt/archive
sudo chmod -R 777 /mnt/upload
- if the /mnt/archive/maccs_def and /mnt/archive/demmaccs_tmp are somehow symlinks to other locations, please ensure that these folders have also the right permissions for user sen2agri-service (recommended rights 777)
- optionally, you can remove the folder /mnt/archive/demmaccs_tmp
- reset the products that were marked as failed in the database:
sudo -u postgres psql sen2agri -c “delete from l1_tile_history”
sudo -u postgres psql sen2agri -c “update downloader_history set status_id = 2 where status_id = 6”
- start the sen2agri-demmaccs:
sudo systemctl start sen2agri-demmaccs sen2agri-demmaccs.timer
Best regards,
Cosmin
Hello,
Still the same problem. I did everything above. I will reinstall everything from the beginning. Could the problem be that the installation was done as root?
I finally managed MAJA to work. I’ve added sen2agri-service to sudoers in /etc/sudoers under the root user.
##Allow root to run any commands anywhere
root ALL=(ALL) ALL
sen2agri-service ALL=(ALL) ALL
Thank you for help.
Best regards
Przemysław
Hello Przemysław,
I am glad you solved the issue but this was an extreme solution.
Normally, the sen2agri-service user was intentionally created with no rights, the only requirement for it being to be able to write in the folders specified. I guess somewhere in the paths where it needed to write still remained a folder that was with read only attributes for this user.
But if this solution it is OK for you, then you can run this way, normally should be no problem.
Best regards,
Cosmin
In my case the problem was that only root user had privileges to execute MAJA scripts. Maybe something went wrong during the installation. I’m using CentOS on Virtualbox.
I realized now that you actually installed MAJA prior to the system installation but not sure why the MAJA package is installing the files in /opt/maja with the rights only for root (we are not in charge with this package).
Normally, if you copy the MAJA installation package in a subfolder called “maja” inside the installation package, the Sen2Agri installer would automatically install also MAJA and activate the read rights.
If you don’t want sen2agri-service to be in the sudoers, you could also try:
- change the read rights for /opt/maja for others too:
sudo chmod -R a+r /opt/maja
- remove the sen2agri-service from the sudoers list.
Best regards,
Cosmin
Hello,
I’ve reinstalled everything as you wrote above by putting MAJA into Sen2Agri installation package and everything works fine. I also have additional questions about preprocessing Sentinel-1 in Sen2Agri:
- how processing workflow looks
- what are the input and output products (does it merge scenes into slices)
I realize that Sen2Agri processors don’t work with S1 data but even automatic preprocessing could save me a lot of time.
Thank you again for your help
Hello,
Unfortunately, the S1 pre-processing is not yet implemented in Sen2Agri. In Sen2Agri is supported in this moment only the download of the S1 products (disabled by default) but no other operations on them. The pre-processing chain is an extension of the sen2agri-services implemented in another project (Sen4CAP - also open source). This part is still in validation process in Sen4CAP and was not made yet public but it will be soon. Maybe at some point we will take into account the adding of this extension also the Sen2Agri.
Best regards,
Cosmin
Thank you for info.
Best regards.
Dear @cudroiu ,
As we also want to study L3A product, we are testing on Sen2agri system.
I got the similar issue regarding to permission of sen2agri-service when I debugged demmaccs service. I gave permission to MAJA which is installed under /opt/bin/maja and still didn’t work out. Even though I added sen2agri-service to sudoer, I still got permission error. I am running version 2.0. Let me know how we can solve it out.
File “/usr/lib64/python2.7/subprocess.py”, line 711, in init
May 09 16:48:54 hadoop-srv-06 demmaccs_launcher.py[20671]: errread, errwrite)
May 09 16:48:54 hadoop-srv-06 demmaccs_launcher.py[20671]: File “/usr/lib64/python2.7/subprocess.py”, line 1327, in _execute_child
May 09 16:48:54 hadoop-srv-06 demmaccs_launcher.py[20671]: raise child_exception
May 09 16:48:54 hadoop-srv-06 demmaccs_launcher.py[20671]: OSError: [Errno 13] Permission denied
Thanks and Regards,
Henry
Dear Henry,
You should check several aspects:
- did you installed MAJA 3.2.2 and not another version?
- do you have the right path set into the “config” table for the key ‘demmaccs.maccs-launcher’ (’/opt/maja/3.2.2/bin/maja’)?
- you specified the path /opt/bin/maja while actually it should be /opt/maja. Please give the proper rights to the full /opt/maja dir not only to the bin directory.
Hope this helps.
Best regards,
Cosmin
Dear @cudroiu
I confirmed that MAJA version is 3.2.2 and it’s installed under /opt folder and it’s been configured accordingly in the config table as well. Please see below screeshots.
Thanks and Regards,
Henry
What about ?
sudo su -l sen2agri-service
ls -l /opt/bin/maja
Cosmin
Dear @cudroiu ,
Thanks for your tips.
[root@hadoop-srv-06 upload]# sudo su -l sen2agri-service
Last login: Mon May 10 13:44:42 UTC 2021 on pts/0
[sen2agri-service@hadoop-srv-06 ~]$ ls -l /opt/bin/maja
-rwxrwxr-x 1 sen2agri-service 1002 2566 Jun 23 2020 /opt/bin/maja
After I gave permission again to /opt/ folder, permission denied error is gone but no L3A products has been generated in product page. Here is result when I queried in the db.
Thanks and Regards,
Henry
Dear @cudroiu ,
It’s working fine after some time. Thanks a lot for ur support as usual.
Best Regards,
Henry