MAJA processing error "Unknown format found in output directory"

Hello, I have a problem with L2A processor. I’ve installed Sen2Agri 2.0 from the beginning but MAJA does not process Sentinel2 properly. I have such a message in the logs “Unknown format found in output directory”. What can cause this problem? Attached, I am sending logs.
demmaccs.log (1.3 KB)
demmaccs_34UDE.log (4.5 KB)

Hello,

Could you try running the command :

/usr/share/sen2agri/sen2agri-demmaccs/demmaccs.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd --processes-number-dem 1 --processes-number-maccs 1 --gipp-dir /mnt/archive/gipp_maja --working-dir /mnt/archive/demmaccs_tmp/ --maccs-launcher /opt/maja/3.2.2/bin/maja --delete-temp True /mnt/archive/dwn_def/s2/default/ketrzynski_2019/S2A_MSIL1C_20190105T094401_N0207_R036_T34UDE_20190105T101012.SAFE /mnt/archive/maccs_def/ketrzynski_2019/l2a/S2A_MSIL2A_20190105T094401_N0207_R036_T34UDE_20190105T101012.SAFE/ --tiles-to-process 34UDE

under the user sen2agri-service (sudo su -l sen2agri-service) and check the error messages that are given by the demmaccs?

Please let me know.

Best regards,
Cosmin

Here is the log.log.txt (16.1 KB)

Could you please do the following operations (as the previous test execution might have been “polluted” by other executions):

  • Stop the sen2agri-demmaccs services executing:

sudo systemctl stop sen2agri-demmaccs sen2agri-demmaccs.timer

  • Execute the command I specified above
  • Extract the log and send it again to us
  • Restart the sen2agri-demmaccs using:

sudo systemctl start sen2agri-demmaccs sen2agri-demmaccs.timer

Cosmin

Hello,
I did executed the command first as sen2agri user and then as root user. Under the root user processing have been completed but under sen2agri user there was an error. Maybe I should add someware additional permissions to the sen2agri user? Attached I’m sending the logs.
sen2agri_user_log.txt (52.1 KB)
root_user_log.txt (666.9 KB)

Hello,

As the MAJA is processing OK the products under the user root, it seems that you have some issues with the rights to the /mnt/archive.
In this case, you should:

  • stop the sen2agri-demmaccs:

sudo systemctl stop sen2agri-demmaccs sen2agri-demmaccs.timer

  • change recursively the access rights to the /mnt/archive to 777

sudo chmod -R 777 /mnt/archive
sudo chmod -R 777 /mnt/upload

  • if the /mnt/archive/maccs_def and /mnt/archive/demmaccs_tmp are somehow symlinks to other locations, please ensure that these folders have also the right permissions for user sen2agri-service (recommended rights 777)
  • optionally, you can remove the folder /mnt/archive/demmaccs_tmp
  • reset the products that were marked as failed in the database:

sudo -u postgres psql sen2agri -c “delete from l1_tile_history”
sudo -u postgres psql sen2agri -c “update downloader_history set status_id = 2 where status_id = 6”

  • start the sen2agri-demmaccs:

sudo systemctl start sen2agri-demmaccs sen2agri-demmaccs.timer

Best regards,
Cosmin

Hello,

Still the same problem. I did everything above. I will reinstall everything from the beginning. Could the problem be that the installation was done as root?

I finally managed MAJA to work. I’ve added sen2agri-service to sudoers in /etc/sudoers under the root user.

##Allow root to run any commands anywhere
root ALL=(ALL) ALL
sen2agri-service ALL=(ALL) ALL

Thank you for help.

Best regards
Przemysław

Hello Przemysław,

I am glad you solved the issue but this was an extreme solution.
Normally, the sen2agri-service user was intentionally created with no rights, the only requirement for it being to be able to write in the folders specified. I guess somewhere in the paths where it needed to write still remained a folder that was with read only attributes for this user.
But if this solution it is OK for you, then you can run this way, normally should be no problem.

Best regards,
Cosmin

In my case the problem was that only root user had privileges to execute MAJA scripts. Maybe something went wrong during the installation. I’m using CentOS on Virtualbox.

I realized now that you actually installed MAJA prior to the system installation but not sure why the MAJA package is installing the files in /opt/maja with the rights only for root (we are not in charge with this package).
Normally, if you copy the MAJA installation package in a subfolder called “maja” inside the installation package, the Sen2Agri installer would automatically install also MAJA and activate the read rights.
If you don’t want sen2agri-service to be in the sudoers, you could also try:

  • change the read rights for /opt/maja for others too:

sudo chmod -R a+r /opt/maja

  • remove the sen2agri-service from the sudoers list.

Best regards,
Cosmin

Hello,

I’ve reinstalled everything as you wrote above by putting MAJA into Sen2Agri installation package and everything works fine. I also have additional questions about preprocessing Sentinel-1 in Sen2Agri:

  1. how processing workflow looks
  2. what are the input and output products (does it merge scenes into slices)
    I realize that Sen2Agri processors don’t work with S1 data but even automatic preprocessing could save me a lot of time.

Thank you again for your help

Hello,

Unfortunately, the S1 pre-processing is not yet implemented in Sen2Agri. In Sen2Agri is supported in this moment only the download of the S1 products (disabled by default) but no other operations on them. The pre-processing chain is an extension of the sen2agri-services implemented in another project (Sen4CAP - also open source). This part is still in validation process in Sen4CAP and was not made yet public but it will be soon. Maybe at some point we will take into account the adding of this extension also the Sen2Agri.

Best regards,
Cosmin

Thank you for info.

Best regards.