Failed Process l2A

Hi,
I’m using sen2agri 1.8.1 in automated mode, the process l2A failed for all downloaded products. Here is the content of the demmaccs.log.

2018-05-09 11:03:06.763982:[14320]:Starting demmaccs
2018-05-09 11:03:06.766544:[14320]:Starting command: /usr/share/sen2agri/sen2agri-demmaccs/demmaccs.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd --processes-number-dem 20 --processes-number-maccs 2 --gipp-dir /mnt/archive/gipp/ --working-dir /mnt/archive/demmaccs_tmp/ --maccs-launcher /opt/maccs/core/5.1/bin/maccs --delete-temp True /mnt/archive/dwn_def/l8/default/sito_1/LC08_L1TP_189032_20171129_20171207_01_T1 /mnt/archive/maccs_def/sito_1/l2a/LC08_L2A_189032_20171129_20171207_01_T1/
2018-05-09 11:03:07.001635:[14334]:working_dir = /mnt/archive/demmaccs_tmp/14334
2018-05-09 11:03:07.003012:[14334]:dem_working_dir = /mnt/archive/demmaccs_tmp/14334_DEM_TMP
2018-05-09 11:03:07.004089:[14334]:dem_output_dir = /mnt/archive/demmaccs_tmp/14334_DEM_OUT
2018-05-09 11:03:07.415323:[14334]:The LANDSAT8 product could not be aligned /mnt/archive/dwn_def/l8/default/sito_1/LC08_L1TP_189032_20171129_20171207_01_T1
2018-05-09 11:03:07.450932:[14320]:Command finished NOK (res = 255) in 0:00:00.684463 : /usr/share/sen2agri/sen2agri-demmaccs/demmaccs.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd --processes-number-dem 20 --processes-number-maccs 2 --gipp-dir /mnt/archive/gipp/ --working-dir /mnt/archive/demmaccs_tmp/ --maccs-launcher /opt/maccs/core/5.1/bin/maccs --delete-temp True /mnt/archive/dwn_def/l8/default/sito_1/LC08_L1TP_189032_20171129_20171207_01_T1 /mnt/archive/maccs_def/sito_1/l2a/LC08_L2A_189032_20171129_20171207_01_T1/
2018-05-09 11:03:07.452636:[14320]:demmaccs.py script didn’t work!
2018-05-09 11:03:07.511170:[14320]:Only set the state as processed in downloader_history (no l2a tiles found after maccs) for product /mnt/archive/maccs_def/sito_1/l2a/LC08_L2A_189032_20171129_20171207_01_T1/

Best regards Caterina

Hi,

Can you take a look at:

sudo journalctl -u sen2agri-demmaccs --since "2018-05-09 11:00"

?

I’m interested in the part that precedes this line:

 2018-05-09 11:03:07.415323:[14334]:The LANDSAT8 product could not be aligned /mnt/archive/dwn_def/l8/default/sito_1/LC08_L1TP_189032_20171129_20171207_01_T1

Hi,
thanks for your quick reply. I make journalctl -u sen2agri-demmaccs --since “2018-05-09 11:00” | tee log.txt. The log.txt file is attached here.log.txt.zip (131.4 KB)

May 09 11:03:07 localhost.localdomain demmaccs_launcher.py[14314]: Creating DEMs for /mnt/archive/dwn_def/s2/default/sito_1/S2B_MSIL1C_20171119T095259_N0206_R079_T33TWE_20171120T164650.SAFE
May 09 11:03:07 localhost.localdomain demmaccs_launcher.py[14314]: 2018-05-09 11:03:07.009512:[14335]:Starting command: /usr/share/sen2agri/sen2agri-demmaccs/dem.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd -p 20 -w /mnt/archive/demmaccs_tmp/14335_DEM_TMP /mnt/archive/dwn_def/s2/default/sito_1/S2B_MSIL1C_20171119T095259_N0206_R079_T33TWE_20171120T164650.SAFE /mnt/archive/demmaccs_tmp/14335_DEM_OUT
May 09 11:03:07 localhost.localdomain demmaccs_launcher.py[14314]: The path does not exist ! /mnt/archive/dwn_def/s2/default/sito_1/S2B_MSIL1C_20171119T095259_N0206_R079_T33TWE_20171120T164650.SAFE

It looks like it couldn’t find /mnt/archive/dwn_def/s2/default/sito_1/S2B_MSIL1C_20171119T095259_N0206_R079_T33TWE_20171120T164650.SAFE. Is it possible that you manually moved the files to another directory?

I don’t manually moved files to another directory, but I checked that "/mnt/archive/dwn_def/s2/default/sito_1/S2B_MSIL1C_20171119T095259_N0206_R079_T33TWE_20171120T164650.SAFE " isn’t present in this folder.
I looked in downloader_history table and I checked that the record of the file is present but in the status_reason column I read “No space left on device”.
I’m using a virtual machine centos 7 64 bit with 150 GB of HD, I created /mnt/archive folder as recommended in the user manual but this folder has however small space about 50 GB. I shared /mnt/archive folder with a folder on my host machine with a space of 2 TB aiming to increase the space.

After you fix the underlying issue of the free disk space, it might be a good idea to restart the download. You can also reprocess any failed products, if you prefer that. Feel free to ask here either way.

And – just in case you didn’t see it – the FAQ has some troubleshooting tips which you might find useful.

Hi,
I increased the space on mnt/archive folder. I deleted the old site and I created a new site with downloader_start_offset = 1. The download of S2 is finished while L8 is in progress. I have selected only L2A processor but this processor has not started yet. The folder of new site is not present in maccs_def. In system overview i checked that the CPU don’t work. I attach images here.

Can you try this again?

sudo journalctl -u sen2agri-demmaccs --since "2018-05-11"

I make journalctl -u sen2agri-demmaccs --since “2018-05-11” | tee log.txt. The log.txt file is attached here.log.txt.zip (1.7 KB)

You might have restarted the server in the meanwhile, because the logs begin today. Can you force the reprocessing with something like this (you’ll need to find the site_id beforehand with select id, name from site;), then keep an eye on the journalctl output?

You can also see the output in real time with journalctl -f -u sen2agri-demmaccs.