I specified region at high Northen latitudes (70 deg North) for summer 2016. data were downloaded but atmospheric correction was not performed. What is the reason- too low sun position?
Hello,
Could you provide more information about your site, please?
- the archive with the shapefile you use (we can try simulating your problem on our premises)
- the season you use
- what errors MACCS gives (normally you can find them in the logs of the product or in the journalctl for sen2agri-demmaccs)?
Generally, to see all MACCS execution for a certain product you can do the following (if you cannot send us the shapefile or the messages in journalctl are not available anymore):
- go into the directory of the failed L2A product and get the maccs execution command (something like :
/usr/share/sen2agri/sen2agri-demmaccs/demmaccs.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd --processes-number-dem 20 --processes-number-maccs 2 --gipp-dir /mnt/archive/gipp/ --working-dir /mnt/archive/demmaccs_tmp/ --maccs-launcher /opt/maccs/core/5.1/bin/maccs --delete-temp True ...
- log into the sen2agri-service account by performing :
sudo su -l sen2agri-service - execute the command you extracted above (the one starting with /usr/share/sen2agri/sen2agri-demmaccs/demmaccs.py …)
- Get the log of your command and provide it to us.
Best regards,
Cosmin
ROI.zip (4.4 KB)
Shape file is attached. But why could be this a problem? Sentinel & Landsat data were downloaded- no problem. I checked images- some of them are quite clean without clouds…
Season is July 1-August 25, 2016
I also tried another time interval July 1- July 30, 2015. Again got failure of L2A processing.
demmaccs.log says DEM failed Specifically:
2018-07-03 00:23:01.873297:[11771]:Starting demmaccs
2018-07-03 00:23:01.873418:[11771]:Starting command: /usr/share/sen2agri/sen2agri-demmaccs/demmaccs.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd --processes-number-dem 20 --processes-number-maccs 2 --gipp-dir /mnt/archive/gipp/ --working-dir /mnt/archive/demmaccs_tmp/ --maccs-launcher /opt/maccs/core/5.1/bin/maccs --delete-temp True /mnt/archive/dwn_def/s2/default/hibini/S2A_OPER_PRD_MSIL1C_PDMC_20160818T183341_R079_V20150728T095006_20150728T095006.SAFE /mnt/archive/maccs_def/hibini/l2a/S2A_OPER_PRD_MSIL2A_PDMC_20160818T183341_R079_V20150728T095006_20150728T095006.SAFE/
2018-07-03 00:23:01.993725:[11783]:working_dir = /mnt/archive/demmaccs_tmp/11783
2018-07-03 00:23:01.993778:[11783]:dem_working_dir = /mnt/archive/demmaccs_tmp/11783_DEM_TMP
2018-07-03 00:23:01.993812:[11783]:dem_output_dir = /mnt/archive/demmaccs_tmp/11783_DEM_OUT
2018-07-03 00:23:01.994160:[11783]:Starting command: /usr/share/sen2agri/sen2agri-demmaccs/dem.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd -p 20 -w /mnt/archive/demmaccs_tmp/11783_DEM_TMP /mnt/archive/dwn_def/s2/default/hibini/S2A_OPER_PRD_MSIL1C_PDMC_20160818T183341_R079_V20150728T095006_20150728T095006.SAFE /mnt/archive/demmaccs_tmp/11783_DEM_OUT
2018-07-03 00:23:03.084329:[11783]:Command finished NOK (res = 1) in 0:00:01.090097 : /usr/share/sen2agri/sen2agri-demmaccs/dem.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd -p 20 -w /mnt/archive/demmaccs_tmp/11783_DEM_TMP /mnt/archive/dwn_def/s2/default/hibini/S2A_OPER_PRD_MSIL1C_PDMC_20160818T183341_R079_V20150728T095006_20150728T095006.SAFE /mnt/archive/demmaccs_tmp/11783_DEM_OUT
2018-07-03 00:23:03.084393:[11783]:DEM failed
2018-07-03 00:23:03.092026:[11771]:Command finished NOK (res = 255) in 0:00:01.218568 : /usr/share/sen2agri/sen2agri-demmaccs/demmaccs.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd --processes-number-dem 20 --processes-number-maccs 2 --gipp-dir /mnt/archive/gipp/ --working-dir /mnt/archive/demmaccs_tmp/ --maccs-launcher /opt/maccs/core/5.1/bin/maccs --delete-temp True /mnt/archive/dwn_def/s2/default/hibini/S2A_OPER_PRD_MSIL1C_PDMC_20160818T183341_R079_V20150728T095006_20150728T095006.SAFE /mnt/archive/maccs_def/hibini/l2a/S2A_OPER_PRD_MSIL2A_PDMC_20160818T183341_R079_V20150728T095006_20150728T095006.SAFE/
2018-07-03 00:23:03.092095:[11771]:demmaccs.py script didn’t work!
2018-07-03 00:23:03.092144:[11771]:Only set the state as processed in downloader_history (no l2a tiles found after maccs) for product /mnt/archive/maccs_def/hibini/l2a/S2A_OPER_PRD_MSIL2A_PDMC_20160818T183341_R079_V20150728T095006_20150728T095006.SAFE/
I just recognized the following problem. Failure occurred due to failure of DEM. But my site is at 70 deg North, while CRTM DEN is calculated only till 60 deg North (as far as I know). Any way to work around?
Is there a way to switch from CRTM DEM to ASTER Global DEM v2?
Indeed, the issue seems to be caused by missing DEM tiles:
The following SRTM tiles are missing: ['srtm_41_-1.tif', 'srtm_42_-1.tif']. Please provide them in the SRTM directory (/mnt/archive/srtm). Will try to continue, but unreliable results
The following SRTM tiles are missing: ['srtm_41_00.tif', 'srtm_41_-1.tif', 'srtm_42_00.tif', 'srtm_42_-1.tif']. Please provide them in the SRTM directory (/mnt/archive/srtm). Will try to continue, but unreliable results
The following SRTM tiles are missing: ['srtm_42_-1.tif', 'srtm_43_-1.tif']. Please provide them in the SRTM directory (/mnt/archive/srtm). Will try to continue, but unreliable results
The following SRTM tiles are missing: ['srtm_42_-1.tif']. Please provide them in the SRTM directory (/mnt/archive/srtm). Will try to continue, but unreliable results
The following SRTM tiles are missing: ['srtm_42_00.tif', 'srtm_42_-1.tif', 'srtm_43_00.tif', 'srtm_43_-1.tif']. Please provide them in the SRTM directory (/mnt/archive/srtm). Will try to continue, but unreliable results
The following SRTM tiles are missing: ['srtm_42_00.tif', 'srtm_42_-1.tif']. Please provide them in the SRTM directory (/mnt/archive/srtm). Will try to continue, but unreliable results
The following SRTM tiles are missing: ['srtm_43_-1.tif', 'srtm_44_-1.tif']. Please provide them in the SRTM directory (/mnt/archive/srtm). Will try to continue, but unreliable results
The following SRTM tiles are missing: ['srtm_43_-1.tif']. Please provide them in the SRTM directory (/mnt/archive/srtm). Will try to continue, but unreliable results
The following SRTM tiles are missing: ['srtm_43_00.tif', 'srtm_43_-1.tif']. Please provide them in the SRTM directory (/mnt/archive/srtm). Will try to continue, but unreliable results
The SRTM DEM tile name computation looks like this (from /usr/share/sen2agri/sen2agri-demmaccs/dem.py):
def get_dtm_tiles(points):
"""
Returns a list of dtm tiles covering the given extent
"""
a_x, a_y, b_x, b_y = points
if a_x < b_x and a_y > b_y:
a_bb_x = int(math.floor(a_x / 5) * 5)
a_bb_y = int(math.floor((a_y + 5) / 5) * 5)
b_bb_x = int(math.floor((b_x + 5) / 5) * 5)
b_bb_y = int(math.floor(b_y / 5) * 5)
print("bounding box {} {} {} {}".format(
a_bb_x, a_bb_y, b_bb_x, b_bb_y))
x_numbers_list = [(x + 180) / 5 + 1
for x in range(min(a_bb_x, b_bb_x), max(a_bb_x, b_bb_x), 5)]
x_numbers_list_format = ["%02d" % (x,) for x in x_numbers_list]
y_numbers_list = [(60 - x) / 5
for x in range(min(a_bb_y, b_bb_y), max(a_bb_y, b_bb_y), 5)]
y_numbers_list_format = ["%02d" % (x,) for x in y_numbers_list]
srtm_zips = ["srtm_" + str(x) + "_" + str(y) + ".tif"
for x in x_numbers_list_format
for y in y_numbers_list_format]
return srtm_zips
which indeed seems to give wrong results for latitudes over 60°.
That code would need to be updated in order to be compatible with other DEM formats. Other changes might be necessary if the range of values is different.
Unfortunately, I don’t expect us to be working on this (it’s not planned), but if you end up implementing it, we will be glad to include the changes in a future version.