Error in processing the last tile?

Hello all,

This issue seems to be related to the following posts:


As a background, I submitted a custom job processing for L4B, but this time I didn’t select any L4A mask. I just want to have a raw/unmasked L4B product. However it seems that there is a problem when the processor is dealing with the last tile of my site.

otbcli_Convert -progress false -in /mnt/archive/orchestrator_temp/l4b/256/65492-crop-type-fused/crop_type_map_51QVU_uncompressed.tif -out '/mnt/archive/orchestrator_temp/l4b/256/65492-crop-type-fused/crop_type_map_51QVU.tif?gdal:co:COMPRESS=DEFLATE' int16
Compression_51QVU done at 2019-12-10 00:56:08.588730. Duration: 0:00:09.598611
ImageClassifier_51QVV done at 2019-12-10 01:30:49.728957. Duration: 2:06:49.457334
Executing Compression_51QVV at 2019-12-10 01:30:49.729049
otbcli_Convert -progress false -in /mnt/archive/orchestrator_temp/l4b/256/65492-crop-type-fused/crop_type_map_51QVV_uncompressed.tif -out '/mnt/archive/orchestrator_temp/l4b/256/65492-crop-type-fused/crop_type_map_51QVV.tif?gdal:co:COMPRESS=DEFLATE' int16
Compression_51QVV done at 2019-12-10 01:30:56.956027. Duration: 0:00:07.226978
Performing post-processing for tile: 51QUU
Performing post-processing for tile: Executing Mask by crop mask 51QUU at 2019-12-10 01:30:57.473668
51QUV
Executing Mask by crop mask 51QUV at 2019-12-10 01:30:57.473949
otbcli_BandMath -progress false -exp 'im2b1 == 0 ? 0 : im1b1' -il /mnt/archive/orchestrator_temp/l4b/256/65492-crop-type-fused/crop_type_map_51QUU.tif /mnt/archive/isabela19/l4a//S2AGRI_L4A_PRD_S6_20191202T180353_V20190108_20190108/TILES/S2AGRI_L4A_V20190108_20190108_T51QUU/IMG_DATA/S2AGRI_L4A_CM_V20190108_20190108_T51QUU.TIF -out '/mnt/archive/orchestrator_temp/l4b/256/65492-crop-type-fused/crop_type_map_masked_51QUU.tif?gdal:co:COMPRESS=DEFLATE' int16
otbcli_BandMath -progress false -exp 'im2b1 == 0 ? 0 : im1b1' -il /mnt/archive/orchestrator_temp/l4b/256/65492-crop-type-fused/crop_type_map_51QUV.tif /mnt/archive/isabela19/l4a//S2AGRI_L4A_PRD_S6_20191202T180353_V20190108_20190108/TILES/S2AGRI_L4A_V20190108_20190108_T51QUV/IMG_DATA/S2AGRI_L4A_CM_V20190108_20190108_T51QUV.TIF -out '/mnt/archive/orchestrator_temp/l4b/256/65492-crop-type-fused/crop_type_map_masked_51QUV.tif?gdal:co:COMPRESS=DEFLATE' int16
Mask by crop mask 51QUU done at 2019-12-10 01:31:05.346762. Duration: 0:00:07.873094
Executing Nodata_51QUU at 2019-12-10 01:31:05.389196
gdal_edit.py -a_nodata -10000 /mnt/archive/orchestrator_temp/l4b/256/65492-crop-type-fused/crop_type_map_masked_51QUU.tif
Error running Nodata_51QUU at 2019-12-10 01:31:09.635572. The call returned 1
Performing post-processing for tile: 51QVU
Executing Mask by crop mask 51QVU at 2019-12-10 01:31:09.713976
otbcli_BandMath -progress false -exp 'im2b1 == 0 ? 0 : im1b1' -il /mnt/archive/orchestrator_temp/l4b/256/65492-crop-type-fused/crop_type_map_51QVU.tif /mnt/archive/isabela19/l4a//S2AGRI_L4A_PRD_S6_20191202T180353_V20190108_20190108/TILES/S2AGRI_L4A_V20190108_20190108_T51QVU/IMG_DATA/S2AGRI_L4A_CM_V20190108_20190108_T51QVU.TIF -out '/mnt/archive/orchestrator_temp/l4b/256/65492-crop-type-fused/crop_type_map_masked_51QVU.tif?gdal:co:COMPRESS=DEFLATE' int16
Processor finished in 13:55:29.993479
2019 Dec 10 01:30:58  :  Application.logger  (INFO) Image #1 has 1 components

2019 Dec 10 01:30:58  :  Application.logger  (INFO) Image #2 has 1 components



Traceback (most recent call last):
  File "/usr/local/bin/gdal_edit.py", line 328, in <module>
    sys.exit(gdal_edit(sys.argv))
  File "/usr/local/bin/gdal_edit.py", line 227, in gdal_edit
    ds = gdal.OpenEx(datasetname, gdal.OF_RASTER | gdal.OF_UPDATE, open_options=open_options)
AttributeError: 'module' object has no attribute 'OpenEx'
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/sen2agri_common.py", line 609, in execute
    pool.map(self.internal_postprocess_tile, self.tiles)
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 250, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 554, in get
    raise self._value
Exception: ('Error running Nodata_51QUU', 1)

Actually, I don’t need the crop mask to be applied to my crop type since I’m doing a land cover classification of all classes.

Are there are turnaround/solutions for this?

@cudroiu @lnicola

Regards,
Brent

Hello,

I tried running the CropTypeFused.py manually and remove the parameter such as “maskprod” and “outprops”. I’ll update you with my results.

Brent