MACCS failing on some products

val · December 18, 2017, 9:14am

Hi,

I’ve got the MACCS processor failing on some products, both L8 and S2, and I can’t really figure out why. I’ve re-downloaded the respective L1 products, inspected them and they look good.

Here’s the output of the download_data table, the log for a failed L8 scene and one for a failed S2 tile:

   product_name                            | status_id | status_description 
-------------------------------------------------------------------+-----------+--------------------
 S2A_MSIL1C_20170603T082601_N0205_R021_T35PNK_20170603T084638.SAFE |         5 | processed
 S2A_MSIL1C_20170610T081601_N0205_R121_T35PNK_20170610T083432.SAFE |         5 | processed
 S2A_MSIL1C_20170613T083011_N0205_R021_T35PNK_20170613T084421.SAFE |         5 | processed
 S2A_MSIL1C_20170623T082601_N0205_R021_T35PNK_20170623T083831.SAFE |         5 | processed
 S2A_MSIL1C_20170630T081601_N0205_R121_T35PNK_20170630T083418.SAFE |         5 | processed
 LC08_L1TP_176054_20170915_20170928_01_T1                          |         6 | processing_failed
 LC08_L1GT_176054_20170611_20170627_01_T2                          |         4 | aborted
 LC08_L1GT_176054_20170713_20170726_01_T2                          |         4 | aborted
 S2A_MSIL1C_20170703T083011_N0205_R021_T35PNK_20170703T083833.SAFE |         6 | processing_failed
 S2A_MSIL1C_20170918T081601_N0205_R121_T35PNK_20170918T083613.SAFE |         6 | processing_failed
 S2A_MSIL1C_20170710T082011_N0205_R121_T35PNK_20170710T083235.SAFE |         5 | processed
 LC08_L1TP_176054_20170627_20170714_01_T1                          |         5 | processed
 S2A_MSIL1C_20170720T081601_N0205_R121_T35PNK_20170720T083410.SAFE |         5 | processed
 S2A_MSIL1C_20170723T083011_N0205_R021_T35PNK_20170723T083846.SAFE |         5 | processed
 LC08_L1GT_176054_20170830_20170914_01_T2                          |         4 | aborted
 LC08_L1TP_175054_20170722_20170728_01_T1                          |         5 | processed
 S2A_MSIL1C_20170713T082601_N0205_R021_T35PNK_20170713T083829.SAFE |         6 | processing_failed
 S2A_MSIL1C_20170730T082011_N0205_R121_T35PNK_20170730T083232.SAFE |         5 | processed
 S2A_MSIL1C_20170802T082601_N0205_R021_T35PNK_20170802T083801.SAFE |         5 | processed
 S2A_MSIL1C_20170809T081601_N0205_R121_T35PNK_20170809T083547.SAFE |         5 | processed
 S2A_MSIL1C_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE |         6 | processing_failed
 S2A_MSIL1C_20170819T082011_N0205_R121_T35PNK_20170819T083417.SAFE |         5 | processed
 S2A_MSIL1C_20170822T082601_N0205_R021_T35PNK_20170822T083808.SAFE |         5 | processed
 LC08_L1TP_175054_20170604_20170616_01_T1                          |         5 | processed
 LC08_L1TP_175054_20170807_20170813_01_T1                          |         6 | processing_failed
 LC08_L1TP_175054_20170823_20170912_01_T1                          |         5 | processed
 S2A_MSIL1C_20170812T083011_N0205_R021_T35PNK_20170812T083633.SAFE |         6 | processing_failed
 S2A_MSIL1C_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE |         5 | processed
 LC08_L1TP_176054_20170729_20170811_01_T1                          |         5 | processed
 LC08_L1TP_175054_20170620_20170630_01_T1                          |         5 | processed
 LC08_L1TP_175054_20170908_20170926_01_T1                          |         5 | processed
 S2A_MSIL1C_20170829T081601_N0205_R121_T35PNK_20170829T083601.SAFE |         6 | processing_failed
 LC08_L1TP_175054_20170924_20171013_01_T1                          |         5 | processed
 S2A_MSIL1C_20170928T082001_N0205_R121_T35PNK_20170928T083408.SAFE |         5 | processed
 LC08_L1TP_176054_20170814_20170825_01_T1                          |         5 | processed
 LC08_L1TP_175054_20170706_20170716_01_T1                          |         5 | processed



2017-12-13 14:13:53.033435:[26842]:Symbolic link L2SITE for tile id 176054 GIPP file could not be found. Searching for the common one 
2017-12-13 14:13:53.040106:[26842]:Copied /mnt/archive/gipp//LANDSAT8/L8_TEST_GIP_L2SITE_S_EUCMN00000A00B_00001_00000000_99999999.EEF to /mnt/archive/demmaccs_tmp/26218/176054/L8_TEST_GIP_L2SITE_S_EU17605400A00B_00001_00000000_99999999.EEF
2017-12-13 14:13:53.042569:[26842]:Symbolic link CKEXTL for tile id 176054 GIPP file could not be found. Searching for the common one 
2017-12-13 14:13:53.047109:[26842]:Copied /mnt/archive/gipp//LANDSAT8/L8_TEST_GIP_CKEXTL_S_EUCMN00000A00B_00001_00000000_99999999.EEF to /mnt/archive/demmaccs_tmp/26218/176054/L8_TEST_GIP_CKEXTL_S_EU17605400A00B_00001_00000000_99999999.EEF
2017-12-13 14:13:53.049317:[26842]:Symbolic link CKQLTL for tile id 176054 GIPP file could not be found. Searching for the common one 
2017-12-13 14:13:53.054390:[26842]:Copied /mnt/archive/gipp//LANDSAT8/L8_TEST_GIP_CKQLTL_S_EUCMN00000A00B_00001_00000000_99999999.EEF to /mnt/archive/demmaccs_tmp/26218/176054/L8_TEST_GIP_CKQLTL_S_EU17605400A00B_00001_00000000_99999999.EEF
2017-12-13 14:13:53.071149:[26842]:Creating sym links for NOMINAL MACCS mode: l2a prev tiles ['/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/L8_TEST_L8C_L2VALD_176054_20170814.DBL.DIR', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/L8_TEST_QCK_L2VALD_176054_20170814.DBL.DIR', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/demmaccs.log', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/L8_TEST_QCK_L2VALD_176054_20170814.HDR', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/mosaic.jpg', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/demmaccs_176054.log', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/L8_TEST_L8C_L2VALD_176054_20170814.DBL', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/L8_TEST_PMC_L2REPT_176054___20170814.EEF', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/L8_TEST_L8C_L2VALD_176054_20170814.HDR']
2017-12-13 14:13:53.128250:[26842]:Created sym links for NOMINAL MACCS mode for ['/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/L8_TEST_L8C_L2VALD_176054_20170814.DBL.DIR', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/L8_TEST_QCK_L2VALD_176054_20170814.DBL.DIR', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/demmaccs.log', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/L8_TEST_QCK_L2VALD_176054_20170814.HDR', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/mosaic.jpg', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/demmaccs_176054.log', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/L8_TEST_L8C_L2VALD_176054_20170814.DBL', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/L8_TEST_PMC_L2REPT_176054___20170814.EEF', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170814_20170825_01_T1/L8_TEST_L8C_L2VALD_176054_20170814.HDR']
2017-12-13 14:13:53.130019:[26842]:sat_id = 2 | acq_date = 20170915T000000
2017-12-13 14:13:53.131035:[26842]:Starting MACCS in L2NOMINAL for /mnt/archive/demmaccs_tmp/26218/LC08_L1TP_176054_20170915_20170928_01_T1 | TileID: 176054
2017-12-13 14:13:53.132139:[26842]:MACCS_COMMAND: ['/opt/maccs/core/5.1/bin/maccs', '--input', '/mnt/archive/demmaccs_tmp/26218/176054', '--TileId', '176054', '--output', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054', '--mode', 'L2NOMINAL', '--loglevel', 'DEBUG', '--enableTest', 'false', '--CheckXMLFilesWithSchema', 'false']
2017-12-13 14:13:53.133130:[26842]:Starting command: /opt/maccs/core/5.1/bin/maccs --input /mnt/archive/demmaccs_tmp/26218/176054 --TileId 176054 --output /mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054 --mode L2NOMINAL --loglevel DEBUG --enableTest false --CheckXMLFilesWithSchema false
2017-12-13 14:17:39.479552:[26842]:Command finished OK (res = 0) in 0:03:46.345690 : /opt/maccs/core/5.1/bin/maccs --input /mnt/archive/demmaccs_tmp/26218/176054 --TileId 176054 --output /mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054 --mode L2NOMINAL --loglevel DEBUG --enableTest false --CheckXMLFilesWithSchema false
2017-12-13 14:17:39.481056:[26842]:MACCS mode L2NOMINAL for /mnt/archive/demmaccs_tmp/26218/LC08_L1TP_176054_20170915_20170928_01_T1 tile 176054 finished in: 0:03:46.411879. Location: /mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1/
2017-12-13 14:17:39.484959:[26842]:Searching for report maccs files (REPT) in: /mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054
2017-12-13 14:17:39.486055:[26842]:Report maccs files (REPT) found in: /mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054 : ['/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054/L8_TEST_PMC_L2REPT_176054___20170915.EEF']
2017-12-13 14:17:39.487431:[26842]:Moving /mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054/L8_TEST_PMC_L2REPT_176054___20170915.EEF to /mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//L8_TEST_PMC_L2REPT_176054___20170915.EEF
2017-12-13 14:17:39.492029:[26842]:Searching for valid products in MACCS working dir: /mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054. Following is the content of this dir: ['/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054/L8_TEST_L8C_L2NOTV_176054_20170915.DBL', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054/L8_TEST_L8C_L2NOTV_176054_20170915.HDR', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054/L8_TEST_L8C_L2NOTV_176054_20170915.DBL.DIR', '/mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054/LTC']
2017-12-13 14:17:39.493127:[26842]:No valid products (VALD status) found in: /mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054.
2017-12-13 14:17:39.494108:[26842]:Deleting MACCS working directory: rmtree: /mnt/archive/maccs_def/t35pnk/l2a/LC08_L2A_176054_20170915_20170928_01_T1//maccs_176054

017-12-13 09:53:22.118061:[919]:Symbolic link L2SITE for tile id 35PNK GIPP file could not be found. Searching for the common one 
2017-12-13 09:53:22.125399:[919]:Copied /mnt/archive/gipp//SENTINEL2/S2__TEST_GIP_L2SITE_S_CMN00____00001_20150703_21000101.EEF to /mnt/archive/demmaccs_tmp/558/35PNK/S2__TEST_GIP_L2SITE_S_35PNK____00001_20150703_21000101.EEF
2017-12-13 09:53:22.128234:[919]:Symbolic link CKEXTL for tile id 35PNK GIPP file could not be found. Searching for the common one 
2017-12-13 09:53:22.134216:[919]:Copied /mnt/archive/gipp//SENTINEL2/S2A_TEST_GIP_CKEXTL_S_CMN00____00001_20150703_21000101.EEF to /mnt/archive/demmaccs_tmp/558/35PNK/S2A_TEST_GIP_CKEXTL_S_35PNK____00001_20150703_21000101.EEF
2017-12-13 09:53:22.137097:[919]:Symbolic link CKQLTL for tile id 35PNK GIPP file could not be found. Searching for the common one 
2017-12-13 09:53:22.142647:[919]:Copied /mnt/archive/gipp//SENTINEL2/S2A_TEST_GIP_CKQLTL_S_CMN00____00004_20150703_21000101.EEF to /mnt/archive/demmaccs_tmp/558/35PNK/S2A_TEST_GIP_CKQLTL_S_35PNK____00004_20150703_21000101.EEF
2017-12-13 09:53:22.221908:[919]:Creating sym links for NOMINAL MACCS mode: l2a prev tiles ['/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/S2A_OPER_SSC_L2VALD_35PNK____20170911.DBL.DIR', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/demmaccs_35PNK.log', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/demmaccs.log', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/S2A_OPER_QCK_L2VALD_35PNK____20170911.DBL.DIR', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/mosaic.jpg', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/S2A_OPER_PMC_L2REPT_35PNK____20170911.EEF', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/S2A_OPER_SSC_L2VALD_35PNK____20170911.HDR', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/S2A_OPER_QCK_L2VALD_35PNK____20170911.HDR', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/S2A_OPER_SSC_L2VALD_35PNK____20170911.DBL']
2017-12-13 09:53:22.292584:[919]:Created sym links for NOMINAL MACCS mode for ['/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/S2A_OPER_SSC_L2VALD_35PNK____20170911.DBL.DIR', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/demmaccs_35PNK.log', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/demmaccs.log', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/S2A_OPER_QCK_L2VALD_35PNK____20170911.DBL.DIR', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/mosaic.jpg', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/S2A_OPER_PMC_L2REPT_35PNK____20170911.EEF', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/S2A_OPER_SSC_L2VALD_35PNK____20170911.HDR', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/S2A_OPER_QCK_L2VALD_35PNK____20170911.HDR', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170911T082601_N0205_R021_T35PNK_20170911T083628.SAFE/S2A_OPER_SSC_L2VALD_35PNK____20170911.DBL']
2017-12-13 09:53:22.295071:[919]:sat_id = 1 | acq_date = 20170921T083001
2017-12-13 09:53:22.296832:[919]:Starting MACCS in L2NOMINAL for /mnt/archive/dwn_def/s2/default/t35pnk/S2A_MSIL1C_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE | TileID: 35PNK
2017-12-13 09:53:22.298558:[919]:MACCS_COMMAND: ['/opt/maccs/core/5.1/bin/maccs', '--input', '/mnt/archive/demmaccs_tmp/558/35PNK', '--TileId', '35PNK', '--output', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK', '--mode', 'L2NOMINAL', '--loglevel', 'DEBUG', '--enableTest', 'false', '--CheckXMLFilesWithSchema', 'false', '--conf', '/usr/share/sen2agri/sen2agri-demmaccs/UserConfiguration']
2017-12-13 09:53:22.301437:[919]:Starting command: /opt/maccs/core/5.1/bin/maccs --input /mnt/archive/demmaccs_tmp/558/35PNK --TileId 35PNK --output /mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK --mode L2NOMINAL --loglevel DEBUG --enableTest false --CheckXMLFilesWithSchema false --conf /usr/share/sen2agri/sen2agri-demmaccs/UserConfiguration
2017-12-13 09:57:47.850379:[919]:Command finished OK (res = 0) in 0:04:25.548703 : /opt/maccs/core/5.1/bin/maccs --input /mnt/archive/demmaccs_tmp/558/35PNK --TileId 35PNK --output /mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK --mode L2NOMINAL --loglevel DEBUG --enableTest false --CheckXMLFilesWithSchema false --conf /usr/share/sen2agri/sen2agri-demmaccs/UserConfiguration
2017-12-13 09:57:47.852275:[919]:MACCS mode L2NOMINAL for /mnt/archive/dwn_def/s2/default/t35pnk/S2A_MSIL1C_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE tile 35PNK finished in: 0:04:25.692905. Location: /mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE/
2017-12-13 09:57:47.858675:[919]:Searching for report maccs files (REPT) in: /mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK
2017-12-13 09:57:47.860441:[919]:Report maccs files (REPT) found in: /mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK : ['/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK/S2A_OPER_PMC_L2REPT_35PNK____20170921.EEF']
2017-12-13 09:57:47.862431:[919]:Moving /mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK/S2A_OPER_PMC_L2REPT_35PNK____20170921.EEF to /mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//S2A_OPER_PMC_L2REPT_35PNK____20170921.EEF
2017-12-13 09:57:47.866455:[919]:Searching for valid products in MACCS working dir: /mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK. Following is the content of this dir: ['/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK/S2A_OPER_SSC_L2NOTV_35PNK____20170921.DBL.DIR', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK/S2A_OPER_SSC_L2NOTV_35PNK____20170921.HDR', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK/LTC', '/mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK/S2A_OPER_SSC_L2NOTV_35PNK____20170921.DBL']
2017-12-13 09:57:47.867849:[919]:No valid products (VALD status) found in: /mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK.
2017-12-13 09:57:47.869062:[919]:Deleting MACCS working directory: rmtree: /mnt/archive/maccs_def/t35pnk/l2a/S2A_MSIL2A_20170921T083001_N0205_R021_T35PNK_20170921T084129.SAFE//maccs_35PNK

Thanks,

Val

brentf · January 3, 2018, 12:20pm

Hello there,

I had a same situation with you.
Hope that somebody would reply on this thread.

Brent

val · January 3, 2018, 12:39pm

Hi Brent,

Turns out the reason some products failed to process was the exceedance of the cloud threshold (80% if I remember correctly). I’ve also seen products failing if there was only a small sliver of the orbit which had no or almost no data.

Although, I don’t quite get why the cloudy products are downloaded in the first place and also I think the logs or the processing information could be a bit more transparent - crossing my fingers for a future release.

So I reccomend checking some filenames which have failed on the SciHub/Amazon to look at the images, or just check the metadatafile in the L1C folder.

Hope that helps.

Best,

Val

lnicola · January 3, 2018, 3:26pm

Hi,

I believe the cloud coverage value detected by MACCS can differ from the one available in the L1C metadata, with the former being more precise. More so, the downloaders and the MACCS launcher can be configured (and are, by default) with different cloud thresholds.

The downloader threshold is the downloader.max-cloud-coverage value in the config database table, set to 100 by default. The threshold used by MACCS is the Max_Cloud_Percentage value in SENTINEL2/S2{A,B}_TEST_GIP_L2COMM_L_ALLSITES_10007_20150703_21000101.EEF and LANDSAT8/L8_TEST_GIP_L2COMM_L_ALLSITES_10003_00000000_99999999.EEF, under /mnt/archive/gipp, and is set to ‘90’ by default.

The bigger issue, in my opinion, is that the log files are removed for failed products.

Sorry for the late reply, but for some reason I don’t get all notifications from this forum.