For many tiles I’m getting Error:
vns::Plugin::ERROR: Sentinel2TML2ImageFileReader(0xe600f0): The MUSCATE product “/mnt/archive/demmaccs_tmp/tmpRpCmbO/35VMC/SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0/SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_MTD_ALL.xml” is not valid according to JPI! [vnsMuscateL2ImageFileReader.txx:MuscateDetectL2Products:368]
How can I debug to find the issue ?
Product directory looks like this:
ls -al /mnt/archive/demmaccs_tmp/tmpRpCmbO/35VMC/SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0/
total 2886820
drwxr-xr-x 4 sen2agri-service sen2agri-service 4096 2019-05-29 14:37 .
drwxr-xr-x 3 sen2agri-service sen2agri-service 4096 2019-05-29 14:38 …
drwxr-xr-x 3 sen2agri-service sen2agri-service 4096 2019-05-29 14:37 DATA
drwxr-xr-x 2 sen2agri-service sen2agri-service 4096 2019-05-29 14:36 MASKS
-rw-r–r-- 1 sen2agri-service sen2agri-service 241212436 2019-05-29 14:31 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_ATB_R1.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327916 2019-05-29 14:35 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_ATB_R2.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327904 2019-05-29 14:35 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_FRE_B11.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327904 2019-05-29 14:35 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_FRE_B12.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 241212426 2019-05-29 14:30 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_FRE_B2.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 241212426 2019-05-29 14:30 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_FRE_B3.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 241212426 2019-05-29 14:31 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_FRE_B4.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327904 2019-05-29 14:35 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_FRE_B5.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327904 2019-05-29 14:35 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_FRE_B6.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327904 2019-05-29 14:35 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_FRE_B7.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327904 2019-05-29 14:35 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_FRE_B8A.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 241212426 2019-05-29 14:31 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_FRE_B8.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 607057 2019-05-29 14:37 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_MTD_ALL.xml
-rw-r–r-- 1 sen2agri-service sen2agri-service 259075 2019-05-29 14:37 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_QKL_ALL.jpg
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327904 2019-05-29 14:34 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_SRE_B11.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327904 2019-05-29 14:34 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_SRE_B12.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 241212426 2019-05-29 14:28 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_SRE_B2.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 241212426 2019-05-29 14:28 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_SRE_B3.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 241212426 2019-05-29 14:28 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_SRE_B4.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327904 2019-05-29 14:34 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_SRE_B5.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327904 2019-05-29 14:34 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_SRE_B6.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327904 2019-05-29 14:34 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_SRE_B7.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 60327904 2019-05-29 14:34 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_SRE_B8A.tif
-rw-r–r-- 1 sen2agri-service sen2agri-service 241212426 2019-05-29 14:29 SENTINEL2A_20190122-093446-289_L2A_T35VMC_C_V1-0_SRE_B8.tif
Hello,
We encountered also this problem with some MAJA products (only for some orbits and tiles) that are actually blocking the whole processing chain for that orbit/tile. This is because some products seem to be OK but they are producing errors when used in NOMINAL mode for the next products to be processed.
We corrected this and we’ll come with a new version of the system soon.
Meanwhile, you can use the attached scripts to correct the problem.Demmaccs_Patch.zip (33.8 KB)
You will have to do the following:
- unzip the archive
- execute the following commands from the unzipped directory:
cp demmaccs*.py /usr/share/sen2agri/sen2agri-demmaccs
cp sen2agri_common_db.py /usr/lib/python2.7/site-packages/
You will have also to regenerate the incorrect products and the simplest solution would be to reset the l1_tile_history table and the downloader_history table but this assumes to reset the MAJA production:
sudo -u postgres psql sen2agri -c “delete from l1_tile_history”
sudo -u postgres psql sen2agri -c “update downloader_history set status_id = 2 where status_id = 6 and site_id = <YOUR_SITE_ID>”
sudo -u postgres psql sen2agri -c “delete from product where site_id = <YOUR_SITE_ID>"
If you want to reset only the tile/orbit affected and not the other tiles, it is a little bit more complicated but you can do the following:
- search for the products generating the problem:
grep L2NOTV /mnt/archive/maccs_def//l2a///DATA/_JPI_ALL.xml
You will get a list of entries from where you will have to extract the product names like the following one: S2B_MSIL2A_20181112T091219_N0207_R050_T35TPJ_20181112T112419.SAFEFrom these products extract the bolded information above and execute the following similar SQL queries:
delete from product where name like ‘%R050_T35TPJ%’ and created_timestamp >= ‘2018-11-12’;
delete from l1_tile_history where downloader_history_id in (select id from downloader_history where product_name like ‘%R050_T35TPJ%’ and product_date >= ‘2018-11-12’);
update downloader_history set status_id = 2 where product_name like ‘%R050_T35TPJ%’ and product_date >= ‘2018-11-12’;
Please note the date format in the queries (- added in the date from the product name).
This will solve the issue.
Hope this helps.
Best regards,
Cosmin
1 Like
Thank you for the patch.
I have copied patch files to required folders.
Grep command you gave did not work for me. /mnt/archive/maccs_def/[…] folders did not contain files _JPI_ALL.xml to grep anything.
Example of folder for failed product looks like this:
ls /mnt/archive/maccs_def/lietuva/l2a/S2A_MSIL2A_20190122T093311_N0207_R136_T34UFF_20190122T100611.SAFE/
demmaccs_34UFF.log demmaccs.log MACCS_L2REPT_34UFF.EEF
I have queried l1_tile_history table for tiles that had word “MUSCATE” in field ‘failed_reason’, removed those entries from l1_tile_history,
updated related downloader_history table entries to status_id = 2.
table product did not contain any products related with these entries as they have failed
I restarted services, still getting errors as in thread subject:
Here’s an example of entries in l1_tile_history field ‘failed_reason’:
MACCS/MAJA:
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B02.gml !
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B03.gml !
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B04.gml !
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B08.gml !
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B06.gml !
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B07.gml !
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B8A.gml !
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B11.gml !
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B01.gml !
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B02.gml !
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B03.gml !
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B04.gml !
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B06.gml !
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B07.gml !
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B08.gml !
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B8A.gml !
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B09.gml !
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B10.gml !
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
Sentinel2L1ImageFileReaderBase::GetDetFooIndexShift: The detector indices are not in ascending order or do not have a constant shift with the feature indices in the MSK_DETFOO /mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/S2A_MSIL1C_20190115T094341_N0207_R036_T34UFF_20190115T101707.SAFE/GRANULE/L1C_T34UFF_A018623_20190115T094336/../../GRANULE/L1C_T34UFF_A018623_20190115T094336/QI_DATA/MSK_DETFOO_B11.gml !
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
PersistentMaskConditionalStatisticsVectorImageFilter::Synthetize.No pixel is valid (Not masked nb valid pixels=0 and not NoData nb valids pixels total per band=0). Return null statistics
vns::Plugin::ERROR: Sentinel2TML2ImageFileReader(0x87057b0): The MUSCATE product </mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/SENTINEL2B_20190110-094522-720_L2A_T34UFF_C_V1-0/SENTINEL2B_20190110-094522-720_L2A_T34UFF_C_V1-0_MTD_ALL.xml> is not valid according to JPI! [vnsMuscateL2ImageFileReader.txx:MuscateDetectL2Products:368]
****************************************************************************************************** [vnsMajaMainProcessor.cxx:main:130]
ScientificProcessing: vns::Plugin::ERROR: Sentinel2TML2ImageFileReader(0x87057b0): The MUSCATE product </mnt/archive/demmaccs_tmp/tmplNti7G/34UFF/SENTINEL2B_20190110-094522-720_L2A_T34UFF_C_V1-0/SENTINEL2B_20190110-094522-720_L2A_T34UFF_C_V1-0_MTD_ALL.xml> is not valid according to JPI! [vnsMuscateL2ImageFileReader.txx:368] [MAJA Base Exception] [vnsMajaMainProcessor.cxx:main:130]
[vnsMajaMainProcessor.cxx:main:130]
****************************************************************************************************** [vnsMajaMainProcessor.cxx:main:130]
If i understand correctly - messages regarding MSK_DETFOO are warnings, but message about MUSCATE product validity is still the same error.
Where should I look for more details?
Hello,
The problem with the JPI does not refer to the current product (where you saw the error message) but is actually in the previous processed product that is considered as valid (in your case the product containing the subfolder SENTINEL2B_20190110-094522-720_L2A_T34UFF_C_V1-0).
I think the grep command did not worked properly for you because in my post, the grep command was somehow modified (stars are not displayed). The actual command is:
grep L2NOTV /mnt/archive/maccs_def/* /l2a/* /* /DATA/*_JPI_ALL.xml
Please note that I introduced some spaces after the star but need to be removed when executed.
In any case, in your particular situation, you should do the following delete operations (as the product that is introducing the problem is the one from 20190110 and not the current one from 20190122) :
delete from product where name like ‘% R136_T34UFF %’ and created_timestamp >= ‘ 2019-01-10 ’;
delete from l1_tile_history where downloader_history_id in (select id from downloader_history where product_name like ‘% R136_T34UFF %’ and product_date >= ‘ 2019-01-10 ’);
update downloader_history set status_id = 2 where product_name like ‘% R136_T34UFF %’ and product_date >= ‘ 2019-01-10 ’;
This should solve your problem.
Best regards,
Cosmin
1 Like
Everything works like you said, thanks you!