Do the system logs (journalctl
like above) contain anything after reprocessing?
Hi,
I installed sen2agri system in server with 2,7TB, RAM 64GB and 2 Intel E-5520 Quadcore 2,16 GHz. I uploaded one site in Spain and started to download but when I check monitoring:
I have errors in L3B product and I donât know why.
Thanks,
Manuel.
For the L4A
you can click the three [Copy to clipboard]
links and paste them here.
For L3B
you could try to use journalctl -u sen2agri-orchestrator
to see why it failed.
Hi,
For L4A I can see
Command:
CropMaskFused.py -ratio 0.75 -nbtrsample 40000 -classifier rf -rseed 0 -pixsize 10 -rfnbtrees 100 -rfmax 25 -rfmin 25 -window 6 -lmbd 2 -eroderad 1 -alpha 0.01 -nbcomp 6 -spatialr 10 -ranger 0.65 -minsize 10 -minarea 20 -siteid 1 -outdir /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/ -targetfolder /mnt/archive/carm/l4a/ -outprops /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/product_properties.txt -refr /mnt/archive/reference_data/ESACCI-LC-L4-LCCS-Map-300m-P5Y-2010-v1.6.1.tif -input /mnt/archive/maccs_def/carm/l2a/S2A_MSIL2A_20170422T110651_N0204_R137_T29SQB_20170422T111300.SAFE/S2A_OPER_SSC_L2VALD_29SQB____20170422.HDR -lut /usr/share/sen2agri/crop-mask.lut
Output:
2018 May 04 22:42:58 : Application.logger (DEBUG) Starting trimming
2018 May 04 22:42:59 : Application.logger (DEBUG) Splitting pixels into classes!
2018 May 04 22:50:55 : Application.logger (DEBUG) Splitting done!
2018 May 04 22:50:55 : Application.logger (DEBUG) Performing Trimming!
2018 May 04 22:50:55 : Application.logger (DEBUG) Trimming done!
2018 May 04 22:51:00 : Application.logger (INFO) Input dataset extent is (-6.75468, 36.907) (-5.47745, 37.9256)
2018 May 04 22:51:00 : Application.logger (INFO) Input dataset projection reference system is: GEOGCS[âGCS_WGS_1984â,DATUM[âWGS_1984â,SPHEROID[âWGS_84â,6378137,298.257223563]],PRIMEM[âGreenwichâ,0],UNIT[âDegreeâ,0.017453292519943295]]
2018 May 04 22:51:00 : Application.logger (INFO) Output projection reference system is: PROJCS[âWGS 84 / UTM zone 29Nâ,
GEOGCS[âWGS 84â,
DATUM[âWGS_1984â,
SPHEROID[âWGS 84â,6378137,298.257223563,
AUTHORITY[âEPSGâ,â7030â]],
AUTHORITY[âEPSGâ,â6326â]],
PRIMEM[âGreenwichâ,0],
UNIT[âdegreeâ,0.0174532925199433],
AUTHORITY[âEPSGâ,â4326â]],
PROJECTION[âTransverse_Mercatorâ],
PARAMETER[âlatitude_of_originâ,0],
PARAMETER[âcentral_meridianâ,-9],
PARAMETER[âscale_factorâ,0.9996],
PARAMETER[âfalse_eastingâ,500000],
PARAMETER[âfalse_northingâ,0],
UNIT[âmetreâ,1,
AUTHORITY[âEPSGâ,â9001â]],
AUTHORITY[âEPSGâ,â32629â]]
2018 May 04 22:51:00 : Application.logger (INFO) Output origin: [699965, 4.2e+06]
2018 May 04 22:51:00 : Application.logger (INFO) Output size: [10980, 10980]
2018 May 04 22:51:00 : Application.logger (INFO) Output spacing: [10, -10]
2018 May 04 22:51:07 : Application.logger (INFO) Image #1 has 1 components
2018 May 04 22:51:07 : Application.logger (INFO) Image #2 has 1 components
Main mission: SENTINEL
Tiles: 1
29SQB 1
Single stratum: True
Increasing NOFILE soft limit from 1024 to 512000
Processing 1 tiles at once
Using 16 threads for each tile
Performing tile preparation (1/2) for tile: 29SQB
Executing PrepareReference_29SQB at 2018-05-04 22:31:13.629294
gdalwarp -q -dstnodata 0 -t_srs âPROJCS[âWGS 84 / UTM zone 29Nâ,
GEOGCS[âWGS 84â,
DATUM[âWGS_1984â,
SPHEROID[âWGS 84â,6378137,298.257223563,
AUTHORITY[âEPSGâ,â7030â]],
AUTHORITY[âEPSGâ,â6326â]],
PRIMEM[âGreenwichâ,0],
UNIT[âdegreeâ,0.0174532925199433],
AUTHORITY[âEPSGâ,â4326â]],
PROJECTION[âTransverse_Mercatorâ],
PARAMETER[âlatitude_of_originâ,0],
PARAMETER[âcentral_meridianâ,-9],
PARAMETER[âscale_factorâ,0.9996],
PARAMETER[âfalse_eastingâ,500000],
PARAMETER[âfalse_northingâ,0],
UNIT[âmetreâ,1,
AUTHORITY[âEPSGâ,â9001â]],
AUTHORITY[âEPSGâ,â32629â]]â -te 699960.0 4090200.0 809760.0 4200000.0 -tr 10 10 -overwrite /mnt/archive/reference_data/ESACCI-LC-L4-LCCS-Map-300m-P5Y-2010-v1.6.1.tif /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/reference-29SQB.tif
PrepareReference_29SQB done at 2018-05-04 22:31:18.343074. Duration: 0:00:04.713780
Executing Erosion_29SQB at 2018-05-04 22:31:18.343227
otbcli Erosion ââ -progress false -radius 1 -in /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/reference-29SQB.tif -out /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/reference-eroded-29SQB.tif
Erosion_29SQB done at 2018-05-04 22:37:13.635760. Duration: 0:05:55.292533
Executing SpectralFeatures_29SQB at 2018-05-04 22:37:13.635956
otbcli SpectralFeaturesExtraction ââ -progress false -mission SENTINEL -pixsize 10 -lambda 2 -out /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/spectral-features-29SQB.tif -rededge true -il /mnt/archive/maccs_def/carm/l2a/S2A_MSIL2A_20170422T110651_N0204_R137_T29SQB_20170422T111300.SAFE/S2A_OPER_SSC_L2VALD_29SQB____20170422.HDR
SpectralFeatures_29SQB done at 2018-05-04 22:42:56.561200. Duration: 0:05:42.925244
Performing tile preparation (2/2) for tile: 29SQB
Executing Trimming_29SQB at 2018-05-04 22:42:56.610220
otbcli Trimming ââ -progress false -alpha 0.01 -nbsamples 0 -seed 0 -feat /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/spectral-features-29SQB.tif -ref /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/reference-eroded-29SQB.tif -out /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/reference-trimmed-29SQB.tif
Trimming_29SQB done at 2018-05-04 22:50:57.706061. Duration: 0:08:01.095841
Building model for stratum: 0
Reference raster for tile: /mnt/archive/maccs_def/carm/l2a/S2A_MSIL2A_20170422T110651_N0204_R137_T29SQB_20170422T111300.SAFE/S2A_OPER_SSC_L2VALD_29SQB____20170422.DBL.DIR/S2A_OPER_SSC_PDTIMG_L2VALD_29SQB____20170422_FRE_R1.DBL.TIF
Executing Rasterize mask at 2018-05-04 22:50:57.785004
otbcli_Rasterization -progress false -in /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/tile-mask-0-29SQB.shp -im /mnt/archive/maccs_def/carm/l2a/S2A_MSIL2A_20170422T110651_N0204_R137_T29SQB_20170422T111300.SAFE/S2A_OPER_SSC_L2VALD_29SQB____20170422.DBL.DIR/S2A_OPER_SSC_PDTIMG_L2VALD_29SQB____20170422_FRE_R1.DBL.TIF -out â/mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/classification-mask-0-29SQB.tif?gdal:co:COMPRESS=DEFLATEâ uint8 -mode binary
Rasterize mask done at 2018-05-04 22:51:05.368532. Duration: 0:00:07.583528
Executing BandMath_29SQB at 2018-05-04 22:51:05.368719
otbcli_BandMath -progress false -exp âim1b1 > 0 ? im2b1 : -10000â -il /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/classification-mask-0-29SQB.tif /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/reference-trimmed-29SQB.tif -out â/mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/reference-trimmed-0-29SQB.tif?gdal:co:COMPRESS=DEFLATEâ int16
BandMath_29SQB done at 2018-05-04 22:51:14.301648. Duration: 0:00:08.932929
Executing TrainImagesClassifier at 2018-05-04 22:51:14.301886
otbcli TrainImagesClassifierNew ââ -nodatalabel -10000 -rand 0 -sample.bm 0 -io.confm2018 May 04 22:51:16 : Application.logger (DEBUG) Entering DoExecute
2018 May 04 22:51:17 : Application.logger (INFO) Processing input 0
2018 May 04 22:51:17 : Application.logger (FATAL) The following error occurred during application execution : /home/cudroiu/sen2agri/sen2agri-processors/Common/TrainImagesClassifierNew/otbListSampleGeneratorRaster.txx:315:
itk::ERROR: No training sample found inside image
atout /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/confusion-matrix-training-0.csv -io.out /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/model-0.txt -sample.mt 40000 -sample.mv 1000 -sample.vfn CROP -sample.vtr 0.01 -classifier rf -classifier.rf.nbtrees 100 -classifier.rf.min 25 -classifier.rf.max 25 -io.rs /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/reference-trimmed-0-29SQB.tif -io.il /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/spectral-features-29SQB.tif
Error running TrainImagesClassifier at 2018-05-04 22:51:17.674916. The call returned 1
Processor finished in 0:20:04.059105
Can you compare the reference map /mnt/archive/reference_data/ESACCI-LC-L4-LCCS-Map-300m-P5Y-2010-v1.6.1.tif
against your tile extent?
Or even better, you can take a look at /mnt/archive/orchestrator_temp/l4a/10/3-crop-mask-fused/reference-trimmed-0-29SQB.tif
.
Itâs possible that the processor didnât extract enough training samples over your tile.
By one hand, if I compare the ESACCI-LC-L4-LCCS-Map-300m-P5Y-2010-v1.6.1.tif and my file extent (CARM):
By other, in /mnt/archive/orchestrator_temp/l4a/ I donât have anything:
Anyway, for L4A product Error also:
Computing class counts
Traceback (most recent call last):
File â/usr/lib/python2.7/site-packages/sen2agri_common.pyâ, line 581, in execute
pool.map(self.internal_train_stratum, self.strata)
File â/usr/lib64/python2.7/multiprocessing/pool.pyâ, line 250, in map
return self.map_async(func, iterable, chunksize).get()
File â/usr/lib64/python2.7/multiprocessing/pool.pyâ, line 554, in get
raise self._value
Exception: (âError running TrainImagesClassifierâ, 1)
Finally, if the processor didnât extract enough training samples over your tile, which is it a possible solution?
It doesnât actually matter, but can you also overlay the 29SQB
tile extent? (or send my the coordinates of CARM
)? It seems to be a little far from your area, the following is 29SQB
:
Also, note that the processor is only looking for pixels with values 10
, 11
and 20
in that reference map. Iâve highlighted those in the image above. There seem to be a lot of green them in that area.
There is, however, a filtering step which uses a Mahalanobis trimming process to exclude some of those pixels. I noticed youâre only using two acquisitions, which isnât a lot, although I would be surprised if every last one was removed.
I would suggest either:
- keeping the temporary files so that we can check the trimming output (run
sudo -u postgres sen2agri psql
and in there runupdate config set value = '1' where key = 'executor.processor.l4a.keep_job_folders';
) - or using more acquisitions and maybe more tiles (
30STG
and maybe30SUG
?) as input
CARM has a 200.000ha and the coordinates are:
I donât know why is 29SQBâŚYes, it is far from my area.
Can you check the site extent and tile filter in the database? The site is in the site
table, and the tiles are in the site_tiles
table. You can connect with sudo -u postgres psql sen2agri
. Feel free to ask if you need help with that.
CARM has a 200.000ha and the coordinates are:
Is that EPSG:32630
?
I noticed youâre only using two acquisitions, which isnât a lot
I misread the logs, you only have one. You really need more acquisitions for the processor to work.
Sen2agri downloaded automatic all acquisitonsâŚ
the projection is ETRS89 (30N) and EPSG:3042
Ok⌠but now what I do?
Thanks for all
Anyway, if the problem is the size of site Iâm going to add one shapefile of Sevilla province and I will check if all products can be generated.
So itâs something like this:
You could check:
-- mage the output easier to read
\x
\pset pager off
-- the site extent, as defined in the database
select id, name from site;
select st_astext(geog) from site where id = 72; -- replace 72 with the id from above
-- check the site tiles, if filtered
select * from site_tiles where site_id = 72; -- replace 72 with the site id
-- look at the available L2A products
select name from product where site_id = 72 and processor_id = 1;
DId the job start automatically? Itâs possible that not enough L2A
were available at the time.
I got now:
**(Sin filas) = (No rows)
Do you see any strange?
By other hand, I uploaded 4 provinces of Spain and I am waitiing to check what the system is going to do.
Many thanks for all.
Hi Inicola,
Here is the system logs after forcing reprocessing
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[3999]: 2018-06-01 13:50:41.732431:[4279]:Starting command: /data/maccs/ --input /mnt/archive/demmaccs_tmp/4015/169078 --TileId 169078 --output /mnt/archive/maccs_def/test_11/l2a/LC08_L2A_169078_20161114_20170318_01_T1//maccs_169078 --mode L2INIT --loglevel DEBUG --enableTest false --CheckXMLFilesWithSchema false
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[3999]: Traceback (most recent call last):
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[3999]: File â/usr/share/sen2agri/sen2agri-demmaccs/demmaccs.pyâ, line 456, in
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[3999]: pool_outputs = pool.map(maccs_launcher, demmaccs_contexts)
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[3999]: File â/usr/lib64/python2.7/multiprocessing/pool.pyâ, line 250, in map
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[3999]: return self.map_async(func, iterable, chunksize).get()
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[3999]: File â/usr/lib64/python2.7/multiprocessing/pool.pyâ, line 554, in get
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[3999]: raise self._value
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[3999]: OSError: [Errno 13] Permission denied
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[3999]: 2018-06-01 13:50:41.756148:[4006]:Command finished NOK (res = 1) in 0:02:41.446013 : /usr/share/sen2agri/sen2agri-demmaccs/demmaccs.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd --processes-number-dem 20 --processes-number-maccs 2 --gipp-dir /mnt/archive/gipp/ --working-dir /mnt/archive/demmaccs_tmp/ --maccs-launcher /data/maccs/ --delete-temp True /mnt/archive/dwn_def/l8/default/test_11/LC08_L1TP_169078_20161114_20170318_01_T1 /mnt/archive/maccs_def/test_11/l2a/LC08_L2A_169078_20161114_20170318_01_T1/
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[3999]: 2018-06-01 13:50:41.757405:[4006]:demmaccs.py script didnât work!
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[3999]: 2018-06-01 13:50:41.758218:[4006]:Only set the state as processed in downloader_history (no l2a tiles found after maccs) for product /mnt/archive/maccs_def/test_11/l2a/LC08_L2A_169078_20161114_20170318_01_T1/
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za systemd[1]: Started Runs MACCS on L1C products.
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za systemd[1]: Starting Runs MACCS on L1C productsâŚ
Jun 01 13:50:41 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[4304]: 2018-06-01 13:50:41.958993:[4304]:No unprocessed L1C found in DB
Jun 01 13:51:00 mitkldsa01.ict.my-it.co.za systemd[1]: Started Runs MACCS on L1C products.
Jun 01 13:51:00 mitkldsa01.ict.my-it.co.za systemd[1]: Starting Runs MACCS on L1C productsâŚ
Jun 01 13:52:00 mitkldsa01.ict.my-it.co.za systemd[1]: Started Runs MACCS on L1C products.
Jun 01 13:52:00 mitkldsa01.ict.my-it.co.za systemd[1]: Starting Runs MACCS on L1C productsâŚ
Jun 01 13:53:00 mitkldsa01.ict.my-it.co.za systemd[1]: Started Runs MACCS on L1C products.
Jun 01 13:53:00 mitkldsa01.ict.my-it.co.za systemd[1]: Starting Runs MACCS on L1C productsâŚ
Jun 01 13:53:00 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[4401]: 2018-06-01 13:53:00.583977:[4401]:No unprocessed L1C found in DB
Jun 01 13:54:00 mitkldsa01.ict.my-it.co.za systemd[1]: Started Runs MACCS on L1C products.
Jun 01 13:54:00 mitkldsa01.ict.my-it.co.za systemd[1]: Starting Runs MACCS on L1C productsâŚ
Jun 01 13:54:00 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[4490]: 2018-06-01 13:54:00.650697:[4490]:No unprocessed L1C found in DB
Jun 01 13:55:00 mitkldsa01.ict.my-it.co.za systemd[1]: Started Runs MACCS on L1C products.
Jun 01 13:55:00 mitkldsa01.ict.my-it.co.za systemd[1]: Starting Runs MACCS on L1C productsâŚ
Jun 01 13:55:00 mitkldsa01.ict.my-it.co.za demmaccs_launcher.py[4538]: 2018-06-01 13:55:00.777881:[4538]:No unprocessed L1C found in DB
Hi,
can you check the permissions on /mnt/archive/demmaccs_tmp/
and /mnt/archive/maccs_def/
? They should be owned by sen2agri-service
.