MACCS mode L2INIT

SMIEJ_CRTS · June 6, 2018, 3:13pm

We have a NFS mount on /mnt/archive, MACCS does’nt work for any downloaded S2 L8 product, is it an access problem.

There is a log from MACCS:

2018-06-06 02:52:13.562160:[8280]:Symbolic link L2SITE for tile id 203038 GIPP file could not be found. Searching for the common one 
2018-06-06 02:52:13.574225:[8280]:Copied /mnt/archive/gipp//LANDSAT8/L8_TEST_GIP_L2SITE_S_EUCMN00000A00B_00001_00000000_99999999.EEF to /mnt/archive/demmaccs_tmp/6709/203038/L8_TEST_GIP_L2SITE_S_EU20303800A00B_00001_00000000_99999999.EEF
2018-06-06 02:52:13.579637:[8280]:Symbolic link CKEXTL for tile id 203038 GIPP file could not be found. Searching for the common one 
2018-06-06 02:52:13.589777:[8280]:Copied /mnt/archive/gipp//LANDSAT8/L8_TEST_GIP_CKEXTL_S_EUCMN00000A00B_00001_00000000_99999999.EEF to /mnt/archive/demmaccs_tmp/6709/203038/L8_TEST_GIP_CKEXTL_S_EU20303800A00B_00001_00000000_99999999.EEF
2018-06-06 02:52:13.594792:[8280]:Symbolic link CKQLTL for tile id 203038 GIPP file could not be found. Searching for the common one 
2018-06-06 02:52:13.605614:[8280]:Copied /mnt/archive/gipp//LANDSAT8/L8_TEST_GIP_CKQLTL_S_EUCMN00000A00B_00001_00000000_99999999.EEF to /mnt/archive/demmaccs_tmp/6709/203038/L8_TEST_GIP_CKQLTL_S_EU20303800A00B_00001_00000000_99999999.EEF
2018-06-06 02:52:13.632772:[8280]:No previous processed l2a tile found for 203038 in product LC08_L1TP_203038_20171030_20171109_01_T1. Running MACCS in L2INIT mode
2018-06-06 02:52:13.637095:[8280]:sat_id = 2 | acq_date = 20171030T000000
2018-06-06 02:52:13.640054:[8280]:Starting MACCS in L2INIT for /mnt/archive/demmaccs_tmp/6709/LC08_L1TP_203038_20171030_20171109_01_T1 | TileID: 203038
2018-06-06 02:52:13.643361:[8280]:MACCS_COMMAND: ['/opt/maccs/core/5.1/bin/maccs', '--input', '/mnt/archive/demmaccs_tmp/6709/203038', '--TileId', '203038', '--output', '/mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1//maccs_203038', '--mode', 'L2INIT', '--loglevel', 'DEBUG', '--enableTest', 'false', '--CheckXMLFilesWithSchema', 'false']
2018-06-06 02:52:13.646727:[8280]:Starting command: /opt/maccs/core/5.1/bin/maccs --input /mnt/archive/demmaccs_tmp/6709/203038 --TileId 203038 --output /mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1//maccs_203038 --mode L2INIT --loglevel DEBUG --enableTest false --CheckXMLFilesWithSchema false
2018-06-06 02:52:14.445460:[8280]:Command finished NOK (res = 133) in 0:00:00.797955 : /opt/maccs/core/5.1/bin/maccs --input /mnt/archive/demmaccs_tmp/6709/203038 --TileId 203038 --output /mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1//maccs_203038 --mode L2INIT --loglevel DEBUG --enableTest false --CheckXMLFilesWithSchema false
2018-06-06 02:52:14.449142:[8280]:MACCS mode L2INIT didn't work for /mnt/archive/demmaccs_tmp/6709/LC08_L1TP_203038_20171030_20171109_01_T1 | TileID: 203038. Location /mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1/
2018-06-06 02:52:14.459535:[8280]:Searching for report maccs files (REPT) in: /mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1//maccs_203038
2018-06-06 02:52:14.463006:[8280]:No report maccs files (REPT) found in: /mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1//maccs_203038.
2018-06-06 02:52:14.467163:[8280]:Searching for valid products in MACCS working dir: /mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1//maccs_203038. Following is the content of this dir: ['/mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1//maccs_203038/PMC_LxREPT.EEF']
2018-06-06 02:52:14.470279:[8280]:No valid products (VALD status) found in: /mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1//maccs_203038.
2018-06-06 02:52:14.473480:[8280]:Deleting MACCS working directory: rmtree: /mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1//maccs_203038

We found this when we check permission :

drwxrwxrwx. 2 nobody     nobody       4096 17 mai   13:44 demmaacs_tmp
drwxrwxrwx. 2 nobody     nobody      49152  6 juin  14:47 demmaccs_tmp
drwxrwxrwx. 2 nobody     nobody         64 15 mai   18:07 dwn_def
drwxrwxrwx. 2 nobody     nobody         64 15 mai   16:58 gipp
drwxrwxrwx. 2 nobody     nobody         64  1 mars  09:45 L3B_GeneratedModels
drwxrwxrwx. 2 nobody     nobody         64  5 juin  11:24 maccs_def
drwxrwxrwx. 2 nobody     nobody         64 27 févr. 14:23 orchestrator_temp
drwxrwxrwx. 2 nobody     nobody         64 15 mai   16:55 reference_data
drwxrwxrwx. 2 nobody     nobody       4096 15 mai   16:55 site
drwxrwxrwx. 2 nobody     nobody     262144 15 mai   16:57 srtm
drwxrwxrwx. 2 nobody     nobody         64 27 févr. 16:13 swbd
drwxrwxrwx. 2 nobody     nobody         64 15 mai   17:00 temp
drwxrwxrwx. 2 nobody     nobody         64 15 mai   17:00 temp_maccs

We tried to change owner :

[root@localhost archive]# chown -R sen2agri-service gipp
chown: changing ownership of ‘gipp/LANDSAT8/L8_TEST_GIP_CKEXTL_S_EU93056200A00B_50001_00000000_99999999.EEF’: Permission denied
chown: changing ownership of ‘gipp/LANDSAT8/L8_TEST_GIP_CKEXTL_S_EUCMN00000A00B_00001_00000000_99999999.EEF’: Permission denied
chown: changing ownership of ‘gipp/LANDSAT8/L8_TEST_GIP_CKQLTL_S_EU93056200A00B_50001_00000000_99999999.EEF’: Permission denied
chown: changing ownership of ‘gipp/LANDSAT8/L8_TEST_GIP_CKQLTL_S_EUCMN00000A00B_00001_00000000_99999999.EEF’: Permission denied

We don’t know what to do now ???

lnicola · June 11, 2018, 8:59am

I’m not sure it’s a permissions problem.

Do you still have the MACCS log files? E.g.
/mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1//maccs_203038/PMC_LxREPT.EEF in this case.

SMIEJ_CRTS · June 11, 2018, 10:45am

Many thanks for your response.
Unfortunately in the folder /mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1
I don’t have any subfolder only two files : demmaccs.log and demmaccs_203038.log joined below
demmaccs.log (5.8 KB)
demmaccs_203038.log (7.4 KB)

lnicola · June 11, 2018, 11:03am

All right, please try this:

$ sudo su -l sen2agri-service # avoid issues with permissions
# note the --delete-temp False
$ /usr/share/sen2agri/sen2agri-demmaccs/demmaccs.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd --processes-number-dem 20 --processes-number-maccs 2 --gipp-dir /mnt/archive/gipp/ --working-dir /mnt/archive/demmaccs_tmp/ --maccs-launcher /opt/maccs/core/5.1/bin/maccs --delete-temp False /mnt/archive/dwn_def/l8/default/t29sps/LC08_L1TP_203038_20171030_20171109_01_T1 /mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1/

This should start the processing, then fail immediately after running MACCS. There’ll be a demmaccs_203038.log file with a line like:

/opt/maccs/core/5.1/bin/maccs --input /mnt/archive/demmaccs_tmp/6709/203038 --TileId 203038 --output /mnt/archive/maccs_def/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1//maccs_203038 --mode L2INIT --loglevel DEBUG --enableTest false --CheckXMLFilesWithSchema false

The first number there (6709) is a PID. It will be different in your case. Copy that line and run it, again as the sen2agri-service.

Post here the console output or any other error messages.

SMIEJ_CRTS · June 12, 2018, 12:12pm

Many thanks
I tried relaunching demmaccs.py as you mentionned, the processing doesn’t fail immediately but finished with errors. I attached the command output file the demmaccs.log , demmaccs_203038.log , /mnt/archive/t29sps/l2a/LC08_L2A_203038_20171030_20171109_01_T1/PMC_LxRPT.EEF and screenshots for the folders /mnt/demmacs_tmp/15563, /mnt/demmacs_tmp/15563/203038 and /mnt/demmacs_tmp/15563/LC08_L1TP_203038_20171030_20171109_01_T1 demmaccs_tmp_15563

!
Note that I downloaded all swbd shapefiles and put it in the /mnt/archive/swbd folder.

SMIEJ_CRTS · June 12, 2018, 12:16pm

OOPS I fergot log files
run_demmaccs.txt (404 Bytes)
demmaccs.log (7.3 KB)
demmaccs_203038.log (11.1 KB)