No L2A products on web after fresh 2.0 install

JoaoUrbano · July 1, 2019, 2:24pm

Hello,

Having problems after a fresh install of sen2agri 2 with a site running a single tile (29SQT).

Getting the following sen2agri-demmaccs.timer - Sentinel-2 DEM/MACCS runner

Loaded: loaded (/usr/lib/systemd/system/sen2agri-demmaccs.timer; enabled; vendor preset: disabled)

Active: active (waiting) since Mon 2019-07-01 09:45:09 EDT; 23min ago

Jul 01 09:45:09 localhost.localdomain systemd[1]: Started Sentinel-2 DEM/MACCS runner.

● sen2agri-demmaccs.service - Runs MACCS on L1C products

Loaded: loaded (/usr/lib/systemd/system/sen2agri-demmaccs.service; static; vendor preset: disabled)

Active: inactive (dead) since Mon 2019-07-01 10:08:12 EDT; 46s ago

Process: 11886 ExecStart=/usr/share/sen2agri/sen2agri-demmaccs/demmaccs_launcher.py (code=exited, status=0/SUCCESS)

Main PID: 11886 (code=exited, status=0/SUCCESS)

Jul 01 10:08:12 localhost.localdomain demmaccs_launcher.py[11886]: Starting thread dmworker_0

Jul 01 10:08:12 localhost.localdomain demmaccs_launcher.py[11886]: Starting thread dmworker_1

Jul 01 10:08:12 localhost.localdomain demmaccs_launcher.py[11886]: Waiting for queue to join…

Jul 01 10:08:12 localhost.localdomain demmaccs_launcher.py[11886]: dmworker_0 will consume: None | l2a | /mnt/archive/maccs_def/{site}/l2a/

Jul 01 10:08:12 localhost.localdomain demmaccs_launcher.py[11886]: 2019-07-01 10:08:12.386385:[11886]:dmworker_0: No tile to process. Gracefully closing…

Jul 01 10:08:12 localhost.localdomain demmaccs_launcher.py[11886]: dmworker_1 will consume: None | l2a | /mnt/archive/maccs_def/{site}/l2a/

Jul 01 10:08:12 localhost.localdomain demmaccs_launcher.py[11886]: 2019-07-01 10:08:12.386492:[11886]:dmworker_1: No tile to process. Gracefully closing…

Jul 01 10:08:12 localhost.localdomain demmaccs_launcher.py[11886]: 2019-07-01 10:08:12.386704:[11886]:dmworker_0: Exit thread

Jul 01 10:08:12 localhost.localdomain demmaccs_launcher.py[11886]: 2019-07-01 10:08:12.386782:[11886]:dmworker_1: Exit thread

Jul 01 10:08:12 localhost.localdomain demmaccs_launcher.py[11886]: All the workers finished their job. Exiting…

Can someone help?

Regards,

Joao

firmanhadi · July 12, 2019, 1:17pm

Hi,

I had the same problem.

I read another thread in this links MAJA not processing downloaded images? and Automatic download of images doesn't start.

One of the problems was that MAJA folder owned by root. I changed it to user and group sen2agri-service.

Now, the demmaccs process is running.

Regards,

Firman.

The steps :

Ownership and folder permission
- chown -Rf sen2agri-service:sen2agri-service /opt/maja/3.2.2/
- chmod 777 /opt/maja/3.2.2
Tested demmaccs manually
- /usr/share/sen2agri/sen2agri-demmaccs/demmaccs.py --srtm /mnt/archive/srtm --swbd /mnt/archive/swbd --processes-number-dem 1 --processes-number-maccs 1 --gipp-dir /mnt/archive/gipp_maja --working-dir /mnt/archive/demmaccs_tmp/ --maccs-launcher /opt/maja/3.2.2/bin/maja --delete-temp True /mnt/archive/dwn_def/s2/default/mitr_pohl/S2A_MSIL1C_20171216T035141_N0206_R104_T47PNS_20171216T072628.SAFE /mnt/archive/maccs_def/mitr_pohl/l2a/S2A_MSIL2A_20171216T035141_N0206_R104_T47PNS_20171216T072628.SAFE/ --tiles-to-process 47PNS
- it worked
Update status_id from downloader_history
- update downloader_history set status_id = 2 where status_id = 6;
Reset l1_tile_history
- delete from l1_tile_history;
Restart demmaccs
- systemctl restart sen2agri-demmaccs
Check demmaccs status
- systemctl status sen2agri-demmaccs -l

firmanhadi · July 12, 2019, 2:23pm

Hi,
I forgot to mention that I updated the software to 2.0.1 before running those steps.

Firman.

JoaoUrbano · July 30, 2019, 7:08pm

Hi Firman,

Not having much luck with version 2.01 either, all downloads are currently stopped.

I am getting the following from systemctl
sen2agri-services.service - Services for Sen2Agri
Loaded: loaded (/usr/lib/systemd/system/sen2agri-services.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2019-07-30 19:25:02 +01; 41min ago
Main PID: 10363 (start.sh)
Tasks: 100
CGroup: /system.slice/sen2agri-services.service
├─10363 /bin/bash /usr/share/sen2agri/sen2agri-services/bin/start.sh
└─10369 java -cp …/modules/:…/lib/:…/config/* org.esa.sen2agri.ServicesStartup

Jul 30 20:05:21 localhost.localdomain start.sh[10363]: 2019-Jul-30 20:05:21.798 INFO [USGS-L8-3 ] o.e.s.scheduling.RetryJob - [site ‘zaers’,sensor ‘Landsat8’] No products to retry
Jul 30 20:05:21 localhost.localdomain start.sh[10363]: 2019-Jul-30 20:05:21.799 INFO [USGS-L8-3 ] o.e.s.scheduling.RetryJob - [site ‘zaers’,sensor ‘Landsat8’] No products to retry
Jul 30 20:05:22 localhost.localdomain start.sh[10363]: 2019-Jul-30 20:05:22.044 INFO [USGS-L8-2 ] o.e.s.scheduling.RetryJob - [site ‘chaouia’,sensor ‘Landsat8’] No products to retry
Jul 30 20:05:22 localhost.localdomain start.sh[10363]: 2019-Jul-30 20:05:22.183 INFO [USGS-L8-2 ] o.e.s.scheduling.RetryJob - [site ‘chaouia’,sensor ‘Landsat8’] No products to retry
Jul 30 20:05:25 localhost.localdomain start.sh[10363]: 2019-Jul-30 20:05:25.032 INFO [USGS-L8-1 ] r.c.t.d.usgs.Landsat8Query - Query returned 68 products
Jul 30 20:05:26 localhost.localdomain start.sh[10363]: 2019-Jul-30 20:05:26.366 INFO [USGS-L8-3 ] r.c.t.d.usgs.Landsat8Query - Query returned 69 products
Jul 30 20:05:27 localhost.localdomain start.sh[10363]: 2019-Jul-30 20:05:27.873 INFO [USGS-L8-1 ] r.c.t.d.usgs.Landsat8Query - Query returned 70 products
Jul 30 20:05:31 localhost.localdomain start.sh[10363]: 2019-Jul-30 20:05:31.092 ERROR [DefaultQuartzScheduler_Worker-1] o.e.s.scheduling.LookupJob - ro.cs.tao.datasource.QueryException: ro.cs.tao.datasource.QueryException: java.lang.NullPointerException @ org.esa.sen2agri.scheduling.LookupJob.lookupAndDownload(LookupJob.java:258) < org.esa.sen2agri.scheduling.AbstractJob.execute(AbstractJob.java:65) < org.quartz.core.JobRunShell.run(JobRunShell.java:202) < java.util.concurrent.FutureTask.get(FutureTask.java:192) < java.util.concurrent.FutureTask.report(FutureTask.java:122) < org.esa.sen2agri.scheduling.LookupJob.executeImpl(LookupJob.java:154) < org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Jul 30 20:05:32 localhost.localdomain start.sh[10363]: 2019-Jul-30 20:05:32.394 INFO [USGS-L8-1 ] r.c.t.d.usgs.Landsat8Query - Query returned 69 products
Jul 30 20:05:35 localhost.localdomain start.sh[10363]: 2019-Jul-30 20:05:35.986 ERROR [DefaultQuartzScheduler_Worker-2] o.e.s.scheduling.LookupJob - ro.cs.tao.datasource.QueryException: ro.cs.tao.datasource.QueryException: java.lang.NullPointerException @ org.esa.sen2agri.scheduling.LookupJob.lookupAndDownload(LookupJob.java:258) < org.esa.sen2agri.scheduling.AbstractJob.execute(AbstractJob.java:65) < org.quartz.core.JobRunShell.run(JobRunShell.java:202) < java.util.concurrent.FutureTask.get(FutureTask.java:192) < java.util.concurrent.FutureTask.report(FutureTask.java:122) < org.esa.sen2agri.scheduling.LookupJob.executeImpl(LookupJob.java:154) < org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
[jonaszed@localhost sen2agri]$

I am really frustrated.

Can you help?

Thanks,

Joao

JoaoUrbano · July 31, 2019, 2:40pm

Hi,

I am currently having the following:

and as the result of systemctl status sen2agri-services -l

I get

● sen2agri-services.service - Services for Sen2Agri
Loaded: loaded (/usr/lib/systemd/system/sen2agri-services.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2019-07-31 14:56:13 +01; 43min ago
Main PID: 3418 (start.sh)
Tasks: 102
CGroup: /system.slice/sen2agri-services.service
├─3418 /bin/bash /usr/share/sen2agri/sen2agri-services/bin/start.sh
└─3423 java -cp …/modules/:…/lib/:…/config/* org.esa.sen2agri.ServicesStartup

Jul 31 15:38:51 localhost.localdomain start.sh[3418]: 2019-Jul-31 15:38:51.009 WARN [thread_pool_executor-2] o.s.o.j.v.HibernateJpaDialect - JDBC Connection to reset not identical to originally prepared Connection - please make sure to use connection release mode ON_CLOSE (the default) and to run against Hibernate 4.2+ (or switch HibernateJpaDialect’s prepareConnection flag to false
Jul 31 15:38:51 localhost.localdomain start.sh[3418]: 2019-Jul-31 15:38:51.009 DEBUG [thread_pool_executor-2] r.c.t.s.p.i.ProgressReportServiceImpl - Completed S2B_MSIL1C_20170815T110619_N0205_R137_T29SQT_20170815T111826:S2B_MSIL1C_20170815T110619_N0205_R137_T29SQT_20170815T111826.zip
Jul 31 15:38:51 localhost.localdomain start.sh[3418]: 2019-Jul-31 15:38:51.016 DEBUG [Scientific Data Hub-S2-1] r.c.t.d.r.s.d.Sentinel2ArchiveDownloadStrategy - Begin download for S2A_MSIL1C_20170820T110651_N0205_R137_T29SQT_20170820T111220.zip
Jul 31 15:38:51 localhost.localdomain start.sh[3418]: 2019-Jul-31 15:38:51.016 WARN [thread_pool_executor-1] o.s.o.j.v.HibernateJpaDialect - JDBC Connection to reset not identical to originally prepared Connection - please make sure to use connection release mode ON_CLOSE (the default) and to run against Hibernate 4.2+ (or switch HibernateJpaDialect’s prepareConnection flag to false
Jul 31 15:38:51 localhost.localdomain start.sh[3418]: 2019-Jul-31 15:38:51.016 DEBUG [thread_pool_executor-1] r.c.t.s.p.i.ProgressReportServiceImpl - Started S2A_MSIL1C_20170820T110651_N0205_R137_T29SQT_20170820T111220
Jul 31 15:38:51 localhost.localdomain start.sh[3418]: 2019-Jul-31 15:38:51.017 WARN [thread_pool_executor-2] o.s.o.j.v.HibernateJpaDialect - JDBC Connection to reset not identical to originally prepared Connection - please make sure to use connection release mode ON_CLOSE (the default) and to run against Hibernate 4.2+ (or switch HibernateJpaDialect’s prepareConnection flag to false
Jul 31 15:38:51 localhost.localdomain start.sh[3418]: 2019-Jul-31 15:38:51.017 DEBUG [thread_pool_executor-2] r.c.t.s.p.i.ProgressReportServiceImpl - Started S2A_MSIL1C_20170820T110651_N0205_R137_T29SQT_20170820T111220:S2A_MSIL1C_20170820T110651_N0205_R137_T29SQT_20170820T111220.zip
Jul 31 15:38:51 localhost.localdomain start.sh[3418]: 2019-Jul-31 15:38:51.543 DEBUG [Scientific Data Hub-S2-1] r.c.t.d.r.s.d.Sentinel2ArchiveDownloadStrategy - (S2A_MSIL1C_20170820T110651_N0205_R137_T29SQT_20170820T111220,Archive) S2A_MSIL1C_20170820T110651_N0205_R137_T29SQT_20170820T111220.zip [size: 810789kB]
Jul 31 15:38:51 localhost.localdomain start.sh[3418]: 2019-Jul-31 15:38:51.543 DEBUG [Scientific Data Hub-S2-1] r.c.t.d.r.s.d.Sentinel2ArchiveDownloadStrategy - Local temporary file /mnt/archive/dwn_def/s2/default/zaers/S2A_MSIL1C_20170820T110651_N0205_R137_T29SQT_20170820T111220.zip created
Jul 31 15:38:51 localhost.localdomain start.sh[3418]: 2019-Jul-31 15:38:51.543 DEBUG [Scientific Data Hub-S2-1] r.c.t.d.r.s.d.Sentinel2ArchiveDownloadStrategy - Begin reading from input stream
[jonaszed@localhost mnt]$

I am completely clueless…

SMIEJ_CRTS · September 18, 2019, 10:41am

Hello,
I tried to launch demmaccs manually (with MACCS/MAJA) but it seems that when i changed the owner of /opt/Maja/ to sen2agri-service, the system could not load a shared library:libMajachain.so.3.2, even if the library exist on the folder /opt/ Maja/lib/.
PLease find bellow the error:

2019-09-18 11:08:22.952514:[13081]:Running command: /opt/MAJA/bin/maja --input /mnt/archive/demmaccs_tmp/tmpivdMWw/29SMR --TileId 29SMR --output /mnt/archive/maccs_def/morocco/l2a/S2A_MSIL2A_20190611T110621_N0207_R137_T29SMR_20190611T133459.SAFE/maccs_29SMR --mode L2INIT --loglevel DEBUG --enableTest false --CheckXMLFilesWithSchema false --conf /usr/share/sen2agri/sen2agri-demmaccs/UserConfiguration
/opt/MAJA/bin/maja-processing-3.2.2: error while loading shared libraries: libMajaChain.so.3.2: cannot open shared object file: No such file or directory
2019-09-18 11:08:22.973749:[13081]:Command finished NOK (res = 127) in 0:00:00.021231 : /opt/MAJA/bin/maja --input /mnt/archive/demmaccs_tmp/tmpivdMWw/29SMR --TileId 29SMR --output /mnt/archive/maccs_def/morocco/l2a/S2A_MSIL2A_20190611T110621_N0207_R137_T29SMR_20190611T133459.SAFE/maccs_29SMR --mode L2INIT --loglevel DEBUG --enableTest false --CheckXMLFilesWithSchema false --conf /usr/share/sen2agri/sen2agri-demmaccs/UserConfiguration