Hi all,
I don’t know why for the new site (see below),there is no landsat image downloaded for the latest system. in the last version landsat image was correctly downloaded.
Could anyone help me for solving this problem
Regards
Hi all,
I don’t know why for the new site (see below),there is no landsat image downloaded for the latest system. in the last version landsat image was correctly downloaded.
Could anyone help me for solving this problem
Regards
you can see the log by “sudo systemctl ststus sen2agri-services.service -l”
Many thanks for your answer. Please find result for systemctl command
sen2agri-services.service - Services for Sen2Agri
Loaded: loaded (/usr/lib/systemd/system/sen2agri-services.service; enabled; vendor preset: disabled)
Active: active (running) since lun. 2018-10-29 14:34:26 WET; 1 months 7 days ago
Main PID: 3741 (start.sh)
CGroup: /system.slice/sen2agri-services.service
├─3741 /bin/bash /usr/share/sen2agri/sen2agri-services/bin/start.sh
└─3754 java -cp …/modules/:…/lib/ org.esa.sen2agri.ServicesStartup
déc. 06 09:20:13 localhost.localdomain start.sh[3741]: 2018-Dec-06 09:20:13.916 INFO [Scientific Data HubSentinel2-1] r.c.t.d.r.s.SciHubDataQuery - [29RQQ, 29SPS, 29RNQ, 29SPR, 29SQS, 29SQR, 29RPQ, 29SNS, 29SNR] POLYGON((-7.9465 31.5277,-6.7664 31.5277,-6.7664 32.5328,-7.9465 32.5328,-7.9465 31.5277)) footprint:“Intersects(POLYGON((-7.9465 31.5277,-6.7664 31.5277,-6.7664 32.5328,-7.9465 32.5328,-7.9465 31.5277)))” AND beginPosition:[2018-12-06T00:00:00.000Z TO 2018-12-06T09:20:07.000Z] AND endPosition:[2018-12-06T00:00:00.000Z TO 2018-12-06T09:20:07.000Z] AND tileId:29SPR AND cloudcoverpercentage:[0.0 TO 100.0] AND platformName:Sentinel-2 AND productType:S2MSI1C
déc. 06 09:20:13 localhost.localdomain start.sh[3741]: 2018-Dec-06 09:20:13.916 DEBUG [Scientific Data HubSentinel2-1] r.c.t.d.r.s.SciHubDataQuery - Executing query: https://scihub.copernicus.eu/apihub/search?rows=50&start=1&q=footprint%3A"Intersects(POLYGON((-7.9465%2031.5277%2C-6.7664%2031.5277%2C-6.7664%2032.5328%2C-7.9465%2032.5328%2C-7.9465%2031.5277)))"%20AND%20beginPosition%3A[2018-12-06T00%3A00%3A00.000Z%20TO%202018-12-06T09%3A20%3A07.000Z]%20AND%20endPosition%3A[2018-12-06T00%3A00%3A00.000Z%20TO%202018-12-06T09%3A20%3A07.000Z]%20AND%20tileId%3A29SPR%20AND%20cloudcoverpercentage%3A[0.0%20TO%20100.0]%20AND%20platformName%3ASentinel-2%20AND%20productType%3AS2MSI1C&orderby=beginposition%20asc
déc. 06 09:20:14 localhost.localdomain start.sh[3741]: 2018-Dec-06 09:20:14.624 INFO [Scientific Data HubSentinel2-1] r.c.t.d.r.s.SciHubDataQuery - Query returned 0 products
déc. 06 09:20:14 localhost.localdomain start.sh[3741]: 2018-Dec-06 09:20:14.642 INFO [Scientific Data HubSentinel2-1] r.c.t.d.r.s.SciHubDataQuery - [30STC, 30STB, 29SPS, 29SQT, 29SPR, 29SQS, 29SQR, 30STA, 29SPT] POLYGON((-6.8732 32.4061,-5.6699 32.4061,-5.6699 33.4209,-6.8732 33.4209,-6.8732 32.4061)) footprint:“Intersects(POLYGON((-6.8732 32.4061,-5.6699 32.4061,-5.6699 33.4209,-6.8732 33.4209,-6.8732 32.4061)))” AND beginPosition:[2018-12-06T00:00:00.000Z TO 2018-12-06T09:20:07.000Z] AND endPosition:[2018-12-06T00:00:00.000Z TO 2018-12-06T09:20:07.000Z] AND tileId:29SQS AND cloudcoverpercentage:[0.0 TO 100.0] AND platformName:Sentinel-2 AND productType:S2MSI1C
déc. 06 09:20:14 localhost.localdomain start.sh[3741]: 2018-Dec-06 09:20:14.642 DEBUG [Scientific Data HubSentinel2-1] r.c.t.d.r.s.SciHubDataQuery - Executing query: https://scihub.copernicus.eu/apihub/search?rows=50&start=1&q=footprint%3A"Intersects(POLYGON((-6.8732%2032.4061%2C-5.6699%2032.4061%2C-5.6699%2033.4209%2C-6.8732%2033.4209%2C-6.8732%2032.4061)))"%20AND%20beginPosition%3A[2018-12-06T00%3A00%3A00.000Z%20TO%202018-12-06T09%3A20%3A07.000Z]%20AND%20endPosition%3A[2018-12-06T00%3A00%3A00.000Z%20TO%202018-12-06T09%3A20%3A07.000Z]%20AND%20tileId%3A29SQS%20AND%20cloudcoverpercentage%3A[0.0%20TO%20100.0]%20AND%20platformName%3ASentinel-2%20AND%20productType%3AS2MSI1C&orderby=beginposition%20asc
déc. 06 09:20:14 localhost.localdomain start.sh[3741]: 2018-Dec-06 09:20:14.989 INFO [Scientific Data HubSentinel2-1] r.c.t.d.r.s.SciHubDataQuery - Query returned 0 products
déc. 06 09:20:15 localhost.localdomain start.sh[3741]: 2018-Dec-06 09:20:15.013 INFO [Scientific Data HubSentinel2-1] r.c.t.d.r.s.SciHubDataQuery - [29RQQ, 30STB, 30RTV, 29SPS, 29SPR, 29SQS, 29SQR, 30STA, 29RPQ] POLYGON((-6.8939 31.5052,-5.7035 31.5052,-5.7035 32.5193,-6.8939 32.5193,-6.8939 31.5052)) footprint:“Intersects(POLYGON((-6.8939 31.5052,-5.7035 31.5052,-5.7035 32.5193,-6.8939 32.5193,-6.8939 31.5052)))” AND beginPosition:[2018-12-06T00:00:00.000Z TO 2018-12-06T09:20:07.000Z] AND endPosition:[2018-12-06T00:00:00.000Z TO 2018-12-06T09:20:07.000Z] AND tileId:29SQR AND cloudcoverpercentage:[0.0 TO 100.0] AND platformName:Sentinel-2 AND productType:S2MSI1C
déc. 06 09:20:15 localhost.localdomain start.sh[3741]: 2018-Dec-06 09:20:15.013 DEBUG [Scientific Data HubSentinel2-1] r.c.t.d.r.s.SciHubDataQuery - Executing query: https://scihub.copernicus.eu/apihub/search?rows=50&start=1&q=footprint%3A"Intersects(POLYGON((-6.8939%2031.5052%2C-5.7035%2031.5052%2C-5.7035%2032.5193%2C-6.8939%2032.5193%2C-6.8939%2031.5052)))"%20AND%20beginPosition%3A[2018-12-06T00%3A00%3A00.000Z%20TO%202018-12-06T09%3A20%3A07.000Z]%20AND%20endPosition%3A[2018-12-06T00%3A00%3A00.000Z%20TO%202018-12-06T09%3A20%3A07.000Z]%20AND%20tileId%3A29SQR%20AND%20cloudcoverpercentage%3A[0.0%20TO%20100.0]%20AND%20platformName%3ASentinel-2%20AND%20productType%3AS2MSI1C&orderby=beginposition%20asc
déc. 06 09:20:15 localhost.localdomain start.sh[3741]: 2018-Dec-06 09:20:15.158 INFO [Scientific Data HubSentinel2-1] r.c.t.d.r.s.SciHubDataQuery - Query returned 0 products
déc. 06 09:20:15 localhost.localdomain start.sh[3741]: 2018-Dec-06 09:20:15.158 INFO [DefaultQuartzScheduler_Worker-2] o.e.s.scheduling.LookupJob - [site ‘Morocco_Centre’,sensor ‘Sentinel2’] Query returned 0 products
As you can see, in the running services there is no query made for USGS
can you show you /usr/share/sen2agri/sen2agri-service/services.properties, i think maybe it has some problem. when system run, it will read this config. i think you modifed by the website but the file no update.
Hi @xiashenzhen,
I have a problem too, I got no downloads (no Landsat 8 or Senntinel 2 images downloaded!), and here is what I got for the command systemctl :
sen2agri-services.service - Services for Sen2Agri
Loaded: loaded (/usr/lib/systemd/system/sen2agri-services.service; enabled; vendor preset: disabled)
Active: active (running) since jeu. 2018-12-06 13:17:07 WET; 1 day 22h ago
Main PID: 12586 (start.sh)
CGroup: /system.slice/sen2agri-services.service
├─12586 /bin/bash /usr/share/sen2agri/sen2agri-services/bin/start.sh
└─12592 java -cp …/modules/:…/lib/:…/config/* org.esa.sen2agri.ServicesStartup
déc. 08 11:17:29 lim.sudmed start.sh[12586]: at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
déc. 08 11:17:29 lim.sudmed start.sh[12586]: Caused by: ro.cs.tao.datasource.QueryException: The supplied credentials are invalid!
déc. 08 11:17:29 lim.sudmed start.sh[12586]: at ro.cs.tao.datasource.remote.scihub.SciHubDataQuery.getCount(SciHubDataQuery.java:182)
déc. 08 11:17:29 lim.sudmed start.sh[12586]: at org.esa.sen2agri.services.internal.DownloadServiceImpl.count(DownloadServiceImpl.java:139)
déc. 08 11:17:29 lim.sudmed start.sh[12586]: at org.esa.sen2agri.scheduling.LookupJob.lambda$lookupAndDownload$1(LookupJob.java:203)
déc. 08 11:17:29 lim.sudmed start.sh[12586]: at java.util.concurrent.FutureTask.run(FutureTask.java:266)
déc. 08 11:17:29 lim.sudmed start.sh[12586]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
déc. 08 11:17:29 lim.sudmed start.sh[12586]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
déc. 08 11:17:29 lim.sudmed start.sh[12586]: at java.lang.Thread.run(Thread.java:748)
déc. 08 11:17:29 lim.sudmed start.sh[12586]: 2018-déc.-08 11:17:29.687 ERROR [DefaultQuartzScheduler_Worker-7] o.e.s.scheduling.LookupJob - ro.cs.tao.datasource.QueryException: The supplied credentials are invalid! @ org.esa.sen2agri.scheduling.LookupJob.lookupAndDownload(LookupJob.java:204) < org.quartz.core.JobRunShell.run(JobRunShell.java:202) < java.util.concurrent.FutureTask.get(FutureTask.java:192) < org.esa.sen2agri.scheduling.AbstractJob.execute(AbstractJob.java:48) < org.esa.sen2agri.scheduling.LookupJob.executeImpl(LookupJob.java:112) < java.util.concurrent.FutureTask.report(FutureTask.java:122) < org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
is there any problem?
Thanks in advance
yes, this config also has problem. And you can see "DefaultQuartzScheduler_Worker-7] o.e.s.scheduling.LookupJob - ro.cs.tao.datasource.QueryException: The supplied credentials are invalid! "
you can restart all service .
sudo systemctl restart sen2agri-demmaccs.service
sen2agri-executor.service
sen2agri-monitor-agent.service
sen2agri-scheduler.service
sen2agri-demmaccs.timer
sen2agri-http-listener.service
sen2agri-orchestrator.service
sen2agri-services.service
Then check the executor and services
sudo systemctl status
sen2agri-executor.service
sen2agri-monitor-agent.service
sen2agri-scheduler.service
sen2agri-http-listener.service
sen2agri-orchestrator.service
sen2agri-services.service -l
Dear xiashenzhen
Please find below the content of services.properties file :
server.port = 8081
spring.datasource.url=jdbc:postgresql://localhost:5432/sen2agri?stringtype=unspecified
spring.datasource.username=admin
spring.datasource.password=sen2agri
log.level=WARNING
###########################
SciHubDataSource.username=SMIEJ_CRTS
SciHubDataSource.password=xxxxxxxxxxxxxxxxxx
USGSDataSource.username=smiej@crts.gov.ma
USGSDataSource.password=xxxxxxxxxxxxxxxxxxx
################################################
SciHubDataSource.Sentinel1.scope=3
SciHubDataSource.Sentinel2.scope=3
AWSDataSource.Sentinel2.scope=2
SciHubDataSource.Sentinel2.local_archive_path=
SciHubDataSource.Sentinel2.fetch_mode=1
USGSDataSource.Landsat8.scope=3
AWSDataSource.Landsat8.scope=2
SciHubDataSource.Sentinel1.enabled=true
SciHubDataSource.Sentinel2.enabled=true
AWSDataSource.Sentinel2.enabled=false
USGSDataSource.Landsat8.enabled=true
AWSDataSource.Landsat8.enabled=false
Hello @xiashenzhen,
Many thanks for you replies!
I executed the commands, I restaeted all services and I got for systemctl :
sen2agri-services.service - Services for Sen2Agri
Loaded: loaded (/usr/lib/systemd/system/sen2agri-services.service; enabled; vendor preset: disabled)
Active: active (running) since lun. 2018-12-10 10:03:55 WET; 17min ago
Main PID: 18146 (start.sh)
CGroup: /system.slice/sen2agri-services.service
├─18146 /bin/bash /usr/share/sen2agri/sen2agri-services/bin/start.sh
└─18152 java -cp …/modules/:…/lib/:…/config/* org.esa.sen2agri.ServicesStartup
déc. 10 10:19:18 lim.sudmed start.sh[18146]: at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
déc. 10 10:19:18 lim.sudmed start.sh[18146]: Caused by: ro.cs.tao.datasource.QueryException: The supplied credentials are invalid!
déc. 10 10:19:18 lim.sudmed start.sh[18146]: at ro.cs.tao.datasource.remote.scihub.SciHubDataQuery.getCount(SciHubDataQuery.java:182)
déc. 10 10:19:18 lim.sudmed start.sh[18146]: at org.esa.sen2agri.services.internal.DownloadServiceImpl.count(DownloadServiceImpl.java:139)
déc. 10 10:19:18 lim.sudmed start.sh[18146]: at org.esa.sen2agri.scheduling.LookupJob.lambda$lookupAndDownload$1(LookupJob.java:203)
déc. 10 10:19:18 lim.sudmed start.sh[18146]: at java.util.concurrent.FutureTask.run(FutureTask.java:266)
déc. 10 10:19:18 lim.sudmed start.sh[18146]: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
déc. 10 10:19:18 lim.sudmed start.sh[18146]: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
déc. 10 10:19:18 lim.sudmed start.sh[18146]: at java.lang.Thread.run(Thread.java:748)
déc. 10 10:19:18 lim.sudmed start.sh[18146]: 2018-déc.-10 10:19:18.502 ERROR [DefaultQuartzScheduler_Worker-1] o.e.s.scheduling.LookupJob - ro.cs.tao.datasource.QueryException: The supplied credentials are invalid! @ org.esa.sen2agri.scheduling.LookupJob.lookupAndDownload(LookupJob.java:204) < org.quartz.core.JobRunShell.run(JobRunShell.java:202) < java.util.concurrent.FutureTask.get(FutureTask.java:192) < org.esa.sen2agri.scheduling.AbstractJob.execute(AbstractJob.java:48) < org.esa.sen2agri.scheduling.LookupJob.executeImpl(LookupJob.java:112) < java.util.concurrent.FutureTask.report(FutureTask.java:122) < org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Then I executed my sen2agri, but still have no downloads:
hello, this is my config. it run well and you can check!
server.port = 8081
spring.datasource.url=jdbc:postgresql://localhost:5432/sen2agri?stringtype=unspecified
spring.datasource.username=admin
spring.datasource.password=sen2agri
################################################
database.config.polling=15
################################################
network.connexions.timeout=300
###########################
SciHubDataSource.username=*********
SciHubDataSource.password=*********
USGSDataSource.username=********
USGSDataSource.password=******
################################################
SciHubDataSource.Sentinel1.scope=3
SciHubDataSource.Sentinel2.scope=3
AWSDataSource.Sentinel2.scope=1
USGSDataSource.Landsat8.scope=3
AWSDataSource.Landsat8.scope=1
#SciHubDataSource.Sentinel2.local_archive_path=/mnt/share/sentinel/tiles/
#SciHubDataSource.Sentinel2.fetch_mode=4
SciHubDataSource.Sentinel1.enabled=false
SciHubDataSource.Sentinel2.enabled=true
AWSDataSource.Sentinel2.enabled=false
USGSDataSource.Landsat8.enabled=true
AWSDataSource.Landsat8.enabled=false
AWSDataSource.Sentinel2.usePadding=true
AWSDataSource.Landsat8.usePadding=true
AWSDataSource.Sentinel2.local.archive.path.format = yyyy/M/d
you can check the download table
sudo -u postgres psql sen2agri -c
“select downloader_history.id, product_name, status_description
from downloader_history inner
join downloader_status
on downloader_status.id = downloader_history.status_id;”
Many thanks @xiashenzhen for your replies and support!
I write the command then I got a lot of warnings:
sudo -u postgres psql sen2agri -c “select downloader_history.id, product_name, status_description from downloader_history inner join downloader_status on downloader_status.id =downloader_history.status_id;”
could not change directory to “/root”
psql: warning: extra command-line argument “product_name,” ignored
psql: warning: extra command-line argument “status_description” ignored
psql: warning: extra command-line argument “from” ignored
psql: warning: extra command-line argument “downloader_history” ignored
psql: warning: extra command-line argument “inner” ignored
psql: warning: extra command-line argument “join” ignored
psql: warning: extra command-line argument “downloader_status” ignored
psql: warning: extra command-line argument “on” ignored
psql: warning: extra command-line argument “downloader_status.id” ignored
psql: warning: extra command-line argument “=downloader_history.status_id” ignored
psql: FATAL: authentification peer échouée pour l’utilisateur « downloader_history.id, »
-bash: ” : commande introuvable
i think the last one “psql: FATAL: authentification peer échouée pour l’utilisateur « downloader_history.id” is the reason. you can update the software for the table. the update script in Sen2AgriDistribution/install_script/update.sh . it maybe can help you.
Hello,
Thank you much for your help and support, I will try this suggestion too, hoping it will work!
Hi @xiashenzhen,
I already run “update.sh” file, and I got this:
Failed to stop sen2agri-sentinel-downloader.service: Unit sen2agri-sentinel-downloader.service not loaded.
Failed to stop sen2agri-landsat-downloader.service: Unit sen2agri-landsat-downloader.service not loaded.
Failed to stop sen2agri-sentinel-downloader.timer: Unit sen2agri-sentinel-downloader.timer not loaded.
Failed to stop sen2agri-landsat-downloader.timer: Unit sen2agri-landsat-downloader.timer not loaded.
Modules complémentaires chargés : fastestmirror, langpacks
Loading mirror speeds from cached hostfile
epel/x86_64/metalink | 51 kB 00:00:00
Thank you for your replies!
you can check the next step:
a. modifed the sen2agri-services.properties
b. check the user “sen2agri-services”
c. shut down your firewall and selinux
d. restart system sen2agri-services.service and show log by do **systemctl status sen2agri-services.service -l
Hi @xiashenzhen,
Many thanks for your replies and help.
I already uninstalled sen2agri system, so during this procedure, I got the following display when I want to stop slurm and munge
$ systemctl stop Slurmd Slurmctld Slurmdbd munge
Failed to stop Slurmd.service: Unit Slurmd.service not loaded.
Failed to stop Slurmctld.service: Unit Slurmctld.service not loaded.
Failed to stop Slurmdbd.service: Unit Slurmdbd.service not loaded.
Failed to stop munge.service: Unit munge.service not loaded.
the same thing for stoping Apatche
Then I started re-installing sen2agri system, using the command
sudo ./sen2agriPlatformInstallAndConfig.sh
but I got this
Checking paths…
Creating /mnt/archive/reference_data
Copying reference data
Disabling SELinux
The Sen2Agri system is not inherently incompatible with SELinux, but relabelling the file system paths is not implemented yet in the installer.
Disabling the firewall
Warning: ZONE_ALREADY_SET: trusted
success
success
Modules complémentaires chargés : fastestmirror, langpacks
Loading mirror speeds from cached hostfile
I don’t know what does this warning mean? and if I should replace as demanded or not?
Any suggestion?
Thank you in advance
you can deleted the /usr/share/sen2agri/ path, then
sudo ./sen2agriPlatformInstallAndConfig.sh