The supplied credentials are invalid!

cudroiu · September 24, 2019, 6:56am

Already answered on other threads but just to have the answer also on this one

Although the account is OK and you can download from the website, for using the download API you need to have activated at USGS the right for downloading the products. To do this, you need to perform a request to USGS in order to activate the downloads for your account:

-	Login to your account on USGS
-	Go to https://ers.cr.usgs.gov/profile/access
-	Push the “Request Access” button and fill the information in the form that will be displayed:
o	“Datasets Required” – Landsat Collection 1 Level-1
o	“Data Products Required” - Landsat 8 OLI/TIRS C1 Level -1
o	“Scenes required” – In your case you can fill 2 monthly or 24 yearly (depends on your site dimension). You can also put more if you consider you will have additional sites to monitor
o	“Scripting Capabilities”: python, java
o	For the other fields, you will have to fill the requested information

After that, you should receive after several hours an email from USGS confirming the activation. Once this is done, the L8 should start also being downloaded.

Hope this helps,

Best regards,
Cosmin

nicolas-sencrop · October 10, 2019, 6:59am

Hello !

I have followed your steps to request a Machine to Machine access from USGS and they have accepted it yesterday but even with that it seems to not work and download of L1 doesn’t start.

DB version 2.0.1

Logs :

oct. 10 06:57:10 Sen2Agri-VM start.sh[73384]: 2019-Oct-10 06:57:10.994 WARN  [USGS-L8-1 ] r.c.t.d.usgs.Landsat8Query     - ro.cs.tao.datasource.QueryException: ro.cs.tao.datasource.QueryException: The supplied credentials are invalid!
oct. 10 06:57:11 Sen2Agri-VM start.sh[73384]: 2019-Oct-10 06:57:11.528 WARN  [USGS-L8-1 ] r.c.t.d.usgs.Landsat8Query     - ro.cs.tao.datasource.QueryException: ro.cs.tao.datasource.QueryException: The supplied credentials are invalid!
oct. 10 06:57:12 Sen2Agri-VM start.sh[73384]: 2019-Oct-10 06:57:12.050 ERROR [DefaultQuartzScheduler_Worker-6] o.e.s.scheduling.LookupJob     - ro.cs.tao.datasource.QueryException: ro.cs.tao.datasource.QueryException: ro.cs.tao.datasource.QueryException: ro.cs.tao.datasource.QueryException: The supplied credentials are invalid! @ org.esa.sen2agri.scheduling.LookupJob.lookupAndDownload(LookupJob.java:258) <         org.esa.sen2agri.scheduling.AbstractJob.execute(AbstractJob.java:65) < org.quartz.core.JobRunShell.run(JobRunShell.java:202) < java.util.concurrent.FutureTask.get(FutureTask.java:192) < java.util.concurrent.FutureTask.report(FutureTask.java:122) < org.esa.sen2agri.scheduling.LookupJob.executeImpl(LookupJob.java:154) < org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)

Thank you for your help !

Edit -----

Also for Sentinel2 it seems that the download is blocked also (no more logs after the last line, and no current download in interface) :

oct. 10 12:01:04 Sen2Agri-VM start.sh[92275]: 2019-Oct-10 12:01:04.394 INFO  [DefaultQuartzScheduler_Worker-1] o.e.s.scheduling.RetryJob      - Job 'Retry.france2-Sentinel2' started
oct. 10 12:01:04 Sen2Agri-VM start.sh[92275]: 2019-Oct-10 12:01:04.409 INFO  [DefaultQuartzScheduler_Worker-2] o.e.s.scheduling.LookupJob     - Job 'Lookup.france2-Sentinel2' started
oct. 10 12:01:04 Sen2Agri-VM start.sh[92275]: 2019-Oct-10 12:01:04.422 INFO  [Amazon Web Services-S2-1] o.e.s.scheduling.RetryJob      - [site 'france2',sensor 'Sentinel2'] No products to retry
oct. 10 12:01:04 Sen2Agri-VM start.sh[92275]: 2019-Oct-10 12:01:04.427 INFO  [Amazon Web Services-S2-1] o.e.s.scheduling.RetryJob      - [site 'france2',sensor 'Sentinel2'] No products to retry
oct. 10 12:01:04 Sen2Agri-VM start.sh[92275]: 2019-Oct-10 12:01:04.437 WARN  [DefaultQuartzScheduler_Worker-2] o.s.o.j.v.HibernateJpaDialect  - JDBC Connection to reset not identical to originally prepared Connection - please make sure to use connection release mode ON_CLOSE (the default) and to run against Hibernate 4.2+ (or switch HibernateJpaDialect's prepareConnection flag to false
oct. 10 12:01:04 Sen2Agri-VM start.sh[92275]: 2019-Oct-10 12:01:04.522 INFO  [Scientific Data Hub-S2-1]                                - Loading Sentinel-2 tiles extents

cudroiu · October 11, 2019, 9:00am

Hello,
It seems that the USGS changed (again) the API for accessing the products (you might have received a mail about that 2 weeks ago).
We will come with an update soon to adapt to the new changes.

For the S2 part, I see that you have configured the AWS as datasource for downloading the products. Unless you are not running on an AWS machine, you should disable the datasources for AWS and keep only SciHub as Enabled. Also, ensure that the Scihub Datasource is configured to have:

Scope as Query and Download
Fetch Mode - Overwrite

Also, a restart of the services using sudo systemctl restart sen2agri-services will not harm

Hope this helps.

Best regards,
Cosmin

SMIEJ_CRTS · October 11, 2019, 9:20am

Thank you for you response and your efforts @cudroiu
Could you give us the script that is responsable of downloading landsat images from usgs api, because we want to use the platform for a national case, and we need landsat images for processing, as soon as possible.

nicolas-sencrop · October 11, 2019, 9:22am

Yes it seems that S2 downloads have started !
Waiting for the fix for L8, but thank you for the help !

cudroiu · October 23, 2019, 8:01am

Hello,

For L8 and even an issue for S2 you can find a solution here :
Paging in sentinel product downloads is not correct.
Nevertheless, please note that this is a temporary version of the services that was not fully tested.

Best regards,
Cosmin

SMIEJ_CRTS · October 30, 2019, 3:03pm

I’ll try this solution and i’ll let you know.But if i have some issue and i want to go back to last version, what should i do?

SMIEJ_CRTS · November 14, 2019, 3:22pm

@cudroiu i tested the updated version of sen2agri-service, and when i run status of sen2agri-services, it gives:
Caused by: ro.cs.tao.datasource.QueryException: java.lang.NullPointerException
nov. 14 15:02:28 localhost.localdomain start.sh[20883]: at ro.cs.tao.datasource.usgs.Landsat8Query.setupRemoteFields(Landsat8Query.java:196)
nov. 14 15:02:28 localhost.localdomain start.sh[20883]: at ro.cs.tao.datasource.usgs.Landsat8Query.authenticate(Landsat8Query.java:162)
nov. 14 15:02:28 localhost.localdomain start.sh[20883]: … 12 more
nov. 14 15:02:28 localhost.localdomain start.sh[20883]: Caused by: java.lang.NullPointerException
nov. 14 15:02:28 localhost.localdomain start.sh[20883]: at ro.cs.tao.datasource.remote.result.json.JsonResponseParser.parse(JsonResponseParser.java:85)
nov. 14 15:02:28 localhost.localdomain start.sh[20883]: at ro.cs.tao.datasource.usgs.Landsat8Query.setupRemoteFields(Landsat8Query.java:184)
nov. 14 15:02:28 localhost.localdomain start.sh[20883]: … 13 more
nov. 14 15:02:28 localhost.localdomain start.sh[20883]: 2019-Nov-14 15:02:28.966 INFO [DefaultQuartzScheduler_Worker-5] o.e.s.scheduling.LookupJob - Job ‘Lookup.morocco-Landsat8’ completed
nov. 14 15:12:19 localhost.localdomain start.sh[20883]: 2019-Nov-14 15:12:19.351 INFO [pool-4-thread-1] o.e.s.services.ScheduleManager - Running scheduled jobs: 0
Please let me know what mean’s these line.

cudroiu · November 26, 2019, 3:08pm

Could you please try the new services from here:

And let me know if there is any change.

Best regards,
Cosmin

SMIEJ_CRTS · December 2, 2019, 12:36pm

when i updated the system , the sen2agri-services are inactive.

sen2agri-services.service - Services for Sen2Agri
Loaded: loaded (/usr/lib/systemd/system/sen2agri-services.service; enabled; vendor preset: disabled)
Active: inactive (dead) since lun. 2019-12-02 12:34:01 WET; 1min 5s ago
Process: 6349 ExecStart=/usr/share/sen2agri/sen2agri-services/bin/start.sh (code=exited, status=0/SUCCESS)
Main PID: 6349 (code=exited, status=0/SUCCESS)

déc. 02 12:34:01 localhost.localdomain start.sh[6349]: 2019-Dec-02 12:34:01.384 INFO [main ] o.s.b.a.l.ConditionEvaluationReportLoggingListener -
déc. 02 12:34:01 localhost.localdomain start.sh[6349]: Error starting ApplicationContext. To display the conditions report re-run your application with ‘debug’ enabled.
déc. 02 12:34:01 localhost.localdomain start.sh[6349]: 2019-Dec-02 12:34:01.385 ERROR [main ] o.s.b.d.LoggingFailureAnalysisReporter -
déc. 02 12:34:01 localhost.localdomain start.sh[6349]: ***************************
déc. 02 12:34:01 localhost.localdomain start.sh[6349]: APPLICATION FAILED TO START
déc. 02 12:34:01 localhost.localdomain start.sh[6349]: ***************************
déc. 02 12:34:01 localhost.localdomain start.sh[6349]: Description:
déc. 02 12:34:01 localhost.localdomain start.sh[6349]: The bean ‘org.springframework.transaction.config.internalTransactionalEventListenerFactory’, defined in null, could not be registered. A bean with that name has already been defined in class path resource [org/springframework/transaction/annotation/ProxyTransactionManagementConfiguration.class] and overriding is disabled.
déc. 02 12:34:01 localhost.localdomain start.sh[6349]: Action:
déc. 02 12:34:01 localhost.localdomain start.sh[6349]: Consider renaming one of the beans or enabling overriding by setting spring.main.allow-bean-definition-overriding=true

cudroiu · December 2, 2019, 12:51pm

Did you renamed/deleted the /usr/share/sen2agri/sen2agri-services/lib and /usr/share/sen2agri/sen2agri-services/modules directories before copying the new lib and modules in /usr/share/sen2agri/sen2agri-services ?

SMIEJ_CRTS · December 2, 2019, 12:57pm

yes i renamed the old modules and lib to lib_bkp2 and modules_bkp2, then i copied the updated one.

cudroiu · December 2, 2019, 1:07pm

Could you please copy also the application.properties into /usr/share/sen2agri/sen2agri-services/config?
Optionally, you can change the INFO values into TRACE for the following 2 keys:

logging.level.ro.cs.tao=INFO
logging.level.org.esa.sen2agri=INFO

SMIEJ_CRTS · December 2, 2019, 2:42pm

now sen2agri-services are running, how much time would take system to start downloading landsat images(of course if it work).

SMIEJ_CRTS · December 2, 2019, 2:46pm

thank you @cudroiu , landsat start downloading

cudroiu · December 2, 2019, 2:52pm

you’re welcome ;-). Glad that I could help you.

henryhein · March 2, 2020, 7:55am

Seems that sen2agri-services archive/patch helped to fix the issue of downloading both Landsat and Sentinel. Unfortunately, link to download the archive is already expired. Could you share the link?

Thanks,
Henry

kserradell · March 2, 2020, 12:19pm

Hello,

Same as here. I’m also looking for the patch.

Could you please provide it again?

Thank you,

KiM

henryhein · March 11, 2020, 3:01pm

it’s working after I upgraded to latest version(2.0.2). it’s able to download Landsat data but then my custom job is not running. Wonder if there’s known issue. Thanks

kserradell · March 13, 2020, 8:25am

Hello.

I’m still having issues to download any product.

@henryhein, how did you update to 2.0.2? Using update.sh script?

Thanks,