I am having a problem in downloading sentinel images. Each time I disabled the site in the web interface, the downloading skips the current one and fetches a new download. From the database, the previously downloading image will be marked with status_id = 3.
Can I delete safely the images with status_id of 3 from both from the database and disk?
How can I make sure that the system will download again the images and register it as status_id 1 in the images? Or will the downloader ignore it since it was already deleted in the disk?
Normally, the services are built to resume the download from date of the last product that is found in the downloader_history table for that site (no matter the status of that product is).
The status 3 (FAILED) of a product means that it gave an error but will be retried, according to the retry policy that is configured in the datasource table for that product source. So basically, if the site is re-enabled, that product will be retried at some point. Only in the case when the number of retries is getting to the maximum configured, in this case the product status will be set to 4 (ABORTED) and it will never be retried.
To summarize:
you can leave the products with the status 3 and the services will retry to download them;
if the product is on disk but the status is 3, the application will still try to re-download it.
if you delete older products with status 3 from the database, they will not be retried by the system due to the “resume from last product date” mode. Nevertheless, you can force re-considering the missing products starting from the start of the season by setting to “true” the flag “downloader.S2.forcestart” in the “config” table and setting your site id in the column “site_id”. This way, the services will check again ONLY ONCE all the products from the start of the season for filling the missing gaps. After that the flag “downloader.S2.forcestart” will be automatically reset to false but can be set again manually to true, if you need this again.
You should try to add it manually in the database as it is present also in version 1.8.3 but not adding automatically in the database. You can try to restart application or wait for around 15 minutes to take into account this key. But as explained above, this is only for checking if some L1C products are missing from the database.
Thanks again. Currently re-downloading the products with status_id = 3 after following your instruction. Does the sen2agri-service need to download first the remaining dates of the entire season before filling the missing/deleted images?
No, normally, after setting the force start the products should be downloaded from the beginning of the season and then continuing with the remaining dates from the season.
