Hi!
I’m testing out the system using a small site (25km x 25km). I created a short season from June 1 2017 to June 30 2017 for testing purposes. I’m keeping all the testing parameters smaller scale because of performance limitations on the VM. I have some questions about the processing results for L2A products:
For the site, only S2 products were processed to L2A, all L8 tiles show status of “processing_failed” or “aborted”. Is there are reason that the system is not processing L8 at all, or is there something wrong with my system or settings?
A fair amount of S2 tiles say “processing_failed” or “aborted” as well. Are all “processing_failed” statuses the same (as in the MACCS failed for some reason) or are there underlying statuses that are hidden and grouped under “processing_failed”, like “too much cloud” or “data extent not above test site”?
The main reason we are testing this system is to gauge its way of processing L8 and S2 into L2A products, but its tough without enough information on why the processing is failing. Any info on this would be great, or any advice on the best places to look in the logs or database would be very appreciated.
Thanks!
Hi,
Can you please run /opt/maccs/core/5.1/bin/maccs?
S2 products usually get marked as failed if the cloud coverage is too high.
It is attached below, would text or image be better? I think the MACCS processor is working because a lot of S2 L2A products are being generated, just that no L8 products and some S2 products are failing.
You’ll need to download and install MACCS again. MACCS 5.1.3 does not support the new L8 naming convention (e.g. LC08_L1TP_171058_20161011_20170320_01_T1). The latest version should be 5.1.5.
Looks like the latest version is 5.1.3 from this URL: https://logiciels.cnes.fr/en/license/60/146
Is this the correct site to download the software from? There is the MAJA download that I could try here:
https://logiciels.cnes.fr/en/license/58/145
But as of 5 minutes ago, the first URL is version 5.1.3.
Sorry, it looks like you’re right. I have no idea where the latest version can be downloaded from. I’ll ask around.
I left a message on the CNES site. Hopefully someone will put the new version there. But it seems like we had this issue before (last June, with 5.1.4), contacted CNES, yet the site is still on 5.1.3. I’ll see what I can do.
We do have an installation package for 5.1.5, but we’re not supposed to distribute it, as far as I know.
Pinging @nicolas.bellemans.
Thank you very much for your help!
I think you can find the good version on https://logiciels.cnes.fr/fr/node/59?type=desc. The English page contains the old version.
Version 5.1.5 did the trick, L8 is being processed now. Thanks!
I have the same problem, I installed the 5.1.5 version over 5.1.3 (no uninstall) and what should I do, to process the already downloaded files which are in the downloader_history in status_id marked as 6. Is it correct to change the status_id to 2 and they will be processed?
@jan.semrad Yes, that should work.
Thank you for reply. I have done it and the Landsat data fall in downloader_history in status_id ti value 4 with 3 retries. I have tried to download them again with status_id 1 but with the same result. Any suggestion?
I have done it and the Landsat data fall in downloader_history in status_id ti value 4 with 3 retries.
That’s a bit surprising, since if MACCS failed they should go to the processing_failed (6) state. aborted means that the download failed, which could make sense for downloading (1) products, but not for downloaded (2) ones. Can you attach the some downloader logs?
I have tried to download them again with status_id 1 but with the same result.
To re-download them, it might be better to the rows instead.
Sorry to bother you. I restarted the server and the L2 processing is up and running as well for Landsat.
But just to clarify the states (if I understand them correctly):
1 - downloading
2 - L2 processing
3 - ??
4 - aborted - failed download
5 - done
6 - processing failed
Which means when status is 6 and would like to reprocess downloaded data should I change it to 2. And when status is 4 I should chnge it to 1 to restart download? Should I before it delete data from disk?
Thank in advance for help.
id | status_description
----+--------------------
1 | downloading
2 | downloaded
3 | [download] failed
4 | [download] aborted
5 | processed
6 | [L2A] processing_failed
7 | [L2A] processing
41 | download_ignored
Which means when status is 6 and would like to reprocess downloaded data should I change it to 2.
Yes.
And when status is 4 I should chnge it to 1 to restart download?
That might work, in the sense that the downloader might think it crashed and restarts the download. I’m not sure how this is handled, and the implementation will completely change in the next version.
The alternative (in the current version, at least), is to remove the row. The downloader will then notice the product in the search results, see that it wasn’t downloaded yet, and download it again. Deleting data from disk shouldn’t be necessary.
Note that there’s a caveat here: the L2A processor uses information from the past product or products to improve cloud detection accuracy, at least. If the products are processed out of order, the results may be different.
Thank you very much, now I can handle everythink based on your information and will wait for new release.
1 Like