Hello
I have been able to download most of the S2 and L8 images for 2017. However, the S2 images in the period 2016-11-30 to 2017-04-30 have been ignored, even though they are available in Copernicus Sentinel Hub.
I think this might be related with a problem described by the developers of sentinelsat
The tileid parameter only works for products from April 2017 onward due to missing metadata in SciHub’s DHuS catalogue.
Does anybody know a solution for this problem?
Kind
Hello.
There should be a different cause, since we are not using the tileid parameter, but rather the tile extents (footprint). Could you please provide us with the polygon of your site, so that we can trace the cause for the missing Sentinel-2 products?
Best,
Cosmin.
Hello
Here is the footprint file. Sorry about the assumption about the tileid, it was the only common pattern I saw
Kind regards,
Ignacio Site35TLK35TLL.zip (1.6 KB)
I have tested using your site definition and a test season between 2016-11-29 and 2017-05-01.
The services from version 1.8.3 found 315 products over the area, and the download started without any issues. Are you using the same system version (1.8.3)? Or an older one? If latter, please upgrade your deployment to 1.8.3.
If not, maybe that was a temporary SciHub issue. In order to restart the download, please do the following:
sudo -u postgres psql sen2agri
SELECT id, name FROM site;
Look for your site and note down the id. Then issue the following query, using the previously noted id:
DELETE FROM downloader_history WHERE site_id = [your site id] AND satellite_id = 1 AND product_date BETWEEN CAST(‘2016-11-30 00:00:00’ as timestamp with time zone) AND CAST(‘2017-04-30 23:59:59’ as timestamp with time zone);
\q
curl http://localhost:8081/downloader/forcestart/[your site id]/1
It should write ‘Force message sent’ if everything ok. This would force, only for one time, the services to restart the download from the beginning of the season.
Cosmin.
I am using the system version 1.8.3. To clarify, I issued the job for the period 2017-01-01 to 2017-12-31, for the tiles 35 TLK and 35TLL, excluding the rest of S2 tiles.
I have issued the commands you recommended, and, as you say at the end I get the message ‘Force message sent’. However, there are no new entries in the downloads_history table, when I issue
sudo -u postgres psql sen2agri -c "select product_name from downloader_history order by product_date" > history.txt
I get the following (just pasted the part showing the problem):
LC08_L1TP_184028_20161123_20180527_01_T1
S2A_MSIL1C_20161127T091322_N0204_R050_T35TLK_20161127T091741.SAFE
S2A_MSIL1C_20161127T091322_N0204_R050_T35TLL_20161127T091741.SAFE
S2A_MSIL1C_20161127T091322_N0204_R050_T35TLL_20161127T091321.SAFE
LC08_L1TP_183029_20161202_20170317_01_T1
LC08_L1TP_183028_20161202_20170317_01_T1
LC08_L1TP_184029_20161209_20170317_01_T1
LC08_L1TP_183029_20161218_20170315_01_T1
LC08_L1TP_183028_20161218_20170315_01_T1
LC08_L1TP_183029_20170103_20170312_01_T1
LC08_L1TP_183028_20170103_20170312_01_T1
LC08_L1TP_184029_20170110_20170311_01_T1
LC08_L1TP_184028_20170110_20170311_01_T1
LC08_L1TP_183028_20170119_20170311_01_T1
LC08_L1TP_184028_20170126_20170510_01_T1
LC08_L1TP_184029_20170126_20170510_01_T1
LC08_L1TP_183029_20170204_20170216_01_T1
LC08_L1TP_183028_20170204_20170216_01_T1
LC08_L1TP_184028_20170211_20170217_01_T1
LC08_L1TP_184029_20170211_20170217_01_T1
LC08_L1TP_184029_20170227_20170316_01_T1
LC08_L1TP_184028_20170227_20170316_01_T1
LC08_L1TP_183029_20170308_20170317_01_T1
LC08_L1TP_183028_20170308_20170317_01_T1
LC08_L1TP_184028_20170315_20170328_01_T1
LC08_L1TP_184029_20170315_20170328_01_T1
LC08_L1TP_183029_20170324_20180527_01_T1
LC08_L1TP_183028_20170324_20180527_01_T1
LC08_L1TP_184028_20170331_20170414_01_T1
LC08_L1TP_184029_20170331_20170414_01_T1
LC08_L1TP_183029_20170409_20170414_01_T1
LC08_L1TP_183028_20170409_20170414_01_T1
LC08_L1TP_184029_20170416_20170501_01_T1
LC08_L1TP_184028_20170416_20170501_01_T1
LC08_L1TP_183029_20170425_20170502_01_T1
LC08_L1TP_183028_20170425_20170502_01_T1
LC08_L1TP_184029_20170502_20170515_01_T1
LC08_L1TP_184028_20170502_20170515_01_T1
S2A_MSIL1C_20170506T090601_N0205_R050_T35TLL_20170506T091302.SAFE
S2A_MSIL1C_20170506T090601_N0205_R050_T35TLK_20170506T091302.SAFE
S2A_MSIL1C_20170509T092031_N0205_R093_T35TLK_20170509T092606.SAFE
S2A_MSIL1C_20170509T092031_N0205_R093_T35TLL_20170509T092606.SAFE
LC08_L1TP_183029_20170511_20170516_01_T1It would be a possibility to download the S2 images independantly, process them with maccs and then importing them with insert_l2a_product_to_db.py. If it is possible, which command should I execute to run MACCS in nominal mode?
Hi,
I am experiencing the same problem. I have an AOI covering Uganda and season starting in August 2016 and going until August 2017. After letting the system run for some time I noticed that there is a gap in S2 downloaded images between 20161129 and 20170518. I tried to fill in the gap by setting scheduled.lookup.all_products.enabled = true flag in the config table. However, this only downloaded new images from 20161203, 20170422, 20170505, 20170508, 20170512 and 20170515 so there is still a gap between December 2016 and April 2017.
I there anything else that I could try to download and process the missing images?
Thanks!
Rado
Hello
I am afraid I was unable to find a solution, so I ended up using other system (MAJA, an improved version of the MACCS system included in sen2agri) for making the corrections. I will create the composites by using WASP when said software becomes available.
You can find information about these two programmes in the CNES github.
Hope it helps,
Ignacio
We found that our tile filtering does not work for some of the SciHub products because they don’t have the tileId attribute filled in. We changed the downloader to query the filename instead, however I’m not sure whether that got in the latest release. The next one should contain the change, which might fix these issues.