Create new site and download/visualise products

System Installation and Configuration
1

Hello,
I have been trying to understand the Sen2Agri system and installed the software on a system with CentOS7. Installation went smooth and now I can see the application running in the browser. As per instructions, I added a site by providing a shape file which I generated using an online tool.
I am still not able to see any products downloaded and my download statistics show its 100 %, but all failed downloads. I saw one of the earlier discussion posts about requesting access to the EROS system and I am awaiting the response from there. Before that I had some general questions to address here and hope you guys can provide some insight. Please consider that this is a very new topic and I am venturing into the system for the first time. So noob alert :slight_smile:

  1. I have a small PC at the moment, the processor is an AMD, and the RAM is also low (Planning to upgrade or switch to a cloud system like AWS) . At the moment, can i still run on a small site and see the results without problem? I have enough hardware space for downloads.
  2. Is there a document that explains the usage of the system other than the installation instruction document (Sen2Agri_SUM_v3.0) ?
  3. As a small site, I selected a farm land in the Netherlands (area of around 3000 sq km). At the site creation page, is it mandatory to have the strata data and insitu data for using the Sen2Agri system? If so, is there a sample I can use or to understand the XML format? Could it be autogenerated ?
  4. Is there any sample dataset for a site I can try on, with strata and onsite data included ?
    Please feel free to link me to another discussion post that answers some or all of the questions. I went through some of the discussion topics here, I could not find some relevant ones and hence this post.

EDIT: I also executed sudo journalctl -u sen2agri-orchestrator -u sen2agri-scheduler -u sen2agri-executor --since 2020-01-01 and piped the output into a text file. I will attach it here for any reference.
output.txt (317.1 KB)

Thanks in advance for your time.
K

2

Please make sure that your uploaded site polygon are not made of 150 points. See SUM page 52.

Yes as long as your site covers a few tiles and your mapping duration (season) is short. A 32-46 GB might be enough for small sites, lower CPU speed results in longer processing time.

Yes, see page 51 (How to use Sen2Agri System).

As long as the farmland is quite homogenous in land cover, there is no need for strata. Strata is used if your site has varied landscapes in terms of land use, topography, climate, farming systems, geology etc. See SUM page 98.

You can browse my answers. I posted mostly in L4B crop type section of this forum. Hope that would help.

My sense from your log files is perhaps, there were no properly generated L2A (Atmospherically corrected products) files available so the L3A processor did not run. Perhaps this has something to do with the permission on running MAJA/MACCS processors (that’s why it failed). Though it was just my assumption, feel free to check the processors in terminal: sudo systemctl status -l sen2agri-services sen2agri-demmaccs if they are enabled and running. From my experience if the downloads and L2A product generation are ok, producing higher level products will be easy.

Regards,
Brent

3

Thanks @brentf for the detailed answer. I will add some points as a response here.

  1. I realize that the current PC setup is inadequate now. I have only 8 gigabytes. Could that possibly hinder the services from starting at all ?

  2. I ran the script to check the status of the services that are running using the command sudo systemctl status -l sen2agri-services sen2agri-demmaccs. I will put the results here for your reference. out.txt (5.3 KB)
    The most concerning one is the error “The supplied credentials are invalid!”. I had already tried what was suggested in this post.

i am still wondering if I did something wrong with the installation. My mnt/upload ad mnt/archive should not have any read-write issues and it is accessible.

Would you suggest to try a cloud service that supports CentOS and try a fresh install? Any links or references to do this, perhaps in AWS?

Thanks in advance for your time and response.
Kiran

4

I’m not sure whether that spec is preventing you. Some users here have tried to use virtual machines (perhaps allocating lower specs) and successfully ran some product processing.

It seems that there were no downloaded products because of the invalid credentials for Landsat 8 only, I’m not sure for Sentinel 2. Also please take note that the Amazon Web Services (for both L8 and S2) should be disabled by default (see data sources tab of Sen2Agri’s web interface). Can you look at the database downloader_history whether you have valid downloaded images and were successfully processed? Products which are processed by maccss should be visible in the L2A products tab of the web interface:
image

For commands:

sudo -u postgres psql sen2agri -c "select product_name, status_id, status_description from downloader_history inner join downloader_status on downloader_status.id = status_id ";

see FAQ

Also, if you want to repeat the whole downloading process,

sudo -u postgres psql sen2agri -c “update downloader_history set no_of_retries = '0' where status_id = '4'; “
sudo -u postgres psql sen2agri -c “update downloader_history set status_id = '3' where status_id = '4' ;“

or

sudo -u postgres psql sen2agri -c "update config set value = 'true' where key = 'downloader.s2.forcestart' and site_id = 1";

your ‘site_id’ is probably 1, if you only have one site.

Please see FAQ for more details on commands.