L3A generation error

Biophysical Indicators Processor
21

Hi- Over weekend 2 L3A granules were generated. But L3B (LAI product)- still did not come out. The [output] link does not provide details. Also I gor Error on L4A processing. But mostly I need L3B. Please help!

22

The [output] link does not provide details.

Do you mean it’s empty? Can you add a screenshot?

Please try sudo journalctl -u sen2agri-orchestrator --since "2018-05-11" or something similar to that.

23

Log repeats this statement:

май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: Scheduler L3A: Getting season dates for site 2 for scheduled date пн июля 31 00:00:00 2017!
май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: GetSiteSeasons took 1 ms
май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: IsInSeason: Date not in season (start = пн мая 1 2017, end = ср мая 31 2017, current=пн июля 31 2017)
май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: IsInSeason: Date not in season (start = пн мая 1 2017, end = ср мая 31 2017, current=пн июля 31 2017)
май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: GetConfigurationParameters took 1 ms
май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: GetSiteSeasons took 1 ms
май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: IsInSeason: Date not in season (start = пн мая 1 2017, end = ср мая 31 2017, current=пн июля 10 2017)
май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: IsInSeason: Date not in season (start = пн мая 1 2017, end = ср мая 31 2017, current=пн июля 10 2017)
май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: Scheduler L3B: Error getting season start dates for site 2 for scheduled date пн июля 10 00:00:00 2017!
май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: GetSiteSeasons took 1 ms
май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: IsInSeason: Date not in season (start = пн мая 1 2017, end = ср мая 31 2017, current=пн июля 31 2017)
май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: IsInSeason: Date not in season (start = пн мая 1 2017, end = ср мая 31 2017, current=пн июля 31 2017)
май 14 00:13:45 proc-s2a sen2agri-orchestrator[6374]: Scheduler CropMask: Error getting season start dates for site 2 for scheduled date пн июля 31 00:00:00 2017!

24

@cudroiu maybe you have some idea?

25

What’s the output of select * from season where site_id = 2;? You can run that with sudo -u postgres psql sen2agri.

26

[root@proc-s2a archive]# sudo -u postgres psql sen2agri
psql (9.4.17)
Введите “help”, чтобы получить справку.

sen2agri=# select * from season where site_id = 2;
id | site_id | name | start_date | end_date | mid_date | enabled
----±--------±---------±-----------±-----------±-----------±--------
5 | 2 | May_2017 | 2017-05-01 | 2017-05-31 | 2017-05-15 | t
(1 строка)

27

IsInSeason: Date not in season (start = пн мая 1 2017, end = ср мая 31 2017, current=пн июля 31 2017)

So it looks like it’s trying to make a product for 31 July, but the season ends on 31 May. The code seems to allow up to a month between the season end and the product date. Can you try with 30 June instead?

28

I Didnt get your point. System generated 2 composites (L3A) products. Why not to generate L3B from them? Or time interval May 1- May 31 is too short? What specific time interval do you suggest (start- mid-end dates)?

29

You’re trying to create a product for the date of 31 July. The system, however, only has data up to 31 May, so that doesn’t work.

30

No! My interval is May 1-May 31. L3A product was generated for May 11. Why not to generate L3B for the same date?

31

Why not to generate L3B for the same date?

Oh, okay, I assumed you entered the 31 July date yourself.

@cudroiu

32

Hello,

Could you do the following operations, please:

  • Start log the messages from sen2agri-orchestrator and sen2agri-executor into 2 distinct files
  • Go to Custom Jobs (in the Sen2Agri Web Interface) -> L3B LAI Processor
  • Select the site that you are testing
  • Select one product L2A product in the “Available input files” area (one L2A product from May 2017)
  • Press “Submit”
  • Monitor the application from the Web UI and see if the job occurred in the “Monitoring” tab and also in the “System overview” tab.
  • If nothing happens, wait for several minutes and send us the log files for the sen2agri-executor and sen2agri-orchestrator above.
  • If on the contrary, the processing is started, you can go to the “Dashboard” tab in Web UI and try adding a new scheduled job in L3B - LAI/NDVI sub-tab. The scheduled job should start on 10th of May 2017, be “Cycle” job and with a repeat rate of 10 days.

Question: all L1C products were processed by your system as L2A products?

Please let us know,
Cosmin

33

What is the specific command to start log the messages from sen2agri-orchestrator and sen2agri-executor into files ?

34

I didn’t start log but started L3B processing as instructed for a single date. I do see job status Running in the Monitoring section. I also see in the system overview tab.

35

sudo journalctl -f -u sen2agri-orchestrator | tee orchestrator.log will stay running and save the log to a file. The command for the executor is similar (sudo journalctl -f -u sen2agri-executor | tee executor.log).

36

Thanks- will do. In fact, I got custom job for L3B (LAI) processing just finished successfully! But to my understanding this is a manual execution. But is it possible to execute L2C-L3B processing in a fully automated manner, just selecting shape file and time frame in the sites tab in the web interface?

37

You should be able to do that from the bottom of the dashboard tab.

38

As described above, you can do automated processing of a season from a site (these correspond to the timeframe and the extent that you have defined in the sites tab of the web interface) by:

  • go to the “Dashboard” tab in Web UI and add a new scheduled job in L3B - LAI/NDVI sub-tab. The scheduled job should be defined in the following manner:
    • Choose a name of the job
    • Select the Site that you want to process
    • Select the Season that you want to process
    • Select L3B product
    • Select Cycle for Schedule type
    • Select First run time to be on 10th of May 2017 (in your case the season starts on 1st of May 2017 so you can put it 10 days later)
    • Set Repeat to 10 days.
  • Submit the job.

Please note that it should be better if you process sites from the last year that you already have the L2A processed before doing this operation.
Once you have done this, the system should automatically process all L2A products into L3B products for your site and season.
Please see also the user manual for additional information.

Cosmin

39

Ok- I understand that I can add new job at the “Dashboard”. But my old job was already in the Dashboard, but stuck. Why? Also- I though if I want full processing (L2C-L3B) from “Sites” tab.

Im worrying if Im able to do full processing in fully automated fashion, without resorting to custom jobs. Why it stuck-what I did wrong?

40

Hello,

Unfortunately, this is a known issue in the Sen2Agri system when processing seasons from the past (not near real-time seasons). As the system is processing L2A products 3 months before the season (and this takes a lot of time), the scheduled jobs for L3B are executed before L2A processor produces the L2A products for the scheduler target interval. Having no L2A products, the scheduler executes (with error) till the end of the season.
The solution is to wait for all L2A products to be created and then add again a scheduled job as described above.
This issue will be solved in the next version of the system.
Sorry for this inconvenient.

Best regards,
Cosmin