I was trying to upload my in situ shp file zip in the custom jobs section of the web interface.
However, it says that there was an error and the upload is not successful.
I’m sharing my in situ shapefile here: https://ufile.io/t1irz
Is there a way to check the validity/integrity of the shp file zipped before uploading it?
I tried to upload again my fixed shapefile in the custom jobs of the web interface, but I encountered the same problem.
Here are the results of fixing my shapefile:
I checked it:
/var/www/html/scripts/check_shp.py /home/brentf/Downloads/PH_BUK_LC_PI_201712/PH_BUK_LC_PI_201712.shp
960 feature(s) found
Warning 1: Self-intersection at or near point 740359.92763765831 830476.7972673676 0
Invalid feature: ID=21
Warning 1: Self-intersection at or near point 717357.97096149693 881168.98342688382 0
Invalid feature: ID=433
Warning 1: Self-intersection at or near point 725221.06657073402 883105.55843447871 0
Invalid feature: ID=448
Warning 1: Self-intersection at or near point 686774.41494734993 863222.37843357422 0
Invalid feature: ID=528
ERROR 1: TopologyException: Input geom 0 is invalid: Self-intersection at or near point 740359.92763765831 830476.7972673676 0 at 740359.92763765831 830476.7972673676 0
Traceback (most recent call last):
File “/var/www/html/scripts/check_shp.py”, line 135, in
sys.exit(main())
File “/var/www/html/scripts/check_shp.py”, line 91, in main
if g0.Overlaps(g1) and g0.Intersection(g1).Area() > 0:
AttributeError: ‘NoneType’ object has no attribute ‘Area’
Fixed it:
/var/www/html/scripts/check_shp.py -f /home/brentf/Downloads/PH_BUK_LC_PI_201712/PH_BUK_LC_PI_201712.shp
960 feature(s) found
Warning 1: Self-intersection at or near point 740359.92763765831 830476.7972673676 0
Invalid feature: ID=21
Fixed feature: ID=21
Warning 1: Self-intersection at or near point 717357.97096149693 881168.98342688382 0
Invalid feature: ID=433
Fixed feature: ID=433
Warning 1: Self-intersection at or near point 725221.06657073402 883105.55843447871 0
Invalid feature: ID=448
Fixed feature: ID=448
Warning 1: Self-intersection at or near point 686774.41494734993 863222.37843357422 0
Invalid feature: ID=528
Fixed feature: ID=528
And checked it again:
[brentf@nirvana-ESSC_MNL mnt]$ /var/www/html/scripts/check_shp.py /home/brentf/Downloads/PH_BUK_LC_PI_201712/PH_BUK_LC_PI_201712.shp
960 feature(s) found
I tried using your version, but still the same problem occurs. Yes, the repacked it again after fixing the shapefile. Maybe I could just delete the features that were fixed? I’ll update you with this after deleting the said features.
Unfortunately, I accidentally closed the terminal tab that contained the fixed feature IDs of my shapefile.
I’m sharing here my fixed version of shapefile (https://ufile.io/eca3s). Again, if it not worked for me, perhaps there was a problem in my system’s web interface?
If the said shp files worked for you and not with my system, I will try the manual processing through the CLI.
Thanks for you help. I really appreciated it.
Actually I’m running Sen2Agri on two separate computers, one is a local one while the other one is controlled remotely.
My local machine has successfully uploaded the shop file, but the remote computer has not.
Does it have something to do with the minimum disk size required? My remote host has only 1.7 TB space.
Though I’m only processing 4 tiles for both Landsat and Sentinel, with the season of June 1 to Sept. 30 of 2017.
I read that slurm is needed in custom jobs (in my case, L4A and L4B). The slurm in my remote machine is not active, therefore not running, perhaps this is the reason I can’t upload my shapefile?
No I only have one full install of Sen2agri system on each machine.
I think you already answered my question with a similar issue here: How to enable slurm services
I’m experiencing the same issue with the in-situ shapefile. You mentioned the validation tool has been fixed?
Also, using the -f flag returns Could not open ...
The first one will fix a crashing bug (AttributeError: ‘NoneType’ object has no attribute ‘Area’) and the second one will enable -f to be used by default.
I downloaded and replaced said files, and the error persist.
Now when I run check_shp.py, I’m receiving this error message:
python /var/www/html/scripts/check_shp.py SSD_WARR_LC_VM_20171128.shp
1414 feature(s) found
ERROR 1: Error in fread() reading object of size 328 at offset 100 from .shp file
Traceback (most recent call last):
File "/var/www/html/scripts/check_shp.py", line 141, in <module>
sys.exit(main())
File "/var/www/html/scripts/check_shp.py", line 50, in main
if not geometry.IsValid():
AttributeError: 'NoneType' object has no attribute 'IsValid'
You have some features with no geometry. I’ll send you a new fix, but can you share the shapefile so I can test it? You can mail it to me at lnicola@c-s.ro.