Shapefile upload error for L4A even after using check_shp.py

Hello,

I have a shapefile to be used as training data for the L4A Crop mask product. I already did the fixing process using the python /var/www/html/scripts/check_shp.py -f PH_BUK_LC_PI_201803.shp, and it said the overlapping areas were fixed.

Is there a way to check if there is something wrong with my system if not with my shapefile?

PS: Actually I’m using a remote desktop server and accessing it through remmina desktop.
I’m wondering because I used the same shp file in another local desktop, and it uploaded
successfully.

Also, I had a problem with nginx and slurm using this remote desktop server (but eventually solved it): Cannot Access the Sen2Agir web interface, tried starting httpd but failed

Regards.

Hi,

I’m not sure I understand. After fixing the shapefile, do you still get an error saying that it’s invalid? And it’s working on another installation (or on the same installation, but uploaded from a different computer)?

Hello,

Yes I still get an error saying that the shape files are invalid after running the check_shp.py to fix it.
On the other hand, the shape files used are working on another installation (local desktop computer).

I also had an access to the Universite Catholique de Louvain Sen2Agri web interface, and the said shape files successfully uploaded.

Regards.

That’s strange, and it shouldn’t happen. Do other shapefiles work on that system?

I made a couple of fixes to that script a while ago. Just to check, can you either send me the version you have, or compare it to the one on the computer where it’s working?

As of now, I can’t try another shape files, because we only have one set of training data for now.

In addition, I recently updated from 1.7 to 1.8 version, and the problem still persists.

Yes I can send it to your messages inbox in this forum.

As of now, I can’t try another shape files, because we only have one set of training data for now.

Just use an empty one, or one from another region. Ironically, these things aren’t checked.

Yes I can send it to your messages inbox in this forum.

Sorry, I meant the check_shapefile.py script.

Hello,

Yes, I’ll follow your suggestions.
I’m sending the .py script in your message inbox.

Regards

Anyway, here is the result after the shape file upload:

Screenshot%20at%202018-04-12%2016-05-28

Here I used the old versions of training data.

I tried uploading a shp file for a new site and it worked. I guess the uploading error only happens in the L4A and L4B custom jobs section?

That looks like a permissions problem. Try sudo chmod 777 /mnt/upload.

Already tried it, but the same thing happens. It is possible that this problem is related to my installation or other issues?

Yes, it sounds like something specific to your system.

I had an issue like this once, and restarting the httpd service fixed it. The operating system provides it with a private temporary directory, and for some reason the process was no longer able to access it.

Please try something like sudo yum install strace, then sudo strace -fp "$(pidof httpd)" 2>&1 | tee log.txt while you upload the file. The log will be large, but it should contain information about any OS-level system calls that fail.

Hello,

Apparently, the shp file upload is now ok after trying strace.
I’ll update this thread if there are any progress with this issue.

Regards.

Hello there again,

It’s weird that I successfully uploaded my shp file for L4A on my first try.
However, on my second try I came back again with the same upload error.

Here is the log of the strace command.

strace should have no effect on the upload, besides making everything a lot slower.

Does the download link work for you? It’s saying “File not yet uploaded”.

Here it is: https://files.fm/u/uq9mtdg7

[pid  3114] stat("/mnt/upload/Pantaronv2/sen2agri_1523868143/", 0x7ffefb057780) = -1 ENOENT (No such file or directory)
[pid  3114] stat("/mnt/upload/Pantaronv2/sen2agri_1523868143", 0x7ffefb0567a0) = -1 ENOENT (No such file or directory)
[pid  3114] stat("/mnt/upload/Pantaronv2", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid  3114] mkdir("/mnt/upload/Pantaronv2/sen2agri_1523868143", 0755) = -1 EACCES (Permission denied)

Is this with a different site that the one that worked?

How do the permissions on those directories look?

ls -la /mnt/upload

No it’s the same that worked.

Here is the result of ls -la /mnt/upload:

drwxrwxrwx 5 sen2agri-service sen2agri-service 4096 Apr 16 13:31 .
drwxr-xr-x 4 sen2agri-service sen2agri-service 4096 Jan 24 17:26 ..
drwxr-xr-x 3 apache           apache           4096 Mar 16 10:30 Bukidnon_NP_only
drwxr-xr-x 3 apache           apache           4096 Apr 12 16:13 Isabela
drwxr-xr-x 3 sen2agri-service sen2agri-service 4096 Feb 22 12:07 Pantaronv2

Regards.

sudo chown -R sen2agri-service: /mnt/upload

Hello,

Still had the same upload error after sudo chown command.
Here’s the log: https://files.fm/u/tsev9bew

Regards.