[MapProxy] timeout for slow WMS server

[Jaak Laineste]

Hi,
I'm trying to use MapProxy as global OSM WMS server. I have full planet in imposm database and Mapserver as WMS server. The problem is that low-zoom queries (especially with zooms 4-6 and metatiles) are very slow, take many minutes. The plan is to pre-seed low zoom levels to cache, so slowness would not be really visible for enduser. But the problem seems to be that mapproxy times out with these queries and never really seeds cache, and I cannot find a way to extend backend timeout in mapproxy configuration. Quickfix in code level would work for me also, but I don't really know Python enough to start to mess with this.

I know that the better way would be to fix the map layout and imposm configuration to make these zooms just work faster, but in short run I'd prefer some mapproxy level solution here.

Jaak

[Oliver Tonnhofer]

Hi Jaak,

Post by Jaak Laineste
I'm trying to use MapProxy as global OSM WMS server. I have full planet in imposm database and Mapserver as WMS server. The problem is that low-zoom queries (especially with zooms 4-6 and metatiles) are very slow, take many minutes. The plan is to pre-seed low zoom levels to cache, so slowness would not be really visible for enduser. But the problem seems to be that mapproxy times out with these queries and never really seeds cache, and I cannot find a way to extend backend timeout in mapproxy configuration. Quickfix in code level would work for me also, but I don't really know Python enough to start to mess with this.

There is the http.cient_timeout option for WMS sources. See: http://mapproxy.org/docs/nightly/sources.html#http

Regards,
Oliver

[Jaak Laineste]

Hi Oliver,

Post by Oliver Tonnhofer

Post by Jaak Laineste
I'm trying to use MapProxy as global OSM WMS server. I have full planet in imposm database and Mapserver as WMS server. The problem is that low-zoom queries (especially with zooms 4-6 and metatiles) are very slow, take many minutes. The plan is to pre-seed low zoom levels to cache, so slowness would not be really visible for enduser. But the problem seems to be that mapproxy times out with these queries and never really seeds cache, and I cannot find a way to extend backend timeout in mapproxy configuration. Quickfix in code level would work for me also, but I don't really know Python enough to start to mess with this.

There is the http.cient_timeout option for WMS sources. See: http://mapproxy.org/docs/nightly/sources.html#http

Ok, I assumed it is in seconds and put it to 10 minutes. Now I get another mapproxy internal timeout exception. I assume that it is coming from the fact that client is OpenLayers, it creates more concurrent requests than mapproxy can handle, and internal request queue timeout should be also larger, just as with wms backend query timeout?
...
File "/usr/local/lib/python2.7/dist-packages/mapproxy/platform/cpython/lock.py", line 72, in lock
raise LockTimeout('another process is still running with our lock')
LockTimeout: another process is still running with our lock

Fast forward to my next problem:

I try to get TMS front-end working and am confused. I have following tile : Loading Image... and I try to get same via mapproxy. If I give same z/x/y then backend WMS request very wrong. From demo service it turnes out that the same tile is /tms/1.0.0/osm_EPSG900913/17/149681/184421.png . Here zoom is one different and y is inverted (which is typical difference between web mapping xyz and TMS). Can it be fixed with config? My grid is:
base: GLOBAL_MERCATOR
num_levels: 19

My target is to use the TMS service in Potlach2 which supports only "web standard" tile xyz system.

Jaak

[Oliver Tonnhofer]

Post by Jaak Laineste
Ok, I assumed it is in seconds and put it to 10 minutes. Now I get another mapproxy internal timeout exception. I assume that it is coming from the fact that client is OpenLayers, it creates more concurrent requests than mapproxy can handle, and internal request queue timeout should be also larger, just as with wms backend query timeout?

The cache locking uses the global client timeout (from globals.http.client_timeout). But you should use the seed tool if you have sources that are that slow.

Post by Jaak Laineste
base: GLOBAL_MERCATOR
num_levels: 19
My target is to use the TMS service in Potlach2 which supports only "web standard" tile xyz system.

The OpenStreetMap tile server is not a TMS, but a copy of the Google Maps tiling schema. See http://mapproxy.org/docs/nightly/services.html#openlayers

This should work in your case: /tiles/osm_EPSG900913/18/149681/77722.png?origin=nw


Regards,
Oliver