A lot of factors can affect the efficiency of generating tiles, including tile
types (original, compact, MongoDB), data, storage locations (local or remote
directories), and the way of generating tiles. After many tests, we organize
four classes of scenarios. Through contrasting the efficiencies of generating
tiles in different scenarios, we obtain the following **conclusion to **
provide some references and suggestions for users.
1. Stand-alone multi-task : Contrast the efficiency of generating tiles on a machine when the tile types are original, compact or MongoDB respectively.
2. Multi-machine multi-task : Contrast the efficiency of generating tiles on multiple machines when the tile types are original, compact and MongoDB respectively.
3. Local and remote directory : Contrast the impact of efficiency of generating tiles when the original data and tiles are saved in a local directory and a remote directory respectively.
4. Vector/raster maps : Contrast the efficiency of generating tiles when the original maps are a vector map and a raster map respectively.
The test data and environment
The vector data, tile ranges, tile scales and tile parameters used by the four classes of scenarios above are the same:
- Test data : Select the vector map. The maximum number of objects in vector datasets is 8324287 and the size of file are 12.4G and number of layers is 326.
- Tile range : [left:11427641.47674698, bottom:2817774.6107045934, right:12993071.816027392, top:4383204.9499850012], the number of child tasks is 100 and the number of result tiles is 1638400.
- Tile scales : 1:18,055.9909335062 (only one scale)
- Set tile key parameters : Version: 5.0, Format: JPG, Tile Size: 256*256, Compression: 75.
Stand-alone multi-task Comparison
- MongoDB tiles : Storing tiles on the local machine; local map; 64-bit computer; four cores CPU. By adding tiles, the number of tiles generated per second is as follows:
- Original tiles : Storing tiles on the local machine; local map; 32-bit computer; two cores CPU. By adding tiles, the number of tiles generated per second is as follows:
- Compact tiles : Storing tiles on the local machine; local map; 64-bit computer; four cores CPU. By adding tiles, the number of tiles generated per second is as follows:
-
Result analysis :
- For the original, compact or MongoDB tiles, the efficiency of generating tiles is greatly improved by multiple tasks on one machine than by one task on one machine.
- The most proper number of processes is 2 times the number of CPU cores, for example opening 4 processes is the best for two cores CPU.
- The improvement efficiency of MongoDB tiles is highest. The improved efficiency from high to low is MongoDB tiles, compact tiles, original tiles.
The efficiency comparison of generating tiles by multiple tasks on multiple computers
- MongoDB tiles : Storing tiles on a remote server; local map; The test procedure is: gradually add processes on the host from one to eight, and then increase processes on other machines.
- Original tiles : Storing tiles on host; local map; The test procedure is: gradually add processes on the host from one to eight, and then increase processes on other machines.
- Compact tiles : Storing tiles on a local machine; local map; The test procedure is: gradually add processes on the host from one to eight, and then increase processes on other machines.
- Result analysis :
- For generating MongoDB tiles, on the basis of opening the best proper processes on the host, adding new processes on another machine can improve the efficiency continuously.
- For generating original tiles, on the basis of opening the best proper processes on the host, adding a new process on another machine can not improve the efficiency.
- For generating compact tiles, on the basis of opening the best proper processes on the host, adding new processes on another machine can improve the efficiency slightly.
The comparison of efficiency of generating tiles when the original data is from a local machine or a remote server and the result data will be saved on a local machine or a remote server
There are four scenarios for generating tiles:
- Local map with local tiles : both the original data and the result data are saved on the local machine.
- Local map with remote tiles : the original data is saved on the host, and the result data is saved on a remote server.
- Remote map with local tiles : the original data is saved on a remote server, and the result data is saved on the local machine.
- Remote map with remote tiles : both the original data and the result data are saved remotely
Open four sub tasks for every machine, and the comparison of efficiency of generating tiles is:
-
Result analysis :
- For generating compact tiles,the efficiency is highest when both the original data and the result data are saved on a local machine.
- For generating original tiles,the efficiency is a little lower when the original data is from a shared directory but still the same with the efficiency of generating MongoDB tiles.
- For generating MongoDB tiles,the source and storage postion of data have little effect on the efficiency.
The comparison of efficiency of generating tiles when the original data is vector or raster
Both the vector map and the raster map are saved on the local machine, and the raster map is a mosaic dataset consisting of 822 images with the resolution is 90 meters and the number of columns and rows of every img image is 6000*6000, and the map size is 44.7G. The number of result tiles is 1638400 and the scales of tiles from two different kinds of maps are the same.
Three test scenarios: local MongoDB tiles, remote MongoDB tiles and local compact tiles. One host and 4 processes for generating tiles. The comparison of efficiency of generating tiles is as follows:
Result analysis :
- Generating tiles when the original map is raster takes more time obviously than when the original map is vector.
Conclusions
Through the above four scenarios, we can summarize the following conclusions:
- The feature of single-machine multiple tasks can improve the speed of generating tiles effectively.
- To every machine for generating tiles, opening tasks that number is 2 times the number of CPU cores is suggested.
- When generating MongoDB tiles, the feature allows you to deploy more computers working in parallel. So for an extensive operation of generating tiles, generating MongoDB tiles by multiple tasks on multiple machines is recommended. With unlimited physical resources, the performance of generating and saving up to 5000 tiles per second can be achieved.
- For generating compact tiles, deploying the original map on the local machine is suggested, and saving the result tiles on the local machine is the best. Transferring original tiles is inappropriate.
- With the same processes, the efficiency of generating compact tiles is highest. If the amount of tiles is not large, the deployment recommended is using the feature of multiple tasks on a high-performance machine.
- For generating compact or original tiles, the feature of multiple tasks on a machine is recommended.
- For generating MongoDB tiles, suggest deploying the MongoDB server on the machine which will not involve in the operation of generating tiles to improve the working efficiency.
According to original data, the tile types, and performance of machines, select an appropriate scheme to generate tiles. During the procedure of generating tiles, you can increase or decrease processes to use the computer resource reasonably and effectively thereby improve greatly the efficiency of generating tiles.
Related Topics
Generate Map Tiles (Single-task)