第二部分：空間SQL的神奇世界

空間SQL（結構化查詢語言）能夠加速你的處理（當被廣泛運用的時候）。下圖是使用SHP和QGIS，以及使用PostGIS來處理500萬個隨機點的對比。

（數據庫相關的博文總是會有個柱狀圖來對比處理時間的。PostGIS = 非常快。柱狀圖不說謊。）

在這個對比中我使用了芬蘭的郵政編碼數據和每個郵編區域的人口數據。在SHP文件和本地的數據庫中我都存有這份數據。在每個多邊形中我穿件隨機點來表示人口。我使用了QGIS處理SHP文件（QGIS的矢量處理工具），在PostGIS中使用了下面這條簡單的SQL:

SELECT ST_GeneratePoints(geom, he_vakiy) from paavo.paavo

正如你從前面的圖中看見的，做相同的分析，PostGIS花了QGIS和SHP方案不到10%的時間。如果你是一個GIS分析員，你每天都會做類似的工作，那麼一年下來PostGIS能夠幫助你節省相當多的時間。

除了快速處理外，你能享受PostGIS提供的一大推空間函數(spatial functions)。具體哪些函數對你有用取決於你的業務需求。

在Voronoi分析和更加傳統的GIS分析（緩存分析，重疊分析，相交分析，剪切分析等等）你能做更高級的分析。例如：

路由用 pgRouting 和道路數據你能找到優化的路由，而且能做不同的網絡分析；
多邊形骨架化這個功能使得你能夠輕鬆地構造多邊形的中軸; （譯者注：我不熟悉GIS分析方面，學習中，歡迎指正）
幾何形狀分割爲後續處理而劃分幾何形狀能夠加速你的GIS處理過程;
聚類從你的數據中找到聚類和模式。現在AI正在風口上，對有的人來說k-means算法可能比以前更有吸引力。。。。。。

你要多邊形骨架化有什麼用？大多數人可能會有這個問題，但是當你的空間分析需要它的時候，你會非常開心，因爲已經有人幫你搞定最難的部分了（嗯，就是數學）。組合各種空間函數和Postgres自身的函數，使你能夠完成高級的空間分析。

一些複雜而有趣的問題（空間連接，聚合等等），能用一條數據庫SQL語句表達，但他們需要很多的計算，這正是PostGIS帶給你的。如果你用編程的方式訪問文件（SHP）的話，那麼解決同樣的問題，你可能需要編寫幾百行特定功能的代碼。

PostGIS和可視化

在我過去的可視化項目中，PostGIS在可視化中扮演了一份角色。我常常預處理數據，然後在QGIS中做真正的可視化。

讓我們來看一個例子。

火車的動態Voronoi線。古怪，但是滿足我的需求。

這個火車和Voronois的動圖是展現PostGIS能力的一個好玩例子。我在本地數據庫中存有幾百萬的或者GPS點數據，用這些點的移動我創建了動畫。但是我想測試下Voronoi線的動畫會是什麼樣。

首先，每列火車每分鐘都有幾個GPS的點數據，所以我需要將這些點組合成一個點來代表或者每分鐘所在的位置。我先是手動創建了一張表來存儲這些結果點。我寫了這樣的一條SQL語句。

INSERT INTO trains.voronoipoints 
SELECT '2018–01–15 09:00:00' AS t, 
       geom 
FROM   (SELECT St_centroid(St_collect(geom)) AS geom, 
               trainno 
        FROM   (SELECT geom, 
                       trainno 
                FROM   trains.week 
                WHERE  time > '2018–01–15 09:00:00' 
                       AND time < '2018–01–15 09:01:00') AS a 
        GROUP  BY trainno) AS b

如果我們拆解這條語句我們能夠得到:

你能看到一些SQL查詢的常規元素 (INSERT INTO, SELECT, AS, FROM, WHERE, AND, GROUP BY)
geom, trainno 和 time 是我在“trains"大綱，“week”表中的列名
子查詢a返回dd returns all GPS points which have been tracked within the requested timeframe.
Because I select all GPS points tracked inside one minute, I might get several points for each train. I only wanted one, so that the voronoi lines would look more sensible. That’s why I use ST_Collect to group the points together and to create a multipoint geometry from them. ST_Centroid replaces the multipoint geometry with a single point located at the centroid (subquery b) and the data is grouped by train numbers.

To do the same thing multiple times, I had a simple Python script to loop over the same query for a few hundred times where I had the start and end times as parameters. After successfully finding one representative point for each minute, I just ran the following command (in 11.5 seconds):

SELECT t, ST_VoronoiLines(geom) from trains.voronoipoints

Then I added the result to QGIS and visualized it with Time Manager. This might be a bit hacky way to achieve the result and a more experienced SQL user might’ve done it completely with a single SQL command, but I’m still pretty happy with the result. Although it might be pointless.

Eventually pretty simple, but the result looks like higher level math (and it is!), as all the hard work is done by PostGIS. Also because I was able to make the Voronoi analysis for only one point per train, the processing time was only seconds for hundreds of thousands of points.

Often the processing time of your queries grows exponentially as the data amounts grow. This is why you have to be smart with your queries.

Hey look! I made a SQL meme!

As a rule of thumb, the more data a query has to fetch and more operations the database has to do (ordering, grouping etc), it becomes slower and thus less efficient. An efficient SQL query only fetches the rows and columns it really needs. SQL can work like a logical puzzle, where you really have to think thoroughly what you want to achieve.

I must also note that tweaking the performance of your queries is a slippery slope and you can get lost in the world of endless optimization. Finding the balance between an “optimal query” and an optimal query is really important. Especially if you are not building an application for a million users, a few milliseconds here or there won’t probably rock your boat.

如何開始？

我敢說學習SQL對於GIS用戶來說，比學習JavaScript, Python或者R語言更加有用。多少年過去了，SQL的語法只有很微小的變化，SQL技能是非常保值的。

I have found that the learning curve in SQL isn’t really steep to do the basics, but it might take you some time to really see the benefits that it can bring to your spatial analysis. But I encourage to be patient and try more complicated analytics and aim for faster processing. Eventually you will see the difference.

First when you are learning SQL basics you will learn how to query data from a single table using basic data selection techniques such as selecting columns, sorting result set, and filtering rows. Then, you will learn about the advanced queries such as joining multiple tables, using set operations, and constructing a subquery. Finally, you will learn how to manage database tables such as creating new a table or modifying an existing table’s structure.

But there also also tools to help you out!

QGIS有一個很讚的工具叫做DB Manager。（譯者注：我們有一個很讚的QQ羣“開源GIS技術交流羣”，羣號767137544，歡迎加入）DB Manager提供了一個數據庫圖形界面，它在QGIS裏邊而且更容易理解。你使用右鍵點點就能修改和增添數據庫表，添加索引，以及做一些其他的基本操作。

【QGIS DB Manager截圖.】

你也應該看看pgAdmin，他是PostgreSQL數據庫最流行的管理和開發平臺。有多重方法能夠將你的數據導入到PostGIS(例如org2org, shp2pgsql)。通常來說我鼓勵你使用不同的工具和方法來跟數據打交道。

我結合Python和PostGIS做了很多實驗。讓Python(或者R)和PostGIS合作，能夠讓你的數據處理和自動化上一個臺階。使用psycopg2，結合Python基本的腳本能力，連接到PostGIS是上手的好方法。

這麼好的東東？你是不是想開始擼了？

下載PostGIS並且安裝到你的機器上。鏈接：download the installers
加載數據到PostGIS。你可以從單個的SHP文件入手，使用QGIS DB Manager把它導入到數據庫；也可以參考教程（this tutorial）中的例子，把Natural Earth data數據導入到PostGIS。
開始擼SQL。從基礎的選擇，過濾和修改更新數據開始，慢慢地你會看到PostGIS能給你工作帶來的好處。