原文: https://vyskocil.org/blog/implement-sql-database-driver-in-100-lines-of-go/
--------------------
Implement Sql Database Driver in 100 Lines of Go
2019.02.18
Go database/sql defines interfaces for SQL databases. Actual driver must be implemented in own package. And dependency injection is done as a part of import and build system. Lets go deeper to see how it is actually implemented.
This exercise will result in a simple driver backed up by csv files. Who do not want to SELECT
from csv file?
Disclaimer: I wrote it in order to learn database/sql
and to learn more about design of APIs for go. I regularly use deprecated methods, which have advanced variants in recent standard library. Consult documentation if you want to know more.
I have had three goals in mind
- Learning
- Simplicity
- 100 lines of code
database/sql
intro
Here is small snippet of Go program using Postgres https://github.com/lib/pq/ driver to talk to database. It use generic methods of database/sql
interfaces, so switching the driver and fixing SQL dialects, we can talk to any database system without a need to change much of the code.
|
|
Magic inside
There is a magic inside
|
|
|
|
Driver is magically registered during module import. It sounds like a magic, isn’t? And it turns out it is actually very simple. Function init()
is package initializer, which can run arbitrary code before all other code. In case of SQL driver, it contains the code to register itself.
|
|
Similary for mysql
|
|
Registering SQL Driver
With all those knowledge one can implement dummy SQL driver
|
|
And to test it has been registered …
Drivers=[]string{"dummy"}
Of course it is really dummy driver as it does not offer ANY functionality.
Connection
As we see, Driver
interface returns driver.Conn
, which is yet another interface. This defines three methods, which can be skipped for the csv case.
First make Open
real function, so let is open the file
|
|
Here one can see the beauty of Go error reporting. File operation errors the same way as database server errors.
This is a bit uncommon for real SQL driver, however we do not have server to connect to. So Open
tries to open the file at least to return file access errors as early as possible. All the heavy lifting is done below.
Here is bunch of methods not interesting for the use case … all those returns not implemented error.
|
|
So for read-only file access the connection is really dumb method, there is only one file path, which needs to be passed.
Do the query
SQL queries are base of the system. Here is a constraint by supporting only one query string. However I do prefer brevity over anything else, so here we are. The driver.Queryer
interface
|
|
Again, nothing really complicated. One needs to handle errors and then return correct structure with proper implemented interface methods.
Query methods returns an object, which allows read of the result. Real database systems deal well with the read and write locks, concurrent access and more. You have maybe heard about database cursor
. In a case of read only csv file, the situation is way more simpler.
As each Query
can pass independent results, opened file is a part of results
. So each results
will have independent file handle and file position, which is an equivalent of database cursor
.
Read the results
One needs to implement next interface. driver.Querier
returns object with driver.Rows
interface. Here is structure results
used to read data through Query
and Scan
methods. It provides enough to implement Rows
interface
|
|
And implementation is fairly straightforward. We do assume that first line of csv file contains names of columns. So this is initialized as a part of query. It simplify Columns[]
method a lot!
|
|
And to not leak file handles, close them once results are read
|
|
And the most important method. Under the hood it reads data from csv file and put them to internal buffer dest[]
, which is then managed by database/sql
code. This is the way how data ends up in Scan
method later on.
func (r *results) Next(dest []driver.Value) error {
d, err := r.reader.Read()
if err != nil {
return err
}
for i := 0; i != len(r.columns); i++ {
dest[i] = driver.Value(d[i])
}
return nil
}
Running together
|
|
And a proof, result of running program
|
|
Conclusion
Complete program is available at github.com/vyskocilm/gazpacho/dbmagic. And as last method, Next
ends at line 100. The goal to implement dummy
sql driver in 100 lines of Go code was achieved.
After going through the article reader shall be able
- To understand the automagic registration of SQL drivers
- To understand the design of
database/sql
better - Hopefully enjoy the simplicity, but expressiveness of Go interfaces
- Learn that design of interfaces is hard and
database/sql
offer V2 interfaces in some cases.
Logo by travis_warren123@Flickr: [https://www.flickr.com/photos/travis_warren123/4229031035/]