Spring Data Mongodb多表關聯查詢

Spring Data Mongodb多表關聯查詢

前言

額瑞巴蒂，好。

最近公司的項目採用Mongodb作爲數據庫，我也是一頭霧水，因爲MongoDB是最近幾年才火起來，沒有什麼太多的學習資料。只有看Mongodb官網，Spring Data Mongodb官網文檔，看起也比較吃力。所以對Mongodb也是摸着石頭過河，有什麼不對的地方還請各位老鐵多多指教。

開始吧！

一、實例

爲了演示對象間一對一、一對多關係，現在創建三張表：公司（Company）、部門（Department）、員工（Employee）

1、數據準備

// 公司
public class Company {
    @Id
    private String id;

    private String companyName;

    private String mobile;
}

// 部門
public class Department {
    @Id
    private String id;

    private String departmentName;

    @DBRef
    private Company company;

    @DBRef
    private List<Employee> employeeList;
}

// 員工
public class Employee {
    @Id
    private String id;
    
    private String employeeName;
    
    private String phone;
    
    @DBRef
    private Department department;
}

創建測試所需的數據：

	@Autowired
    private MongoTemplate mongoTemplate;

    @Test
    public void initData() {
        // 公司
        Company company = new Company();
        company.setCompanyName("XXX公司");
        company.setMobile("023-66668888");
        mongoTemplate.save(company);

        // 部門
        Department department = new Department();
        department.setDepartmentName("XXX信息開發系統");
        department.setCompany(company);
        department.setEmployeeList(Collections.emptyList());
        mongoTemplate.save(department);

        // 員工
        List<Employee> employeeList = new ArrayList<>();
        Employee employee1 = new Employee();
        employee1.setEmployeeName("張一");
        employee1.setPhone("159228359xx");
        employee1.setDepartment(department);
        employeeList.add(employee1);

        Employee employee2 = new Employee();
        employee2.setEmployeeName("張二");
        employee2.setPhone("159228358xx");
        employee2.setDepartment(department);
        employeeList.add(employee2);
        mongoTemplate.insert(employeeList, Employee.class);

        department.setEmployeeList(employeeList);
        mongoTemplate.save(department);
    }

2、 一對一：兩表關聯查詢

RemoveDollarOperation ：自定義的Mongodb aggregation管道操作，在稍後的內容中會介紹

   /**
     * 員工表關聯部門表
     */
    @Test
    public void twoTableQuery() {
                // 1、消除@DBRef引用對象中的"$id"的"$"符號
        RemoveDollarOperation removeDollarOperation = new RemoveDollarOperation("newDepartmentFieldName", "department");

        // 2、使用mongodb $lookup實現左連接部門表
        LookupOperation lookupOperation = LookupOperation.newLookup().from("department")
                .localField("newDepartmentFieldName.id").foreignField("_id").as("newDepartment");

        // $match條件篩選
		// MatchOperation matchOperation = new MatchOperation(Criteria.where("newDepartment.departmentName").is("信息開發系統"));
        
        // 3、Aggregation管道操作（還可以加入$match、$project等其他管道操作，但是得注意先後順序）
        TypedAggregation aggregation = Aggregation.newAggregation(Employee.class, removeDollarOperation, lookupOperation);
		// TypedAggregation aggregation = Aggregation.newAggregation(Employee.class, removeDollarOperation, lookupOperation, matchOperation);
        AggregationResults<Document> results = mongoTemplate.aggregate(aggregation, Document.class);

        System.out.println(JSONArray.toJSONString(results.getMappedResults()));
    }

3、一對一：多表關聯查詢

/**
  * 員工表關聯部門表，部門表關聯公司表
  */
@Test
public void threeTableQuery() {
        // 1、消除@DBRef引用對象中的"$id"的"$"符號
        RemoveDollarOperation removeDollarOperation1 = new RemoveDollarOperation("newDepartmentFieldName", "department");

        // 2、使用mongodb $lookup實現左連接部門表
        LookupOperation lookupOperation1 = LookupOperation.newLookup().from("department")
                .localField("newDepartmentFieldName.id").foreignField("_id").as("newDepartment");

        // 3、使用$unwind展平步驟二中的左連接的department表的"newDepartment"
        UnwindOperation unwindOperation = new UnwindOperation(Fields.field("$newDepartment"));

        // 4、消除@DBRef引用對象中的"$id"的"$"符號
        RemoveDollarOperation removeDollarOperation2 = new RemoveDollarOperation("newCompanyFieldName", "newDepartment.company");

        // 5、使用mongodb $lookup實現左連接公司表
        LookupOperation lookupOperation2 = LookupOperation.newLookup().from("company")
                .localField("newCompanyFieldName.id").foreignField("_id").as("newCompany");

        MatchOperation matchOperation = new MatchOperation(Criteria.where("newCompany.companyName").is("XXX公司"));

        // 4、Aggregation管道操作（還可以加入$match、$project等其他管道操作，但是得注意先後順序）
        TypedAggregation aggregation = Aggregation.newAggregation(Employee.class,
                removeDollarOperation1, lookupOperation1,
                unwindOperation,
                removeDollarOperation2, lookupOperation2,
                matchOperation);

        AggregationResults<Document> results = mongoTemplate.aggregate(aggregation, Document.class);

        System.out.println(JSONArray.toJSONString(results.getMappedResults()));
}

4、一對多：關聯查詢

	/**
     * 查詢部門中的所有員工，部門關聯多個員工
     */
    @Test
	public void oneToManyTableQuery() {
        // 1、展平“多”的一方
        UnwindOperation unwindOperation = new UnwindOperation(Fields.field("employeeList"));

        // 2、消除@DBRef引用對象中的"$id"的"$"符號
        RemoveDollarOperation removeDollarOperation1 = new RemoveDollarOperation("newEmployeeFieldName", "employeeList");

        // 3、使用mongodb $lookup實現左連接員工表
        LookupOperation lookupOperation1 = LookupOperation.newLookup().from("employee")
                .localField("newEmployeeFieldName.id").foreignField("_id").as("newEmployee");

        // 篩選條件（非必須，看自己是否需要篩選）
        MatchOperation matchOperation = new MatchOperation(Criteria.where("newEmployee.employeeName").is("張一"));

        // 4、Aggregation管道操作（還可以加入$match、$project等其他管道操作，但是得注意先後順序）
        TypedAggregation aggregation = Aggregation.newAggregation(Employee.class,
                unwindOperation,
                removeDollarOperation1, lookupOperation1,
                matchOperation);

        AggregationResults<Document> results = mongoTemplate.aggregate(aggregation, Document.class);

        System.out.println(JSONArray.toJSONString(results.getMappedResults()));
    }

二、講道理

1、自定義RemoveDollarOperation管道操作的作用

先談談mongodb原生$lookup

我們先來看下mongodb的$lookup操作，這是mongodb $lookup的原生語法

{
   $lookup:
     {
       from: "collection to join（集合名）",
       localField: "field from the input documents（外鍵）",
       foreignField: "field from the documents of the "from" collection（被左連接的表的關聯主鍵）",
       as: "output array field（存放連接獲得的結果的列名）"
     }
}

然後使用原生語法進行lookup關聯操作，我們來看下員工表與部門表在Mongodb中的數據

// employee
{
    "_id": ObjectId("5c244aafc8fbfb40c02d830c"),
    "employeeName": "張一",
    "phone": "159228359xx",
    "department": DBRef("department", ObjectId("5c244aafc8fbfb40c02d830b")),
    "_class": "com.example.mongo.domain.company.Employee"
}

// department
{
    "_id": ObjectId("5c244aafc8fbfb40c02d830b"),
    "departmentName": "信息開發系統",
    "company": DBRef("company", ObjectId("5c244aafc8fbfb40c02d830a")),
    "employeeList": [
        DBRef("employee", ObjectId("5c244aafc8fbfb40c02d830c")),
        DBRef("employee", ObjectId("5c244aafc8fbfb40c02d830d"))
    ],
    "_class": "com.example.mongo.domain.company.Department"
}

你以爲可以直接通過下面方式進行表連接操作嗎，那就錯了

執行上面的mongo語句，會報以下錯誤

錯誤原因：field的名稱不支持以"$"開頭

那問題就來了，既然mongo原生lookup都不支持這一的操作，更何況Spring data mongodb了呢，那"localField"到底該填什麼才能實現表關聯呢？

去掉DBRef中"$id"的"$"

既然不能以"$“開頭，那我就把”$"去掉唄：

MongoDB官方提供的一個方法：https://jira.mongodb.org/browse/SERVER-14466

db.collection.aggregate({$addFields:{"newFieldName":
     {$arrayToObject:{$map:{
          input:{$objectToArray:"$localFieldName"}, 
          in:{
             k:{$cond:[ 
                     {$eq:[{"$substrCP":["$$this.k",0,1]},{$literal:"$"}]},
                     {$substrCP:["$$this.k",1,{$strLenCP:"$$this.k"}]},
                     "$$this.k"
             ]},
             v:"$$this.v"
           }
         }}}
}})

使用前：

"department": DBRef("department", ObjectId("5c244aafc8fbfb40c02d830b"))

使用後：

"department": {"ref":"department", "id": "5c244aafc8fbfb40c02d830b"}

去除"$“的方式是通過在結果中新追加一列"newFieldName”,這列的值是來至"$localFieldName"。

所以我們在使用過程中只需替換上面兩處的值即可。

來，我們按這方式操作一波：(修改爲"newDepartmentFieldName","$department")

db.employee.aggregate([{
    "$addFields": {
        "newDepartmentFieldName": {
            "$arrayToObject": {
                "$map": {
                    "input": {
                        "$objectToArray": "$department"
                    },
                    "in": {
                        "k": {
                            "$cond": [{
                                "$eq": [{
                                    "$substrCP": ["$$this.k", 0, 1]
                                }, {
                                    "$literal": "$"
                                }]
                            }, {
                                "$substrCP": ["$$this.k", 1, {
                                    "$strLenCP": "$$this.k"
                                }]
                            }, "$$this.k"]
                        },
                        "v": "$$this.v"
                    }
                }
            }
        }
    }
}, {
    "$lookup": {
        "from": "department",
        "localField": "newDepartmentFieldName.id",
        "foreignField": "_id",
        "as": "newDepartment"
    }
}])

結果出來咯，老鐵們

{
    "_id": ObjectId("5c244aafc8fbfb40c02d830c"),
    "employeeName": "張一",
    "phone": "159228359xx",
    "department": DBRef("department", ObjectId("5c244aafc8fbfb40c02d830b")),
    "_class": "com.example.mongo.domain.company.Employee",
    "newDepartmentFieldName": {
        "ref": "department",
        "id": ObjectId("5c244aafc8fbfb40c02d830b")
    },
    "newDepartment": [
        {
            "_id": ObjectId("5c244aafc8fbfb40c02d830b"),
            "departmentName": "信息開發系統",
            "company": DBRef("company", ObjectId("5c244aafc8fbfb40c02d830a")),
            "employeeList": [
                DBRef("employee", ObjectId("5c244aafc8fbfb40c02d830c")),
                DBRef("employee", ObjectId("5c244aafc8fbfb40c02d830d"))
            ],
            "_class": "com.example.mongo.domain.company.Department"
        }
    ]
}

自定義RemoveDollarOperation管道操作

前面說了這麼多，就是想告訴你們，我爲什麼要自定義一個RemoveDollarOperation管道操作。就是爲了解決Mongodb $lookup的"localField"的值不支持以"$"開頭

以下是RemoveDollarOperation的實現：

只需implements AggregationOperation，實現toDocument()方法即可

/**
 * @author : zhangmeng
 * Date : 2018/12/27 11:13
 * Description : 自定義的Spring data mongodb的Aggregation Operation
 */
public class RemoveDollarOperation implements AggregationOperation {
    /**
     * 查詢結果新追加的列名
     */
    private String newField;

    /**
     * 需要關聯的表中的外鍵
     */
    private String localField;

    public RemoveDollarOperation(String newField, String localField) {
        this.newField = newField;
        this.localField = localField;
    }

    @Override
    public Document toDocument(AggregationOperationContext context) {
        List<Object> eqObjects = new ArrayList<>();
        eqObjects.add(new Document("$substrCP", Arrays.asList("$$this.k", 0, 1)));
        eqObjects.add(new Document("$literal", "$"));

        List<Object> substrCPObjects = new ArrayList<>();
        substrCPObjects.add("$$this.k");
        substrCPObjects.add(1);
        substrCPObjects.add(new Document("$strLenCP", "$$this.k"));

        List<Object> objects = new ArrayList<>();
        objects.add(new Document("$eq", eqObjects));
        objects.add(new Document("$substrCP", substrCPObjects));
        objects.add("$$this.k");

        Document operation = new Document(
                "$addFields",
                new Document(newField,
                        new Document("$arrayToObject",
                                new Document("$map",
                                        new Document("input",new Document("$objectToArray", "$"+localField))
                                                .append("in", new Document("k",new Document("$cond", objects))
                                                        .append("v", "$$this.v")))
                        )
                )
        );

        return context.getMappedObject(operation);
    }
}

你看到那麼多的Document 拼接，其實就是爲了實現

db.collection.aggregate({$addFields:{"newFieldName":
     {$arrayToObject:{$map:{
          input:{$objectToArray:"$localFieldName"},  ...

注意事項：
在實現過程中，可能因爲Spring-data-mongodb版本不同，

	
	// Spring-data-mongodb 2.0以上使用Org.bson的Document （具體版本不確定）

	@Override
    public Document toDocument(AggregationOperationContext context) {
    	...
    	...
		Document operation = new Document(
                "$addFields",
                new Document(newField,"")
                ...
                ...
        );

        return context.getMappedObject(operation);
	}

	// Spring-data-mongodb 2.0以下使用com.mongodb.BasicDBObject
	
	@Override
    public DBObject toDBObject(AggregationOperationContext context) {
		...
    	...
		DBObject operation = new DBObject (
                "$addFields",
                new DBObject (newField,"")
                ...
                ...
        );

        return context.getMappedObject(operation);
}

2、實例中的一對一多表關聯查詢中的第4步使用UnwindOperation的原因

可能當我們實現了實例1中的一對一兩表關聯查詢後，順理成章就覺得如果要再關聯第三張表的話，直接再使用

一次RemoveDollarOperation，LookupOperation進行關聯

db.employee.aggregate([{
    "$addFields": {
        "newDepartmentFieldName": {
            "$arrayToObject": {
                "$map": {
                    "input": {
                        "$objectToArray": "$department"
                    },
                    "in": {
                        "k": {
                            "$cond": [{
                                "$eq": [{
                                    "$substrCP": ["$$this.k", 0, 1]
                                }, {
                                    "$literal": "$"
                                }]
                            }, {
                                "$substrCP": ["$$this.k", 1, {
                                    "$strLenCP": "$$this.k"
                                }]
                            }, "$$this.k"]
                        },
                        "v": "$$this.v"
                    }
                }
            }
        }
    }
}, {
    "$lookup": {
        "from": "department",
        "localField": "newDepartmentFieldName.id",
        "foreignField": "_id",
        "as": "newDepartment"
    }
}, {
    "$addFields": {
        "newCompanyFieldName": {
            "$arrayToObject": {
                "$map": {
                    "input": {
                        "$objectToArray": "$newDepartment.company"
                    },
                    "in": {
                        "k": {
                            "$cond": [{
                                "$eq": [{
                                    "$substrCP": ["$$this.k", 0, 1]
                                }, {
                                    "$literal": "$"
                                }]
                            }, {
                                "$substrCP": ["$$this.k", 1, {
                                    "$strLenCP": "$$this.k"
                                }]
                            }, "$$this.k"]
                        },
                        "v": "$$this.v"
                    }
                }
            }
        }
    }
}, {
    "$lookup": {
        "from": "company",
        "localField": "newCompanyFieldName.id",
        "foreignField": "_id",
        "as": "newCompany"
    }
}])

但是，執行後就錯了：

來來來，我們一步一步分析下

這是Employee關聯Department後，得到的結果，“newDepartment"是關聯後得到的結果：

我們如果要進一步Department關聯Company的話，直接再使用RemoveDollarOperation，LookupOperation是不行的，因爲在消除”$"操作時入參需要一個非數組對象，而前一步的結果的"newDepartment"是一個數組，所以報錯了

爲了得到一個非數組對象，我們就要使用$unwind將"newDepartment"展平

然後就可以使用"newDepartment"繼續RemoveDollarOperation，LookupOperation操作了。最終得到Employee關

聯Department關聯Company的結果了。如果還想繼續關聯，就以此類推。

最終得出的模型：

一對一兩表關聯的步驟：
1、RemoveDollarOperation
2、LookupOperation

一對一多表關聯的步驟：
1、RemoveDollarOperation  2、LookupOperation
3、UnwindOperation
4、RemoveDollarOperation  5、LookupOperation
...

一對多表關聯的步驟：
1、UnwindOperation
2、RemoveDollarOperation
3、LookupOperation