Tungsten Fabric解決方案指南-Kubernetes集成

作者：Tony Liu 譯者：TF編譯組

1 Kubernetes與TF的集成

集成方案中，在Kubernetes和Tungsten Fabric（編者按：原文爲Contrail，其開源版已更名爲Tungsten Fabric，本文出現Contrail之處均以Tungsten Fabric替換）之間有兩個連接。

• contrail-kube-manager和kube-api-server

• Contrail CNI

1.1 contrail-kube-manager

此服務連接到kube-api-server以接收更新。然後，它會連接到Tungsten Fabric配置API服務器，來創建必要的配置（VM，VMI /端口，IP等），以將容器連接到overlay層。它還會將更新發送到kube-api-server。

1.2 Contrail CNI

每個node/minion上的Kubelet都使用CNI參數運行。啓動容器時，kubelet調用CNI來建立網絡。Contrail CNI連接到vRouter代理REST API：

1）獲得必要的配置
2）將容器網絡接口插入vRouter

1.3 Gateway

Tungsten Fabric使用gateway連接overlay和underlay網絡，以提供外部訪問。我們需要使用gateway來支持Kubernetes的暴露服務和ingress功能。

必須在Tungsten Fabric中創建一個浮動IP池。

在/etc/contrailctl/kubemanager.conf中配置這個FIP池的FQ名稱，以進行配置（provisioning）。

[KUBERNETES_VNC]public_fip_pool = {'domain': 'default-domain', 'project': 'default', 'network': 'public', 'name': 'public-fip-pool'}

在容器condir-kube-manager中，浮動IP池在/etc/contrail/contrail-kubernetes.conf中進行配置。

[VNC]public_fip_pool = {'domain': 'default-domain', 'project': 'default', 'network': 'public', 'name': 'public-fip-pool'}

在Kubernetes中公開service或創建ingress時，將從該池中分配一個FIP作爲外部IP。

2 Namespace

與Tungsten Fabric集成時，Kubernetes命名空間（namespace）可以映射到項目/租戶（project/tenant）或虛擬網絡。

2.1 單租戶（Single-tenant）

如果有設置/etc/contrail/contrail-kubernetes.conf中的[KUBERNETES].cluster_project，它是單租戶（single-tenant），Kubernetes命名空間將映射到Tungsten Fabric中的虛擬網絡。所有非隔離命名空間都映射到默認虛擬網絡“cluster-network”。而每個隔離命名空間都映射到一個單獨的虛擬網絡“-vn”。

這裏有一個示例，說明在/etc/contrailctl/kubemanager.conf中設置[KUBERNETES].cluster_project以啓用單租戶的情形。

[KUBERNETES]cluster_project = {'domain': 'default-domain', 'project': 'kubernetes'}

以下是由Condir-kube-manager在初始化期間創建的。

• Flat IPAM <cluster_project>：具有子網的pod-ipam

• IPAM <cluster_project>：service-ipam

• 虛擬網絡“cluster-network”的安全組k8s-default-default-default和k8s-default-default-sg

• 虛擬網絡<cluster_project>：具有pod-ipam和service-ipam的cluster-network

（參見附錄A.1）

2.1.1 非隔離的命名空間

創建一個非隔離的命名空間。

apiVersion: v1kind: Namespacemetadata: name: "dev-unisolated"

當Kubernetes創建一個非隔離命名空間時，Tungsten Fabric將創建兩個SG，即k8s-default--sg和k8s-default--sg。這裏不創建虛擬網絡。所有非隔離的NS中的容器都將位於cluster-network上。

在非隔離命名空間中啓動一個Pod。

apiVersion: v1kind: Podmetadata:  name: nginx-1spec:  containers:  - name: nginx    image: docker.io/nginx    imagePullPolicy: IfNotPresentkubectl create -f nginx-1.yaml -n <namespace>kubectl get pods -n <namespace>

在非隔離命名空間中啓動Pod時，Tungsten Fabric（contrail-kube-manager）將執行以下操作。

不同非隔離命名空間中的Pod可以相互連接，因爲它們位於Tungsten Fabric中的同一虛擬網絡上。

2.1.2 隔離的命名空間

創建一個隔離的命名空間。

apiVersion: v1kind: Namespacemetadata: name: "dev-isolated" annotations: {   "opencontrail.org/isolation" : "true" }

在Kubernetes中創建隔離命名空間時，Tungsten Fabric將創建以下內容。

• 虛擬網絡<cluster_project>:-vn
• 安全組k8s-default--default和k8s-default--sg

在隔離命名空間中啓動Pod時，Tungsten Fabric將執行以下操作。

由於端口位於不同的虛擬網絡上，因此不同的隔離命名空間中的Pods無法相互連接。

2.2 多租戶（Multi-tenant）

如果未設置/etc/contrail/contrail-kubernetes.conf中的[KUBERNETES].cluster_project，它就是多租戶，Kubernetes命名空間將映射到Tungsten Fabric中的租戶/項目（tenant/project）。非隔離命名空間中的Pod在默認虛擬網絡“cluster-network”上啓動。而每個隔離命名空間都映射到一個單獨的虛擬網絡“-vn”。

在初始化時，contrail-kube-manager創建以下內容：

• 爲每個現有的Kubenetes命名空間（如default、kube-public和kube-system）提供一個項目/租戶。
• Flat IPAM 默認域：default:pod-ipam
• IPAM 默認域：default:service-ipam
• 每一個命名空間的安全組：k8s-default--sg和k8s-default--sg
• 虛擬網絡默認域名：具有pod-ipam的default:cluster-network和service-ipam

2.2.1 非隔離的命名空間

創建一個非隔離的命名空間。

apiVersion: v1kind: Namespacemetadata: name: "dev-unisolated"

Contrail-kube-manager將創建以下內容。

當在非隔離命名空間中啓動Pod時，conventil-kube-manager將創建端口：

不同非隔離命名空間中的Pod可以相互連接，因爲它們位於Tungsten Fabric中的同一虛擬網絡上。

2.2.2 隔離的命名空間

創建一個隔離的命名空間。

apiVersion: v1kind: Namespacemetadata: name: "dev-isolated" annotations: {   "opencontrail.org/isolation" : "true" }

Contrail-cube-manager將創建以下內容。

• 項目缺省域：
• 虛擬網絡default-domain::-vn與default-domain:default:pod-ipam關聯。
• 安全組default-domain::k8s-default--default和default-domain::k8s-default--sg

當在非隔離命名空間中啓動Pod時，conventil-kube-manager將創建端口。

• 在項目default-domain:中
• 在虛擬網絡default-domain::-vn上
• 從IPAM default-domain:default:pod-ipam獲取地址
• 附帶安全組default-domain::k8s-default--default和default-domain::k8s-default--sg

由於端口位於不同的虛擬網絡上，因此不同的隔離命名空間中的Pods無法相互連接。

2.3 自定義命名空間

創建一個自定義命名空間。

apiVersion: v1kind: Namespacemetadata: name: "dev-customized" annotations: {   "opencontrail.org/network": '{"domain": "default-domain", "project": "demo", "name": "red"}' }

在自定義命名空間中啓動Pod時，contrail-kube-manager將創建端口。

• 在項目default-domain:default中
• 在虛擬網絡上映射到自定義命名空間
• 從與該虛擬網絡關聯的IPAM上獲取地址
• 安全組？

2.4指定虛擬網絡上的Pod

在指定的虛擬網絡上啓動Pod。

當在指定的虛擬網絡上啓動Pod時，conutil-kube-manager將創建端口。

• 在項目中映射到指定或默認的命名空間
• 在指定的虛擬網絡上
• 從與特定虛擬網絡關聯的IPAM上獲取地址
• 安全組？

2.5 Kubernetes網絡策略

Kubernetes網絡策略將照常運行，它由Tungsten Fabric中的安全組實現。該版本將與4.0.1一起發佈。

2.6 POD SNAT

Tungsten Fabric支持該功能，可以在Tungsten Fabric中配置一個路由器（配置對象），使其成爲啓動容器的虛擬網絡的外部網關。這與支持OpenStack的外部網關是一樣的。

3 Service

Kubernetes service支持ClusterIP，NodePort，LoadBalancer和ExternalName。它還支持使用ExternalIP指定IP。Tungsten Fabric支持ClusterIP和LoadBalancer，以及ExternalIP。

在Kubernetes中創建service時，Tungsten Fabric中會創建一個負載均衡器（loadbalancer）。負載均衡器的提供者爲“native”，而ECMP負載均衡由vRouter實現。浮動IP被創建爲VIP。

創建具有多個實例的應用程序。

apiVersion: v1kind: ReplicationControllermetadata:  name: web-qaspec:  replicas: 2  selector:    app: web-qa  template:    metadata:      name: web-qa      labels:        app: web-qa    spec:      containers:      - name: web        image: docker.io/nginx        imagePullPolicy: IfNotPresent

3.1 ClusterIP

在這些應用程序前面創建service。默認的service類型是ClusterIP。

kind: ServiceapiVersion: v1metadata:  name: web-qaspec:  selector:    app: web-qa  ports:    - protocol: TCP      port: 80      targetPort: 80

當service被創建後，conventil-kube-manager將執行以下操作。

當LB被創建後，“原生”LB驅動程序將執行以下操作。

• 在FIP中設置端口映射。
• 將所有成員的VMI添加到FIP。

當service類型爲ClusterIP時，只能在集羣內訪問該service。FIP從集羣網絡（cluster-network）中的service FIP池中分配，並映射到所有的Pod地址。當訪問集羣內的service地址時，vRouter將在Pod之間平衡流量。

3.2 Loadbalancer

創建一個LoadBalancer類型的service。

kind: ServiceapiVersion: v1metadata:  name: web-qaspec:  selector:    app: web-qa  ports:    - protocol: TCP      port: 80      targetPort: 80  type: LoadBalancer

對於服務類型LoadBalancer，服務被暴露於外部。從服務FIP池中分配FIP，用於集羣內的訪問，同時從公共FIP池中分配FIP，映射到所有POD地址。該FIP將被通告給網關，網關將在POD之間進行ECMP負載均衡。

附錄A 單租戶（Single-tenant）

A.1 IPAM

<cluster_project>:pod-ipam

{    "fq_name": [        "default-domain",        "kubernetes",        "pod-ipam"    ],    "uuid": "c9641741-c785-456e-845b-a14a253c3572",    "ipam_subnet_method": "flat-subnet",    "parent_type": "project",    "perms2": {        "owner": "None",        "owner_access": 7,        "global_access": 0,        "share": []    },    "ipam_subnets": {        "subnets": [            {                "subnet": {                    "ip_prefix": "10.32.0.0",                    "ip_prefix_len": 12                },                "dns_server_address": "10.47.255.253",                "enable_dhcp": true,                "created": null,                "default_gateway": "10.47.255.254",                "dns_nameservers": [],                "dhcp_option_list": null,                "subnet_uuid": null,                "alloc_unit": 1,                "last_modified": null,                "host_routes": null,                "addr_from_start": null,                "subnet_name": null,                "allocation_pools": []            }        ]    },    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2017-12-27T18:45:33.957901",        "uuid": {            "uuid_mslong": 14511749470582293870,            "uuid_lslong": 9537393975711511922        },        "user_visible": true,        "last_modified": "2017-12-27T18:45:33.957901",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "display_name": "pod-ipam"}

<cluster_project>:service-ipam

{    "fq_name": [        "default-domain",        "kubernetes",        "service-ipam"    ],    "uuid": "526f554a-0bf4-47c6-a8e4-768a3f98cef4",    "parent_type": "project",    "perms2": {        "owner": "None",        "owner_access": 7,        "global_access": 0,        "share": []    },    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2017-12-27T18:45:34.000690",        "uuid": {            "uuid_mslong": 5940060210041472966,            "uuid_lslong": 12169982429206466292        },        "user_visible": true,        "last_modified": "2017-12-27T18:45:34.000690",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "display_name": "service-ipam"}

A.2 安全組

k8s-default-dev-share-default

{    "fq_name": [        "default-domain",        "kubernetes",        "k8s-default-dev-share-default"    ],    "uuid": "ad29de07-5ef6-4f55-86bb-52c44827c09d",    "parent_type": "project",    "perms2": {        "owner": "46c31b9b-d21c-4c27-9445-6c94db948b6d",        "owner_access": 7,        "global_access": 0,        "share": []    },    "security_group_id": 8000010,    "id_perms": {        "enable": true,        "description": "Default security group",        "creator": null,        "created": "2018-01-12T09:02:15.110429",        "uuid": {            "uuid_mslong": 12477748365846007637,            "uuid_lslong": 9708444424704868509        },        "user_visible": true,        "last_modified": "2018-01-12T15:45:08.899388",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "security_group_entries": {        "policy_rule": [            {                "direction": ">",                "protocol": "any",                "dst_addresses": [                    {                        "security_group": "local",                        "subnet": null,                        "virtual_network": null,                        "subnet_list": [],                        "network_policy": null                    }                ],                "action_list": null,                "created": null,                "rule_uuid": "dc13bb48-e2a7-4c59-a0b8-740ecfcb9a2c",                "dst_ports": [                    {                        "end_port": 65535,                        "start_port": 0                    }                ],                "application": [],                "last_modified": null,                "ethertype": "IPv4",                "src_addresses": [                    {                        "security_group": null,                        "subnet": {                            "ip_prefix": "0.0.0.0",                            "ip_prefix_len": 0                        },                        "virtual_network": null,                        "subnet_list": [],                        "network_policy": null                    }                ],                "rule_sequence": null,                "src_ports": [                    {                        "end_port": 65535,                        "start_port": 0                    }                ]            },            {                "direction": ">",                "protocol": "any",                "dst_addresses": [                    {                        "security_group": "local",                        "subnet": null,                        "virtual_network": null,                        "subnet_list": [],                        "network_policy": null                    }                ],                "action_list": null,                "created": null,                "rule_uuid": "a84e2d98-2b8f-45ba-aa75-88494da73b11",                "dst_ports": [                    {                        "end_port": 65535,                        "start_port": 0                    }                ],                "application": [],                "last_modified": null,                "ethertype": "IPv6",                "src_addresses": [                    {                        "security_group": null,                        "subnet": {                            "ip_prefix": "::",                            "ip_prefix_len": 0                        },                        "virtual_network": null,                        "subnet_list": [],                        "network_policy": null                    }                ],                "rule_sequence": null,                "src_ports": [                    {                        "end_port": 65535,                        "start_port": 0                    }                ]            },            {                "direction": ">",                "protocol": "any",                "dst_addresses": [                    {                        "security_group": null,                        "subnet": {                            "ip_prefix": "0.0.0.0",                            "ip_prefix_len": 0                        },                        "virtual_network": null,                        "subnet_list": [],                        "network_policy": null                    }                ],                "action_list": null,                "created": null,                "rule_uuid": "b7752ec1-6037-4c7f-97a9-291893fbed64",                "dst_ports": [                    {                        "end_port": 65535,                        "start_port": 0                    }                ],                "application": [],                "last_modified": null,                "ethertype": "IPv4",                "src_addresses": [                    {                        "security_group": "local",                        "subnet": null,                        "virtual_network": null,                        "subnet_list": [],                        "network_policy": null                    }                ],                "rule_sequence": null,                "src_ports": [                    {                        "end_port": 65535,                        "start_port": 0                    }                ]            },            {                "direction": ">",                "protocol": "any",                "dst_addresses": [                    {                        "security_group": null,                        "subnet": {                            "ip_prefix": "::",                            "ip_prefix_len": 0                        },                        "virtual_network": null,                        "subnet_list": [],                        "network_policy": null                    }                ],                "action_list": null,                "created": null,                "rule_uuid": "ea5cd2a8-2d47-47c4-a9ab-390de2317246",                "dst_ports": [                    {                        "end_port": 65535,                        "start_port": 0                    }                ],                "application": [],                "last_modified": null,                "ethertype": "IPv6",                "src_addresses": [                    {                        "security_group": "local",                        "subnet": null,                        "virtual_network": null,                        "subnet_list": [],                        "network_policy": null                    }                ],                "rule_sequence": null,                "src_ports": [                    {                        "end_port": 65535,                        "start_port": 0                    }                ]            }        ]    },    "annotations": {        "key_value_pair": [            {                "key": "namespace",                "value": "dev-share"            },            {                "key": "cluster",                "value": "k8s-default"            },            {                "key": "kind",                "value": "Namespace"            },            {                "key": "project",                "value": "kubernetes"            },            {                "key": "name",                "value": "k8s-default-dev-share-default"            },            {                "key": "owner",                "value": "k8s"            }        ]    },    "display_name": "k8s-default-dev-share-default"}

k8s-default-dev-share-sg

{    "fq_name": [        "default-domain",        "kubernetes",        "k8s-default-dev-share-sg"    ],    "uuid": "791f1c7e-a66e-4c47-ba05-409f00ee2c8e",    "parent_type": "project",    "perms2": {        "owner": "46c31b9b-d21c-4c27-9445-6c94db948b6d",        "owner_access": 7,        "global_access": 0,        "share": []    },    "security_group_id": 8000017,    "id_perms": {        "enable": true,        "description": "Namespace security group",        "creator": null,        "created": "2018-01-12T09:02:15.236401",        "uuid": {            "uuid_mslong": 8727725933151013959,            "uuid_lslong": 13404190917597736078        },        "user_visible": true,        "last_modified": "2018-01-12T09:02:15.275407",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "display_name": "k8s-default-dev-share-sg",    "annotations": {        "key_value_pair": [            {                "key": "namespace",                "value": "dev-share"            },            {                "key": "cluster",                "value": "k8s-default"            },            {                "key": "kind",                "value": "Namespace"            },            {                "key": "project",                "value": "kubernetes"            },            {                "key": "name",                "value": "k8s-default-dev-share-sg"            },            {                "key": "owner",                "value": "k8s"            }        ]    }}

A.3 虛擬網絡

<cluster_project>:cluster-network

{    "virtual_network_properties": {        "forwarding_mode": "l3",        "allow_transit": null,        "network_id": null,        "mirror_destination": false,        "vxlan_network_identifier": null,        "rpf": null    },    "fq_name": [        "default-domain",        "kubernetes",        "cluster-network"    ],    "uuid": "1b9f7f74-17f0-493a-9108-729f91b43598",    "address_allocation_mode": "user-defined-subnet-only",    "mac_aging_time": 300,    "parent_type": "project",    "perms2": {        "owner": "None",        "owner_access": 7,        "global_access": 0,        "share": []    },    "display_name": "cluster-network",    "pbb_evpn_enable": false,    "mac_learning_enabled": false,    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2017-12-27T18:45:34.062865",        "uuid": {            "uuid_mslong": 1990449696915605818,            "uuid_lslong": 10450728964983109016        },        "user_visible": true,        "last_modified": "2017-12-29T10:29:20.685414",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "flood_unknown_unicast": false,    "layer2_control_word": false,    "port_security_enabled": true,    "network_ipam_refs": [        {            "to": [                "default-domain",                "kubernetes",                "service-ipam"            ],            "href": "http://127.0.0.1:8082/network-ipam/526f554a-0bf4-47c6-a8e4-768a3f98cef4",            "attr": {                "ipam_subnets": [                    {                        "subnet": {                            "ip_prefix": "10.167.0.0",                            "ip_prefix_len": 16                        },                        "dns_server_address": "10.167.255.253",                        "enable_dhcp": true,                        "created": null,                        "default_gateway": "10.167.255.254",                        "dns_nameservers": [],                        "dhcp_option_list": null,                        "subnet_uuid": "10a8de65-9de8-419b-b14c-180bf2ab3dc9",                        "alloc_unit": 1,                        "last_modified": null,                        "host_routes": null,                        "addr_from_start": null,                        "subnet_name": null,                        "allocation_pools": []                    }                ],                "host_routes": null            },            "uuid": "526f554a-0bf4-47c6-a8e4-768a3f98cef4"        },        {            "to": [                "default-domain",                "kubernetes",                "pod-ipam"            ],            "href": "http://127.0.0.1:8082/network-ipam/c9641741-c785-456e-845b-a14a253c3572",            "attr": {                "ipam_subnets": [                    {                        "subnet": null,                        "dns_server_address": null,                        "enable_dhcp": true,                        "created": null,                        "default_gateway": null,                        "dns_nameservers": [],                        "dhcp_option_list": null,                        "subnet_uuid": "d2b090ce-cbcc-4b00-b50a-cc1ed5468b00",                        "alloc_unit": 1,                        "last_modified": null,                        "host_routes": null,                        "addr_from_start": null,                        "subnet_name": null,                        "allocation_pools": []                    }                ],                "host_routes": null            },            "uuid": "c9641741-c785-456e-845b-a14a253c3572"        }    ],    "pbb_etree_enable": false,    "virtual_network_network_id": 5}

<cluster_project>:dev-vn

{    "virtual_network_properties": {        "forwarding_mode": "l3",        "allow_transit": null,        "network_id": null,        "mirror_destination": false,        "vxlan_network_identifier": null,        "rpf": null    },    "fq_name": [        "default-domain",        "kubernetes",        "dev-vn"    ],    "uuid": "ce01826b-e3e6-407f-8798-80612018e89c",    "address_allocation_mode": "flat-subnet-only",    "mac_aging_time": 300,    "parent_type": "project",    "perms2": {        "owner": "None",        "owner_access": 7,        "global_access": 0,        "share": []    },    "display_name": "dev-vn",    "pbb_evpn_enable": false,    "mac_learning_enabled": false,    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-09T11:40:06.196335",        "uuid": {            "uuid_mslong": 14844289246686494847,            "uuid_lslong": 9770700546218977436        },        "user_visible": true,        "last_modified": "2018-01-09T12:18:55.796399",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "flood_unknown_unicast": false,    "layer2_control_word": false,    "port_security_enabled": true,    "network_ipam_refs": [        {            "to": [                "default-domain",                "kubernetes",                "pod-ipam"            ],            "href": "http://127.0.0.1:8082/network-ipam/c9641741-c785-456e-845b-a14a253c3572",            "attr": {                "ipam_subnets": [                    {                        "subnet": null,                        "dns_server_address": null,                        "enable_dhcp": true,                        "created": null,                        "default_gateway": null,                        "dns_nameservers": [],                        "dhcp_option_list": null,                        "subnet_uuid": "48ed8235-efcd-44a1-998c-659e4f5840f4",                        "alloc_unit": 1,                        "last_modified": null,                        "host_routes": null,                        "addr_from_start": null,                        "subnet_name": null,                        "allocation_pools": []                    }                ],                "host_routes": null            },            "uuid": "c9641741-c785-456e-845b-a14a253c3572"        }    ],    "annotations": {        "key_value_pair": [            {                "key": "cluster",                "value": "k8s-default"            },            {                "key": "kind",                "value": "Namespace"            },            {                "key": "namespace",                "value": "dev"            },            {                "key": "isolated",                "value": "True"            },            {                "key": "project",                "value": "kubernetes"            },            {                "key": "name",                "value": "dev"            },            {                "key": "owner",                "value": "k8s"            }        ]    },    "pbb_etree_enable": false,    "virtual_network_network_id": 11}

A.4虛擬機接口和實例IP

非隔離端口

{    "fq_name": [        "default-domain",        "kubernetes",        "dev-web-k528t__5a1fc03e-f7ab-11e7-8f66-52540065dced"    ],    "virtual_machine_interface_mac_addresses": {        "mac_address": [            "02:5a:1f:c0:3e:f7"        ]    },    "display_name": "dev-share__dev-web-k528t",    "security_group_refs": [        {            "to": [                "default-domain",                "kubernetes",                "k8s-default-dev-share-default"            ],            "href": "http://127.0.0.1:8082/security-group/ad29de07-5ef6-4f55-86bb-52c44827c09d",            "attr": null,            "uuid": "ad29de07-5ef6-4f55-86bb-52c44827c09d"        },        {            "to": [                "default-domain",                "kubernetes",                "k8s-default-dev-share-sg"            ],            "href": "http://127.0.0.1:8082/security-group/791f1c7e-a66e-4c47-ba05-409f00ee2c8e",            "attr": null,            "uuid": "791f1c7e-a66e-4c47-ba05-409f00ee2c8e"        }    ],    "routing_instance_refs": [        {            "to": [                "default-domain",                "kubernetes",                "cluster-network",                "cluster-network"            ],            "href": "http://127.0.0.1:8082/routing-instance/5ed7608a-28bb-4735-a8d8-2e9132b03d62",            "attr": {                "direction": "both",                "protocol": null,                "ipv6_service_chain_address": null,                "dst_mac": null,                "mpls_label": null,                "vlan_tag": null,                "src_mac": null,                "service_chain_address": null            },            "uuid": "5ed7608a-28bb-4735-a8d8-2e9132b03d62"        }    ],    "virtual_machine_interface_disable_policy": false,    "parent_type": "project",    "perms2": {        "owner": "None",        "owner_access": 7,        "global_access": 0,        "share": []    },    "virtual_network_refs": [        {            "to": [                "default-domain",                "kubernetes",                "cluster-network"            ],            "href": "http://127.0.0.1:8082/virtual-network/1b9f7f74-17f0-493a-9108-729f91b43598",            "attr": null,            "uuid": "1b9f7f74-17f0-493a-9108-729f91b43598"        }    ],    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-12T15:14:58.189964",        "uuid": {            "uuid_mslong": 6494120564367233511,            "uuid_lslong": 10333036915785587949        },        "user_visible": true,        "last_modified": "2018-01-12T15:14:58.253769",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "virtual_machine_refs": [        {            "to": [                "dev-web-k528t__708154c6-f7ab-11e7-a9df-98f2b3a36be0"            ],            "href": "http://127.0.0.1:8082/virtual-machine/708154c6-f7ab-11e7-a9df-98f2b3a36be0",            "attr": null,            "uuid": "708154c6-f7ab-11e7-a9df-98f2b3a36be0"        }    ],    "vlan_tag_based_bridge_domain": false,    "port_security_enabled": true,    "annotations": {        "key_value_pair": [            {                "key": "cluster",                "value": "k8s-default"            },            {                "key": "kind",                "value": "Pod"            },            {                "key": "namespace",                "value": "dev-share"            },            {                "key": "project",                "value": "kubernetes"            },            {                "key": "name",                "value": "dev-web-k528t"            },            {                "key": "owner",                "value": "k8s"            }        ]    },    "uuid": "5a1fc03e-f7ab-11e7-8f66-52540065dced"}

IP實例

{    "fq_name": [        "dev-web-k528t__5a2f9cde-f7ab-11e7-8f66-52540065dced"    ],    "uuid": "5a2f9cde-f7ab-11e7-8f66-52540065dced",    "service_health_check_ip": false,    "instance_ip_address": "10.47.255.251",    "perms2": {        "owner": "cloud-admin",        "owner_access": 7,        "global_access": 0,        "share": []    },    "annotations": {        "key_value_pair": [            {                "key": "cluster",                "value": "k8s-default"            },            {                "key": "kind",                "value": "Pod"            },            {                "key": "namespace",                "value": "dev-share"            },            {                "key": "project",                "value": "kubernetes"            },            {                "key": "name",                "value": "dev-web-k528t"            },            {                "key": "owner",                "value": "k8s"            }        ]    },    "subnet_uuid": "d2b090ce-cbcc-4b00-b50a-cc1ed5468b00",    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-12T15:14:58.323069",        "uuid": {            "uuid_mslong": 6498585268770771431,            "uuid_lslong": 10333036915785587949        },        "user_visible": true,        "last_modified": "2018-01-12T15:14:58.363792",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "virtual_machine_interface_refs": [        {            "to": [                "default-domain",                "kubernetes",                "dev-web-k528t__5a1fc03e-f7ab-11e7-8f66-52540065dced"            ],            "href": "http://127.0.0.1:8082/virtual-machine-interface/5a1fc03e-f7ab-11e7-8f66-52540065dced",            "attr": null,            "uuid": "5a1fc03e-f7ab-11e7-8f66-52540065dced"        }    ],    "service_instance_ip": false,    "instance_ip_local_ip": false,    "virtual_network_refs": [        {            "to": [                "default-domain",                "kubernetes",                "cluster-network"            ],            "href": "http://127.0.0.1:8082/virtual-network/1b9f7f74-17f0-493a-9108-729f91b43598",            "attr": null,            "uuid": "1b9f7f74-17f0-493a-9108-729f91b43598"        }    ],    "instance_ip_secondary": false,    "display_name": "dev-share__dev-web-k528t"}

隔離端口

{    "fq_name": [        "default-domain",        "kubernetes",        "dev-client__c64b3b12-f7b5-11e7-8f66-52540065dced"    ],    "virtual_machine_interface_mac_addresses": {        "mac_address": [            "02:c6:4b:3b:12:f7"        ]    },    "display_name": "dev__dev-client",    "security_group_refs": [        {            "to": [                "default-domain",                "kubernetes",                "k8s-default-dev-sg"            ],            "href": "http://127.0.0.1:8082/security-group/579019d5-038e-4901-b6ab-ed146022dd70",            "attr": null,            "uuid": "579019d5-038e-4901-b6ab-ed146022dd70"        },        {            "to": [                "default-domain",                "kubernetes",                "k8s-default-dev-default"            ],            "href": "http://127.0.0.1:8082/security-group/e43caf6e-6b35-40c3-b336-83c155078efe",            "attr": null,            "uuid": "e43caf6e-6b35-40c3-b336-83c155078efe"        }    ],    "routing_instance_refs": [        {            "to": [                "default-domain",                "kubernetes",                "dev-vn",                "dev-vn"            ],            "href": "http://127.0.0.1:8082/routing-instance/45173786-a1b4-4c75-8ef0-590de67d2d05",            "attr": {                "direction": "both",                "protocol": null,                "ipv6_service_chain_address": null,                "dst_mac": null,                "mpls_label": null,                "vlan_tag": null,                "src_mac": null,                "service_chain_address": null            },            "uuid": "45173786-a1b4-4c75-8ef0-590de67d2d05"        }    ],    "virtual_machine_interface_disable_policy": false,    "parent_type": "project",    "perms2": {        "owner": "None",        "owner_access": 7,        "global_access": 0,        "share": []    },    "virtual_network_refs": [        {            "to": [                "default-domain",                "kubernetes",                "dev-vn"            ],            "href": "http://127.0.0.1:8082/virtual-network/ce01826b-e3e6-407f-8798-80612018e89c",            "attr": null,            "uuid": "ce01826b-e3e6-407f-8798-80612018e89c"        }    ],    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-12T16:29:34.640295",        "uuid": {            "uuid_mslong": 14288579195414319591,            "uuid_lslong": 10333036915785587949        },        "user_visible": true,        "last_modified": "2018-01-12T16:29:34.708511",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "virtual_machine_refs": [        {            "to": [                "dev-client__c64878a1-f7b5-11e7-9dbb-98f2b3a33b90"            ],            "href": "http://127.0.0.1:8082/virtual-machine/c64878a1-f7b5-11e7-9dbb-98f2b3a33b90",            "attr": null,            "uuid": "c64878a1-f7b5-11e7-9dbb-98f2b3a33b90"        }    ],    "vlan_tag_based_bridge_domain": false,    "port_security_enabled": true,    "annotations": {        "key_value_pair": [            {                "key": "cluster",                "value": "k8s-default"            },            {                "key": "kind",                "value": "Pod"            },            {                "key": "namespace",                "value": "dev"            },            {                "key": "project",                "value": "kubernetes"            },            {                "key": "name",                "value": "dev-client"            },            {                "key": "owner",                "value": "k8s"            }        ]    },    "uuid": "c64b3b12-f7b5-11e7-8f66-52540065dced"}{    "fq_name": [        "dev-client__c65c2a12-f7b5-11e7-8f66-52540065dced"    ],    "uuid": "c65c2a12-f7b5-11e7-8f66-52540065dced",    "service_health_check_ip": false,    "instance_ip_address": "10.47.255.250",    "perms2": {        "owner": "cloud-admin",        "owner_access": 7,        "global_access": 0,        "share": []    },    "annotations": {        "key_value_pair": [            {                "key": "cluster",                "value": "k8s-default"            },            {                "key": "kind",                "value": "Pod"            },            {                "key": "namespace",                "value": "dev"            },            {                "key": "project",                "value": "kubernetes"            },            {                "key": "name",                "value": "dev-client"            },            {                "key": "owner",                "value": "k8s"            }        ]    },    "subnet_uuid": "4b421367-165a-4555-80ab-2cff90cb9401",    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-12T16:29:34.763793",        "uuid": {            "uuid_mslong": 14293345578320728551,            "uuid_lslong": 10333036915785587949        },        "user_visible": true,        "last_modified": "2018-01-12T16:29:34.810063",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "virtual_machine_interface_refs": [        {            "to": [                "default-domain",                "kubernetes",                "dev-client__c64b3b12-f7b5-11e7-8f66-52540065dced"            ],            "href": "http://127.0.0.1:8082/virtual-machine-interface/c64b3b12-f7b5-11e7-8f66-52540065dced",            "attr": null,            "uuid": "c64b3b12-f7b5-11e7-8f66-52540065dced"        }    ],    "service_instance_ip": false,    "instance_ip_local_ip": false,    "virtual_network_refs": [        {            "to": [                "default-domain",                "kubernetes",                "dev-vn"            ],            "href": "http://127.0.0.1:8082/virtual-network/ce01826b-e3e6-407f-8798-80612018e89c",            "attr": null,            "uuid": "ce01826b-e3e6-407f-8798-80612018e89c"        }    ],    "instance_ip_secondary": false,    "display_name": "dev__dev-client"}

附錄B Service

B.1 LB VMI

{    "fq_name": [        "default-domain",        "kubernetes",        "svc-dev-web__20c27603-2d0f-45f5-9647-defe4adaba9a"    ],    "virtual_machine_interface_mac_addresses": {        "mac_address": [            "02:20:c2:76:03:2d"        ]    },    "display_name": "dev-share__svc-dev-web",    "security_group_refs": [        {            "to": [                "default-domain",                "kubernetes",                "k8s-default-dev-share-sg"            ],            "href": "http://127.0.0.1:8082/security-group/791f1c7e-a66e-4c47-ba05-409f00ee2c8e",            "attr": null,            "uuid": "791f1c7e-a66e-4c47-ba05-409f00ee2c8e"        },        {            "to": [                "default-domain",                "kubernetes",                "k8s-default-dev-share-default"            ],            "href": "http://127.0.0.1:8082/security-group/ad29de07-5ef6-4f55-86bb-52c44827c09d",            "attr": null,            "uuid": "ad29de07-5ef6-4f55-86bb-52c44827c09d"        }    ],    "routing_instance_refs": [        {            "to": [                "default-domain",                "kubernetes",                "cluster-network",                "cluster-network"            ],            "href": "http://127.0.0.1:8082/routing-instance/5ed7608a-28bb-4735-a8d8-2e9132b03d62",            "attr": {                "direction": "both",                "protocol": null,                "ipv6_service_chain_address": null,                "dst_mac": null,                "mpls_label": null,                "vlan_tag": null,                "src_mac": null,                "service_chain_address": null            },            "uuid": "5ed7608a-28bb-4735-a8d8-2e9132b03d62"        }    ],    "virtual_machine_interface_disable_policy": false,    "parent_type": "project",    "perms2": {        "owner": "None",        "owner_access": 7,        "global_access": 0,        "share": []    },    "virtual_network_refs": [        {            "to": [                "default-domain",                "kubernetes",                "cluster-network"            ],            "href": "http://127.0.0.1:8082/virtual-network/1b9f7f74-17f0-493a-9108-729f91b43598",            "attr": null,            "uuid": "1b9f7f74-17f0-493a-9108-729f91b43598"        }    ],    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-12T15:21:05.324801",        "uuid": {            "uuid_mslong": 2360578910708516341,            "uuid_lslong": 10828869012794555034        },        "user_visible": true,        "last_modified": "2018-01-12T15:21:05.365345",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "vlan_tag_based_bridge_domain": false,    "virtual_machine_interface_device_owner": "K8S:LOADBALANCER",    "port_security_enabled": true,    "uuid": "20c27603-2d0f-45f5-9647-defe4adaba9a"}

B.2 LB IP實例和浮動IP

IP實例

{    "fq_name": [        "svc-dev-web__ff9782ea-f79d-423e-af9e-cde45ef847f2"    ],    "uuid": "ff9782ea-f79d-423e-af9e-cde45ef847f2",    "service_health_check_ip": false,    "instance_ip_address": "10.167.87.84",    "perms2": {        "owner": "cloud-admin",        "owner_access": 7,        "global_access": 0,        "share": []    },    "virtual_network_refs": [        {            "to": [                "default-domain",                "kubernetes",                "cluster-network"            ],            "href": "http://127.0.0.1:8082/virtual-network/1b9f7f74-17f0-493a-9108-729f91b43598",            "attr": null,            "uuid": "1b9f7f74-17f0-493a-9108-729f91b43598"        }    ],    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-12T15:21:05.433006",        "uuid": {            "uuid_mslong": 18417333146843169342,            "uuid_lslong": 12654778383687239666        },        "user_visible": true,        "last_modified": "2018-01-12T15:21:05.433006",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "virtual_machine_interface_refs": [        {            "to": [                "default-domain",                "kubernetes",                "svc-dev-web__20c27603-2d0f-45f5-9647-defe4adaba9a"            ],            "href": "http://127.0.0.1:8082/virtual-machine-interface/20c27603-2d0f-45f5-9647-defe4adaba9a",            "attr": null,            "uuid": "20c27603-2d0f-45f5-9647-defe4adaba9a"        }    ],    "service_instance_ip": false,    "instance_ip_local_ip": false,    "instance_ip_secondary": false,    "display_name": "svc-dev-web"}

Floating IP

{    "project_refs": [        {            "to": [                "default-domain",                "kubernetes"            ],            "href": "http://127.0.0.1:8082/project/46c31b9b-d21c-4c27-9445-6c94db948b6d",            "attr": null,            "uuid": "46c31b9b-d21c-4c27-9445-6c94db948b6d"        }    ],    "fq_name": [        "svc-dev-web__ff9782ea-f79d-423e-af9e-cde45ef847f2",        "dee62bd0-ed5a-4ac5-b7d7-dc6f329cdba7"    ],    "uuid": "dee62bd0-ed5a-4ac5-b7d7-dc6f329cdba7",    "floating_ip_port_mappings": {        "port_mappings": [            {                "protocol": "TCP",                "src_port": 80,                "dst_port": 80            }        ]    },    "parent_type": "instance-ip",    "perms2": {        "owner": "cloud-admin",        "owner_access": 7,        "global_access": 0,        "share": []    },    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-12T15:21:05.562790",        "uuid": {            "uuid_mslong": 16061573297398762181,            "uuid_lslong": 13247299199082224551        },        "user_visible": true,        "last_modified": "2018-01-12T15:21:06.073466",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "floating_ip_address": "10.167.87.84",    "virtual_machine_interface_refs": [        {            "to": [                "default-domain",                "kubernetes",                "dev-web-669n0__59f3d2a8-f7ab-11e7-8f66-52540065dced"            ],            "href": "http://127.0.0.1:8082/virtual-machine-interface/59f3d2a8-f7ab-11e7-8f66-52540065dced",            "attr": null,            "uuid": "59f3d2a8-f7ab-11e7-8f66-52540065dced"        },        {            "to": [                "default-domain",                "kubernetes",                "dev-web-k528t__5a1fc03e-f7ab-11e7-8f66-52540065dced"            ],            "href": "http://127.0.0.1:8082/virtual-machine-interface/5a1fc03e-f7ab-11e7-8f66-52540065dced",            "attr": null,            "uuid": "5a1fc03e-f7ab-11e7-8f66-52540065dced"        }    ],    "floating_ip_port_mappings_enable": true,    "display_name": "dee62bd0-ed5a-4ac5-b7d7-dc6f329cdba7",    "floating_ip_traffic_direction": "ingress"}

B.3 LB

Loadbalancer

{    "fq_name": [        "default-domain",        "kubernetes",        "svc-dev-web__34f826d8-f7ac-11e7-9dbb-98f2b3a33b90"    ],    "uuid": "34f826d8-f7ac-11e7-9dbb-98f2b3a33b90",    "service_appliance_set_refs": [        {            "to": [                "default-global-system-config",                "native"            ],            "href": "http://127.0.0.1:8082/service-appliance-set/d5cf94dd-6556-40fc-b3dd-0020dacf7cfc",            "attr": null,            "uuid": "d5cf94dd-6556-40fc-b3dd-0020dacf7cfc"        }    ],    "parent_type": "project",    "perms2": {        "owner": "None",        "owner_access": 7,        "global_access": 0,        "share": []    },    "loadbalancer_properties": {        "status": null,        "provisioning_status": "ACTIVE",        "admin_state": true,        "vip_address": "10.167.87.84",        "vip_subnet_id": null,        "operating_status": "ONLINE"    },    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-12T15:21:05.486093",        "uuid": {            "uuid_mslong": 3816843397506535911,            "uuid_lslong": 11365846252762905488        },        "user_visible": true,        "last_modified": "2018-01-12T15:21:05.514920",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "virtual_machine_interface_refs": [        {            "to": [                "default-domain",                "kubernetes",                "svc-dev-web__20c27603-2d0f-45f5-9647-defe4adaba9a"            ],            "href": "http://127.0.0.1:8082/virtual-machine-interface/20c27603-2d0f-45f5-9647-defe4adaba9a",            "attr": null,            "uuid": "20c27603-2d0f-45f5-9647-defe4adaba9a"        }    ],    "display_name": "dev-share__svc-dev-web",    "loadbalancer_provider": "native",    "annotations": {        "key_value_pair": [            {                "key": "cluster",                "value": "k8s-default"            },            {                "key": "kind",                "value": "Service"            },            {                "key": "namespace",                "value": "dev-share"            },            {                "key": "project",                "value": "kubernetes"            },            {                "key": "name",                "value": "svc-dev-web"            },            {                "key": "owner",                "value": "k8s"            }        ]    }}

LB Listener

{    "loadbalancer_listener_properties": {        "default_tls_container": null,        "protocol": "TCP",        "connection_limit": null,        "admin_state": true,        "sni_containers": [],        "protocol_port": 80    },    "fq_name": [        "default-domain",        "kubernetes",        "svc-dev-web__34f826d8-f7ac-11e7-9dbb-98f2b3a33b90-TCP-80-331d4fc1-7e80-47a7-a6a0-6cef54c37b6c"    ],    "uuid": "331d4fc1-7e80-47a7-a6a0-6cef54c37b6c",    "parent_type": "project",    "perms2": {        "owner": "None",        "owner_access": 7,        "global_access": 0,        "share": []    },    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-12T15:21:05.564006",        "uuid": {            "uuid_mslong": 3683187762728552359,            "uuid_lslong": 12006716381744823148        },        "user_visible": true,        "last_modified": "2018-01-12T15:21:05.564006",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "loadbalancer_refs": [        {            "to": [                "default-domain",                "kubernetes",                "svc-dev-web__34f826d8-f7ac-11e7-9dbb-98f2b3a33b90"            ],            "href": "http://127.0.0.1:8082/loadbalancer/34f826d8-f7ac-11e7-9dbb-98f2b3a33b90",            "attr": null,            "uuid": "34f826d8-f7ac-11e7-9dbb-98f2b3a33b90"        }    ],    "display_name": "svc-dev-web__34f826d8-f7ac-11e7-9dbb-98f2b3a33b90-TCP-80-331d4fc1-7e80-47a7-a6a0-6cef54c37b6c"}

LB Pool

{    "fq_name": [        "default-domain",        "kubernetes",        "svc-dev-web__34f826d8-f7ac-11e7-9dbb-98f2b3a33b90-TCP-80-331d4fc1-7e80-47a7-a6a0-6cef54c37b6c"    ],    "uuid": "3ed542dc-cbc5-4b47-aeb7-c35f8443a672",    "parent_type": "project",    "perms2": {        "owner": "None",        "owner_access": 7,        "global_access": 0,        "share": []    },    "loadbalancer_listener_refs": [        {            "to": [                "default-domain",                "kubernetes",                "svc-dev-web__34f826d8-f7ac-11e7-9dbb-98f2b3a33b90-TCP-80-331d4fc1-7e80-47a7-a6a0-6cef54c37b6c"            ],            "href": "http://127.0.0.1:8082/loadbalancer-listener/331d4fc1-7e80-47a7-a6a0-6cef54c37b6c",            "attr": null,            "uuid": "331d4fc1-7e80-47a7-a6a0-6cef54c37b6c"        }    ],    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-12T15:21:05.646375",        "uuid": {            "uuid_mslong": 4527598516469844807,            "uuid_lslong": 12589746098345846386        },        "user_visible": true,        "last_modified": "2018-01-12T15:21:05.646375",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "loadbalancer_pool_properties": {        "status": null,        "protocol": "TCP",        "subnet_id": null,        "session_persistence": null,        "admin_state": true,        "persistence_cookie_name": null,        "status_description": null,        "loadbalancer_method": null    },    "display_name": "svc-dev-web__34f826d8-f7ac-11e7-9dbb-98f2b3a33b90-TCP-80-331d4fc1-7e80-47a7-a6a0-6cef54c37b6c"}

LB Member

{    "fq_name": [        "default-domain",        "kubernetes",        "svc-dev-web__34f826d8-f7ac-11e7-9dbb-98f2b3a33b90-TCP-80-331d4fc1-7e80-47a7-a6a0-6cef54c37b6c",        "53d85c7f-6b13-482e-8706-92142bfa2543"    ],    "uuid": "53d85c7f-6b13-482e-8706-92142bfa2543",    "parent_type": "loadbalancer-pool",    "perms2": {        "owner": "None",        "owner_access": 7,        "global_access": 0,        "share": []    },    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-12T15:21:05.811773",        "uuid": {            "uuid_mslong": 6041680602444548142,            "uuid_lslong": 9729624660315350339        },        "user_visible": true,        "last_modified": "2018-01-12T15:21:05.830431",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "display_name": "53d85c7f-6b13-482e-8706-92142bfa2543",    "loadbalancer_member_properties": {        "status": null,        "status_description": null,        "weight": 1,        "admin_state": true,        "address": null,        "protocol_port": 80    },    "annotations": {        "key_value_pair": [            {                "key": "vm",                "value": "708154c6-f7ab-11e7-a9df-98f2b3a36be0"            },            {                "key": "vmi",                "value": "5a1fc03e-f7ab-11e7-8f66-52540065dced"            }        ]    }}

B.4 外部FIP

{    "project_refs": [        {            "to": [                "default-domain",                "kubernetes"            ],            "href": "http://127.0.0.1:8082/project/46c31b9b-d21c-4c27-9445-6c94db948b6d",            "attr": null,            "uuid": "46c31b9b-d21c-4c27-9445-6c94db948b6d"        }    ],    "fq_name": [        "default-domain",        "kubernetes",        "BGP",        "BGP",        "svc-dev-web__1526aa69-f7bf-11e7-9dbb-98f2b3a33b90120.136.134.67-externalIP"    ],    "uuid": "ac091da2-28d7-467f-bd49-10edb2885219",    "floating_ip_port_mappings": {        "port_mappings": [            {                "protocol": "TCP",                "src_port": 80,                "dst_port": 80            }        ]    },    "parent_type": "floating-ip-pool",    "perms2": {        "owner": "None",        "owner_access": 7,        "global_access": 0,        "share": []    },    "id_perms": {        "enable": true,        "description": null,        "creator": null,        "created": "2018-01-12T17:36:13.280888",        "uuid": {            "uuid_mslong": 12396472031621105279,            "uuid_lslong": 13639451559556829721        },        "user_visible": true,        "last_modified": "2018-01-12T17:36:13.424379",        "permissions": {            "owner": "cloud-admin",            "owner_access": 7,            "other_access": 7,            "group": "cloud-admin-group",            "group_access": 7        }    },    "floating_ip_address": "120.136.134.67",    "virtual_machine_interface_refs": [        {            "to": [                "default-domain",                "kubernetes",                "dev-web-669n0__59f3d2a8-f7ab-11e7-8f66-52540065dced"            ],            "href": "http://127.0.0.1:8082/virtual-machine-interface/59f3d2a8-f7ab-11e7-8f66-52540065dced",            "attr": null,            "uuid": "59f3d2a8-f7ab-11e7-8f66-52540065dced"        },        {            "to": [                "default-domain",                "kubernetes",                "svc-dev-web__78f5adca-cbfe-422a-810c-bb3be9c15589"            ],            "href": "http://127.0.0.1:8082/virtual-machine-interface/78f5adca-cbfe-422a-810c-bb3be9c15589",            "attr": null,            "uuid": "78f5adca-cbfe-422a-810c-bb3be9c15589"        },        {            "to": [                "default-domain",                "kubernetes",                "dev-web-k528t__5a1fc03e-f7ab-11e7-8f66-52540065dced"            ],            "href": "http://127.0.0.1:8082/virtual-machine-interface/5a1fc03e-f7ab-11e7-8f66-52540065dced",            "attr": null,            "uuid": "5a1fc03e-f7ab-11e7-8f66-52540065dced"        }    ],    "floating_ip_port_mappings_enable": true,    "display_name": "svc-dev-web__1526aa69-f7bf-11e7-9dbb-98f2b3a33b90120.136.134.67-externalIP",    "floating_ip_traffic_direction": "ingress"}

推薦閱讀

Tungsten Fabric解決方案指南-Gateway MX

“Tungsten Fabric+K8s集成指南”系列文章——
第一篇：部署準備與初始狀態
第二篇：創建虛擬網絡
第三篇：創建安全策略
第四篇：創建隔離命名空間

“Tungsten Fabric+K8s輕鬆上手”系列文章——
第一篇：TF Carbide 評估指南–準備篇
第二篇：通過Kubernetes的服務進行基本應用程序連接
第三篇：通過Kubernetes Ingress進行高級外部應用程序連接
第四篇：通過Kubernetes命名空間實現初步的應用程序隔離
第五篇：通過Kubernetes網絡策略進行應用程序微分段