docker命令之pull

pull

1 usage

NAME
       docker-pull - Pull an image or a repository from the registry
SYNOPSIS
       docker pull NAME[:TAG]
DESCRIPTION
       This  command  pulls down an image or a repository from the registry.  If there is more than one image for a repository
       (e.g.  fedora) then all images for that repository name are pulled down including any tags.

docker client:1.0

docker registry:0.9

2 代碼分析

以docker pull example.com/ns/repo爲例

2.1docker客戶端

docker客戶端執行的操作有兩個動作：pull和login

首先做pull，如果registry返回的狀態碼是401，那麼docker客戶端就要執行login，如果login成功，就可以再次做pull。

func (cli *DockerCli) CmdPull(args ...string) error {
	cmd := cli.Subcmd("pull", "NAME[:TAG]", "Pull an image or a repository from the registry")
	allTags := cmd.Bool([]string{"a", "-all-tags"}, false, "Download all tagged images in the repository")
	if err := cmd.Parse(args); err != nil {
		return nil
	}

	if cmd.NArg() != 1 {
		cmd.Usage()
		return nil
	}
	var (
		v         = url.Values{}
		remote    = cmd.Arg(0)
		newRemote = remote
	)

解析出來的taglessRemote：example.com/ns/repo，tag：空。如何沒有tag，使用默認的latest

	taglessRemote, tag := parsers.ParseRepositoryTag(remote)

<span style="font-family: Arial, Helvetica, sans-serif;">		</span><span style="font-family: Arial, Helvetica, sans-serif;">if tag == "" && !*allTags { </span>

<pre>		newRemote = taglessRemote + ":latest"
	}
	if tag != "" && *allTags {

return fmt.Errorf("tag can't be used with --all-tags/-a")}v.Set("fromImage", newRemote) //fromImage = newRemote:example.com/namespace/repo:latest

獲取HOME下的.dockercfg，並解析之

<span style="white-space:pre">	</span>// Resolve the Repository name from fqn to hostname + name
<span style="white-space:pre">	</span>hostname, _, err := registry.ResolveRepositoryName(taglessRemote)//hostname:example.com
<span style="white-space:pre">	</span>if err != nil {return err}cli.LoadConfigFile()
<span style="white-space:pre">	</span>// Resolve the Auth config relevant for this server
<span style="white-space:pre">	</span>authConfig := cli.configFile.ResolveAuthConfig(hostname)  

pull動作的執行體。主要指發起POST方法，路由爲/images/create的請求

<span style="white-space:pre">
</span>
<span style="white-space:pre">	</span>pull := func(authConfig registry.AuthConfig) error {
<span style="white-space:pre">		</span>buf, err := json.Marshal(authConfig)
<span style="white-space:pre">			</span>if err != nil {
<span style="white-space:pre">				</span>return err
<span style="white-space:pre">			</span>}

		registryAuthHeader := []string{
			base64.URLEncoding.EncodeToString(buf),
		}
		return cli.stream("POST", "/images/create?"+v.Encode(), nil, cli.out, map[string][]string{
			"X-Registry-Auth": registryAuthHeader,
		})
	} 

向docker httpserver發起pull請求，如果httpserver返回狀態碼401，docker客戶端執行login，loging成功，再次發起pull請求

	if err := pull(authConfig); err != nil {

		if strings.Contains(err.Error(), "Status 401") {
			fmt.Fprintln(cli.out, "\nPlease login prior to pull:")

			if err := cli.CmdLogin(hostname); err != nil {
				return err
			}

			authConfig := cli.configFile.ResolveAuthConfig(hostname)
			return pull(authConfig)
		}
		return err
	}

	return nil
}

---------------------------------------------------------------------------------

分析：cli.stream("POST", "/images/create?"+v.Encode(), nil, cli.out, map[string][]string{"X-Registry-Auth": registryAuthHeader})
代碼位置：api/client/utils.go,這個streamHelper主要的功能是構建http包內容，併發起請求，處理請求的響應

func (cli *DockerCli) stream(method, path string, in io.Reader, out io.Writer, headers map[string][]string) error {
	return cli.streamHelper(method, path, true, in, out, nil, headers)
}

在api/server/server.go中createRouter方法中可以找到POST /images/create對應的handler：postImagesCreate
// Creates an image from Pull or from Import
func postImagesCreate(engWriter *engine.Engine, version version.Version, w http.Response, r *http.Request, vars map[string]string) error {

</pre><p><span style="font-family:Arial,Helvetica,sans-serif;">解析http request請求內容</span></p>

<span style="font-family: Arial, Helvetica, sans-serif;">if err := parseForm(r); err != nil {</span>

		return err
	}

	var (
		image = r.Form.Get("fromImage")  //  example.com
		repo  = r.Form.Get("repo")  //  namespace/repo
		tag   = r.Form.Get("tag")   //  latest
		job   *engine.Job
	)

<span style="font-family: Arial, Helvetica, sans-serif;">//認證相關</span>

	authEncoded := r.Header.Get("X-Registry-Auth")
	authConfig := &registry.AuthConfig{}
	if authEncoded != "" {
		authJson := base64.NewDecoder(base64.URLEncoding, strings.NewReader(authEncoded))
		if err := json.NewDecoder(authJson).Decode(authConfig); err != nil {
			// for a pull it is not an error if no auth was given
			// to increase compatibility with the existing api it is defaulting to be empty
			authConfig = &registry.AuthConfig{}
		}
	}

<span style="white-space:pre">	</span>//如果image不爲空，執行pull
	if image != "" { //pull
		if tag == "" {
			image, tag = parsers.ParseRepositoryTag(image)
		}
		metaHeaders := map[string][]string{}
		for k, v := range r.Header {
			if strings.HasPrefix(k, "X-Meta-") {
				metaHeaders[k] = v
			}
		}
		job = eng.Job("pull", image, tag)
		job.SetenvBool("parallel", version.GreaterThan("1.3"))
		job.SetenvJson("metaHeaders", metaHeaders)
		job.SetenvJson("authConfig", authConfig)
	} else { //import
		if tag == "" {
			repo, tag = parsers.ParseRepositoryTag(repo)
		}
		job = eng.Job("import", r.Form.Get("fromSrc"), repo, tag)
		job.Stdin.Add(r.Body)
	}

	if version.GreaterThan("1.0") {
		job.SetenvBool("json", true)
		streamJSON(job, w, true)
	} else {
		job.Stdout.Add(utils.NewWriteFlusher(w))
	}

<span style="white-space:pre">	</span>//執行pull操作
	if err := job.Run(); err != nil {
		if !job.Stdout.Used() {
			return err
		}
		sf := utils.NewStreamFormatter(version.GreaterThan("1.0"))
		w.Write(sf.FormatError(err))
	}

	return nil
}

job具體由registry/session.go中的GetRepositoryData執行

func (r *Session) GetRepositoryData(remote string) (*RepositoryData, error) {
	indexEp := r.indexEndpoint
	repositoryTarget := fmt.Sprintf("%srepositories/%s/images", indexEp, remote)

	log.Debugf("[registry] Calling GET %s", repositoryTarget)

	req, err := r.reqFactory.NewRequest("GET", repositoryTarget, nil)
	if err != nil {
		return nil, err
	}
	if r.authConfig != nil && len(r.authConfig.Username) > 0 {
		req.SetBasicAuth(r.authConfig.Username, r.authConfig.Password)
	}
	req.Header.Set("X-Docker-Token", "true")

	res, _, err := r.doRequest(req)
	if err != nil {
		return nil, err
	}
	defer res.Body.Close()
	if res.StatusCode == 401 {
		return nil, errLoginRequired
	}
	// TODO: Right now we're ignoring checksums in the response body.
	// In the future, we need to use them to check image validity.
	if res.StatusCode != 200 {
		return nil, utils.NewHTTPRequestError(fmt.Sprintf("HTTP code: %d", res.StatusCode), res)
	}

	var tokens []string
	if res.Header.Get("X-Docker-Token") != "" {
		tokens = res.Header["X-Docker-Token"]
	}

	var endpoints []string
	if res.Header.Get("X-Docker-Endpoints") != "" {
		endpoints, err = buildEndpointsList(res.Header["X-Docker-Endpoints"], indexEp)
		if err != nil {
			return nil, err
		}
	} else {
		// Assume the endpoint is on the same host
		u, err := url.Parse(indexEp)
		if err != nil {
			return nil, err
		}
		endpoints = append(endpoints, fmt.Sprintf("%s://%s/v1/", u.Scheme, req.URL.Host))
	}

	checksumsJSON, err := ioutil.ReadAll(res.Body)
	if err != nil {
		return nil, err
	}
	remoteChecksums := []*ImgData{}
	if err := json.Unmarshal(checksumsJSON, &remoteChecksums); err != nil {
		return nil, err
	}

	// Forge a better object from the retrieved data
	imgsData := make(map[string]*ImgData)
	for _, elem := range remoteChecksums {
		imgsData[elem.ID] = elem
	}

	return &RepositoryData{
		ImgList:   imgsData,
		Endpoints: endpoints,
		Tokens:    tokens,
	}, nil
}

2.2 registry端

docker-registry端使用的是flask框架，docker客戶端使用GET方法，路由/v1/repositories/<path:repository>/images向docker registry發起pull請求

處理的handler在docker_registry/index.py下的get_repository_images方法

入口參數是namespace和repository，本例中namespace=ns，repository=repo，返回給docker客戶端的pull請求分三步完成

1、獲取鏡像的存儲路徑

path = store.index_images_path(namespace, repository)

2、在存儲路徑下找到鏡像的真實數據

data = store.get_content(path)

3、返回請求數據

headers = generate_headers(namespace, repository, "read")
return toolket.response(data, 200, headers, True)

3.pull流程圖譜

1.在docker client要pull image之前，也login一樣，也是要做ping操作

2.registry返回給client 200 OK

3.client端攜帶需要pull的鏡像信息和授權信息：basic authorization，向registry請求鏡像

4.registry返回給client請求鏡像的所有分層id

5.client向registry請求此鏡像的tag信息

6.registry返回給client此鏡像的tag信息和此tag對應的lay id

 

7.client獲取祖先lay信息

 

8.registry返回此祖先對應的所有lay信息

如果lay id在本地已經存在的話，client就不會download此lay了，如果不存在，那麼先請求此lay的json信息，然後請求lay tar包