Gitlab从源码安装版本迁移到docker版本

Gitlab源码版本介绍

1. 版本：gitlab-ce:11.11.3-ce.0
2. 有LDAP登录功能
3. 创建的有runner

迁移前准备工作

1. 备份旧gitlab数据
2. 备份configure文件

源码安装的gitlab配置文件	docker容器的gitlab配置文件	说明
gitlab.yml	gitlab.rb	gitlab基本配置包括LDAP等
secrets.yml	gitlab-secrets.json	包含秘钥，防止登录runner报500错误

3. 获取和源码相同版本的docker镜像

   sudo docker pull gitlab/gitlab-ce:11.11.3-ce.0
   

使用Docker安装Gitlab

sudo docker run --detach \
  --hostname 172.20.xxx.xxx \
  --publish 443:443 \
  --publish 8090:80 \
  --publish 2222:22 \
  --name gitlab \
  --restart always \
  --privileged=true \
  --volume /srv/gitlab/config:/etc/gitlab \
  --volume /srv/gitlab/logs:/var/log/gitlab \
  --volume /srv/gitlab/data:/var/opt/gitlab \
  gitlab/gitlab-ce:11.11.3-ce.0

备注：如果出现端口被占用，修改端口，删除重新创建容器

迁移数据

1. 迁移备份文件
   将备份的gitlab数据文件复制到挂载的 /srv/gitlab/data/backups目录中（如：1587580178_2020_04_23_11.11.3_gitlab_backup.tar），最好将文件权限设置为777.

   进入gitlab容器进行数据恢复
   先关闭相关数据服务

   gitlab-ctl stop unicorn
   gitlab-ctl stop sidekiq
   

   进行数据数据恢复

   gitlab-rake gitlab:backup:restore BACKUP=1587580178_2020_04_23_11.11.3 --trace
   

2. 迁移配置文件
   参照历史配置文件修改挂载目录/srv/gitlab/config中的gitlab.rb和gitlab-secrets.json
   重新加载配置文件

   gitlab-ctl reconfigure
   

可能遇到的问题

1. 使用LDAP登录失败，提示id冲突
   原因：备份文件的包含有users数据，恢复时已经插入数据库，但数据库默认自增id还是从1开始，进行LDAP登录时插入数据的id会重复。
   解决方法：将数据库的所有表的自增seq进行更新，更新为当前表总数据条数+1

   ALTER TABLE public.tags OWNER TO gitlab;
   CREATE SEQUENCE IF NOT EXISTS public.tags_id_seq
       START WITH 1
       INCREMENT BY 1
       NO MINVALUE
       NO MAXVALUE
       CACHE 1;
   ALTER TABLE public.tags_id_seq OWNER TO gitlab;
   ALTER SEQUENCE public.tags_id_seq OWNED BY public.tags.id;
   SELECT pg_catalog.setval('public.tags_id_seq', COALESCE((SELECT MAX(id)+1 FROM public.tags), 1), false);
   

   附加自动生成sql语句的Python代码

   tables = ["table1", "table2"]
   sqlfile = open('sqlfile.sql', 'w')
   for item in tables:
       print("ALTER TABLE public."+item+" OWNER TO gitlab;",file=sqlfile)
       print("CREATE SEQUENCE IF NOT EXISTS public."+item+"_id_seq"+"\n"+
        "    START WITH 1"+"\n"+
        "    INCREMENT BY 1"+"\n"+
        "    NO MINVALUE"+"\n"+
        "    NO MAXVALUE"+"\n"+
        "    CACHE 1;",file=sqlfile)
       print("ALTER TABLE public."+item+"_id_seq OWNER TO gitlab;",file=sqlfile)
       print("ALTER SEQUENCE public."+item+"_id_seq OWNED BY public."+item+".id;",file=sqlfile)
       print("SELECT pg_catalog.setval('public."+item+"_id_seq', COALESCE((SELECT MAX(id)+1 FROM public."+item+"), 1), false);"+"\n",file=sqlfile)
       print("ALTER TABLE ONLY public."+item+" ALTER COLUMN id SET DEFAULT nextval('public."+item+"_id_seq'::regclass);",file=sqlfile)
   sqlfile.close()
   print(len(tables))
   

   备注：tables就自己在数据库中查吧。
   docker容器进入postgresql

   gitlab-rails dbconsole
   

2. Gitlab runner界面报500错误
   原因：恢复文件中有runner数据，但是没有恢复gitlab secret配置，导致无法解密runner中的token。
   解决办法：恢复和以前一样的secret，或者重置token。

   -- Clear project tokens
   UPDATE projects SET runners_token = null, runners_token_encrypted = null;
   -- Clear group tokens
   UPDATE namespaces SET runners_token = null, runners_token_encrypted = null;
   -- Clear instance tokens
   UPDATE application_settings SET runners_registration_token_encrypted = null;
   -- Clear runner tokens
   UPDATE ci_runners SET token = null, token_encrypted = null;
   

3. 恢复数据时报数据库表无权限
   源码安装的gitlab备份文件部分：

   ALTER SCHEMA public OWNER TO postgres;
   
   --
   -- Name: SCHEMA public; Type: COMMENT; Schema: -; Owner: postgres
   --	
   
   

   --
   -- Name: abuse_reports; Type: TABLE; Schema: public; Owner: git
   --
   
   CREATE TABLE public.abuse_reports (
       id integer NOT NULL,
       reporter_id integer,
       user_id integer,
       message text,
       created_at timestamp without time zone,
       updated_at timestamp without time zone,
       message_html text,
       cached_markdown_version integer
   );
   
   
   ALTER TABLE public.abuse_reports OWNER TO git;
   
   --
   -- Name: abuse_reports_id_seq; Type: SEQUENCE; Schema: public; Owner: git
   --
   
   CREATE SEQUENCE public.abuse_reports_id_seq
       AS integer
       START WITH 1
       INCREMENT BY 1
       NO MINVALUE
       NO MAXVALUE
       CACHE 1;
   
   
   ALTER TABLE public.abuse_reports_id_seq OWNER TO git;
   
   --
   -- Name: abuse_reports_id_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: git
   --
   
   ALTER SEQUENCE public.abuse_reports_id_seq OWNED BY public.abuse_reports.id;
   
   

   docker中gitlab备份文件部分：

   ALTER SCHEMA public OWNER TO "gitlab-psql";
   
   --
   -- Name: SCHEMA public; Type: COMMENT; Schema: -; Owner: gitlab-psql
   --
   
   

   --
   -- Name: abuse_reports; Type: TABLE; Schema: public; Owner: gitlab
   --
   
   CREATE TABLE public.abuse_reports (
       id integer NOT NULL,
       reporter_id integer,
       user_id integer,
       message text,
       created_at timestamp without time zone,
       updated_at timestamp without time zone,
       message_html text,
       cached_markdown_version integer
   );
   
   
   ALTER TABLE public.abuse_reports OWNER TO gitlab;
   
   --
   -- Name: abuse_reports_id_seq; Type: SEQUENCE; Schema: public; Owner: gitlab
   --
   
   CREATE SEQUENCE public.abuse_reports_id_seq
       START WITH 1
       INCREMENT BY 1
       NO MINVALUE
       NO MAXVALUE
       CACHE 1;
   
   
   ALTER TABLE public.abuse_reports_id_seq OWNER TO gitlab;
   
   --
   -- Name: abuse_reports_id_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: gitlab
   --
   
   ALTER SEQUENCE public.abuse_reports_id_seq OWNED BY public.abuse_reports.id;
   
   

   对比两者的备份文件，发现：

   1. 源码安装的gitlab，public schema的Owner是postgres。docker中gitlab的Owner是gitlab-psql。所以迁移数据后需要执行

      ALTER SCHEMA public OWNER TO "gitlab-psql";
      

   2. 源码安装的gitlab，tables的Owner是git。docker中gitlab的Owner是gitlab。所以迁移数据后需要执行上面问题1提到的LDAP登录问题的解决方法提到的语句。