[Linux系統] 使用WSL2下的Ubuntu20.04部署AI環境

一、安裝WSL2下的Ubuntu20.04

 1）打開windows WSL服務

# 以管理員身份運行 PowerShell
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart

# 使用powershell將wsl2設置爲默認版本
wsl --set-default-version 2

2）安裝linux子系統

# 安裝操作系統 wsl --install -d <可安裝的版本名> 例如，我安裝版本名爲Ubuntu-18.04，命令爲：wsl --install -d Ubuntu-20.04
wsl --install -d Ubuntu-20.04

# 查看可以安裝的linux版本
wsl  -l -o

3）給Ubuntu換源

# 備份源列表
sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak
# 命令行打開sources.list文件
sudo vim /etc/apt/sources.list

deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-updates main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-updates main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-backports main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-backports main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-security main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ focal-security main restricted universe multiverse multiverse

sudo apt-get update

二、遷移子系統到其他磁盤

wsl --shutdown Ubuntu-20.04
wsl --export Ubuntu-20.04 D:\WSL2\WSL2Ubuntu20.04.bak
wsl --unregister Ubuntu-20.04
wsl --import Ubuntu-20.04 D:\WSL2Ubuntu20.04LTS D:\WSL2\WSL2Ubuntu20.04.bak --version 2

三、修改爲靜態IP地址

在 /etc/netplan 下創建 01-network-manager-all.yaml 文件，包含以下內容：

# Let NetworkManager manage all devices on this system
network:
  version: 2
  renderer: NetworkManager
  ethernets:
    eth0:    #網卡名
      dhcp4: no    #關閉ipv4動態分配ip地址
      dhcp6: no    #關閉ipv6動態分配ip地址
      addresses: [172.19.222.100/24]    #前面是ip地址，子網掩碼是255.255.240.0
      gateway4: 172.19.208.1    #網關
      nameservers:
        addresses: [114.114.114.114, 8.8.8.8] #DNS

四、在WSL下啓用 systemctl 命令

參考：https://www.cnblogs.com/leokale-zz/p/17579564.html

五、安裝 openssh 

sudo apt-get install openssh-server -y

修改 /etc/ssh/sshd_config 配置內容

port 22
AllowUsers root leo
PermitRootLogin yes
PasswordAuthentication yes

啓動服務：

sudo service ssh restart
#或
sudo systemctl start ssh
# 開機啓動
sudo systemctl enable ssh

六、安裝anaconda

下載：https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/
版本：Anaconda3-2023.07-1-Linux-x86_64.sh  （Python3.11）

bash Anaconda3-2023.07-1-Linux-x86_64.sh

七、安裝nginx

下載使用：http://tengine.taobao.org/download/tengine-2.3.0.tar.gz

wget http://tengine.taobao.org/download/tengine-2.3.0.tar.gz
tar zxf tengine-2.3.0.tar.gz

編譯安裝

cd tengine-2.3.0
./configure --prefix=/usr/local/nginx
make -j16
sudo make install

# 將nginx用systemctl管理
vi /usr/lib/systemd/system/nginx.service

[Unit]
Description=The nginx HTTP and reverse proxy server
After=syslog.target network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
PIDFile=/opt/nginx/logs/nginx.pid
ExecStartPre=/opt/nginx/sbin/nginx -t
ExecStart=/opt/nginx/sbin/nginx -c /opt/nginx/conf/nginx.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

systemctl enable nginx.service
systemctl start nginx.service

Nginx配置：

參考：https://www.cnblogs.com/leokale-zz/p/11773349.html

八、安裝redis

 下載：http://download.redis.io/releases/redis-5.0.7.tar.gz

# 下載redis
wget http://download.redis.io/releases/redis-5.0.7.tar.gz
# 解壓縮
tar xzf redis-5.0.7.tar.gz
# 移動文件夾到/usr/local中
sudo mv redis-5.0.7 /usr/local/
# 進入redis目錄
cd /usr/local/redis-5.0.7

sudo make all
sudo make install

cd /etc
sudo mkdir redis
cd /usr/local/redis-5.0.7
sudo cp redis.conf /etc/redis/redis.conf

修改redis.conf內容：

cd /etc/redis/
sudo nano redis.conf

bind 172.19.222.100 127.0.0.1 # 服務IP地址
port 6379  # 服務端口
daemonize yes  # 是否以守護進程形式運行
dbfilename dump.rdb  # 持久化數據文件名稱
dir /var/lib/redis  # 持久化數據文件存放位置
logfile /var/log/redis/redis-server.log  # 日誌文件存放位置
database 16  # 一共16個數據庫，編號1~15
slaveof  # 設置主從複製（用於分佈式）

cd /var/lib
sudo mkdir redis
cd /var/log
sudo mkdir redis

設置redis服務：

# 編輯文件
vim /usr/lib/systemd/system/redis.service

# 在redis.service文件中加入以下內容 並保存

##########################

[Unit]
Description=Redis-5.0.7-6379
After=network.target

[Service]
Type=forking
PIDfile=/var/run/redis-6379.pid
ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

###########################

sudo systemctl daemon-reload

sudo systemctl enable redis
sudo systemctl start redis

九、安裝 cuda toolkit

CUDA toolkit版本12.1

通過deb方式安裝，下載鏈接：https://developer.nvidia.com/cuda-12-1-0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local

wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.1.0/local_installers/cuda-repo-wsl-ubuntu-12-1-local_12.1.0-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-12-1-local_12.1.0-1_amd64.deb
sudo cp /var/cuda-repo-wsl-ubuntu-12-1-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda

配置cuda環境變量

export CUDA_HOME=/usr/local/cuda-12.1
export PATH=/usr/local/cuda-12.1/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-12.1/lib64:$LD_LIBRARY_PATH

十、安裝cudnn

cudnn版本8.9.3.28

下載：cudnn-local-repo-ubuntu2004-8.9.3.28_1.0-1_amd64.deb

sudo dpkg -i cudnn-local-repo-ubuntu2004-8.9.3.28_1.0-1_amd64.deb
sudo cp /var/cudnn-local-repo-ubuntu2004-8.9.3.28/cudnn-local-AAD7FE56-keyring.gpg /usr/share/keyrings/
apt-get update
sudo apt-get install libcudnn8=8.9.3.28-1+cuda12.1
sudo apt-get install libcudnn8-dev=8.9.3.28-1+cuda12.1
sudo apt-get install libcudnn8-samples=8.9.3.28-1+cuda12.1

驗證是否成功：

sudo apt-get install libfreeimage3 libfreeimage-dev -y
cp -r /usr/src/cudnn_samples_v8/ $HOME
cd  $HOME/cudnn_samples_v8/mnistCUDNN
make clean && make
./mnistCUDNN
#Test Passed！

十一、編譯安裝ffmpeg

1）安裝pkg-config

sudo apt-get install pkg-config

2）安裝nv-codec-headers

git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git
cd nv-codec-headers 
sudo make install 
# 驗證是否安裝成功
pkg-config --modversion ffnvcodec

3）源碼安裝libx264（可選apt安裝）

git clone https://code.videolan.org/videolan/x264.git
cd x264
./configure --disable-asm --enable-shared --enable-pic
# 編譯 & 安裝
make -j32
sudo make install

4）安裝依賴庫

sudo apt-get install -y aptitude 
sudo aptitude install -y build-essential
sudo aptitude install -y yasm nasm
sudo apt-get install -y libssl-dev librtmp-dev libvorbis-dev
sudo apt-get install -y libgmp3-dev libaom-dev libtheora-dev
sudo apt-get install -y libass-dev libbluray-dev libfdk-aac-dev
sudo apt-get install -y libmp3lame-dev libopencore-amrnb-dev
sudo apt-get install -y libopenmpt-dev libopus-dev libxvidcore-dev liblzma-dev
sudo apt-get install -y libshine-dev libsnappy-dev libsoxr-dev
sudo apt-get install -y libspeex-dev libtheora-dev libtwolame-dev
sudo apt-get install -y libvo-amrwbenc-dev libvpx-dev libwavpack-dev
sudo apt-get install -y libwebp-dev libx264-dev libx265-dev 

5）下載ffmpeg源碼

wget https://ffmpeg.org/releases/ffmpeg-4.4.tar.gz
tar zxvf ffmpeg-4.4.tar.gz

6）編譯安裝

./configure  --prefix="/usr/local/ffmpeg" \
--enable-static --enable-shared \
--enable-version3 --enable-gpl \
--enable-nonfree --enable-small --enable-libmp3lame \
--enable-libx264 --enable-libx265 \
--enable-libvpx --enable-libtheora --enable-libvorbis \
--enable-libopus --enable-libfdk-aac --enable-libass --enable-libwebp \
--enable-librtmp --enable-postproc --enable-libfreetype --enable-pthreads \
--enable-openssl --enable-shared --disable-debug --disable-doc --disable-ffplay \
--enable-cuvid --enable-cuda --enable-cuda-nvcc \
--extra-cflags="-I/usr/local/cuda-12.1/include" --extra-ldflags="-L/usr/local/cuda-12.1/lib64" \
--extra-cflags="-I/usr/local/include" --extra-ldflags="-L/usr/local/lib" --extra-libs="-lpthread -lm"

sudo make -j16
sudo make install

問題：

ERROR: failed checking for nvcc.
#將configure文件中的sm_30爲sm_90（cuda12對應90）

 十二、安裝AI環境

1）安裝python依賴

# requirements.txt文件內容
requests
wget
numpy
prettytable
tqdm
thop
Pillow
seaborn
redis
tensorboard
soundfile
librosa
Polygon3
zhconv
pydub==0.25.1
scikit-learn==1.2.1
typeguard==2.13.3
opencv-python==4.4.0.46
Flask==2.2.3
gpustat==1.0.0
psutil==5.9.4

pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

2）安裝cudatoolkit

cudatoolkit版本11.8

conda install cudatoolkit=11.8 -c conda-forge

3）安裝torch

torch版本2.0.0

下載鏈接：https://download.pytorch.org/whl/cu118/torch-2.0.0%2Bcu118-cp39-cp39-linux_x86_64.whl

pip install torch-2.0.0+cu118-cp39-cp39-linux_x86_64.whl -i https://pypi.tuna.tsinghua.edu.cn/simple

4）安裝paddlepaddle-gpu

python -m pip install paddlepaddle-gpu==2.5.0.post117 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html

 

 

 

 

 

= =