Windows 10 安裝 NVIDIA cuDNN v7.6.0 for CUDA 10.1

Windows 10 安裝 NVIDIA cuDNN v7.6.0 for CUDA 10.1

NVIDIA 引領人工智能計算 - NVIDIA
https://www.nvidia.cn/

NVIDIA 開發者 - NVIDIA DEVELOPER
https://developer.nvidia.com/

NVIDIA cuDNN
https://developer.nvidia.com/cudnn

cuDNN Archive
https://developer.nvidia.com/rdp/cudnn-archive

DEEP LEARNING SDK DOCUMENTATION - cuDNN Archives
https://docs.nvidia.com/deeplearning/sdk/cudnn-archived/index.html

NVIDIA Deep Learning SDK
https://docs.nvidia.com/deeplearning/sdk/index.html

cuDNN Release 7.6.0 Documentation - cuDNN Installation Guide
https://docs.nvidia.com/deeplearning/sdk/cudnn-archived/cudnn_760/cudnn-install/index.html

1. Download cuDNN v7.6.0 (May 20, 2019), for CUDA 10.1

The NVIDIA CUDA® Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks.

Library for Windows, Mac, Linux, Ubuntu and RedHat/Centos (x86_64 architecture) cuDNN Library for Windows 10

primitive ['prɪmɪtɪv]：adj. 原始的，遠古的，簡單的，粗糙的 n. 原始人

使用 cudnn-10.1-windows10-x64-v7.6.0.64.zip 安裝。

2. cuDNN Installation Guide - Overview

The NVIDIA CUDA Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, pooling, normalization, and activation layers. cuDNN is part of the NVIDIA Deep Learning SDK.
NVIDIA CUDA Deep Neural Network library (cuDNN) 是 GPU 加速的用於深度神經網絡的原語庫。cuDNN 爲標準例程提供了高度優化的實現，such as forward and backward convolution, pooling, normalization, and activation layers。cuDNN 是 NVIDIA Deep Learning SDK 的一部分。

Deep learning researchers and framework developers worldwide rely on cuDNN for high-performance GPU acceleration. It allows them to focus on training neural networks and developing software applications rather than spending time on low-level GPU performance tuning. cuDNN accelerates widely used deep learning frameworks, including Caffe, Caffe2, TensorFlow, Theano, Torch, PyTorch, MXNet, and Microsoft Cognitive Toolkit. cuDNN is freely available to members of the NVIDIA Developer Program.
全球的深度學習研究人員和框架開發人員都依賴 cuDNN 來實現高性能 GPU 加速。它使他們可以專注於訓練神經網絡和開發軟件應用程序，而不必花時間在底層 GPU 性能調整上。cuDNN 加速了廣泛使用的深度學習框架，including Caffe, Caffe2, TensorFlow, Theano, Torch, PyTorch, MXNet, and Microsoft Cognitive Toolkit。cuDNN is freely available to members of the NVIDIA Developer Program.

3. Downloading cuDNN

4. Installing cuDNN on Windows

4.1. Prerequisites

• NVIDIA GPU
  https://developer.nvidia.com/cuda-gpus
  

• Supported NVIDIA hardware, CUDA, and CUDA driver versions for cuDNN v7.5.1 - v7.6.2
  https://docs.nvidia.com/deeplearning/sdk/cudnn-support-matrix/index.html
  

• One of the following supported CUDA versions and NVIDIA graphics driver:
  NVIDIA graphics driver R418 or newer for CUDA 10.1
  NVIDIA graphics driver R410 or newer for CUDA 10.0
  NVIDIA graphics driver R396 or newer for CUDA 9.2
  NVIDIA graphics driver R384 or newer for CUDA 9
  NVIDIA graphics driver R377 or newer for CUDA 8

4.1.1. Installing NVIDIA Graphics Drivers

4.1.2. Installing CUDA

4.2. Installing cuDNN on Windows

0. cudnn-10.1-windows10-x64-v7.6.0.64.zip -> 解壓到當前文件夾

• Your CUDA directory path is referred to as C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1

• Your cuDNN directory path is referred to as <installpath> = E:\software

• E:\software\cuda
  

1. Navigate to your <installpath> directory containing cuDNN.

2. Unzip the cuDNN package.
   cudnn-10.1-windows10-x64-v7.6.0.64.zip

3. Copy the following files into the CUDA Toolkit directory.
   a. Copy <installpath>\cuda\bin\cudnn64_7.dll to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin.
   E:\software\cuda\bin\cudnn64_7.dll -> C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin
   b. Copy <installpath>\cuda\include\cudnn.h to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include.
   E:\software\cuda\include\cudnn.h -> C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include
   c. Copy <installpath>\cuda\lib\x64\cudnn.lib to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\lib\x64.
   E:\software\cuda\lib\x64\cudnn.lib -> C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\lib\x64

4. Set the following environment variables to point to where cuDNN is located. To access the value of the $(CUDA_PATH) environment variable, perform the following steps:
   a. Open a command prompt from the Start menu.
   b. Type Run and hit Enter.
   c. Issue the control sysdm.cpl command.
   d. Select the Advanced tab at the top of the window.
   e. Click Environment Variables at the bottom of the window.
   f. Ensure the following values are set:

Variable Name: CUDA_PATH 
Variable Value: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1

5. Include cudnn.lib in your Visual Studio project.
   a. Open the Visual Studio project and right-click on the project name.
   b. Click Linker > Input > Additional Dependencies.
   c. Add cudnn.lib and click OK.

cuDNN v7 can coexist with previous versions of cuDNN, such as v5 or v6.

References

cuDNN Support Matrix - CUDA And NVIDIA Hardware For cuDNN
https://docs.nvidia.com/deeplearning/sdk/cudnn-support-matrix/index.html

CUDA Installation Guide for Microsoft Windows
https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html