訓練kaldi chian模型,Failed to allocate a memory和Failed to write matrix to stream錯誤解決方案

訓練kaldi chian模型,出現如下錯誤
ERROR (nnet3-chain-train[5.5]:AllocateNewRegion():cu-allocator.cc:498) Failed to allocate a memory region of 5627707392 bytes.  Possibly this is due to sharing the GPU.  Try switching the GPUs to exclusive mode (nvidia-smi -c 3) and using the option --use-gpu=wait to scripts like steps/nnet3/chain/train.py.  Memory info: free:10732M, used:440M, total:11172M, free/total:0.960617

同時伴隨提示“ERROR (nnet3-am-copy[5.5]:Write():kaldi-matrix.cc:1403) Failed to write matrix to stream”

按照提示信息,需要設置GPU爲exclusive mode,使用命令

sudo nvidia-smi -c 3

重新運行腳本,即可正常訓練

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章