java 離線中文語音文字識別

轉載註明出處：https://www.cnblogs.com/rolayblog/p/15237099.html

項目需要，要實現類似小愛同學的語音控制功能，並且要離線，不能花公司一分錢。第一步就是需要把音頻文字化。經過各種資料蒐集後，選擇了vosk。這是vosk的官方介紹：

Vosk is a speech recognition toolkit. The best things in Vosk are:

Supports 19+ languages and dialects - English, Indian English, German, French, Spanish, Portuguese, Chinese, Russian, Turkish, Vietnamese, Italian, Dutch, Catalan, Arabic, Greek, Farsi, Filipino, Ukrainian, Kazakh. More to come.
Works offline, even on lightweight devices - Raspberry Pi, Android, iOS
Installs with simple pip3 install vosk
Portable per-language models are only 50Mb each, but there are much bigger server models available.
Provides streaming API for the best user experience (unlike popular speech-recognition python packages)
There are bindings for different programming languages, too - java/csharp/javascript etc.
Allows quick reconfiguration of vocabulary for best accuracy.
Supports speaker identification beside simple speech recognition.

選擇它的理由，開源、可離線、可使用第三方的訓練結果，本次使用的官方提供的中文訓練集，如果有需要可自行訓練，不過成本太大。具體見官網：https://alphacephei.com/vosk/，官方demo：https://github.com/alphacep/vosk-api。

本次使用springboot +maven實現，官方demo爲springboot+gradle。

1、pom文件如下：

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.5.4</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.example</groupId>
    <artifactId>voice</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>voice-ai</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <repositories>
        <repository>
            <id>com.alphacephei</id>
            <name>vosk</name>
            <url>https://alphacephei.com/maven/</url>
        </repository>
    </repositories>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>net.java.dev.jna</groupId>
            <artifactId>jna</artifactId>
            <version>5.7.0</version>
        </dependency>
        <dependency>
            <groupId>com.alphacephei</groupId>
            <artifactId>vosk</artifactId>
            <version>0.3.30</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.8</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

pom

特別說明一下，vosk的包在常見的maven倉庫裏面是沒有的，所以需要指定下載地址。

2、工程結構：

3、語音識別工具類

public class VoiceUtil {

    @Value("${leenleda.vosk.model}")
    private String VOSKMODELPATH;

    public String getWord(String filePath) throws IOException, UnsupportedAudioFileException {
        Assert.isTrue(StringUtils.hasLength(VOSKMODELPATH), "無效的VOS模塊！");
        byte[] bytes = Files.readAllBytes(Paths.get(filePath));
        // 轉換爲16KHZ
        reSamplingAndSave(bytes, filePath);
        File f = new File(filePath);
        RandomAccessFile rdf = null;
        rdf = new RandomAccessFile(f, "r");
        log.info("聲音尺寸:{}", toInt(read(rdf, 4, 4)));
        log.info("音頻格式:{}", toShort(read(rdf, 20, 2)));
        short track=toShort(read(rdf, 22, 2));
        log.info("1 單聲道 2 雙聲道: {}", track);
        log.info("採樣率、音頻採樣級別 16000 = 16KHz: {}", toInt(read(rdf, 24, 4)));
        log.info("每秒波形的數據量：{}", toShort(read(rdf, 22, 2)));
        log.info("採樣幀的大小：{}", toShort(read(rdf, 32, 2)));
        log.info("採樣位數：{}", toShort(read(rdf, 34, 2)));
        rdf.close();
        LibVosk.setLogLevel(LogLevel.WARNINGS);

        try (Model model = new Model(VOSKMODELPATH);
             InputStream ais = AudioSystem.getAudioInputStream(new BufferedInputStream(new FileInputStream(filePath)));
             // 採樣率爲音頻採樣率的聲道倍數
             Recognizer recognizer = new Recognizer(model, 16000*track)) {

            int nbytes;
            byte[] b = new byte[4096];
            int i = 0;
            while ((nbytes = ais.read(b)) >= 0) {
                i += 1;
                if (recognizer.acceptWaveForm(b, nbytes)) {
//                    System.out.println(recognizer.getResult());
                } else {
//                    System.out.println(recognizer.getPartialResult());
                }
            }
            String result = recognizer.getFinalResult();
            log.info("識別結果：{}", result);
            if (StringUtils.hasLength(result)) {
                JSONObject jsonObject = JSON.parseObject(result);
                return jsonObject.getString("text").replace(" ", "");
            }
            return "";
        }
    }

    public static int toInt(byte[] b) {
        return (((b[3] & 0xff) << 24) + ((b[2] & 0xff) << 16) + ((b[1] & 0xff) << 8) + ((b[0] & 0xff) << 0));
    }

    public static short toShort(byte[] b) {
        return (short) ((b[1] << 8) + (b[0] << 0));
    }


    public static byte[] read(RandomAccessFile rdf, int pos, int length) throws IOException {
        rdf.seek(pos);
        byte result[] = new byte[length];
        for (int i = 0; i < length; i++) {
            result[i] = rdf.readByte();
        }
        return result;
    }

    public static void reSamplingAndSave(byte[] data, String path) throws IOException, UnsupportedAudioFileException {
        WaveFileReader reader = new WaveFileReader();
        AudioInputStream audioIn = reader.getAudioInputStream(new ByteArrayInputStream(data));

        AudioFormat srcFormat = audioIn.getFormat();
        int targetSampleRate = 16000;

        AudioFormat dstFormat = new AudioFormat(srcFormat.getEncoding(),
                targetSampleRate,
                srcFormat.getSampleSizeInBits(),
                srcFormat.getChannels(),
                srcFormat.getFrameSize(),
                srcFormat.getFrameRate(),
                srcFormat.isBigEndian());

        AudioInputStream convertedIn = AudioSystem.getAudioInputStream(dstFormat, audioIn);
        File file = new File(path);
        WaveFileWriter writer = new WaveFileWriter();
        writer.write(convertedIn, AudioFileFormat.Type.WAVE, file);
    }
}

語音識別工具類

有幾點需要說明一下，官方demo裏面對採集率是寫死了的，爲16000。這是以16KHz來算的，所以我把所有拿到的音頻都轉成了16KHz。還有采集率的設置，需要設置爲聲道數的倍數。

4、前端交互

@RestController
public class VoiceAiController {

    @Autowired
    VoiceUtil voiceUtil;

    @PostMapping("/getWord")
    public String getWord(MultipartFile file) {
        String path = "G:\\leenleda\\application\\voice-ai\\" + new Date().getTime() + ".wav";
        File localFile = new File(path);
        try {
            file.transferTo(localFile); //把上傳的文件保存至本地
            System.out.println(file.getOriginalFilename() + " 上傳成功");
            // 上傳成功，開始解析
            String text = voiceUtil.getWord(path);
            localFile.delete();
            return text;
        } catch (IOException | UnsupportedAudioFileException e) {
            e.printStackTrace();
            localFile.delete();
            return "上傳失敗";
        }
    }
}

VoiceAiController

5、前端頁面

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">

<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    <title>聲音轉換</title>
</head>

<body>
    <div>
        <audio controls autoplay></audio>
        <input id="start" type="button" value="錄音" />
        <input id="stop" type="button" value="停止" />
        <input id="play" type="button" value="播放" />
        <input id="upload" type="button" value="提交" />
        <div id="text">

        </div>
    </div>
    <script src="http://libs.baidu.com/jquery/2.1.4/jquery.min.js"></script>
    <script type="text/javascript" src="HZRecorder.js"></script>
    <script>
        var recorder;
        var audio = document.querySelector('audio');
        $("#start").click(function () {
            HZRecorder.get(function (rec) {
                recorder = rec;
                recorder.start();
            });
        })

        $("#stop").click(function () {
            recorder.stop();
        })

        $("#play").click(function () {
            recorder.play(audio);
        })

        $("#upload").click(function () {
            recorder.upload("/admin/getWord", function (state, e) {
                switch (state) {
                    case 'uploading':
                        //var percentComplete = Math.round(e.loaded * 100 / e.total) + '%';
                        break;
                    case 'ok':
                        //alert(e.target.responseText);
                        // alert("上傳成功");
                        break;
                    case 'error':
                        alert("上傳失敗");
                        break;
                    case 'cancel':
                        alert("上傳被取消");
                        break;
                }
            });
        })


    </script>
</body>

</html>

index.html

(function (window) {
    //兼容
    window.URL = window.URL || window.webkitURL;
    navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;


    var HZRecorder = function (stream, config) {
        config = config || {};
        config.sampleBits = 16;      //採樣數位 8, 16
        config.sampleRate = 16000;   //採樣率(1/6 44100)


        var context = new AudioContext();
        var audioInput = context.createMediaStreamSource(stream);
        var recorder = context.createScriptProcessor(4096, 1, 1);


        var audioData = {
            size: 0          //錄音文件長度
            , buffer: []     //錄音緩存
            , inputSampleRate: context.sampleRate    //輸入採樣率
            , inputSampleBits: 16       //輸入採樣數位 8, 16
            , outputSampleRate: config.sampleRate    //輸出採樣率
            , oututSampleBits: config.sampleBits       //輸出採樣數位 8, 16
            , input: function (data) {
                this.buffer.push(new Float32Array(data));
                this.size += data.length;
            }
            , compress: function () { //合併壓縮
                //合併
                var data = new Float32Array(this.size);
                var offset = 0;
                for (var i = 0; i < this.buffer.length; i++) {
                    data.set(this.buffer[i], offset);
                    offset += this.buffer[i].length;
                }
                //壓縮
                var compression = parseInt(this.inputSampleRate / this.outputSampleRate);
                var length = data.length / compression;
                var result = new Float32Array(length);
                var index = 0, j = 0;
                while (index < length) {
                    result[index] = data[j];
                    j += compression;
                    index++;
                }
                return result;
            }
            , encodeWAV: function () {
                var sampleRate = Math.min(this.inputSampleRate, this.outputSampleRate);
                var sampleBits = Math.min(this.inputSampleBits, this.oututSampleBits);
                var bytes = this.compress();
                var dataLength = bytes.length * (sampleBits / 8);
                var buffer = new ArrayBuffer(44 + dataLength);
                var data = new DataView(buffer);


                var channelCount = 1;//單聲道
                var offset = 0;


                var writeString = function (str) {
                    for (var i = 0; i < str.length; i++) {
                        data.setUint8(offset + i, str.charCodeAt(i));
                    }
                }

                // 資源交換文件標識符 
                writeString('RIFF'); offset += 4;
                // 下個地址開始到文件尾總字節數,即文件大小-8 
                data.setUint32(offset, 36 + dataLength, true); offset += 4;
                // WAV文件標誌
                writeString('WAVE'); offset += 4;
                // 波形格式標誌 
                writeString('fmt '); offset += 4;
                // 過濾字節,一般爲 0x10 = 16 
                data.setUint32(offset, 16, true); offset += 4;
                // 格式類別 (PCM形式採樣數據) 
                data.setUint16(offset, 1, true); offset += 2;
                // 通道數 
                data.setUint16(offset, channelCount, true); offset += 2;
                // 採樣率,每秒樣本數,表示每個通道的播放速度 
                data.setUint32(offset, sampleRate, true); offset += 4;
                // 波形數據傳輸率 (每秒平均字節數) 單聲道×每秒數據位數×每樣本數據位/8 
                data.setUint32(offset, channelCount * sampleRate * (sampleBits / 8), true); offset += 4;
                // 快數據調整數 採樣一次佔用字節數 單聲道×每樣本的數據位數/8 
                data.setUint16(offset, channelCount * (sampleBits / 8), true); offset += 2;
                // 每樣本數據位數 
                data.setUint16(offset, sampleBits, true); offset += 2;
                // 數據標識符 
                writeString('data'); offset += 4;
                // 採樣數據總數,即數據總大小-44 
                data.setUint32(offset, dataLength, true); offset += 4;
                // 寫入採樣數據 
                if (sampleBits === 8) {
                    for (var i = 0; i < bytes.length; i++, offset++) {
                        var s = Math.max(-1, Math.min(1, bytes[i]));
                        var val = s < 0 ? s * 0x8000 : s * 0x7FFF;
                        val = parseInt(255 / (65535 / (val + 32768)));
                        data.setInt8(offset, val, true);
                    }
                } else {
                    for (var i = 0; i < bytes.length; i++, offset += 2) {
                        var s = Math.max(-1, Math.min(1, bytes[i]));
                        data.setInt16(offset, s < 0 ? s * 0x8000 : s * 0x7FFF, true);
                    }
                }


                return new Blob([data], { type: 'audio/wav' });
            }
        };


        //開始錄音
        this.start = function () {
            audioInput.connect(recorder);
            recorder.connect(context.destination);
        }


        //停止
        this.stop = function () {
            recorder.disconnect();
        }


        //獲取音頻文件
        this.getBlob = function () {
            this.stop();
            return audioData.encodeWAV();
        }


        //回放
        this.play = function (audio) {
            audio.src = window.URL.createObjectURL(this.getBlob());
        }


        //上傳
        this.upload = function (url, callback) {
            var fd = new FormData();
            fd.append("file", this.getBlob());
            var xhr = new XMLHttpRequest();
            if (callback) {
                xhr.upload.addEventListener("progress", function (e) {
                    callback('uploading', e);
                }, false);
                xhr.addEventListener("load", function (e) {
                    callback('ok', e);
                }, false);
                xhr.addEventListener("error", function (e) {
                    callback('error', e);
                }, false);
                xhr.addEventListener("abort", function (e) {
                    callback('cancel', e);
                }, false);
            }
            xhr.open("POST", url);
            xhr.send(fd);
            xhr.onreadystatechange = function () {
                console.log("語音識別結果："+xhr.responseText)
                $("#text").append('<h2>'+xhr.responseText+'</h2>');
            }
        }

        //音頻採集
        recorder.onaudioprocess = function (e) {
            audioData.input(e.inputBuffer.getChannelData(0));
            //record(e.inputBuffer.getChannelData(0));
        }


    };
    //拋出異常
    HZRecorder.throwError = function (message) {
        alert(message);
        throw new function () { this.toString = function () { return message; } }
    }
    //是否支持錄音
    HZRecorder.canRecording = (navigator.getUserMedia != null);
    //獲取錄音機
    HZRecorder.get = function (callback, config) {
        if (callback) {
            if (navigator.getUserMedia) {
                navigator.getUserMedia(
                    { audio: true } //只啓用音頻
                    , function (stream) {
                        var rec = new HZRecorder(stream, config);
                        callback(rec);
                    }
                    , function (error) {
                        switch (error.code || error.name) {
                            case 'PERMISSION_DENIED':
                            case 'PermissionDeniedError':
                                HZRecorder.throwError('用戶拒絕提供信息。');
                                break;
                            case 'NOT_SUPPORTED_ERROR':
                            case 'NotSupportedError':
                                HZRecorder.throwError('瀏覽器不支持硬件設備。');
                                break;
                            case 'MANDATORY_UNSATISFIED_ERROR':
                            case 'MandatoryUnsatisfiedError':
                                HZRecorder.throwError('無法發現指定的硬件設備。');
                                break;
                            default:
                                HZRecorder.throwError('無法打開麥克風。異常信息:' + (error.code || error.name));
                                break;
                        }
                    });
            } else {
                HZRecorder.throwErr('當前瀏覽器不支持錄音功能。'); return;
            }
        }
    }


    window.HZRecorder = HZRecorder;


})(window);

HZRecorder

6、運行效果

java 離線中文語音文字識別

java 違規圖片、視頻自動識別（色情、血腥、暴力）

element-ui 文件上傳問題記錄

springboot 統一處理請求非法參數

wkhtmltopdf 使用記錄

html 頁面打開exe方法

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結