SonarQube 因爲中文文件名報錯
SonarQube 因爲中文文件名報錯
centos7, Jenkins, SonarQube 環境
01:46:43.972 INFO: ------------------------------------------------------------------------
01:46:43.972 INFO: EXECUTION FAILURE
01:46:43.972 INFO: ------------------------------------------------------------------------
01:46:43.973 INFO: Total time: 8.409s
01:46:44.053 INFO: Final Memory: 19M/553M
01:46:44.053 INFO: ------------------------------------------------------------------------
01:46:44.053 ERROR: Error during SonarQube Scanner execution
01:46:44.053 ERROR: Malformed input or input contains unmappable characters: doc/???????????????????????????.xlsx
01:46:44.053 ERROR:
01:46:44.053 ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.
原因：沒有設置相關的環境變量爲UTF-8
一、配置Jenkins
Dockerfile
FROM jenkins/jenkins:lts
USER root
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update
# Install docker community edition according to:
# https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce
RUN apt-get install -y \
apt-utils \
apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
RUN apt-key fingerprint 0EBFCD88
RUN add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/debian \
$(lsb_release -cs) \
stable"
RUN apt-get update
RUN apt-get install -y docker-ce
# Install utilities
RUN apt-get install -y nmap
# Install docker composer according to:
# https://docs.docker.com/compose/install/#install-compose
RUN curl -L "https://github.com/docker/compose/releases/download/1.23.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
RUN chmod +x /usr/local/bin/docker-compose
# Set timezone
ENV TZ=Asia/Shanghai
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
ENV JENKINS_OPTS --httpPort=8089 --prefix=/jenkins
# Install plugins
COPY plugins.txt /usr/share/jenkins/plugins.txt
RUN /usr/local/bin/install-plugins.sh < /usr/share/jenkins/plugins.txt
Plugin.txt
ace-editor:1.1
ant:1.9
antisamy-markup-formatter:1.5
apache-httpcomponents-client-4-api:4.5.5-3.0
authentication-tokens:1.3
blueocean:1.16.0
blueocean-autofavorite:1.2.4
blueocean-bitbucket-pipeline:1.16.0
blueocean-commons:1.16.0
blueocean-config:1.16.0
blueocean-core-js:1.16.0
blueocean-dashboard:1.16.0
blueocean-display-url:2.3.0
blueocean-events:1.16.0
blueocean-github-pipeline:1.16.0
blueocean-git-pipeline:1.16.0
blueocean-i18n:1.16.0
blueocean-jira:1.16.0
blueocean-jwt:1.16.0
blueocean-personalization:1.16.0
blueocean-pipeline-api-impl:1.16.0
blueocean-pipeline-editor:1.16.0
blueocean-pipeline-scm-api:1.16.0
blueocean-rest:1.16.0
blueocean-rest-impl:1.16.0
blueocean-web:1.16.0
bouncycastle-api:2.17
branch-api:2.5.1
build-name-setter:2.0.1
build-pipeline-plugin:1.5.8
build-timeout:1.19
built-on-column:1.1
cloudbees-bitbucket-branch-source:2.4.4
cloudbees-folder:6.8
command-launcher:1.3
conditional-buildstep:1.3.6
config-file-provider:3.6
copyartifact:1.42
credentials:2.1.19
credentials-binding:1.18
dashboard-view:2.10
display-url-api:2.3.1
docker-commons:1.15
docker-workflow:1.18
durable-task:1.29
email-ext:2.66
emailext-template:1.1
embeddable-build-status:2.0.1
envinject:2.1.6
envinject-api:1.5
extended-choice-parameter:0.78
external-monitor-job:1.7
favorite:2.3.2
git:3.10.0
git-client:2.7.7
github:1.29.4
github-api:1.95
github-branch-source:2.5.2
github-oauth:0.32
git-parameter:0.9.10
git-server:1.7
gradle:1.31
handlebars:1.1.1
handy-uri-templates-2-api:2.1.7-1.0
htmlpublisher:1.18
influxdb:1.21
jackson2-api:2.9.9
javadoc:1.5
jdk-tool:1.2
jenkins-design-language:1.16.0
jenkins-multijob-plugin:1.32
jira:3.0.7
jquery:1.12.4-0
jquery-detached:1.2.1
jquery-ui:1.0.2
jsch:0.1.55
junit:1.28
ldap:1.20
locale:1.4
localization-zh-cn:0.0.15
lockable-resources:2.5
login-theme:1.0
mailer:1.23
mapdb-api:1.0.9.0
matrix-auth:2.4.2
matrix-project:1.14
maven-plugin:3.2
mercurial:2.6
momentjs:1.1.1
pam-auth:1.5.1
parameterized-trigger:2.35.2
persistent-parameter:1.1
pipeline-build-step:2.9
pipeline-github-lib:1.0
pipeline-graph-analysis:1.10
pipeline-input-step:2.10
pipeline-milestone-step:1.3.1
pipeline-model-api:1.3.8
pipeline-model-declarative-agent:1.1.1
pipeline-model-definition:1.3.8
pipeline-model-extensions:1.3.8
pipeline-rest-api:2.11
pipeline-stage-step:2.3
pipeline-stage-tags-metadata:1.3.8
pipeline-stage-view:2.11
pipeline-utility-steps:2.3.0
plain-credentials:1.5
publish-over:0.22
publish-over-ssh:1.20.1
pubsub-light:1.12
rebuild:1.31
repo:1.10.7
resource-disposer:0.12
role-strategy:2.10
run-condition:1.2
scm-api:2.4.1
script-security:1.58
simple-theme-plugin:0.5.1
sse-gateway:1.17
ssh:2.6.1
ssh-agent:1.17
ssh-credentials:1.16
ssh-slaves:1.29.4
structs:1.19
subversion:2.12.1
throttle-concurrents:2.0.1
timestamper:1.9
token-macro:2.7
variant:1.2
windows-slaves:1.4
workflow-aggregator:2.6
workflow-api:2.34
workflow-basic-steps:2.16
workflow-cps:2.68
workflow-cps-global-lib:2.13
workflow-durable-task-step:2.30
workflow-job:2.32
workflow-multibranch:2.21
workflow-scm-step:2.7
workflow-step-api:2.19
workflow-support:3.3
ws-cleanup:0.37
#!/bin/bash
set -x
docker build -t qa/jenkins:latest . \
&& docker run -d --rm \
-p 8088:8089 \
-p 50000:50000 \
-v /data/jenkins_home:/var/jenkins_home \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /etc/hosts:/etc/hosts \
--env "JAVA_OPTS=-Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF-8" \
--name jenkins \
qa/jenkins:latest
二、配置SonarQube
Dockerfile
FROM centos:7
USER root
# Setup http proxy if we are behind a firewall
ARG http_proxy
ENV http_proxy ${http_proxy}
# Install build tools
RUN yum install -y epel-release
RUN yum install -y wget gcc-c++ make cmake3 unzip
# Download and install scanner
RUN (wget -q --timeout=5 --tries=3 https://qa.xindong.com/nexus/repository/raw/sonar/sonar-scanner-cli-3.2.0.1227-linux.zip \
|| wget -q --timeout=5 --tries=3 https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-3.2.0.1227-linux.zip) \
&& unzip sonar-scanner-cli-3.2.0.1227-linux.zip \
&& rm sonar-scanner-cli-3.2.0.1227-linux.zip \
&& mv sonar-scanner-3.2.0.1227-linux /opt/sonar-scanner
ENV LANG="en_US.UTF-8"
ENV LC_NUMERIC="en_US.UTF-8"
ENV LC_TIME="en_US.UTF-8"
ENV LC_COLLATE="en_US.UTF-8"
ENV LC_MONETARY="en_US.UTF-8"
ENV LC_MESSAGES="en_US.UTF-8"
ENV LC_PAPER="en_US.UTF-8"
ENV LC_NAME="en_US.UTF-8"
ENV C_ADDRESS="en_US.UTF-8"
ENV LC_TELEPHONE="en_US.UTF-8"
ENV LC_MEASUREMENT="en_US.UTF-8"
ENV LC_IDENTIFICATION="en_US.UTF-8"
# Add script
ARG script=script
ADD $script/ /opt/script
# Add project config
ARG conf=conf
ADD $conf/ /opt/conf
# Start analysis
ARG ws=/var/workspace
ENV WORKSPACE $ws
WORKDIR $WORKSPACE
CMD ["sh", "-c", "/opt/script/start-scan.sh"]
script/start-scan.sh
#!/bin/bash
##
## Scan project workspace using sonar scanner.
##
## Usage: start-scan.sh [options]
##
## Options:
## -u,--sonar-host-url <url>
## URL of SonarQube website to which we send analysis result.
## -w,--workspace <path>
## Path of workspace other than current working directory.
## -c,--proj-conf <path>
## Path of sonar-project.properties.
## -s,--scanner <path>
## Path of sonar-scanner.
##
usage() {
[ "$*" ] && echo "$0: $*"
sed -n '/^##/,/^$/s/^## \{0,1\}//p' "$0"
exit 2
} 2>/dev/null
# Default arguments
sonar_host_url="https://qa.xindong.com/sonar/"
workspace=$(pwd)
proj_conf=/opt/conf/sonar-project.properties
scanner=/opt/sonar-scanner/bin/sonar-scanner
# Parse options
positional=()
while [[ $# -gt 0 ]]
do
key="$1"
case $key in
-u|--sonar-host-url)
sonar_host_url="$2"
shift;
shift;;
-w|--workspace)
workspace=$(realpath "$2")
shift;
shift;;
-c|--proj-conf)
proj_conf=$(realpath "$2")
shift;
shift;;
-s|--scanner)
scanner=$(realpath "$2")
shift;
shift;;
-h|--help)
usage;
shift;;
-*)
usage "$1: unknown option"
shift;
break;;
*)
positional+=("$1")
shift;;
esac
done
set -- "${positional[@]}" # Restore positional arguments.
# Copy sonar project properties file
cp $proj_conf $workspace
# Run scanner
$scanner -Dsonar.host.url=$sonar_host_url
conf/sonar-project.properties
sonar.projectKey=rosita
sonar.projectName=RO Server Interface Testing Automation
sonar.projectVersion=1.0
sonar.sources=.
sonar.exclusions=\
log/**/*,\
venv/**/*,\
bin/**/*,\
obj/**/*,\
test/**/*,\
ci/**/*,\
proto/**/*,\
bootstrap/**/*,\
pblib/proto/**/*,\
report/**/*
sonar.language=py
sonar.sourceEncoding=UTF-8
# sonar.showProfiling=true
# sonar.showSql=true
sonar.verbose=true
sonar.tests=test
sonar.python.xunit.reportPath=report/test*_report.xml
sonar.python.coverage.reportPath=report/coverage.xml
三、總結
主要修復步驟如下：
（1）start-jenkins.sh 添加參數
--env "JAVA_OPTS=-Dsun.jnu.encoding=UTF-8 -Dfile.encoding=UTF-8"
（2）Jenkins 系統管理 》系統設置 》全局屬性
1. file.encoding: UTF-8
2. sun.jnu.encoding: UTF-8
(3) SonarQube：Dockerfile配置ENV
ENV LANG="en_US.UTF-8"
ENV LC_NUMERIC="en_US.UTF-8"
ENV LC_TIME="en_US.UTF-8"
ENV LC_COLLATE="en_US.UTF-8"
ENV LC_MONETARY="en_US.UTF-8"
ENV LC_MESSAGES="en_US.UTF-8"
ENV LC_PAPER="en_US.UTF-8"
ENV LC_NAME="en_US.UTF-8"
ENV C_ADDRESS="en_US.UTF-8"
ENV LC_TELEPHONE="en_US.UTF-8"
ENV LC_MEASUREMENT="en_US.UTF-8"
ENV LC_IDENTIFICATION="en_US.UTF-8"
注：個人遇到的問題是通過修改SonarQube的Dockerfile解決，即添加ENV配置。