How to build libiconv with VC2017

最近解决了VC2017编译libiconv的问题，总结一下经验和收获。
按照GNU libiconv官方的README.windows文档“2) Native binaries, built using the MS Visual C/C++ tool chain.”的说明，应该是在Cygwin环境下使用Visual C/C++ 9.0或以上版本进行。
网上介绍VC编译libiconv的文章很多，采用官方推荐做法的却很少，奇哉怪也！我在解决的过程中发现原来这个过程有好几个坑，可能是因为这个原因吧。

1. fprintf()输出’\n’换行符问题

libiconv/lib/genaliases.c输出’\n’到以文本方式打开的canonical.sh，这导致在Cygwin中生成的Unix脚本文件用的换行符却是DOS格式，然后导致脚本中的sed程序处理出错。
这个问题已在官方代码库修正（在libiconv-1.15之后），改为以二进制方式打开文件。

参考

[1] Avoid end-of-lines problem in generated shell scripts on Cygwin.
[2] Text and Binary modes

2. windres的-F参数问题

在libiconv/lib/Makefile.in中:

libiconv.res.lo : $(srcdir)/../windows/libiconv.rc
    $(LIBTOOL_COMPILE) --tag=RC $(RC) `$(SHELL) $(srcdir)/../windows/windres-options --escape $(PACKAGE_VERSION)` -i $(srcdir)/../windows/libiconv.rc -o libiconv.res.lo --output-format=coff

在libiconv/src/Makefile.in中:

iconv.res : $(srcdir)/../windows/iconv.rc
    $(WINDRES) `$(SHELL) $(srcdir)/../windows/windres-options --escape $(PACKAGE_VERSION)` -i $(srcdir)/../windows/iconv.rc -o iconv.res --output-format=coff

这里$(RC)和$(WINDRES)在生成的Makefile中就对应了Cygwin的windres程序。（不能用Windows平台的rc.exe，因为它不支持output-format参数）。
libiconv的出现远早于64位Windows。在64位Windows的Cygwin64终端上，windres默认将rc资源文件编译成64位的res文件。如果要编译的是32位的libiconv，link链接时就出错了。
解决办法是在”configure”的运行参数中增加

    RC="windres -Fpe-i386" WINDRES="windres -Fpe-i386"

如果是64位的目标，则增加：

    RC="windres -Fpe-x86-64" WINDRES="windres -Fpe-x86-64"

链接器link也有类似问题，需要在LDFLAGS中添加“/MACHINE:X86”或“/MACHINE:X64”。
这个问题只能建议官方更新README.windows文档了。

参考

[1] windres

3. UTF-8字符导致C4819编译警告的问题

先看这个补丁：Use Unicode single-quotes in comments.
对于Windows系统区域和语言设置不是英语的用户，代码中的UTF-8字符会导致MSVC报告C4819编译警告。
关键问题是编译器不能正确处理后续代码了，后面经常就跟着报编译错误，然后是编译失败和链接失败。
关于这个问题网上也有很多解决方案：

a. 修改Windows区域和语言设置

改成英语国家，就是日常使用不太方便了。可能美国同行都没注意到有UTF-8导致C4819这回事。

b. 转码后保存

网上很多介绍怎么通过VC界面手动保存的，不抄了。我自己写的bash版：

    if [ ! `regtool -q check/HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Control/Nls/CodePage/OEMCP` ]; then
        TOCODEPAGE=`regtoolget /HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Control/Nls/CodePage/OEMCP`
        cp include/iconv.h include/iconv.h.backup
        iconv-c -f utf-8 -t cp"${TOCODEPAGE}" include/iconv.h 1>include/iconv.h.tmp
        mv include/iconv.h.tmp include/iconv.h
        make
        make check
        make install
        mv include/iconv.h.backup include/iconv.h
    fi

之前为另一项目做的DOS批处理文件版：

FOR %%X IN (src\*.*, include\artilist.h, dat\*.*, win\win32\*.c) DO (
    iconv.exe -c -f utf-8 -t gb18030 %%X 1> %%X.gb
    mv.exe %%X.gb %%X
)

c. 不修改代码

给CFLAGS和CXXFLAGS加”/utf-8”参数，详见下面的参考[1]。

d. 不要在C/C++代码里用UTF-8

把前面提到的补丁取消掉是最好的，毕竟iconv.h还要导出给库用户使用的，不改的话就是把问题推给用户了。
C/C++源文件最好只用ASCII字符编码。有多语言需求的可以放到程序外去实现，比如配置文件、数据库或其他明确支持UTF-8的脚本语言。

参考

[1] /utf-8 (Set Source and Executable character sets to UTF-8)
[2] Unicode Support in the Compiler and Linker
[3] Compile error with source file containing UTF8 strings (in CJK system locale)
[4] It is impossible to use UTF-8 without BOM in source files.

4. build-vc2017.sh

我写的辅助脚本，仅供参考。

#!/bin/sh
if [ ! -f configure ]; then
    echo Install Cygwin packages: autoconf make automake gcc-core gettext-devel git gperf groff m4 patch
    echo Then run ./autogen.sh before this script.
    exit 1
fi

# Set PlatformTarget
# Remember to "make clean" after switching PlatformTarget, before running this script again.
#PlatformTarget=x64
if [ -z "${PlatformTarget}" ]; then
    PlatformTarget=x86
fi

# Get WindowsSdkDir
#WindowsSdkDir='C:\\Program Files (x86)\\Windows Kits\\10\\'
if [ -z "${WindowsSdkDir}" ]; then
    REGISTRY1=/HKEY_LOCAL_MACHINE/SOFTWARE/Wow6432Node/Microsoft/Microsoft\ SDKs/Windows/v10.0
    REGISTRY2=/HKEY_CURRENT_USER/SOFTWARE/Wow6432Node/Microsoft/Microsoft\ SDKs/Windows/v10.0
    REGISTRY3=/HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Microsoft\ SDKs/Windows/v10.0
    REGISTRY4=/HKEY_CURRENT_USER/SOFTWARE/Microsoft/Microsoft\ SDKs/Windows/v10.0
    if [ ! `regtool -q check "${REGISTRY1}/InstallationFolder"` ]; then
        WindowsSdkDir=`regtool get "${REGISTRY1}/InstallationFolder"`
    elif [ ! `regtool -q check "$REGISTRY2/InstallationFolder"` ]; then
        WindowsSdkDir=`regtool get "${REGISTRY2}/InstallationFolder"`
    elif [ ! `regtool -q check "$REGISTRY3/InstallationFolder"` ]; then
        WindowsSdkDir=`regtool get "${REGISTRY3}/InstallationFolder"`
    elif [ ! `regtool -q check "$REGISTRY4/InstallationFolder"` ]; then
        WindowsSdkDir=`regtool get "${REGISTRY4}/InstallationFolder"`
    else
        echo Cannot find WindowsSdkDir
        exit 1;
    fi
fi

if [ ! -d "${WindowsSdkDir}" ]; then
    echo Cannot find WindowsSdkDir ${WindowsSdkDir}
    exit 1
fi

# Set WindowsSDKVersion
#WindowsSDKVersion='10.0.15063.0'
#WindowsSDKVersion='10.0.14393.0'
#WindowsSDKVersion='10.0.10586.0'
#WindowsSDKVersion='10.0.10240.0'
if [ -z "${WindowsSDKVersion}" ]; then
    WindowsSDKVersion='10.0.16299.0'
fi

# Set VSINSTALLDIR
VSINSTALLDIR='D:\Program Files (x86)\Microsoft Visual Studio\2017\Community\'
#VSINSTALLDIR='E:\Program Files (x86)\Microsoft Visual Studio\2017\Community\'
#VSINSTALLDIR='F:\Program Files (x86)\Microsoft Visual Studio\2017\Community\'
#VSINSTALLDIR='G:\Program Files (x86)\Microsoft Visual Studio\2017\Community\'
if [ -z "${VSINSTALLDIR}" ]; then
    VSINSTALLDIR='C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\'
fi

# Set VSToolsVersion
#VSToolsVersion='14.11.25503'
if [ -z "${VSToolsVersion}" ]; then
    VSToolsVersion='14.11.25503'
fi

echo PlatformTarget=${PlatformTarget}
echo WindowsSdkDir=${WindowsSdkDir}
echo WindowsSDKVersion=${WindowsSDKVersion}
echo VSINSTALLDIR=${VSINSTALLDIR}
echo VSToolsVersion=${VSToolsVersion}
echo

# Verify WindowsSdkIncludeDir and WindowsSdkLibDir
WindowsSdkIncludeDir="${WindowsSdkDir}"'Include\'"${WindowsSDKVersion}"
WindowsSdkLibDir="${WindowsSdkDir}"'Lib\'"${WindowsSDKVersion}"
if [ ! -d "${WindowsSdkIncludeDir}" ]; then
        echo Cannot find WindowsSdkIncludeDir ${WindowsSdkIncludeDir}
    echo Modify WindowsSDKVersion value and run again.
    exit 1
fi
if [ ! -d "${WindowsSdkLibDir}" ]; then
        echo Cannot find WindowsSdkLibDir ${WindowsSdkLibDir}
    echo Modify WindowsSDKVersion value and run again.
    exit 1
fi

# Windows C library headers and libraries.
WindowsCrtIncludeDir="${WindowsSdkIncludeDir}\\ucrt"
WindowsCrtLibDir="${WindowsSdkLibDir}"'\ucrt\'"${PlatformTarget}"
if [ ! -f "${WindowsCrtIncludeDir}\\stdio.h" ]; then 
    echo Cannot locate ${WindowsCrtIncludeDir}\\stdio.h
    exit 1
fi
if [ ! -f "${WindowsCrtLibDir}\\ucrt.lib" ]; then
    echo Cannot locate ${WindowsCrtLibDir}\\ucrt.lib
    exit 1
fi
INCLUDE="${WindowsCrtIncludeDir};$INCLUDE"
LIB="${WindowsCrtLibDir};$LIB"

# Windows API headers and libraries.
INCLUDE="${WindowsSdkIncludeDir}\\um;${WindowsSdkIncludeDir}\\shared;$INCLUDE"
LIB="${WindowsSdkLibDir}"'\um\'"${PlatformTarget};$LIB"

# Visual C++ tools, headers and libraries.
VCToolsInstallDir="${VSINSTALLDIR}"'VC\Tools\MSVC\'"${VSToolsVersion}"
if [ ! -f "${VCToolsInstallDir}\\include\\stdarg.h" ]; then
        echo Cannot locate ${VCToolsInstallDir}\\include\\stdarg.h
    echo Modify VSINSTALLDIR and VSToolsVersion values and run again.
    exit 1
fi
INCLUDE="${VCToolsInstallDir}\\include;$INCLUDE"
LIB="${VCToolsInstallDir}"'\lib\'"${PlatformTarget};$LIB"

export INCLUDE LIB
#echo INCLUDE=${INCLUDE}
#echo LIB=${LIB}
#echo
#read -n1 -p "Verify the above settings. Press CTRL-C to abort, or other keys to continue..."

[ -d $HOME/msvc/ ] || mkdir $HOME/msvc/
cp gnulib/build-aux/ar-lib $HOME/msvc/ && chmod a+x $HOME/msvc/ar-lib
cp gnulib/build-aux/compile $HOME/msvc/ && chmod a+x $HOME/msvc/compile

if [ "${PlatformTarget}" == "x86" ]; then
    export PATH=`cygpath -u "${VCToolsInstallDir}"`/bin/Hostx86/x86:`cygpath -u "${WindowsSdkDir}"`bin/"${WindowsSDKVersion}"/x86:"$PATH"
    win32_target=_WIN32_WINNT_WIN7
    ./configure --host=i686-w64-mingw32 --prefix=/usr/local/msvc32 \
        CC="$HOME/msvc/compile cl -nologo" \
        CFLAGS="-MD /utf-8" \
        CXX="$HOME/msvc/compile cl -nologo" \
        CXXFLAGS="-MD /utf-8" \
        CPPFLAGS="-D_WIN32_WINNT=$win32_target" \
        RC="windres -Fpe-i386" \
        WINDRES="windres -Fpe-i386" \
        LDFLAGS="/MACHINE:X86" \
        LD="link" \
        NM="dumpbin -symbols" \
        STRIP=":" \
        AR="$HOME/msvc/ar-lib lib" \
        RANLIB=":"
        make
        make check
        make install
elif [ "${PlatformTarget}"=="x64" ]; then
    export PATH=`cygpath -u "${VCToolsInstallDir}"`/bin/Hostx64/x64:`cygpath -u "${WindowsSdkDir}"`bin/"${WindowsSDKVersion}"/x64:"$PATH"
    win32_target=_WIN32_WINNT_WIN7
    ./configure --host=x86_64-w64-mingw32 --prefix=/usr/local/msvc64 \
        CC="$HOME/msvc/compile cl -nologo" \
        CFLAGS="-MD /utf-8" \
        CXX="$HOME/msvc/compile cl -nologo" \
        CXXFLAGS="-MD /utf-8" \
        CPPFLAGS="-D_WIN32_WINNT=$win32_target" \
        RC="windres -Fpe-x86-64" \
        WINDRES="windres -Fpe-x86-64" \
        LDFLAGS="/MACHINE:X64" \
        LD="link" \
        NM="dumpbin -symbols" \
        STRIP=":" \
        AR="$HOME/msvc/ar-lib lib" \
        RANLIB=":"
        make
        make check
        make install
else
    echo Modify PlatformTarget value and run again.
    exit 1
fi