Windows下unzip解压缩遇到的warning: xxx appears to use backslashes as path separators

IBM Spectrum Symphony是基于SOA架构的分布式计算框架,它能在自由伸缩的共享集群中,为计算密集型和数据密集型的应用提供强大的企业级管理。

Symphony有一个配置是这样的,在应用的profile里头,有个方法是Register,可以配置成actionOnSI="blockHost"。这样当某service实例在Register方法中由于某种原因(比方说磁盘不足、单点故障等)而失败的话,计算节点就会被阻止加入到计算,那么Symphony会从资源组里分配其他可用的计算节点参与运算。这样做的目的是避免作业一直分配到某个不能成功完成运算的节点,导致时间浪费甚至作业得不到计算。

      <Method name="Register">
        <Timeout actionOnSI="blockHost" duration="600"/>
        <Exit actionOnSI="blockHost"/>
      </Method>

 

某天,客户遇到这样一个问题。他们更新了应用的service代码,要部署到cluster。部署是成功的,但是在解压缩的时候,出错了,结果就是节点被blocked。为了方便理解,先贴上Symphony SOAM这部分的架构图。红框部分是Symphony中间件,客户仅需要开发客户端client和服务端service,client用于提交作业,service实例用于计算作业,至于资源调度和作业调度则由中间件完成,客户不需要考虑。

所以我们需要看SIM(图中的Service Instance Manager)的日志,因为是SIM管理着SI(图中的Service Instance),包括SI的启动、更新、退出等。下面是SIM的日志

 15:51:06.951 GMT DEBUG [5268:17244] CLI.dpl.download.Uncompress - Uncompress::copyArchive: Copying file C:\Windows\TEMP\\SI_Package_Name_Version\SI_Package_Name_Version.zip to D:\grid\ibm/soam\deploy\SI_Package_Name_Version.v1\SI_Package_Name_Version.zip

 15:51:06.966 GMT DEBUG [5268:17244] CLI.dpl.download.Uncompress - Uncompress::execNextStep: Executing uncompress or validate step: D:\grid\ibm\soam\7.2\w2k3_x64-vc7-psdk\bin\..\bin\utils\unzip -o -q "SI_Package_Name_Version.zip"

 15:51:06.966 GMT DEBUG [5268:17244] CLI.dpl.download.Uncompress - Uncompress::execCmd: Execute PATH: D:\grid\ibm/soam/7.2/w2k3_x64-vc7-psdk/lib64;D:\grid\ibm/soam/7.2/w2k3_x64-vc7-psdk/bin;D:\grid\ibm/soam/7.2/w2k3_x64-vc7-psdk/lib;D:\grid\ibm\3.6\lib;D:\grid\ibm\soam\7.2\w2k3_x64-vc7-psdk\bin;D:\grid\ibm\soam\7.2\w2k3_x64-vc7-psdk\lib;D:\grid\ibm\soam\7.2\w2k3_x64-vc7-psdk\lib64;D:\grid\ibm\3.6\bin;D:\grid\ibm\3.6\lib;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\\Hewlett-Packard\HPSUT\bin;C:\Program Files\HP\HP BTO Software\lib;C:\Program Files\HP\HP BTO Software\bin;C:\Program Files\HP\HP BTO Software\bin\win64;C:\Program Files\HP\HP BTO Software\bin\win64\OpC;;D:\grid\ibm\3.6\lib;D:\grid\ibm\3.6\bin

 15:51:06.966 GMT DEBUG [5268:17244] CLI.dpl.download.RunTask - RunTask::execBlock(): Started: D:\grid\ibm\soam\7.2\w2k3_x64-vc7-psdk\bin\..\bin\utils\unzip -o -q "SI_Package_Name_Version.zip" with tid 388881

 15:51:07.869 GMT DEBUG [5268:17244] CLI.dpl.download.RunTask - RunTask::execBlock(): There may be some exceptions or errors during decompressing process, and the status of the process is 1

 15:51:07.869 GMT DEBUG [5268:17244] CLI.dpl.download.Uncompress - Uncompress::execNextStep: Decompress or validate command failed.

 15:51:07.869 GMT INFO [5268:17244] CLI.dpl.deployutils - warning: SI_Package_Name_Version.zip appears to use backslashes as path separators

 15:51:07.869 GMT DEBUG [5268:17244] CLI.dpl.download.Uncompress - Uncompress::extract: Deleting file D:\grid\ibm/soam\deploy\SI_Package_Name_Version.v1\SI_Package_Name_Version.zip 

 15:51:07.869 GMT DEBUG [5268:17244] CLI.dpl.download.InstallModule - InstallModule::copyModuleAndDecompress(): Failed to uncompress: C:\Windows\TEMP\\SI_Package_Name_Version\SI_Package_Name_Version.zip

 15:51:07.869 GMT ERROR [5268:17244] CLI.dpl.download.InstallModule - Domain <Application>: Failed to uncompress the package, C:\Windows\TEMP\\SI_Package_Name_Version\SI_Package_Name_Version.zip. Check if the de-archive utility is available or there is enough disk storage for uncompressing.

 15:51:07.916 GMT DEBUG [5268:17244] CLI.SoamDeploy.CliSoamDeployDownloadImpl - Deleting tmp file: C:\Windows\TEMP\\SI_Package_Name_Version\SI_Package_Name_Version.zip

 15:51:07.916 GMT ERROR [5268:17244] CLI.SoamDeploy.CliSoamDeployDownloadImpl - Domain <Application>: The package <SI_Package_Name_Version> failed to install. The deployed package file could not be decompressed. Check if: (1) the decompression utility is available, (2) there is enough disk storage for decompressing the file, or (3) the package file is too large to be decompressed.

从SIM日志来看,新的SI的package是已经下载到本地了,但是在解压缩的时候,报了个warning级别的消息:"SI_Package_Name_Version.zip appears to use backslashes as path separators",解压缩的命令是unzip -o -q "SI_Package_Name_Version.zip"。根据后边的日志,可能的原因包括:

(1) unzip工具不存在;

(2) 磁盘空间不足;

(3) 包太大。

当然,客户检查下来,这三个怀疑点都不是问题。

 

那会不会是压缩包本身有问题呢?于是让客户在同一台计算节点上,手动去解压缩那个包。果然,手动解压缩也有类似的warning,只不过虽然报了这个warning,解压缩是成功的,所有文件和目录都解压出来了。

D:\grid\ibm\soam\deploy\Applications\SampleAppJava\SI_Package_Name_Version.v1

>D:\grid\ibm\soam\7.2\w2k3_x64-vc7-psdk\bin\..\bin\utils\unzip -o -q SI_Package_Name_Version.zip

warning: SI_Package_Name_Version.zip appears to use backslashes as path separators

至此,可以断定跟Symphony关系不大,要么是压缩包有问题,要么是unzip不能解压缩这个压缩包。

 

通过询问客户得知,客户的压缩文件都是经过Windows的Powershell压缩工具Compress-Archive生成的。于是模仿客户的包的格式,在实验室里可以复现相同问题。原来问题在于,使用Compress-Archive压缩带有文件夹的内容时,压缩没有问题;但用unzip去解压缩时,就会遇到“use backslashes as path separators”问题。进一步测试发现,在Windows中如果通过右击内容,选择"Compress(zipped) folder"的方式,或者通过7-zip来压缩的方式,unzip都可以完美解压。

PS C:\Users\Administrator> Compress-Archive

cmdlet Compress-Archive at command pipeline position 1

Supply values for the following parameters:

Path[0]: C:\Users\Administrator\file1.log

Path[1]: C:\Users\Administrator\lab\

Path[2]:

DestinationPath: C:\Users\Administrator\package.zip

PS C:\Users\Administrator> C:\symphony\soam\7.3\w2k3_x64-vc7-psdk\bin\utils\unzip.exe -o -q .\package.zip

warning: ./package.zip appears to use backslashes as path separators

PS C:\Users\Administrator> C:\symphony\soam\7.3\w2k3_x64-vc7-psdk\bin\utils\unzip.exe -o -q .\package.zip

warning: ./package.zip appears to use backslashes as path separators

PS C:\Users\Administrator> Compress-Archive

所以,严格地说,这不是Symphony的问题,是Windows的Compress-Archive工具的问题。一顿Google,果然找到Powershell报告这是一个Microsoft.PowerShell.Archive的bug,在版本1.2.3.0得到修复,详情请参考https://github.com/PowerShell/Microsoft.PowerShell.Archive/issues/48

由于客户坚持要用Microsoft.PowerShell.Archive去压缩,所以解决方法只能是升级Microsoft.PowerShell.Archive,方法请参考https://www.powershellgallery.com/packages/Microsoft.PowerShell.Archive/1.2.3.0

 

下面是我在实验室里升级Microsoft.PowerShell.Archive和验证问题得到解决的操作:

PS C:\Users\egoadmin> Install-Module -Name Microsoft.PowerShell.Archive -RequiredVersion 1.2.3.0 

NuGet provider is required to continue

PowerShellGet requires NuGet provider version '2.8.5.201' or newer to interact with NuGet-based repositories. The NuGet

provider must be available in 'C:\Program Files\PackageManagement\ProviderAssemblies' or

'C:\Users\egoadmin\AppData\Local\PackageManagement\ProviderAssemblies'. You can also install the NuGet provider by

running 'Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force'. Do you want PowerShellGet to install

and import the NuGet provider now?

[Y] Yes [N] No [S] Suspend [?] Help (default is "Y"): y

Untrusted repository

You are installing the modules from an untrusted repository. If you trust this repository, change its

InstallationPolicy value by running the Set-PSRepository cmdlet. Are you sure you want to install the modules from

'PSGallery'?

[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "N"): A

PS C:\Users\egoadmin>

PS C:\Users\egoadmin>

PS C:\Users\egoadmin> Get-Module -ListAvailable -Name Microsoft.PowerShell.Archive

Directory: C:\Program Files\WindowsPowerShell\Modules

ModuleType Version Name ExportedCommands

---------- ------- ---- ----------------

Manifest 1.2.3.0 Microsoft.PowerShell.Archive {Compress-Archive, Expand-Archive}

Directory: C:\windows\system32\WindowsPowerShell\v1.0\Modules

 

ModuleType Version Name ExportedCommands

---------- ------- ---- ----------------

Manifest 1.0.1.0 Microsoft.PowerShell.Archive {Compress-Archive, Expand-Archive}

PS C:\Users\egoadmin\Downloads> dir

Directory: C:\Users\egoadmin\Downloads

Mode LastWriteTime Length Name

---- ------------- ------ ----

d----- 6/1/2020 7:53 AM lab

d----- 6/1/2020 7:42 AM unzip552

-a---- 6/1/2020 7:53 AM 38 1234.txt

-a---- 6/1/2020 7:38 AM 1295576 ChromeSetup.exe

-a---- 6/1/2020 7:40 AM 1305781 unzip552.zip

PS C:\Users\egoadmin\Downloads> Compress-Archive 

cmdlet Compress-Archive at command pipeline position 1

Supply values for the following parameters:

Path[0]: lab

Path[1]: 1234.txt

Path[2]:

DestinationPath: 1.zip

PS C:\Users\egoadmin\Downloads> dir

Directory: C:\Users\egoadmin\Downloads

Mode LastWriteTime Length Name

---- ------------- ------ ----

d----- 6/1/2020 7:53 AM lab

d----- 6/1/2020 7:42 AM unzip552

-a---- 6/1/2020 8:03 AM 290 1.zip

-a---- 6/1/2020 7:53 AM 38 1234.txt

-a---- 6/1/2020 7:38 AM 1295576 ChromeSetup.exe

-a---- 6/1/2020 7:40 AM 1305781 unzip552.zip

C:\Users\Administrator\Downloads>E:\SpectrumComputing\soam\7.2\w2k3_x64-vc7-psdk\bin\utils\unzip.exe -o -q 1.zip 

C:\Users\Administrator\Downloads>dir

Volume in drive C has no label.

Volume Serial Number is 74B6-4C2C

Directory of C:\Users\Administrator\Downloads

06/01/2020 12:54 AM <DIR> .

06/01/2020 12:54 AM <DIR> ..

06/01/2020 01:06 AM 290 1.zip

06/01/2020 07:53 AM 38 1234.txt

06/01/2020 12:54 AM <DIR> lab

2 File(s) 328 bytes

3 Dir(s) 4,173,824 bytes free

 

 

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章