Building under Manjaro Linux

[obones]

This ticket is here to discuss what I had to do to get a build working on my Manjaro Linux, on top of the changes that I already submitted as pull requests.
I am not that confident about the changes which is why I want to discuss them here first.

Initial run gives this error:

GitVersion: Process returned an error (exit code 1)

This is fixed by editing Tools\Cake\GitVersion.CommandLine.4.0.0\tools\LibGit2Sharp.dll.config and replacing the os="linux" line with this one:

<dllmap os="linux" dll="git2-15e1193" target="libgit2.so" />

I got a hint of this by looking at this message but I'm not sure this can be fixed by CCNet itself

Then, I had to modify ccnet.build at line 177 because it says mono here. However, doing this, it tries to run a binary called mono from the project folder. The exact error message is this:

'/path/to/checkout/CruiseControl.Net/project/mono' failed to start

I thus replaced mono with ${msbuild.file} just like the line above, but it failed a bit later with msbuild complaining that it was receiving multiple projects to build.
This comes form the fact that there is this line as the first argument for the execution:

<arg value="msbuild.file" if="${platform::is-unix()}"/>

Well, I believe it was intended to be ${msbuild.file} instead of just msbuild.file but in any case, it's not required at all as it gets interpreted as the name of a project to build.
After having removed that arg line, I was able to run the build.sh script to completion.

Any thoughts/opinions on this are welcome

[obones]

Ok, I went further and tried running the tests.
At first, I received this error:

System.Runtime.Remoting.RemotingException : Tcp transport error

Looking around, I found this issue: OmniSharp/omnisharp-roslyn#1548

And sure enough, adding --inprocess to the arguments for nunit at line 266 of ccnet.build allowed the tests to run to completion.

Now, on to figure out why 184 of them failed (91 failures, 93 errors)

[savornicesei]

I'm running latest openSUSE Tumbleweed, 20210113 snapshot with kernel 5.10.5-1 and mono:

Mono JIT compiler version 6.12.0.107 (tarball Mon Dec 28 10:00:47 UTC 2020)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug 
        Interpreter:   yes
        LLVM:          supported, not enabled.
        Suspend:       hybrid
        GC:            sgen (concurrent by default)

I have mono-complete pattern installed and I'm running ./build.sh --target=build

Issue 1: NSIS issue from PR #301
Encountered. Fixed by changes in PR #301

Issue 2: GitVersion: Process returned an error (exit code 1)
Not encountered

Issue 3 <arg value="msbuild.file" if="${platform::is-unix()}"/>
You are correct. it was supposed to be ${msbuild.file} because the full command should be mono /usr/lib/mono/msbuild/15.0/bin/MSBuild.dll ccnet.sln

Removing the <arg value="msbuild.file" if="${platform::is-unix()}"/> line gives the following error:

build:

     [echo] Build configuration: Build
     [echo] Source Directory: /data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/project
     [echo] Build Directory: /data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/Build
     [echo] MSBuild path: mono
     [exec] Cannot open assembly '/data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/project/ccnet.sln': File does not contain a valid CIL image.

BUILD FAILED - 0 non-fatal error(s), 1 warning(s)

/data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/ccnet.build(180,10):
External Program Failed: mono (return code was 2)

Removing the basedir="${src.dir}" started the build process just to fail a little later with

     [echo] Build configuration: Build
     [echo] Source Directory: /data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/project
     [echo] Build Directory: /data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/Build
     [echo] MSBuild path: mono
     [exec] Microsoft (R) Build Engine version 16.0.0.0 for Mono
     [exec] Copyright (C) Microsoft Corporation. All rights reserved.
     [exec] 
     [exec] Build started 16.01.2021 01:17:48.
     [exec] Project "/data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/project/ccnet.sln" on node 1 (Clean;Build target(s)).
     [exec] ValidateSolutionConfiguration:
     [exec]   Building solution configuration "Build|Any CPU".
     [exec] Project "/data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/project/ccnet.sln" (1) is building "/data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/project/PowerShell/PowerShell.csproj" (2) on node 1 (Clean target(s)).
     [exec] /usr/lib/mono/msbuild/15.0/bin/Microsoft.CSharp.CurrentVersion.targets(331,5): error MSB4019: The imported project "/usr/lib/mono/msbuild/15.0/bin/Roslyn/Microsoft.CSharp.Core.targets" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk. [/data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/project/PowerShell/PowerShell.csproj]
     [exec] Done Building Project "/data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/project/PowerShell/PowerShell.csproj" (Clean target(s)) -- FAILED.

This is a problem with RoslynTargetsPath which seems to point to /usr/lib/mono/msbuild/15.0/bin/Roslyn on my system.
It can be fixed by either making a soft link to /usr/lib/mono/msbuild/Current/bin/Roslyn or by sending the right path in MSBuild command line if running on linux:

<arg value="/v:n" /> <!-- verbosity levels: q[uiet], m[inimal], n[ormal] (default), d[etailed], and diag[nostic] -->
<arg value="/p:RoslynTargetsPath=${path::combine(directory::get-parent-directory(msbuild.file), '../../Current/bin/Roslyn')}" if="${platform::is-unix()}" />

Either ways seems to give the same horror:

/usr/lib/mono/msbuild/15.0/bin/Roslyn/Microsoft.Managed.Core.targets(60,27): error MSB4184: The expression ""../CommonAssemblyInfo.cs".GetPathsOfAllDirectoriesAbove()" cannot be evaluated. Method 'System.String.GetPathsOfAllDirectoriesAbove' not found. [/data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/project/WebDashboard/WebDashboard.csproj]
     [exec] 

There are some small additional adjustments to be done here: echo lines from line 170 to 174 should be moved at line 178 and last echo line should be <echo message="MSBuild.path: ${msbuild.file}" />

[obones]

Removing the line gives the following error

Thing is, this arg must be removed when ${msbuild.file} is also replaced to be the value returned by vswhere which on my system is /usr/bin/msbuild and automatically calls mono with the appropriate DLL. So in this case, there is no need to call mono directly.
On top of this, there does not seem to be a search in $PATH for the executable, so using just mono leads to it being searched for in the current folder. And because it is not there, I get the error mentioned above.

Clearly, it appears that depending on the distribution, some things are available and some others are not.

[savornicesei]

Hmmm...
It would not be the first time things are packaged different in openSUSE (thinking at libICU). Seems I need to test the changes on several distros. I'll check them out in the following days.

[savornicesei]

@obones
framework::get-msbuild-path(vswhere.executable)
can't return /usr/bin/msbuild since it checks hardcoded paths in /usr/lib (check Tools/ccnet-utils.inc.build - which follows the same logic as NuGet client in detecting MSBuild on linux).
If it's not too much trouble for you could you paste the script into a console app and check how it returns /usr/bin/msbuild?

L.E: Seems like my mono environment was broken. I was using the official mono-project mono packages and at some moment in time I switched to the openSUSE-provided ones without switching and upgrading msbuild, msbuild-libhostfxr and msbuild-sdkresolver to the ones provided by Mono:Factory.

Issue 3 from previous comment works by removing basedir="${src.dir}"

I'm thinking that perhaps we should probe first for msbuild in $PATH and if it does not exist then we call mono MSBuild.dll ccnet.sln
What do you think?

The tests fail for me (on a local branch containing your PRs #300, #304, #305, #306) with the following error:

/data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/ccnet.build(245,18):
The fileset specified is empty after scanning '/data/Proiecte/OSS/CruiseControl.NET/CruiseControl.NET/Tools/NUnit' for: NAnt.Core.StringCollectionWithGoodToString:
nunit.framework.dll

[obones]

Well, ok, here is the beginning of the log when building:

 [echo] MSBuild path: /files/download/Linux/CruiseControl.Net/Build
 [echo] MSBuild exe: /usr/lib/mono/4.5/../msbuild/15.0/bin/MSBuild.dll
 [exec] Starting '/usr/lib/mono/4.5/../msbuild/15.0/bin/MSBuild.dll (/files/download/Linux/CruiseControl.Net/project/ccnet.sln /t:Clean;Build /p:TargetFrameworkVersion=4.5;Configuration=Build /p:WarningsNotAsErrors=619 /p:SHFBROOT=/files/download/Linux/CruiseControl.Net/Tools/EWSoftware.SHFB.2019.11.17/tools /p:DISABLE_COM=True /p:DISABLE_JSCRIPT=True /p:MONO_IOMAP=all /p:MONO_PATH=/files/download/Linux/CruiseControl.Net/lib)' in '/files/download/Linux/CruiseControl.Net/project'
 [exec] Microsoft (R) Build Engine version 16.8.0-ci-20621-01 pour Mono
 [exec] Copyright (C) Microsoft Corporation. Tous droits réservés.

So clearly, I was wrong about /usr/bin/msbuld it's just that the DLL can be started all by itself, there is no need to put mono in front of it. How and why this works is a mystery to me.

But there is one thing for sure, using just mono as the command line does not work at all because it looks for a mono executable in the current folder, and there is none to be found. The log in that case is this:

     [echo] MSBuild path: /files/download/Linux/CruiseControl.Net/Build
     [echo] MSBuild exe: mono
     [exec] Starting '/files/download/Linux/CruiseControl.Net/project/mono (/files/download/Linux/CruiseControl.Net/project/ccnet.sln /t:Clean;Build /p:TargetFrameworkVersion=4.5;Configuration=Build /p:WarningsNotAsErrors=619 /p:SHFBROOT=/files/download/Linux/CruiseControl.Net/Tools/EWSoftware.SHFB.2019.11.17/tools /p:DISABLE_COM=True /p:DISABLE_JSCRIPT=True /p:MONO_IOMAP=all /p:MONO_PATH=/files/download/Linux/CruiseControl.Net/lib)' in '/files/download/Linux/CruiseControl.Net/project'

BUILD FAILED

/files/download/Linux/CruiseControl.Net/ccnet.build(182,10):
'/files/download/Linux/CruiseControl.Net/project/mono' failed to start.
    ApplicationName='/files/download/Linux/CruiseControl.Net/project/mono', CommandLine='/files/download/Linux/CruiseControl.Net/project/ccnet.sln /t:Clean;Build /p:TargetFrameworkVersion=4.5;Configuration=Build /p:WarningsNotAsErrors=619 /p:SHFBROOT=/files/download/Linux/CruiseControl.Net/Tools/EWSoftware.SHFB.2019.11.17/tools /p:DISABLE_COM=True /p:DISABLE_JSCRIPT=True /p:MONO_IOMAP=all /p:MONO_PATH=/files/download/Linux/CruiseControl.Net/lib', CurrentDirectory='/files/download/Linux/CruiseControl.Net/project', Native error= Cannot find the specified file

This is why I just use the ${msbuild.file} value directly.

[savornicesei]

So just removing the basedir attribute from exec does not work for you?