GeneratorHarness can't tell whether generated sources compile: a duplicate type across two generated files, or a reference to a missing type, comes back with 0 diagnostics #50

What's wrong

GeneratorHarness.Drive (SourceGeneratorToolkit.Testing/GeneratorHarness.cs:187-194) calls driver.RunGenerators(CreateCompilation()).GetRunResult(). The updated compilation is thrown away, so the harness only ever reports:

• GeneratorRunResult.Diagnostics, which holds only the diagnostics the generator reports itself.
• Per-tree SyntaxTree.GetDiagnostics(), which a consumer can check for syntax errors only.

A consumer therefore has no way to check that the generated output compiles. There's no RunGeneratorsAndUpdateCompilation and no access to the output compilation.

There's also no way to pass metadata references, user source trees or parse options. CreateCompilation (:196-201) references only CoreLib, has no syntax trees and uses default parse options.

Reproduction

I wrote a scratch generator that emits two files:

• A.g.cs: namespace N { public class Dup { } }
• B.g.cs: namespace N { public class Dup { public Missing M; } }

That's CS0101 (duplicate type) and CS0246 (missing type). Running it through harness.Run(...) gives:

exception=none diagnostics=0 sources=2
A.g.cs: syntax diags=0
B.g.cs: syntax diags=0

Why it matters

The README presents the harness as "a test harness that ships", so consumers don't have to rebuild the CSharpGeneratorDriver plumbing. But the failures this repo has had to fix are of the "the generated file doesn't compile" kind: #26, and #39 which is still open.

The regression test for #26 falls back to CSharpSyntaxTree.ParseText(source).GetDiagnostics() (SourceGeneratorToolkit.Test/GeneratorBaseTests.cs:232), which only catches syntax errors. Semantic breakage passes every harness-based test and only shows up in the consumer's real build. That includes duplicate types across generated files, unresolved names and a missing partial.

And because user source can't be supplied, a generator that also reads syntax or attributes can't be driven through the harness at all. ForAttributeWithMetadataName is a typical example.

Suggested fix and acceptance criteria

• Add a harness entry point, such as RunAndCompile(...) or an OutputCompilationDiagnostics member on a result wrapper. It should use RunGeneratorsAndUpdateCompilation and return the output compilation's error diagnostics alongside the run result.
• Have the harness compilation reference the trusted platform assemblies, or accept caller-supplied MetadataReferences, so ordinary BCL types resolve. Optionally accept CSharpParseOptions/LangVersion and extra user source trees.
• Add a test with a fixture generator that emits a duplicate type, and assert that CS0101 is reported. Assert that the existing ThingsGenerator and PairGenerator outputs compile with zero errors.
• Document the new member in the README's GeneratorHarness table.