TL;DR

How to reference custom Roslyn code generators from Unity?
- How reliable are local "conventional" projects?
- How to reference "conventional" dotnet projects in Unity?
Can I make a Roslyn code generator a Unity package or at least an asmdef, and how? How would I use it?

In depth

I'm planning to use a custom Roslyn code generator in a Unity project.

I have found information on how to set up a code generator correctly using CodeGeneration.Roslyn. With this setup, it is easy to export the code generator as a NuGet package and import it in any other project.

The first question is whether the code generation via a NuGet package like that will even work under Unity 2020.2 LTS. I wonder how much of a hassle is it to achieve MSBuild project references like below within a Unity project. I'd like to avoid trying this myself, given it might not work at all, but I will do it anyway while awaiting an answer:

<ItemGroup>
    <PackageReference Include="CodeGeneration.Roslyn.Tool" Version="0.7.63">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers;buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    
    <!-- This ProjectReference to the generator needs to have the OutputItemType metadata -->
    <ProjectReference Include="..\Duplicator.Generators\Duplicator.Generators.csproj"
                      ReferenceOutputAssembly="false"
                      SetTargetFramework="TargetFramework=net4.7.2"
                      OutputItemType="CodeGenerationRoslynPlugin" />

    <ProjectReference Include="..\Duplicator.Attributes\Duplicator.Attributes.csproj"
                      ExcludeAssets="runtime"
                      SetTargetFramework="TargetFramework=net4.7.2"
                      PrivateAssets="all" />
    
</ItemGroup>

The problem with this approach is that the code generator will then not be within the Unity project. The NuGet export indirectness will make it harder to debug the generator.

So, the second idea is to reference a local project with the code generator within the Unity repository, however I am not sure if this is at all possible. All .csproj files in Unity are autogenerated and I do not know how well it behaves with such embedded "conventional" projects.

Also, it is not clear where such sub-projects should be located within the project structure. I guess the simplest solution would be to put the code generator projects at a level with the entire Unity project folder to avoid conflicts. But then it is not clear whether one may reference cs projects from outside the project folder and how that plays out with Unity. I know Godot does this well, but I'm unsure about Unity.

Another issue with this is that such project structure is unconventional and just confusing. AFAIK Unity has its own package manager and encourages structuring the project into smaller .asmdef sub-projects and/or packages, however, I'm unsure whether one could distribute a Roslyn analyzer, let alone a Roslyn code generator, this way. However, this solution seems most appealing to me, since it would allow me to keep the code generator as a sub-project within the main Unity project, without having to nest repositories (at least in the beginning of development, it could be trivially refactored into a separate repo via git submodule afterwards), and with the ability for my code generator to depend on Unity API's if needed. The question here is whether it is possible to write a Roslyn code generator as a Unity package.

Now, I know the new 2021 beta has C# 9 support along with the new Code Generator feature, however, I'm reluctant to use the beta because it sure is buggy and therefore frustrating.

I'm also wondering whether there is some Unity built-in API for this task, perhaps an undocumented one. I know that Unity uses Roslyn to compile the code, so it might as well expose the syntax tree and allow code changes.

If this sort of code generation or referencing other projects is not supported, I could always default to a Roslyn script that would manually scan the sources for specific attributes, and emit code files into a specific folder in Unity. The problem with this approach is that it is unbearably slow and I predict the meta file stuff would cause conflicts eventually. To sync the meta files I would have to check in the autogen folder to source control, which I would like to avoid. So I would hate to have to fall back to this option (I'd rather try the beta).

PS. Ok, maybe it isn't necessarily slow. Many other tools use this method like this one. Anyway, it is more work.

The official Unity documentation only has info on how to use the analyzers. There is no advice on creating your own ones, and not a word about code generation.

Update 1

The actual references are going to be in fact a lot simpler. The same repository gives an example on creating and linking a metapackage. So, the code for using the code generator in a consuming project would actually be similar to this:

<ItemGroup>
    <PackageReference Include="Kari" Version="1.0.0"/>
</ItemGroup>

It still ignores some details regarding how to trick NuGet into using a local folder as its package repository. Using their samples, I was able to figure out how to make it work. See this.

Right now I'm faced with the problem of referencing the code generator NuGet package from Unity. I'm going to try NuGetForUnity, but I'm not completely sure if it will work, given the dependency is a build-time one.

Update 2

Tried NuGetForUnity. Apparently all it does is it dumps all dll's of the referenced packages in a folder, and then has Unity blindly import everything from the folder, be it a compile-time dependency or a runtime one. I tried to just place my precompiled nuget packages into the package repository folder, and Unity gave me errors about importing multiple copies of the same dll (because the tool has a copy of the shared project, plus some project target multiple dotnet versions). If I then refresh the packages from the UI, my copied folders simply get deleted.

See the repo.

I'm going to try searching for ways to affect what .csproj unity generates. If that doesn't work, I'll have to default to generating code with a prebuild step.

Update 3

Decided to write my own stand-alone code generator. I started with this repo as the basis, gradually adding my own features.

Then I decided to decouple my code generator from the main project, which I did. Basically, I have a MasterEnvironment, which keeps track of the individual asmdef's in the unity project and splits them into smaller project environments. It has a list of Administrators. Each one of these defines a function for each stage of the process: initialization, analysis (info collection), callbacks (relations to data from other administrators), code generation. I load these dynamically from precompiled dll's, which I call plugins when launching the code generator, by passing paths to these dll's as arguments. Thus the code generator is decoupled from the plugins that actually analyze and generate the code.

This architecture made it really easy to parallelize computation between both plugins and the individual subprojects. As a result, the code generator turned out decently fast.

Update 4

The code generator is in a separate repo now. Find it here.