Protobuf-net on UWP/.NET Native and iOS

Question

I have a Xamarin.Forms App based on .NET Standard 1.4 that uses protobuf-net to store objects in the database that will be sent to a WCF service at a later time.

On Android and UWP "managed" everything works fine but - after searching through repositories, articles and blogposts that can no longer be accessed, and also after trying to get the precompilation tool to work, but failing at that - I have one simple (probably not) question: How do I get protobuf-net to work in "restricted" environments like UWP/.NET Native and iOS/Xamarin?

Sobersided

I've managed to do this once for Xamarin\IOS app (not Forms, but shouldn't matter). Was not easy and I used precompilation tool indeed. Unfortunately I don't have access to this codebase right now, just so that you know it's possible. – Marshy

I know that it must be possible, but I can't figure out how :( – Sobersided

Same here, I did use protobuf-net on iOS, but neither did set it up for the project, nor have access to the code anymore. Did you read this answer? – Thane

Yes, but my problem is, that I can't get the project for the precompile tool to compile. Also any links to his blog are dead. – Sobersided

@Sobersided blog links should work fine - any specific things that are 404? – Waterfowl

@MarcGravell Firefox and Chorme (on different machines) are telling me, that the connection was closed while loading the website. Chrome specifically says ERR_CONNECTION_CLOSED. All Plugins (Like Adblock, Noscript, etc.) are off. – Sobersided

@Sobersided great: what url are you using? – Waterfowl

@MarcGravell I'm using https://blog.marcgravell.com but I just tested it with chorme on my mobile phone which works just fine. I then used an hotspot to connect a machine through my phone to the internet, and it did load aswell. Seems to be an internal issue. – Sobersided

@Sobersided yeah, I'm seeing traffic to that, so if there's a problem: it isn't everywhere – Waterfowl

@Sobersided what about going for google implementation of protocol buffers in c#? – Faggoting

Answer 1

Right now I don't have a great solution for this scenario. I know some people have made it work, but I'm not expert enough in UWP / Native / iOS to give you reliable "here's the path to success" instructions.

UWP / .NET Native and iOS share (as you know) a common issue: lack of full runtime emit. I understand why this is. It is just: tricky.

Historically, protobuf-net has tried to solve this problem via a build tool that repeated the existing IL-emit usually done at runtime - as a build-time tool. This was ugly and nasty, but it worked. Kind of. To hack around some platform restrictions, protobuf-net used some of the IKVM tooling to help with this, but as the .NET framework scene has continued to expand this is basically not viable. Plus: the IKVM tool is now abandoned and won't be being maintained.

In parallel with this, there is increasing impetus to investigate some newer concepts:

• full async/await for asynchronous IO sources: note that this is extremely unfriendly to IL emit, but is almost embarrassingly easy to implement in C#
• "pipelines" / "channels" / "streams 2" - whatever it is being called this week; but: the new allocation-free IO concept that is being used in Kestrel (I helped kick this ball around a little bit when it was in the early stages, so I'm familiar with what needs doing) - note that this also ties into async/await
• and of course: how all of the above relates to pre-generation

Right now, I'm very much of the opinion that the best route forward is for the pre-gen scenario to switch to emitting C# via build-time tooling. I have repeatedly petitioned MS for improved automated C# emit based on Roslyn, but so far: no joy (vexingly: the asp.net stuff even had a fully working proof-of-concept, but it is shelved). So right now I'm thinking: we need to assume that isn't going to happen, and basically write it independently. This isn't necessarily as complex as it sounds (and: codegen of various forms is very familiar to me). The advantage of C# emit here is that I don't need to fight the intricacies of every framework - I just need to make it compile (well, and run, obviously).

So: what's holding me back? In theory: nothing. I just need to get this stuff written and deployed. In reality: life, time, etc. I am guilty of prioritising things that impact me daily, and the reality is that I'm not really a daily user of those platforms, which means I'm not feeling the pain that you're feeling. But: I hear you loud and clear, and I am trying to ramp up the v3 work that should address these points. I genuinely want to have a good story for those things - and my aim is that by moving to a C#-emit model (for pre-gen, at least): it helps me. And if it helps me I know it won't be the forgotten toy in the attic / basement that I know is there but which it is hard to find the motivation to go to the trouble of finding.

Waterfowl

So, for short, you are telling me either to get precompile working, or wait until you've finished v3 of Protobuf-net? – Sobersided

@Sobersided I don't know; but my educated guess is that the work for me to revisit precompile in the context of each of those frameworks is comparable or larger than the work for me to get a viable C# emit – Waterfowl

So, do you have an ETA for this new version, which is supposed to work on all plattforms? – Sobersided

@Sobersided no, I don't; that's where the "life, time, etc" comes in; ultimately my day job isn't to write libraries, so this all comes out of my free time (unless it is something that I can very genuinely say "yup, this is hurting my employer and fixing this would help them" - which doesn't really apply to UWP / iOS / etc). I don't think there's a vast amount of work involved, but: the time of year means it is quite busy with family things etc. So, no: I don't have an ETA that I would commit to – Waterfowl

Okay. I have a deadline right ahead,so I'll be switching to like json or something temporarily. But I'll be watching protobuf-net closely and switch as soon as possible. Have a nice time with your family :D – Sobersided