Skip to content

Commit e619a12

Browse files
committed
Static protogen site: WASM core plus TypeScript UI
Replaces the ASP.NET-hosted site in the protobuf-net repo, which was stranded on netcoreapp3.1/netstandard2.1 by changes to how Blazor projects work. The .NET side targets net10.0-browser through Microsoft.NET.Sdk.WebAssembly rather than Blazor, so it is coupled only to the WASM runtime. It exposes three [JSExport] methods and nothing else; the UI is plain TypeScript with CodeMirror 6 for both editors. Publish output is a folder of static files, which is what moves this off active hosting. Schema parsing and code generation come from the published protobuf-net.Reflection package. Imports of google/** and protobuf-net/** resolve from resources embedded in it, so the old server round-trip for import resolution is gone. Payload decomposition is new (WireWalker.cs) and does not use ProtoReader. An analysis tool needs exact byte offsets per component and partial results on malformed input; walking the span directly gives both. Length-delimited fields are ambiguous on the wire, so each field leads with a best guess and keeps the alternatives behind a disclosure, with heuristics to suppress the readings that are technically valid but almost always wrong. C# and VB.NET only. The protoc-backed languages needed a server-side native executable; protoc-as-WASM was investigated and parked, with the reasoning recorded in the README.
0 parents  commit e619a12

34 files changed

Lines changed: 4290 additions & 0 deletions

‎.gitattributes‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
* text=auto eol=lf
2+
3+
*.png binary
4+
*.ico binary
5+
*.woff binary
6+
*.woff2 binary

‎.github/workflows/deploy.yml‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
name: Build and deploy
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
workflow_dispatch:
9+
10+
permissions:
11+
contents: read
12+
13+
concurrency:
14+
group: pages
15+
cancel-in-progress: true
16+
17+
jobs:
18+
build:
19+
runs-on: ubuntu-latest
20+
steps:
21+
- uses: actions/checkout@v4
22+
23+
- uses: actions/setup-dotnet@v4
24+
with:
25+
dotnet-version: '10.0.x'
26+
27+
- uses: actions/setup-node@v4
28+
with:
29+
node-version: '22'
30+
cache: npm
31+
cache-dependency-path: web/package-lock.json
32+
33+
- name: Install
34+
working-directory: web
35+
run: npm ci
36+
37+
# publishes the WASM project, copies _framework into public/, typechecks, then bundles
38+
- name: Build
39+
working-directory: web
40+
run: npm run build
41+
42+
- name: Upload artifact
43+
if: github.event_name != 'pull_request'
44+
uses: actions/upload-pages-artifact@v3
45+
with:
46+
path: web/dist
47+
48+
deploy:
49+
needs: build
50+
if: github.event_name != 'pull_request'
51+
runs-on: ubuntu-latest
52+
permissions:
53+
pages: write
54+
id-token: write
55+
environment:
56+
name: github-pages
57+
url: ${{ steps.deployment.outputs.page_url }}
58+
steps:
59+
- id: deployment
60+
uses: actions/deploy-pages@v4

‎.gitignore‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# .NET
2+
bin/
3+
obj/
4+
*.user
5+
6+
# node
7+
node_modules/
8+
9+
# build output
10+
web/dist/
11+
12+
# generated by web/scripts/build-wasm.mjs from the .NET publish; never commit
13+
web/public/_framework/

‎LICENCE.txt‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
The core Protocol Buffers technology is provided courtesy of Google.
2+
At the time of writing, this is released under the BSD license.
3+
Full details can be found here:
4+
5+
http://code.google.com/p/protobuf/
6+
7+
The protobuf-net logo is Copyright 2021 onwards Marc Gravell; you must
8+
not use this logo (or any derivative) in your own works without explicit
9+
permission.
10+
11+
This .NET implementation is Copyright 2008 onwards Marc Gravell
12+
13+
Licensed under the Apache License, Version 2.0 (the "License");
14+
you may not use this file except in compliance with the License.
15+
You may obtain a copy of the License at
16+
17+
http://www.apache.org/licenses/LICENSE-2.0
18+
19+
Unless required by applicable law or agreed to in writing, software
20+
distributed under the License is distributed on an "AS IS" BASIS,
21+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
22+
See the License for the specific language governing permissions and
23+
limitations under the License.

‎README.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# protogen site
2+
3+
The site behind **[protogen.marcgravell.com](https://protogen.marcgravell.com)**: generate C# and
4+
VB.NET from `.proto` schemas, and pull apart raw protobuf payloads without a schema.
5+
6+
Everything runs in the browser. There is no server, no API and no upload — schemas and payloads
7+
never leave the machine they were pasted on. The site is a folder of static files.
8+
9+
This replaces the older ASP.NET-hosted site that lived in the
10+
[protobuf-net](https://github.com/protobuf-net/protobuf-net) repo under `src/protogen.site`.
11+
12+
## How it works
13+
14+
| Layer | What it is |
15+
| --- | --- |
16+
| `src/ProtoGen.Wasm` | .NET 10 targeting `net10.0-browser` via `Microsoft.NET.Sdk.WebAssembly` — **not** Blazor. Exposes three `[JSExport]` methods and nothing else. |
17+
| `web` | Plain TypeScript + Vite. CodeMirror 6 for both editors. No UI framework. |
18+
19+
The schema work is done by the published [`protobuf-net.Reflection`](https://www.nuget.org/packages/protobuf-net.Reflection)
20+
package — the same parser and generators as the `protogen` command-line tool. Imports of
21+
`google/**` and `protobuf-net/**` resolve from resources embedded in that package, so the common
22+
cases need no network access.
23+
24+
The payload decomposition is local to this repo (`WireWalker.cs`). It deliberately does not use
25+
`ProtoReader`: an analysis tool needs exact byte offsets for every component and partial results on
26+
malformed input, neither of which a serializer's reader is built to give.
27+
28+
Total download is roughly 1.7 MB brotli-compressed, most of which is the .NET runtime, cached
29+
after first visit.
30+
31+
## Building
32+
33+
Requires the .NET 10 SDK and Node 20+.
34+
35+
```sh
36+
cd web
37+
npm install
38+
npm run dev # publishes the WASM project, then starts Vite on :5180
39+
npm run build # -> web/dist, ready to serve as static files
40+
```
41+
42+
`npm run wasm` alone re-publishes the .NET side into `web/public/_framework`. That folder is
43+
generated and git-ignored; it is copied verbatim rather than bundled, because the .NET boot process
44+
resolves its own content-hashed filenames.
45+
46+
## Deployment
47+
48+
GitHub Actions builds on push to `main` and publishes `web/dist` to GitHub Pages
49+
(`.github/workflows/deploy.yml`).
50+
51+
Two details that matter for Pages:
52+
53+
- `web/public/CNAME` carries the custom domain. DNS needs a `CNAME` record for `protogen`
54+
pointing at `protobuf-net.github.io`. Delete the file to test on the default
55+
`*.github.io` URL first.
56+
- `web/public/.nojekyll` stops Jekyll stripping the `_framework` directory, which would otherwise
57+
be ignored for starting with an underscore. The artifact-based deploy does not run Jekyll, but
58+
the file guards against a future switch to branch-based publishing.
59+
60+
## Scope
61+
62+
**C# and VB.NET only.** The old site also offered C++, Java, JavaScript, Objective-C, PHP, Python
63+
and Ruby by shelling out to `protoc` on the server. `protoc` is a native executable and cannot run
64+
client-side, so those targets are gone rather than quietly broken.
65+
66+
Compiling `protoc` itself to WebAssembly was investigated and parked. There is no maintained build:
67+
`kwonoj/protobuf-wasm` and `mjz20/protobuf_wasm` patch the protobuf *runtime* for Emscripten and
68+
explicitly do not build the compiler, and the one recent attempt at the compiler proper
69+
([protobuf#20819](https://github.com/protocolbuffers/protobuf/issues/20819)) produces a `protoc.js`
70+
that builds but fails code generation on path resolution. The real work is MEMFS plumbing for
71+
`--proto_path` and generated outputs, unmaintained upstream, on top of a C++ binary that links
72+
every language generator and would likely dwarf the .NET runtime.
73+
74+
If a target language is ever worth restoring, JavaScript/TypeScript is the cheap one: `protobufjs`
75+
and `@bufbuild/protobuf` generate it in pure JS with no `protoc` involved.
76+
77+
See [`docs/trimming.md`](docs/trimming.md) for why the build suppresses `IL2104`.
78+
79+
## Licence
80+
81+
Apache-2.0, matching protobuf-net.

‎docs/trimming.md‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# Trimming and the `IL2104` suppression
2+
3+
`ProtoGen.Wasm.csproj` publishes with `PublishTrimmed` and `TrimMode=full`, and suppresses
4+
`IL2104` ("assembly produced trim warnings"). This note records what was checked, so the
5+
suppression is a decision rather than a shrug.
6+
7+
## What warns, and why it does not matter here
8+
9+
The warnings come from `protobuf-net.Core`, and every one of them is on the *reflection-based
10+
serializer* — the machinery that builds a serializer for an arbitrary `[ProtoContract]` type at
11+
runtime:
12+
13+
| Location | Warning | Path |
14+
| --- | --- | --- |
15+
| `Internal/DynamicStub.cs` | `IL2055`, `IL2067`, `IL2070` | `MakeGenericType` / `Activator.CreateInstance` for runtime-built serializers |
16+
| `Helpers.cs` | `IL2070` | `Type.GetConstructor` |
17+
| `Internal/TypeHelperT.cs` | `IL2070` | `Type.GetInterfaces` for collection discovery |
18+
| `Meta/TypeModel.cs` | `IL2067` | `Activator.CreateInstance` for list and auxiliary types |
19+
| `ProtoWriter.State.WriteMethods.cs` | `IL2098` | a `DynamicallyAccessedMembers` attribute on a non-`Type` parameter |
20+
21+
This site never reaches that code. It uses two things from the package graph:
22+
23+
1. **`protobuf-net.Reflection`'s parser and code generators** — plain object-graph work over the
24+
descriptor model, no runtime type resolution.
25+
2. **`CustomProtogenSerializer`** — for reading protobuf-net's own custom options out of a
26+
descriptor. That is a hand-written `TypeModel` with explicit `ISerializer<T>` implementations
27+
for every descriptor type, so it resolves serializers statically and never consults the
28+
reflection path above.
29+
30+
The site does not serialize user types, so `RuntimeTypeModel` is never touched.
31+
32+
## The one warning that is on our path
33+
34+
`protobuf-net.Reflection/TokenExtensions.cs` raises `IL2090`: `EnumCache<T>` calls
35+
`Type.GetFields` on `T` without a `[DynamicallyAccessedMembers(PublicFields)]` annotation. This
36+
*is* on the parse path — it backs enum parsing in the `.proto` DSL.
37+
38+
It survives today because the trimmer keeps enum fields in practice, and the enums in question are
39+
referenced directly from code the trimmer can see. It is nonetheless an annotation gap in the
40+
library rather than something this repo can fix locally; the proper fix is a one-line attribute
41+
upstream in protobuf-net.
42+
43+
**If enum-valued options in schemas ever start failing to parse in a published build, look here
44+
first.**
45+
46+
## How to re-check
47+
48+
```sh
49+
dotnet publish src/ProtoGen.Wasm -c Release 2>&1 | grep -E "IL[0-9]{4}"
50+
```
51+
52+
If a warning appears from an assembly or a code path not listed above, do not extend the
53+
suppression — work out whether the new path is actually reachable first.

‎global.json‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"sdk": {
3+
"version": "10.0.100",
4+
"rollForward": "latestFeature",
5+
"allowPrerelease": false
6+
}
7+
}

‎src/ProtoGen.Wasm/Codegen.cs‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
using Google.Protobuf.Reflection;
2+
using ProtoBuf.Reflection;
3+
4+
namespace ProtoGen.Wasm;
5+
6+
/// <summary>
7+
/// Wraps protobuf-net.Reflection's parser and code generators.
8+
/// </summary>
9+
/// <remarks>
10+
/// Imports of <c>google/**</c> and <c>protobuf-net/**</c> resolve from resources embedded in
11+
/// protobuf-net.Reflection, so the common cases need no network access and no server. Any other
12+
/// import fails with protobuf-net's own "unable to find" diagnostic, which is the honest answer:
13+
/// there is no filesystem here to find it on.
14+
/// </remarks>
15+
internal static class Codegen
16+
{
17+
public static GenerateResponse Generate(GenerateRequest request)
18+
{
19+
var response = new GenerateResponse();
20+
try
21+
{
22+
var set = new FileDescriptorSet();
23+
string fileName = string.IsNullOrWhiteSpace(request.FileName) ? "my.proto" : request.FileName.Trim();
24+
set.Add(fileName, includeInOutput: true, new StringReader(request.Schema ?? ""));
25+
set.Process();
26+
27+
bool fatal = false;
28+
foreach (var error in set.GetErrors())
29+
{
30+
response.Errors.Add(new SchemaError
31+
{
32+
IsError = error.IsError,
33+
LineNumber = error.LineNumber,
34+
ColumnNumber = error.ColumnNumber,
35+
Length = error.Text?.Length ?? 0,
36+
Message = error.Message,
37+
ErrorNumber = error.ErrorNumber,
38+
File = error.File ?? "",
39+
});
40+
if (error.IsError) fatal = true;
41+
}
42+
43+
// warnings are worth showing alongside output; errors mean there is no output to show
44+
if (fatal) return response;
45+
46+
var generator = ResolveGenerator(request.Language);
47+
var normalizer = ResolveNormalizer(request.NamingConvention);
48+
foreach (var file in generator.Generate(set, normalizer, BuildOptions(request)))
49+
{
50+
response.Files.Add(new GeneratedFile { Name = file.Name, Text = file.Text });
51+
}
52+
}
53+
catch (Exception ex)
54+
{
55+
response.Exception = $"{ex.GetType().Name}: {ex.Message}";
56+
}
57+
return response;
58+
}
59+
60+
private static CodeGenerator ResolveGenerator(string? language) => language?.ToLowerInvariant() switch
61+
{
62+
"csharp" or "c#" or null or "" => CSharpCodeGenerator.Default,
63+
"vb" or "vbnet" or "vb.net" => VBCodeGenerator.Default,
64+
_ => throw new ArgumentException(
65+
$"'{language}' is not supported. This tool generates C# and VB.NET via protobuf-net's own " +
66+
"generators; the other languages protoc supports need protoc itself, which cannot run in a browser."),
67+
};
68+
69+
private static NameNormalizer ResolveNormalizer(string? convention) => convention?.ToLowerInvariant() switch
70+
{
71+
"original" or "none" => NameNormalizer.Null,
72+
"noplural" => NameNormalizer.NoPlural,
73+
_ => NameNormalizer.Default,
74+
};
75+
76+
private static Dictionary<string, string> BuildOptions(GenerateRequest request)
77+
{
78+
var options = new Dictionary<string, string>();
79+
if (!string.IsNullOrWhiteSpace(request.LanguageVersion)) options["langver"] = request.LanguageVersion!;
80+
if (request.Services) options["services"] = "yes";
81+
if (request.OneOfEnum) options["oneof"] = "enum";
82+
if (request.ListSet) options["listset"] = "yes";
83+
if (request.DisableNullWrappers) options["nullwrappers"] = "false";
84+
if (request.DisableCompatLevel) options["compatlevel"] = "false";
85+
if (request.NullableValueType) options["nullablevaluetype"] = "yes";
86+
if (request.RepeatedAsList) options["repeatedaslist"] = "yes";
87+
return options;
88+
}
89+
}

0 commit comments

Comments
 (0)