gRPC JSON Transcoding in .NET 7

gRPC is a great framework to build APIs. It's Google's take on Remote Procedure Call(RPC) and has official support in .NET. gRPC.NET is now the recommended framework for Remote Procedure Call requirements in the .NET world. gRPC benefits from great performance and is designed for HTTP/2 and beyond. However, it is not a good choice if you wish to have a browser-based client consuming your API. To consume a gRPC API, you need a generated gRPC client, and browsers don't provide the level of control required over web requests to support a gRPC client. But what if you have a browser-based application that wants to make use of this gRPC service? You can reuse/duplicate the code that your gRPC service uses into an HTTP API, but that would mean a lot of code duplicated and a lot of effort to maintain. This is the exact scenario where gRPC JSON Transcoding can prove useful. It lowers that barrier to entry for gRPC APIs, so that browser-based applications can use it but with no code duplication. You can extend any existing gRPC API to act as a HTTP API with this feature.

With a gRPC service, a client app talks to the service using HTTP/2 POST. The request and response are Protobuf with Google.Protobuf used for serialization and deserialization. Most importantly, the clients must use a generated client to talk to the service.

With gRPC JSON Transcoding, any application that can issue an HttpRequest can use the service. The application can issue a HTTP/1.1 or an HTTP/2 request using one of the well-known HTTP Methods. JSON is used as the message exchange format and the built-in System.Text.Json library is used for serialization and deserialization.

With this approach, the gRPC service can continue to function as usual, but it can also act as an HTTP API. There is no need to duplicate your code. You reuse the code that you have in place with your gRPC service but the API is extended to act as a HTTP API as well resulting in a wider reach for your API. In addition, it is a single ASP.NET Core app that functions as a gRPC as well as HTTP API, making your code maintainable and more manageable to host as well. 

Let us see this in action now. We first need a gRPC service in place, we will then extend it to implement gRPC JSON Transcoding. Christmas is almost around the corner, so my API will manage a list of Christmas messages. The messages are translations of  "Merry Christmas!" in various different languages.

I will start with the .proto file, which looks as shown below. My file is called festiveapi.proto. It contains a single service definition called the FestiveAdventService. It has multiple rpcs - to list messages, get a message, create a message, update a message and delete a message. 

syntax = "proto3";


option csharp_namespace = "FestiveAdventApi";

package festiveadvent;

import "google/protobuf/empty.proto";

service FestiveAdventService{

  rpc ListMessages(google.protobuf.Empty) returns (ListReply);
  rpc GetMessage(GetAdventMessageRequest) returns (AdventMessageReply);
  rpc CreateMessage(CreateAdventMessageRequest) returns (AdventMessageReply);
  rpc UpdateMessage(UpdateAdventMessageRequest) returns (AdventMessageReply);
  rpc DeleteMessage(DeleteAdventMessageRequest) returns (google.protobuf.Empty);
}

message CreateAdventMessageRequest{
  int32 id = 1;
  string message=2;
}

message GetAdventMessageRequest{
  int32 id =1;
}

message UpdateAdventMessageRequest{
  int32 id=1; //id is bound from route parameter
  string message=2;
}

message DeleteAdventMessageRequest{
  int32 id =1;
}

message ListReply{
  repeated AdventMessageReply AdventMessages = 1;
}

message AdventMessageReply{
  int32 id = 1;
  string message=2;
}

I have chosen the Visual Studio out-of-the-box Grpc Service Template as my starting point. It has all the Nuget packages in place (Grpc.AspNetCore). The tooling kicks in when I build my project with the above proto file and it generates code for me. I can now give my own implementations to the rpc methods.

using FestiveAdvent.Data;
using FestiveAdvent.Repository;
using FestiveAdventApi;
using Google.Protobuf.WellKnownTypes;
using Grpc.Core;

namespace FestiveAdvent.Services;

public class FestiveAdventApiService : FestiveAdventService.FestiveAdventServiceBase
{
    private readonly FestiveAdventRepository _festiveAdventRepository;


    public FestiveAdventApiService(FestiveAdventRepository festiveAdventRepository)
    {
        _festiveAdventRepository = festiveAdventRepository;
    }

    public override Task<ListReply> ListMessages(Empty request, ServerCallContext context)
    {
        var items = _festiveAdventRepository.GetAdventMessages();
        
        var listReply = new ListReply();
        var messageList = items.Select(item => new AdventMessageReply() { Id = item.Id,Message = item.Message }).ToList();
        listReply.AdventMessages.AddRange(messageList);
        return Task.FromResult(listReply);
    }

    public override Task<AdventMessageReply> GetMessage(GetAdventMessageRequest request, ServerCallContext context)
    {
        var message = _festiveAdventRepository.GetAdventMessage(request.Id);
        return Task.FromResult(new AdventMessageReply() { Id = message.Id, Message = message.Message });
    }

    public override Task<AdventMessageReply> CreateMessage(CreateAdventMessageRequest request, ServerCallContext context)
    {
        var messageToCreate = new AdventMessage()
        {
            Id=request.Id,
            Message = request.Message
        };

        var createdMessage = _festiveAdventRepository.AddAdventMessage(messageToCreate);

        var reply = new AdventMessageReply() { Id = createdMessage.Id, Message = createdMessage.Message};

        return Task.FromResult(reply);
    }

    public override Task<AdventMessageReply> UpdateMessage(UpdateAdventMessageRequest request, ServerCallContext context)
    {
        var existingMessage = _festiveAdventRepository.GetAdventMessage(request.Id);

        if (existingMessage == null)
        {
            throw new RpcException(new Status(StatusCode.NotFound, "Advent message not found"));
        }
        
        var messageToUpdate = new AdventMessage()
        {
            Id=request.Id,
            Message = request.Message
        };

        var createdMessage = _festiveAdventRepository.UpdateAdventMessages(messageToUpdate);

        var reply = new AdventMessageReply() { Id = createdMessage.Id, Message = createdMessage.Message};

        return Task.FromResult(reply);
    }

    public override Task<Empty> DeleteMessage(DeleteAdventMessageRequest request, ServerCallContext context)
    {
        var existingMessage = _festiveAdventRepository.GetAdventMessage(request.Id);

        if (existingMessage == null)
        {
            throw new RpcException(new Status(StatusCode.NotFound, "Advent message not found"));
        }
        
        _festiveAdventRepository.DeleteAdventMessage(request.Id);
        
        return Task.FromResult(new Empty());
    }
}

Note: I am using a repository to manage the in-memory list of Christmas messages. The code is available in the sample repo mentioned in the resources section

Finally, I have to map my class as a gRPC service using the middleware. 

app.MapGrpcService<FestiveAdventApiService>();

And I have got my gRPC service up and running. Now let's extend this gRPC API to be an HTTP API as well.

gRPC JSON Transcoding is an extension to ASP.NET Core. With gRPC JSON Transcoding, the request messages in JSON are deserialized into Protobufs, and the gRPC service is invoked directly. The most important concept to understand here is the HTTP Rule.

Understanding the basics of HTTP Rules

HTTP Rules define the schema of gRPC-HTTP API mapping. Each annotation specifies a route and the HTTP Method for that endpoint. The mapping defines the HTTP Method and a route template(called a pattern) for the mapping. The mapping also drives how route parameters, query strings, and request body get mapped to the gRPC request message. The mapping also drives how the gRPC response message gets mapped to the response body. HTTP Rules are specified using the annotation google.api.http.

A typical HTTP Rule can look like this. 

rpc ListMessages(google.protobuf.Empty) returns (ListReply){
    option (google.api.http) ={
      get : "/message"
    };
  }

In this example, with the annotation in place, the rpc can be invoked as an HTTP API endpoint and responds to an HTTP GET at <api_base_url>/listmessages. It can continue to be invoked as a gRPC service too. 

It is also possible to have route parameters in the route template. For example, the rpc to get a message by its id can be annotated as shown below.

rpc GetMessage(GetAdventMessageRequest) returns (AdventMessageReply){
    option (google.api.http) ={
      get : "/message/{id}"
    };
  }

message GetAdventMessageRequest{
  int32 id =1;
}

message AdventMessageReply{
  int32 id = 1;
  string message=2;
}

In this example, the id field in the GetAdventMessageRequest message gets mapped to the route parameter id. 

You can also pass in query parameters in your route. They get bound to the corresponding fields of the same name in the request message.

You can also specify the request message to be bound to the request body. In this example, to create a message, the request message is bound completely from the JSON request body.

  rpc CreateMessage(AdventMessageRequest) returns (AdventMessageReply){
    option (google.api.http) ={
      post : "/message"
      body:"*"
    };
  }

message AdventMessageRequest{
  int32 id = 1;
  string message=2;
}

message AdventMessageReply{
  int32 id = 1;
  string message=2;
}

The convention "*" specifies that any field not bound by the route template gets bound from the request body.

I have only shown examples of HTTP GET and POST here so that I can cover off binding of Protobuf fields, but HTTP Rules support GET, POST, PUT, PATCH and DELETE (as shown in my sample code below). You can use the custom pattern (in place of the HTTP Method) should you wish to implement HTTP Methods like HEAD.

I am only covering the basics of HTTP Rules here, you can read more about HTTP Rules in the official Google docs.

Let's now bring our knowledge of HTTP Rules and put it into practice. 

To begin with, I need Microsoft.AspNetCore.Grpc.JsonTranscoding Nuget package in place. 

dotnet add package Microsoft.AspNetCore.Grpc.JsonTranscoding --version 7.0.0

I also need to register JsonTranscoding in the Startup.

builder.Services.AddGrpc().AddJsonTranscoding();

My annotated proto file looks as shown below. 

syntax = "proto3";


option csharp_namespace = "FestiveAdventApi";

package festiveadvent;

import "google/protobuf/empty.proto";
import "google/api/annotations.proto";

service FestiveAdventService{

  rpc ListMessages(google.protobuf.Empty) returns (ListReply){
    option (google.api.http) ={
      get : "/message"
    };
  }

  rpc GetMessage(CreateAdventMessageRequest) returns (AdventMessageReply){
    option (google.api.http) ={
      get : "/message/{id}"
    };
  }

  rpc CreateMessage(AdventMessageRequest) returns (AdventMessageReply){
    option (google.api.http) ={
      post : "/message"
      body:"*"
    };
  }

  rpc UpdateMessage(UpdateAdventMessageRequest) returns (AdventMessageReply){
    option (google.api.http) ={
      put : "/message/{id}"
      body:"*" 
    };
  }

  rpc DeleteMessage(DeleteAdventMessageRequest) returns (google.protobuf.Empty){
    option (google.api.http) ={
      delete : "/message/{id}"
    };
  }
}

message CreateAdventMessageRequest{
  int32 id = 1;
  string message=2;
}

message GetAdventMessageRequest{
  int32 id =1;
}

message UpdateAdventMessageRequest{
  int32 id=1; //id is bound from route parameter
  string message=2;
}

message DeleteAdventMessageRequest{
  int32 id =1;
}

message ListReply{
  repeated AdventMessageReply AdventMessages = 1;
}

message AdventMessageReply{
  int32 id = 1;
  string message=2;
}

Each of my rpcs are annotated.

• The ListMessages rpc is mapped to a HTTP GET /messages endpoint
• The GetMessage rpc is mapped to a HTTP GET /message/{id} endpoint. The id field in the GetAdventMessageRequest message is bound from the route parameter id
• The CreateMessage rpc is mapped to a HTTP POST /message endpoint. The CreateAdventMessageRequest message is bound from the request body as specified by the body:"*" . The fields in the request message are bound to corresponding fields of the same name in the request body
• The UpdateMessage rpc is mapped to a HTTP PUT /message/{id} endpoint. The id field in the UpdateAdventMessageRequest message is bound to the route parameter id. Any other field in the request message gets bound from the request body as specified by the body:"*"
• The DeleteMessage rpc is mapped to a HTTP DELETE /message/{id} endpoint. The id field in the DeleteAdventMessageRequest message is bound to the route parameter id.

That is it!! I can now run my project and test the HTTP API that I have created using gRPC JSON Transcoding using Postman. Nothing has changed about my gRPC API, it can continue to serve my existing gRPC clients. But I have extended the API using a few annotations so that my browser-based applications can consume it. And as you can see, no code duplication, and it's a single ASP.NET Core service! Win Win!!!!

But that's not all..

We can also have OpenAPI support for our gRPC JSON Transcoded API. To get this up and running we need the package Microsoft.AspNetCore.Grpc.Swagger.

dotnet add package Microsoft.AspNetCore.Grpc.Swagger --version 0.3.0

 With the package installed, we can configure the startup. We first need to register Swagger into the DI container.

builder.Services.AddGrpcSwagger();

//Register Swagger generator, defining one or more documents to be generated by Swagger generator 
builder.Services.AddSwaggerGen(c =>
{
    c.SwaggerDoc("v1",
        new OpenApiInfo { Title = "gRPC transcoding", Version = "v1" });
});

 We then need to enable the middleware to serve the generated Swagger document as a JSON endpoint and also serve the Swagger UI.

// Enable middleware to serve generated Swagger as a JSON endpoint.
app.UseSwagger();

// Enable middleware to serve swagger-ui (HTML, JS, CSS, etc.)
app.UseSwaggerUI(c =>
{
    c.SwaggerEndpoint("/swagger/v1/swagger.json", "My API V1");
});

 Build, run and browse to /swagger and I can test my gRPC JSON Transcoded HTTP API.

Note: OpenAPI support for gRPC JSON Transcoded API is an experimental feature in .NET 7

So there we go, we have a gRPC API which has now been extended to a HTTP API. This is a very useful feature when you want to extend your existing gRPC service to function like an HTTP API as well. This feature was called gRPC HTTP API in the past and was an experimental feature, but due to tremendous community interest, it is now a full-fledged feature in .NET 7.

That is it from me this year on the Festive Tech Calendar. A very Merry Christmas and a Happy New Year to all the readers and the team at Festive Tech Calendar!!!