Channel ManagedChannelImpl was not shut down properly

Question

If I run following these two tests I get the error.

1st test

@Rule
public GrpcCleanupRule grpcCleanup = new GrpcCleanupRule();

@Test
public void findAll() throws Exception {
    // Generate a unique in-process server name.
    String serverName = InProcessServerBuilder.generateName();

    // Create a server, add service, start, and register for automatic graceful shutdown.
    grpcCleanup.register(InProcessServerBuilder
            .forName(serverName)
            .directExecutor()
            .addService(new Data(mockMongoDatabase))
            .build()
            .start());

    // Create a client channel and register for automatic graceful shutdown.
    RoleServiceGrpc.RoleServiceBlockingStub stub = RoleServiceGrpc.newBlockingStub(
            grpcCleanup.register(InProcessChannelBuilder
                    .forName(serverName)
                    .directExecutor()
                    .build()));

    RoleOuter.Response response = stub.findAll(Empty.getDefaultInstance());
    assertNotNull(response);
}

2nd test

@Test
public void testFindAll() {
    ManagedChannel channel = ManagedChannelBuilder.forAddress("localhost", 8081)
            .usePlaintext()
            .build();

    RoleServiceGrpc.RoleServiceBlockingStub stub = RoleServiceGrpc.newBlockingStub(channel);
    RoleOuter.Response response = stub.findAll(Empty.newBuilder().build());
    assertNotNull(response);
}

io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference cleanQueue SEVERE: ~~~ Channel ManagedChannelImpl{logId=1, target=localhost:8081} was not shutdown properly!!! ~~~ Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.

java.lang.RuntimeException: ManagedChannel allocation site at io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.(ManagedChannelOrphanWrapper.java:94)

If I comment out one of them, then no errors, unit tests pass though but the exception is thrown if both are ran together.

Edit

Based on the suggestion.

@Test
public void testFindAll() {
    ManagedChannel channel = ManagedChannelBuilder.forAddress("localhost", 8081)
            .usePlaintext()
            .build();

    RoleServiceGrpc.RoleServiceBlockingStub stub = RoleServiceGrpc.newBlockingStub(channel);
    RoleOuter.Response response = stub.findAll(Empty.newBuilder().build());
    assertNotNull(response);

    channel.shutdown();
}

Quin

What's grpcCleanup? You should create a server using InProcessServerBuilder (check), and then add the corresponding services in the same chained call. – Ogdon

edited, @Rule public GrpcCleanupRule grpcCleanup = new GrpcCleanupRule();, I'm new to this, using an example – Quin

I would try creating all that stuff inside an @Before and then stopping the server in the @After; in any case, those methods you are testing shouldn't be responsible for starting and stopping the gRPC server(s). – Ogdon

with the GrpcCleanupRule, you don't need @After, @Before for grpc resources. but the 2nd test needs to register the channel it creates. the cleanup rule will call graceful shutdown. – Stupidity

The second test is sort of integration test in it's own file (not the same .java file as the 1st test), can you please help with code on how to write this kind of integration test properly. – Quin

@Stupidity see my edits for 2nd test, is it what is suggested? – Quin

@AppDeveloper, i expect to see GrpcCleanupRule just like test case 1. Adding grpcCleanup.register(channel); should be sufficient – Stupidity

I have a similar issue when implemented an onclicklistener in kotlin. Can someone help me out – Egression

Answer 1

Hey I just faced similar issue using Dialogflow V2 Java SDK where I received the error

 Oct 19, 2019 4:12:23 PM io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference cleanQueue
SEVERE: *~*~*~ Channel ManagedChannelImpl{logId=41, target=dialogflow.googleapis.com:443} was not shutdown properly!!! ~*~*~*
    Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.

Also, Having a huge customer base we started running into out of memory unable to create native thread error.

After performing a lot of Debugging operations and Using Visual VM Thread Monitoring I finally figured out that the problem was because of SessionsClient not closing. So I used the attached code block to solve that issue. Post testing that block I was finally able to free up all the used threads and also the error mentioned earlier was resolved.

SessionsClient sessionsClient = null;
QueryResult queryResult = null;

try {
    SessionsSettings.Builder settingsBuilder = SessionsSettings.newBuilder();
    SessionsSettings sessionsSettings = settingsBuilder
            .setCredentialsProvider(FixedCredentialsProvider.create(credentials)).build();
    sessionsClient = SessionsClient.create(sessionsSettings);
    SessionName session = SessionName.of(projectId, senderId);
    com.google.cloud.dialogflow.v2.TextInput.Builder textInput = TextInput.newBuilder().setText(message)
            .setLanguageCode(languageCode);
    QueryInput queryInput = QueryInput.newBuilder().setText(textInput).build();

    DetectIntentResponse response = sessionsClient.detectIntent(session, queryInput);

    queryResult = response.getQueryResult();
} catch (Exception e) {
    e.printStackTrace();
}
finally {
    sessionsClient.close();
}

The shorter values on the graph highlights the use of client.close(). Without that the threads were stuck in Parking State.

Composite

Answer 2

I had a similar issue recently using Google cloud task API.

Channel ManagedChannelImpl{logId=5, target=cloudtasks.googleapis.com:443} was 
not shutdown properly!!! ~*~*~* (ManagedChannelOrphanWrapper.java:159)
Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.

In this case CloudTasksClient object implements AutoCloseable and we should call its .close() method after its done.

We can use a try block like this which would auto close when it's done.

   try( CloudTasksClient client = CloudTasksClient.create()){
               CloudTaskQueue taskQueue = new CloudTaskQueue(client);
              
}

or Add try/finally

 CloudTasksClient client =null;
          try{
                client = CloudTasksClient.create() ;
                CloudTaskQueue taskQueue = new CloudTaskQueue(client);
           } catch (IOException e) {
                e.printStackTrace();
            } finally {
                client.close();
         }

Trestle

Answer 3

In my case, I just shutdown the channel in try,finally block:

ManagedChannel channel = ManagedChannelBuilder.forAddress...
try{
   ...
}finally {
   channel.shutdown();
}

Weaverbird