The client id has to be included into a request, so the provider knows that the request came from you, as @Matt Lacey already pointed out. Normally, the provider also issues a confidential client secret that is additionally included into the access token request, so the provider can verify that your app is allowed to use that client id.
Chrome extensions run on an open platform and the platform itself provides no methods for either authenticating the extension against a server (which salesforce would then also have to support) or storing properties securely (would be hard, if not impossible on an open platform), so keeping the client secret confidential is unfortunately not possible.
As this is a common problem, it is already considered in the OAuth specification (see section 10.1 Client Authentication and 10.2 Client Impersonation). The provider is therefore required to do additional checks, but on the client side you can't do anything to effectively improve security.
If you want some more insight into how this will be handled on Android devices in the future, check out my answer here.