The app and the service live on the same process, which means when the app is killed so is your service. Changing the return value of onStartCommand doesn't affect this process. It simply tells the Service to either start/stop when you tell it or when it's finished doing what it needs to. As mentioned in your comment to your original post, setting it as a foreground process worked, but that's really just forcing the service to have a high priority, not solving the problem.
To change the Service so that it's killed separately and assuming it's a started service rather than a bound service due to the use of onStartCommand, specify a process name in the manifest for that Service.
From the Process and Threads Developer Guide:
The manifest entry for each type of component element— <activity>, <service>, <receiver>, and <provider>
—
supports an android:process attribute that can specify a
process in which that component should run. You can set
this
attribute so that each component runs in its own process or so
that some components share a process while
others do not. You can also set android:process so that
components of different applications run in the same
process—provided that the applications share the same
Linux user ID and are signed with the same certificates.
Android might decide to shut down a process at some
point, when memory is low and required by other
processes that are more immediately serving the user.
Application components running in the process that's
killed are consequently destroyed. A process is started
again for those components when there's again work for them to do.
From <service>
in Manifest File:
android:process
The name of the process where the service is to run.
Normally, all components of an application run in the default process
created for the application. It has the same name as the application
package. The element's process attribute can set a
different default for all components. But component can override the
default with its own process attribute, allowing you to spread your
application across multiple processes.
If the name assigned to this
attribute begins with a colon (':'), a new process, private to the
application, is created when it's needed and the service runs in that
process. If the process name begins with a lowercase character, the
service will run in a global process of that name, provided that it
has permission to do so. This allows components in different
applications to share a process, reducing resource usage.
Not sure why the other answer that mentioned this was down voted. I've used this method in the past and, today, created a simple one Activity app with a Service on a different process just to make sure I wasn't crazy. I used Android Device Monitor to kill the app's process. You can see both, separate processes in ADM and can see that when the app's process is killed, the Service's is not.