How do I detect if a Windows server is available after a reboot?

Question

I want to automate a Windows 2000+ server reboot process using Task Scheduler or similar tool to remotely reboot a server and wait for it to come back up. I can issue shutdown or psshutdown to remotely reboot, but I want something better than sleep to wait for it to come back. I need to verify it is back online within n minutes or throw an error.

By 'back online', I would like to verify more than just that it can be pinged, but perhaps its RFC service is responding or some other determinate vital sign.

I'd prefer an NT script approach, but I'm not ruling out writing a custom tool to do this.

Any ideas?

Hadhramaut

Answer 1

Your remote restart script could start the server, wait n minutes, then query your RFC service. You could also have a local script on the server do the same thing.

Houseroom

Answer 2

After working on this for a while, I came up with the following VBScript. Feel free to comment/improve.

'
' Remotely reboot a server and
' wait for server to come back up.
'
' Usage:  cscript /nologo /E:VBScript RebootWait.vbs <Server Name>
'
' Shawn Poulson, 2008.09.11
'

'
' Get server name from command line
'
If WScript.Arguments.Count <> 1 Then
   ShowUsage()
   WScript.Quit(1)
End If

ServerName = WScript.Arguments(0)

'
' Verify server is currently up
'
WScript.StdOut.WriteLine Now & ": Verify server '" & ServerName & "' is currently up..."
If Not IsAvailable(ServerName) Then
   WScript.StdOut.WriteLine "Error: Server is down.  Reboot aborted!"
   WScript.Quit(1)
End If
WScript.StdOut.WriteLine Now & ": Server is up."

'
' Reboot server
'
WScript.StdOut.WriteLine Now & ": Rebooting server '" & ServerName & "'..."
RebootStatus = RebootServer(ServerName)
If RebootStatus < 0 Then
   WScript.StdOut.WriteLine "Error: Reboot returned error " & RebootStatus
   WScript.Quit(1)
End If
WScript.StdOut.WriteLine Now & ": Reboot command was successful"

'
' Wait for server to come down
'
WScript.StdOut.Write Now & ": Waiting for server '" & ServerName & "' to go down..."
WaitCount = 0
Do While IsAvailable(ServerName)
   WaitCount = WaitCount + 1
   If WaitCount > 60 Then ' 5 min timeout
      WScript.StdOut.WriteLine "Error: Timeout waiting for server to come down!"
      WScript.Quit(1)
   End If
   WScript.StdOut.Write(".")
   WScript.Sleep(5000)
Loop
WScript.StdOut.WriteLine "Success!"
WScript.StdOut.WriteLine Now & ": Server is down."

'
' Wait for server to come back up
'
WScript.StdOut.Write Now & ": Waiting for server '" & ServerName & "' to come back up..."
WaitCount = 0
Do While Not IsAvailable(ServerName)
   WaitCount = WaitCount + 1
   If WaitCount > 240 Then ' 20 min timeout
      WScript.StdOut.WriteLine "Error: Timeout waiting for server to come back up!"
      WScript.Quit(1)
   End If
   WScript.StdOut.Write(".")
   WScript.Sleep(5000)
Loop
WScript.StdOut.WriteLine "Success!"
WScript.StdOut.WriteLine Now & ": Server is back up after reboot."

'
' Success!
'
WScript.Quit(0)


Sub ShowUsage()
   WScript.Echo "Usage: " & WScript.ScriptName & " <Server name>"
End Sub

' Returns:
' 1 = Successfully issued reboot command
' -2 = Could not reach server
' -3 = Reboot command failed
Function RebootServer(ServerName)
   Dim OpSystem
   On Error Resume Next
   For Each OpSystem in GetObject("winmgmts:{(Shutdown)}!\\" & ServerName & "\root\CIMV2").ExecQuery("select * from Win32_OperatingSystem where Primary=true")
      On Error GoTo 0

      If IsObject(OpSystem) Then
         ' Invoke forced reboot
         If OpSystem.Win32Shutdown(6, 0) = 0 Then
            ' Success
            RebootServer = 1
         Else
            ' Command failed
            RebootServer = -3
         End If

      Else
         RebootServer = -2

      End If
   Next
End Function

' Return True if available
Function IsAvailable(ServerName)
   ' Use Windows RPC service state as vital sign
   IsAvailable = (GetServiceState(ServerName, "RpcSs") = "Running")
End Function

' Return one of:
'  Stopped, Start Pending, Stop Pending,
'  Running, Continue Pending, Pause Pending,
'  Paused, Unknown
Function GetServiceState(ServerName, ServiceName)
   Dim Service
   On Error Resume Next
   Set Service = GetObject("winmgmts:\\" & ServerName & "\root\CIMV2:Win32_Service='" & ServiceName & "'")
   On Error GoTo 0
   If IsObject(Service) Then GetServiceState = Service.State
End Function

Hadhramaut

Nice! I tested this and it works as advertised. You do need to be running with an account that has admin privs over the remote server; ability to provide such creds would be spiffy. – Provo

You could wrap this command with "runas" to provide alternate credentials. – Hadhramaut

Answer 3

You could use psservice to query the status of the RFC or Print Spooler services. The Spooler is usually one of the last services to start. You could use syntax like:

psservice \\someothermachine query spooler

That will return something like this once the service is running.

SERVICE_NAME: Spooler                                                                             
DISPLAY_NAME: Print Spooler                                                                       
Manages all local and network print queues and controls all printing jobs. If this service is stop
ped, printing on the local machine will be unavailable. If this service is disabled, any services 
that explicitly depend on it will fail to start.                                                  
        GROUP             : SpoolerGroup                                                          
        TYPE              : 110 WIN32_OWN_PROCESS INTERACTIVE_PROCESS                             
        STATE             : 4  RUNNING                                                            
                               (STOPPABLE,NOT_PAUSABLE,ACCEPTS_SHUTDOWN)                          
        WIN32_EXIT_CODE   : 0  (0x0)                                                              
        SERVICE_EXIT_CODE : 0  (0x0)                                                              
        CHECKPOINT        : 0x0                                                                   
        WAIT_HINT         : 0x0                                                                   

If the other machine is not ready, you'll get something like

Unable to connect to \\someothermachine:                                                                  
The RPC server is unavailable. 

Katydid

Answer 4

With VBScript (WSH) you could check it with the .state property. This script shows that property being used in a different application but should help illustrate the idea:

http://www.robvanderwoude.com/vbstech_proc_service.html

Waxwork

Answer 5

You can poll some core service to see if it has started:

sc "\\server_name" query EventSystem

Quintain

Answer 6

Use nmap to get a list of open services on the machine and parse the results to make sure what you need is active. It's also useful to make sure that things you don't need are not active.

Templia

Answer 7

The key here is I need to script this. Is there a cleaner way to extract the service status from psservice/sc query? I'm able to pipe it to findstr "RUNNING", but there's got to be a better way.

Hadhramaut