ServerXMLHTTP request returing data but not returning url of final page after 301 redirection

Question

I am trying to make LINK FINDER app in ASP
It working is divided into 5 step

1. Send http request to server at www.foo.com
2. Check status of request
3. If its 200 then move to step 4 otherwise show error
4. Parse all link
5. Send http request to server to parsed link

I am able to do first 4 step, But facing challenge in 5th step

I am getting 3 type of links

1.)absolute link : http://www.foo.com/file.asp
2.)links from root directory, which need domain name eg /folder2/file2.asp
3.)relative link : ../file3.asp

Challenge

When I am requesting www.foo.com , which is 301 redirected to www.foo.com/folder3/folder3/file3.asp

I am getting html content of redirected page, But don't get redirected url and not able to check 3rd type of links

Using following code

Set ServerXmlHttp = Server.CreateObject("MSXML2.ServerXMLHTTP.6.0")
ServerXmlHttp.open "GET", "http://www.foo.com"
ServerXmlHttp.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
ServerXmlHttp.send PostData
If ServerXmlHttp.status = 200 Then
 //My CODE

Hope for quick response... or any other idea for link finder in asp , vb.net

Dight

Answer 1

It's out of the ServerXMLHTTP capabilities.
Instead you have to use IWinHttpRequest or another third-party component that able to manage redirects.
In the following example, req.Option(WHR_URL) returns the current url even if redirected.
Option req.option(WHR_EnableRedirects) is True by default like ServerXMLHTTP.
So, I've added a line commented out showing how to disable redirects.

Const WHR_URL = 1
Const WHR_EnableRedirects = 6
'Enum constants are listed at http://msdn.microsoft.com/en-us/library/windows/desktop/aa384108(v=vs.85).aspx
Dim req
Set req = Server.CreateObject("WinHttp.WinHttpRequest.5.1")
    'req.Option(WHR_EnableRedirects) = False 'don't follow the redirects
    req.open "GET", "http://www.foo.com", False
    req.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
    req.send PostData
If req.Status = 200 Then
    Response.Write "Last URL : " & req.Option(WHR_URL)
End If

Ichor

gr88888.... one thing I wants to know. why you have defined two constant ? Can I also track all redirection from requested url to final url ? – Dight

@Dight Enum constants are listed on goo.gl/b4817X under the Constants section starting at zero and respectively. If you don't use them, of course you can use numeric values also. And yes you can track. Disable redirects and handle 3xx statuses using req.Status, then get the url by given Location header using req.GetResponseHeader("Location"). – Ichor

I am not able to use WinHttp.WinHttpRequest.5.1 object EVENT .I have also posted question regarding this issue at stackoverflow.com/questions/20552183 I need to callback function for asynchronous http request. – Dight

Answer 2

I have been struggling too, but the solution is to verify if the final URL is equal to the start URL.

Calling getOption(-1) right after the 'send' will give you the final URL.

Then resend your request with this final URL.

Here my working example from SQL server using MSXML2.ServerXMLHTTP.3.0:

declare @err_code INT,@response_text VARCHAR(MAX),@oa_object INT, @status int
declare @url varchar(4000)='https://api.insocial.nl/v2/customers/'
declare @final_url varchar(4000)
DECLARE @result_table Table (result varchar(max))
 
EXEC @err_code=sp_OACreate 'MSXML2.ServerXMLHTTP.3.0', @oa_object OUT;
Exec @err_code=sp_OAMethod @oa_object, 'open', NULL, 'GET', @url,'false', null, null
Exec @err_code=sp_OAMethod @oa_object, 'setTimeouts',NULL,5000,5000,10000,1200000
Exec @err_code=sp_OAMethod @oa_object, 'setRequestHeader', NULL, 'X-AUTH-TOKEN', '***'
Exec @err_code=sp_OAMethod @oa_object, 'send', Null, null
 
Exec @err_code=sp_OAMethod @oa_object,'getOption(-1)',@final_url OUT
print @final_url
if (@final_url <> @url)
begin
    Exec @err_code=sp_OAMethod @oa_object, 'open', NULL, 'GET', @final_url,'false', null, null
    Exec @err_code=sp_OAMethod @oa_object, 'setTimeouts',NULL,5000,5000,10000,1200000
    Exec @err_code=sp_OAMethod @oa_object, 'setRequestHeader', NULL, 'X-AUTH-TOKEN', '***'
    Exec @err_code=sp_OAMethod @oa_object, 'send', Null, null
end
 
SET @response_text = null --make sure we don't return garbage
INSERT @result_table (result)
EXEC @err_code = sp_OAGetProperty @oa_object, 'responseText' 
SELECT * FROM @result_table
 
EXEC sp_OADestroy @oa_object

Instal