I am trying to log into this url:
http://www.kalahari.com/marketplace/default.aspx
The two fields that are being submitted are labelled:
ctl00$ctl00$ucMarketPlaceSupportNavigation$txtMPTopSignInEmail
ctl00$ctl00$ucMarketPlaceSupportNavigation$txtMPTopSignInPasswordTextNormal
The code I've tried to use so far:
$username = 'XXXXXXX';
$password = 'XXXXXXX';
$loginUrl = 'http://www.kalahari.com/marketplace/default.aspx';
$cookie = 'cookies.txt';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $loginUrl );
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, TRUE );
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE );
curl_setopt($ch, CURLOPT_COOKIEJAR , $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_HEADER, FALSE );
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$ret = curl_exec($ch); //access login page
// Collecting all POST fields
$postfields = array();
$postfields['ctl00$ctl00$ucMarketPlaceSupportNavigation$txtMPTopSignInEmail'] = $username;
$postfields['ctl00$ctl00$ucMarketPlaceSupportNavigation$txtMPTopSignInPasswordTextNormal'] = $password;
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields);
$ret = curl_exec($ch);//Get result after login page.
print $ret;
This however only takes me back to the original login page... not even with an error message.
I then had a look at what is getting posted, and I saw:
Request URL:http://www.kalahari.com/marketplace/default.aspx
Request Method:POST
Status Code:302 Found
Request Headersview source
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Cache-Control:max-age=0
Connection:keep-alive
Content-Length:2596
Content-Type:application/x-www-form-urlencoded
Cookie:VISITORID=9840A7E31683480CB19A66FB8AA73BFC; ASP.NET_SessionId=foous3ftij3os2vvr1wbm3mm; __utma=160092839.590473234.1362995010.1362995010.1362995010.1; __utmc=160092839; __utmz=160092839.1362995010.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); _SUPERFLY_nosample=1; shopperName=; signin=0; kalahariShopperId=922859656760417F99E83D5B1427115F; surfLang=ENG; prefLanguage=en-ZA; _chartbeat2=1yx62ww1m7xz1o84.1360134968807.1363000295875.00000000000001
Host:www.kalahari.com
Origin:http://www.kalahari.com
Referer:http://www.kalahari.com/marketplace/default.aspx
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.152 Safari/537.22
Form Dataview sourceview URL encoded
__EVENTTARGET:
__EVENTARGUMENT:
__VIEWSTATE:/wEPDwULLTEzODYxODcwMTMPZBYCZg9kFgJmD2QWAgIBD2QWCAICD2QWBAIDDw8WAh4LTmF2aWdhdGVVcmwFNH4vcGlwZWxpbmUvc2lnbmluLmFzcHg/UmV0dXJuVVJMPS9zZWN1cmUvbWFya2V0cGxhY2VkZAIHDxYCHgdWaXNpYmxlaBYCAgEPDxYCHwFoZGQCBA9kFgICAQ9kFgICAQ8WAh8BaBYCAgEPZBYQAgEPDxYGHghDc3NDbGFzcwULYWN0aXZlX2xpbmseB0VuYWJsZWRnHgRfIVNCAgJkZAIDDw8WBh8CBQVsaW5rcx8DZx8EAgJkZAIFDw8WBh8CBQ1kaXNhYmxlX2xpbmtzHwNoHwQCAmRkAgcPDxYGHwIFDWRpc2FibGVfbGlua3MfA2gfBAICZGQCCQ8PFgYfAgUNZGlzYWJsZV9saW5rcx8DaB8EAgJkZAILDw8WBh8CBQ1kaXNhYmxlX2xpbmtzHwNoHwQCAmRkAg0PDxYGHwIFDWRpc2FibGVfbGlua3MfA2gfBAICZGQCDw8PFgYfAgUNZGlzYWJsZV9saW5rcx8DaB8EAgJkZAIFD2QWAgIBD2QWBgIJDxAPZBYCHgdvbkNsaWNrBYQBamF2YXNjcmlwdDpWYWxpZGF0ZUxvZ2luUmFkaW9CdG5ZZXMoY3RsMDBfY3RsMDBfY3BsaE1haW5fY3BsaENvbnRlbnRfcmRsUGFzc3dvcmRZZXMsY3RsMDBfY3RsMDBfY3BsaE1haW5fY3BsaENvbnRlbnRfcmRsUGFzc3dvcmRObyk7ZGRkAgsPEA9kFgIfBQWDAWphdmFzY3JpcHQ6VmFsaWRhdGVMb2dpblJhZGlvQnRuTm8oY3RsMDBfY3RsMDBfY3BsaE1haW5fY3BsaENvbnRlbnRfcmRsUGFzc3dvcmRZZXMsY3RsMDBfY3RsMDBfY3BsaE1haW5fY3BsaENvbnRlbnRfcmRsUGFzc3dvcmRObyk7ZGRkAhUPD2QWAh8FBdMBamF2YXNjcmlwdDpWYWxpZGF0ZUxvZ2luKCdjdGwwMF9jdGwwMF9jcGxoTWFpbl9jcGxoQ29udGVudF9sYmxSZXN1bHQnLGN0bDAwX2N0bDAwX2NwbGhNYWluX2NwbGhDb250ZW50X3R4dFBhc3N3b3JkLGN0bDAwX2N0bDAwX2NwbGhNYWluX2NwbGhDb250ZW50X3JkbFBhc3N3b3JkWWVzLGN0bDAwX2N0bDAwX2NwbGhNYWluX2NwbGhDb250ZW50X3JkbFBhc3N3b3JkTm8pO2QCBg9kFgICBQ8PFgIfAAUafi9waXBlbGluZS9vcmRlcl9saXN0LmFzcHhkZBgBBR5fX0NvbnRyb2xzUmVxdWlyZVBvc3RCYWNrS2V5X18WAwUvY3RsMDAkY3RsMDAkY3BsaE1haW4kY3BsaENvbnRlbnQkcmRsUGFzc3dvcmRZZXMFLmN0bDAwJGN0bDAwJGNwbGhNYWluJGNwbGhDb250ZW50JHJkbFBhc3N3b3JkTm8FLmN0bDAwJGN0bDAwJGNwbGhNYWluJGNwbGhDb250ZW50JHJkbFBhc3N3b3JkTm99/wuPOuNOonYg5XWvf3RGR1YVkw==
__EVENTVALIDATION:/wEWDQLsuI7QDgKnpLoxApD7nfEPAvLAqqUGAp35/akJAqGiqqYPAsXC5NUHAsHJ5OMCAovxoc8LArq0mqAKApm+rVoC9dLe0Q8C5IvEsAlYqPIdcrZvBZcvYav7ATMf4Nhbfg==
ctl00$ctl00$ucMarketPlaceSupportNavigation$txtMPTopSignInEmail:XXXXXXXXX
ctl00$ctl00$ucMarketPlaceSupportNavigation$txtMPTopSignInPasswordTextNormal:Password
ctl00$ctl00$ucMarketPlaceSupportNavigation$txtMPTopSignInPassword:XXXXXXXXXXX
ctl00$ctl00$ucMarketPlaceSupportNavigation$btnSigninTop:Sign in
ctl00$ctl00$cplhMain$cplhContent$txtEmail:Email address
ctl00$ctl00$cplhMain$cplhContent$rdlPasswordYes:rdlPasswordYes
ctl00$ctl00$cplhMain$cplhContent$txtPasswordTextNormal:Password
ctl00$ctl00$cplhMain$cplhContent$txtPassword:
ctl00$ctl00$cplhMain$cplhContent$hdnEmailDefault:Email address
ctl00$ctl00$cplhMain$cplhContent$hdnPasswordDefault:Password
Response Headersview source
Cache-Control:private, no-cache="Set-Cookie"
Content-Length:146
Content-Type:text/html; charset=utf-8
Date:Mon, 11 Mar 2013 11:11:57 GMT
Etag:
Location:/marketplace/default.aspx
Server:Microsoft-IIS/6.0
Set-Cookie:.KALAHARINETAUTH=782A6F442823F8148FB113BA0BAF3A9A8DE253762A4ACFAA5E911E4721166F0EEC6A1891755133AADD28654CF0DAE3880CC2B84260F0B915C07897909CFB071495AF8EF05D1BD678DEE1933FCB08E5ECB1CF76462900681C7D4AE963C151E3079D95FBAD6466F0528787455A951D5EC0DA26F0E6CAA341E4C717D7F3BC01D182F488F47F; domain=.kalahari.com; path=/; HttpOnly
Set-Cookie:surfLang=ENG; domain=.kalahari.com; expires=Sat, 11-Mar-2023 11:11:57 GMT; path=/
Set-Cookie:prefLanguage=en-ZA; domain=www.kalahari.com; path=/
Set-Cookie:signin=1; domain=kalahari.com; path=/
Set-Cookie:tempshopperid=922859656760417F99E83D5B1427115F; domain=kalahari.com; path=/
Set-Cookie:kalahariShopperId=54B14971F72D426BA02DEF3A3D99DC93; domain=kalahari.com; expires=Sun, 17-Jan-2038 22:00:00 GMT; path=/
Set-Cookie:shopperName=XXXX; domain=kalahari.com; path=/
Set-Cookie:[email protected]; domain=kalahari.com; path=/
X-AspNet-Version:2.0.50727
X-Powered-By:ASP.NET
It looks like I'm supposed to be submitting much more than just the username and password. What exactly must I post and how do I post something like a "viewstate" that isn't a fixed value?
__EVENTVALIDATION
is deliberately intended to prevent the kind of scraping you are attempting. A new value will be generated whenever anything runs on the server, not just when the whole page renders, which may include changing individual drop-down box values for instance. – Meritorious