Java – How can I Log into a Website with HtmlUnit?

Question

I am writing a Java program to log into the website my school uses to post grades.

This is the url of the login form: https://ma-andover.myfollett.com/aspen/logon.do

This is the HTML of the login form:

<form name="logonForm" method="post" action="/aspen/logon.do" autocomplete="off"><div><input type="hidden" name="org.apache.struts.taglib.html.TOKEN" value="30883f4c7e25a014d0446b5251aebd9a"></div>
<input type="hidden" id="userEvent" name="userEvent" value="930">
<input type="hidden" id="userParam" name="userParam" value="">
<input type="hidden" id="operationId" name="operationId" value="">
<input type="hidden" id="deploymentId" name="deploymentId" value="ma-andover">
<input type="hidden" id="scrollX" name="scrollX" value="0">
<input type="hidden" id="scrollY" name="scrollY" value="0">
<input type="hidden" id="formFocusField" name="formFocusField" value="username">
<input type="hidden" name="mobile" value="false">
<input type="hidden" name="SSOLoginDone" value="">
<center>
<img src="images/spacer.gif" height="15" width="1">

<script language="JavaScript">
document.forms[0].elements['deploymentId'].value = 'ma-andover';
</script>

<script language="JavaScript">
$(function()
{
$('form').attr('autocomplete', 'off');
var name = $('#username');
var password = $('#password');
name.attr('autocomplete', 'off');
password.attr('autocomplete', 'off');
if (name.val() == '')
{
password.attr('disabled','disabled');
}
});
</script>

<img src="images/spacer.gif" height="30" width="1">
<table border="0" cellpadding="0" cellspacing="0">
<tbody><tr>
<td>
<div id="logonDetailContainer" class="logonDetailContainer">
<table border="0" cellpadding="0" cellspacing="0">

<tbody><tr>
<td>
<label style="text-align: center; margin-bottom: 0px">Andover Public Schools</label>
<img src="images/spacer.gif" height="10" width="1">
<hr class="logonHorizontalRule">
</td>
</tr>

<tr>
<td>
<img src="images/spacer.gif" height="10" width="1">


<input type="text" name="fakeuser" style="display: none">
<input type="password" name="fakepassword" style="display: none">

</td>
</tr>
<tr>
<td class="labelCell">

<label>Login ID</label>
<input type="text" name="username" tabindex="1" value="" onkeypress="$('#password').prop('disabled', false)" id="username" class="logonInput" autocomplete="off">

&nbsp;

</td>
</tr>
<tr>
<td class="labelCell">

<label>Password</label>
<input id="password" type="password" name="password" tabindex="2" value="" class="logonInput" autocomplete="off" disabled="disabled">

<a href="javascript:EmbeddedPopup.popupManager.open('passwordRecovery.do?isSecondary=false&amp;deploymentId=ma-andover', 400, 400, 100)" tabindex="5" style="float: right">
I forgot my password
</a>


</td>
</tr>
<tr>
<td width="1" class="logonTopPadding" style="float: left">
<input type="submit" tabindex="3" value="Log On" class="log-button">
</td>
</tr>

</tbody></table>
</div>
</td>
</tr>
</tbody></table>

</center>
<script>
setTimeout(function(){window.location.reload(true);}, 1800000);
</script>
</form>

I am trying to use the following code to log in:

import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlForm;
import com.gargoylesoftware.htmlunit.html.HtmlPage;

public class LoginAttempt {

    public static void main(String[] args) throws Exception {  
            WebClient webClient = new WebClient();

            HtmlPage page = (HtmlPage) webClient.getPage("https://ma-andover.myfollett.com/aspen/logon.do"); 
            HtmlForm form = page.getFormByName("logonForm"); 
            form.getInputByName("username").setValueAttribute("myUsername"); //works fine 
            form.getInputByName("password").setValueAttribute("myPassword"); //does not work 

            page = form.getInputByValue("Log On").click(); //works fine

            System.out.println(page.asText());
    } 

}

The program fills the username box and clicks the "Log On" button, but it does not fill the password box. What can I change to make this program work? I suspect the "type = 'password'" attribute of the password box has something to do with the problem, but please correct me if I am wrong. Any help is appreciated. Thank you very much.

The target page: https://ma-andover.myfollett.com/aspen/home.do

And this is my output, in case it might be helpful:

Aspen: Log On

Aspen

    About Aspen
Andover Public Schools
Login ID myUsername  
Password I forgot my password
Log On

Copyright © 2003-2014 Follett School Solutions. All rights reserved.
Follett Corporation Follett Software Company Aspen Terms of Use

You must enter a password.
OK

Ne

which version of htmlunit are you using? – Nidorf

Answer 1

The password field is disabled until you type something in the username field. By setting the value in username doesn't trigger the event that manages the enabling of password field.

The below works

public static void main(String[] args) {
    WebClient webClient = new WebClient();
    try {
        HtmlPage page = (HtmlPage) webClient
                .getPage("https://ma-andover.myfollett.com/aspen/logon.do");
        HtmlForm form = page.getFormByName("logonForm");
        form.getInputByName("username").setValueAttribute("myUsername"); 
        HtmlInput passWordInput = form.getInputByName("password");
        passWordInput.removeAttribute("disabled");
        passWordInput.setValueAttribute("myPassword"); 

        page = form.getInputByValue("Log On").click(); // works fine

        System.out.println(page.asText());
    } catch (Exception e) {
        e.printStackTrace();
    } finally {
        webClient.close();
    }
}

The output is

Aspen: Log On

Aspen

    About Aspen
Andover Public Schools
Login ID myUsername  
Password I forgot my password
Log On

Copyright © 2003-2014 Follett School Solutions. All rights reserved.
Follett Corporation Follett Software Company Aspen Terms of Use

Invalid login.  
OK

Nidorf

Thank you for answering my question so quickly. The username box was filled in before the password box, so I don't see why it was disabled, but your line, "passWordInput.removeAttribute("disabled");" seems to have worked. The login now works properly. – Ne

@alkis I tried your code but it throws a version error.Exception in thread "main" java.lang.UnsupportedClassVersionError: com/gargoylesoftware/htmlunit/html/HtmlPage : Unsupported major.minor version 52.0 .I am using java 1.7 and the <dependency> <groupId>net.sourceforge.htmlunit</groupId> <artifactId>htmlunit</artifactId> <version>2.27</version> </dependency> .What can be the cause?Any help is appreciated – Kinghood

@Kinghood This a very old answer. I don't remember the versions I used. Please open a new question with the versions that you are using and the code. Someone else might have experienced the same problem and might have an easy solution. – Nidorf

Answer 2

To automatically handle the JavaScript, you should use type() instead.

try (WebClient webClient = new WebClient()) {

    HtmlPage page = (HtmlPage) webClient.getPage("https://ma-andover.myfollett.com/aspen/logon.do"); 
    HtmlForm form = page.getFormByName("logonForm"); 
    form.getInputByName("username").type("myUsername"); 
    form.getInputByName("password").type("myPassword"); 

    page = form.getInputByValue("Log On").click();

    System.out.println(page.asText());
}

Dixon

Answer 3

I used:

final WebClient webClient = new WebClient())    
HtmlPage page = webClient.getPage("url");

((HtmlTextInput) page.getHtmlElementById("usernameID")).setText("Username");
page.getHtmlElementById("passwordID").setAttribute("value","Password");

page.getElementsByTagName("button").get(0).click();

System.out.println(page.asText());

I clicked the button that way because my button doesn't have an id, name, or value, but luckily its the only button on the page. So I just get all the button tags (all one of them) and select the first element in the List to click.

Autohypnosis