LDAP: How to return more than 1000 results (java)
Asked Answered
I

4

19

I am using the LDAP SDK from this site: https://www.unboundid.com/products/ldap-sdk/ . I would like to make a search operation which returns a lot of entries.

According to the FAQ's site, ( https://www.unboundid.com/products/ldap-sdk/docs/ldapsdk-faq.php#search ) I have to use a SearchResultListener implementation.

So here is what I did:

 public class UpdateThread extends Thread implements SearchResultListener {
 ...
 // create request
 final SearchRequest request = new SearchRequest(this, instance.getBaseDN(),SearchScope.SUB, filter);
 // Setting size limit of results.
 request.setSizeLimit(2000);

 ...

 // Get every result one by one.
 @Override
public void searchEntryReturned(SearchResultEntry arg0) {
    System.out.println("entry "+arg0.getDN());

}

The problem is that "searchEntryReturned" returns a maximum of 1000 results. Even if I set the size limit to "2000".

Incoming answered 3/7, 2012 at 13:16 Comment(0)
M
11

Although it's almost certainly the case that the server is enforcing the size limit of 1000 entries, there are potentially ways to get around that by issuing the request in multiple parts.

If the server supports the use of the simple paged results control (as defined in RFC 2696 and supported in the LDAP SDK as per https://docs.ldap.com/ldap-sdk/docs/javadoc/com/unboundid/ldap/sdk/controls/SimplePagedResultsControl.html), then you can use it to iterate through the results in "pages" containing a specified number of entries.

Alternately, the virtual list view (VLV) request control (https://www.unboundid.com/products/ldap-sdk/docs/javadoc/index.html?com/unboundid/ldap/sdk/controls/VirtualListViewRequestControl.html) could be used, but I would probably only recommend that if the server doesn't support the simple paged results control because the VLV request control also requires that the results be sorted, and that likely either requires special configuration in the server or some pretty expensive processing in order to be able to service the request.

Marentic answered 3/7, 2012 at 17:0 Comment(0)
G
22

It is pretty simple to implement a paged LDAP query using standard java, by using the adding a PagedResultsControl to the LdapContext, without using a third party API as per Neil's answer above.

Hashtable<String, Object> env = new Hashtable<String, Object>(11);
env
    .put(Context.INITIAL_CONTEXT_FACTORY,
        "com.sun.jndi.ldap.LdapCtxFactory");

/* Specify host and port to use for directory service */
env.put(Context.PROVIDER_URL,
    "ldap://localhost:389/ou=People,o=JNDITutorial");

try {
  LdapContext ctx = new InitialLdapContext(env, null);

  // Activate paged results
  int pageSize = 5;
  byte[] cookie = null;
  ctx.setRequestControls(new Control[] { new PagedResultsControl(pageSize,
      Control.NONCRITICAL) });
  int total;

  do {
    /* perform the search */
    NamingEnumeration results = ctx.search("", "(objectclass=*)",
        new SearchControls());

    /* for each entry print out name + all attrs and values */
    while (results != null && results.hasMore()) {
      SearchResult entry = (SearchResult) results.next();
      System.out.println(entry.getName());
    }

    // Examine the paged results control response
    Control[] controls = ctx.getResponseControls();
    if (controls != null) {
      for (int i = 0; i < controls.length; i++) {
        if (controls[i] instanceof PagedResultsResponseControl) {
          PagedResultsResponseControl prrc = (PagedResultsResponseControl) controls[i];
          total = prrc.getResultSize();
          if (total != 0) {
            System.out.println("***************** END-OF-PAGE "
                + "(total : " + total + ") *****************\n");
          } else {
            System.out.println("***************** END-OF-PAGE "
                + "(total: unknown) ***************\n");
          }
          cookie = prrc.getCookie();
        }
      }
    } else {
      System.out.println("No controls were sent from the server");
    }
    // Re-activate paged results
    ctx.setRequestControls(new Control[] { new PagedResultsControl(
        pageSize, cookie, Control.CRITICAL) });

  } while (cookie != null);

  ctx.close();

Example copied from here.

Gca answered 31/7, 2015 at 16:10 Comment(0)
M
11

Although it's almost certainly the case that the server is enforcing the size limit of 1000 entries, there are potentially ways to get around that by issuing the request in multiple parts.

If the server supports the use of the simple paged results control (as defined in RFC 2696 and supported in the LDAP SDK as per https://docs.ldap.com/ldap-sdk/docs/javadoc/com/unboundid/ldap/sdk/controls/SimplePagedResultsControl.html), then you can use it to iterate through the results in "pages" containing a specified number of entries.

Alternately, the virtual list view (VLV) request control (https://www.unboundid.com/products/ldap-sdk/docs/javadoc/index.html?com/unboundid/ldap/sdk/controls/VirtualListViewRequestControl.html) could be used, but I would probably only recommend that if the server doesn't support the simple paged results control because the VLV request control also requires that the results be sorted, and that likely either requires special configuration in the server or some pretty expensive processing in order to be able to service the request.

Marentic answered 3/7, 2012 at 17:0 Comment(0)
V
10

I solved like @PeterK , but with some modifications

    public List<MyUser> listUsers() {
    LOG.info("listUsers() inicio");
    List<MyUser> users = new ArrayList<MyUser>();

    Hashtable env = new Hashtable();
    env.put(Context.INITIAL_CONTEXT_FACTORY, INITIAL_CTX);
    env.put(Context.PROVIDER_URL, 'ldap://192.168.10.10:389');
    env.put(Context.SECURITY_AUTHENTICATION, CONNECTION_TYPE);
    env.put(Context.SECURITY_PRINCIPAL, USER_ADMIN_PASSWORD);
    env.put(Context.SECURITY_CREDENTIALS, USER_ADMIN);

    try {
        LdapContext ctx = new InitialLdapContext(env, null);

        // Activate paged results
        int pageSize = 1000;
        byte[] cookie = null;
        ctx.setRequestControls(new Control[] { new PagedResultsControl(pageSize, Control.NONCRITICAL) });
        int total;

        do {
            /* perform the search */
            SearchControls sc = new SearchControls();
            sc.setSearchScope(SearchControls.SUBTREE_SCOPE);
            String filtro = "(&(sAMAccountName=*)&(objectClass=user))";
            NamingEnumeration results = ctx.search(getBaseDn(ctx), filtro, sc);

            /* for each entry */
            while (results.hasMoreElements()) {
                SearchResult result = (SearchResult) results.nextElement();
                Attributes attributes = result.getAttributes();
                //convert to MyUser class
                MyUser user = toUser(attributes);
                users.add(user);
            }

            // Examine the paged results control response
            Control[] controls = ctx.getResponseControls();
            if (controls != null) {
                for (int i = 0; i < controls.length; i++) {
                    if (controls[i] instanceof PagedResultsResponseControl) {
                        PagedResultsResponseControl prrc = (PagedResultsResponseControl) controls[i];
                        total = prrc.getResultSize();
                        if (total != 0) {
                            System.out.println("***************** END-OF-PAGE " + "(total : " + total + ") *****************\n");
                        } else {
                            System.out.println("***************** END-OF-PAGE " + "(total: unknown) ***************\n");
                        }
                        cookie = prrc.getCookie();
                    }
                }
            } else {
                System.out.println("No controls were sent from the server");
            }
            // Re-activate paged results
            ctx.setRequestControls(new Control[] { new PagedResultsControl(pageSize, cookie, Control.CRITICAL) });

        } while (cookie != null);

        ctx.close();

    } catch (NamingException e) {
        System.err.println("PagedSearch failed.");
        e.printStackTrace();
    } catch (IOException ie) {
        System.err.println("PagedSearch failed.");
        ie.printStackTrace();
    } catch (Exception ie) {
        System.err.println("PagedSearch failed.");
        ie.printStackTrace();
    }

    LOG.info("listUsers() size = " + (users.size()));
    LOG.info("listUsers() fim");

    return users;
}


private MyUser toUser(Attributes attributes) throws NamingException {
    if (attributes != null) {
        String fullName = attributes.get("distinguishedName") != null ? attributes.get("distinguishedName").get().toString() : null;
        String mail = attributes.get("mail") != null ? attributes.get("mail").get().toString() : null;
        String userName = attributes.get("cn") != null ? attributes.get("cn").get().toString() : null;
        String userPrincipalName = attributes.get("userPrincipalName") != null ? attributes.get("userPrincipalName").get().toString() : null;

        if (userPrincipalName != null) {
            String[] user = userPrincipalName.split("@");
            if (user != null && user.length > 0) {
                userName = user[0];
            }
        }

        MyUser user = new MyUser();
        user.setFullName(fullName);
        user.setEmail(mail);
        user.setName(userName);
        user.setUserPrincipalName(userPrincipalName);
        user.setRoles(getRolesUser(attributes));

        return user;
    }

    return null;
}
Vesture answered 8/1, 2016 at 19:19 Comment(6)
PeterK will never see your message as you did itChapland
you would enter a comment under his Answer. Remember: under your Answer the following people are alerted when you post a comment under your Answer ... (1) the Op I think but don't hold me to it, (2) another commenter like myself if that person is the sole person with a comment thread as in no one else is commenting, (3) someone you @theirName that is already under your answer in a comment or the OpChapland
It was only because (2) was the case above that I was alerted that you wrote that comment above "How do this?" ... otherwise if some 3rd person commented, I would not have been alerted. A lot of people think they are being ignored on SO in comments but in reality they have no clue they are being spoken to due to the way it worksChapland
Thanks @Chapland , but my posted code above, does not fit in the comment space.Lutenist
I hear ya. You appear to have a nice answer above. I was just alerting you to the reality of how people get pingedChapland
It is very slowBorchers
M
5

The LDAP client is setting a "client-requested" size limit of 2000. This client-requested limit cannot override the limits set in the configuration of the server. No matter what the client-requested size limit is, the server's size limit overrides it. Contact your directory server administrator and ask that the size limit be increased.

Mopup answered 3/7, 2012 at 15:19 Comment(1)
if you were not able to change it in the request the "Active Directory Users and Computers" will also wont work .when i get the message on the 2000 entries limit on that tool i change the settings to a higher value and it works. so what you wrote does not sound correctly to me.Kumar

© 2022 - 2024 — McMap. All rights reserved.