How to get a link to fetch next records from Azure Search?

Question

currently I am creating an application where I need to call API of azure search.

Here is the API : https://serviceName.search.windows.net/indexes/global-index/docs/search?api-version=2016-09-01

Here is my Java code :

@Override
public JSONObject global(SearchParametersDto searchInTableDto) {
    JSONObject jsonOutput = new JSONObject();
    ArrayList encodedURLList = new ArrayList < > ();
    try {
        String url = "https://" + searchInTableDto.getServiceName().toString() + ".search.windows.net/indexes/" +
            searchInTableDto.getIndexName().toString() + "/docs/search?api-version=2016-09-01";

        HttpClient client = new DefaultHttpClient();
        HttpPost post = new HttpPost(url);
        post.setHeader("Content-Type", "application/json");
        post.setHeader("api-key", searchInTableDto.getApiKey().toString());
        String clientId = searchInTableDto.getClientId().toString();
        String updatedClientId = " \"+" + clientId + "\" ";
        JSONObject jsonInput = new JSONObject();


        jsonInput.put("search", "(test||test||test||test||test||test||test)+ Contacts+Campaigns+Companies+Targets+Complanits+Claims+Activities+Opportunities+Completed Activities");
        //jsonInput.put("top", 1000);

        System.out.println(jsonInput);
        post.setEntity(new StringEntity(jsonInput.toString(), ContentType.create("application/json")));
        HttpResponse response = client.execute(post);
        BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
        StringBuffer resultOutput = new StringBuffer();
        String line = "";
        while ((line = rd.readLine()) != null) {
            resultOutput.append(line);
        }
        JSONParser parser = new JSONParser();
        jsonOutput = (JSONObject) parser.parse(resultOutput.toString());
        org.json.simple.JSONArray valuesArray = (org.json.simple.JSONArray) jsonOutput.get("value");
        //jsonOutput.put("searchResult", valuesArray);
        System.out.println(valuesArray.size());
    } catch (Exception e) {
        e.printStackTrace();
        String errorMessageAndClassWithMethod = getErrorContainingClassAndMethod();
        throw new SpringAppRuntimeException(errorMessageAndClassWithMethod + e.toString());
    }
    return jsonOutput;
}

when I run my search query on Azure search , I get link for next result like : "@odata.nextLink": "https://serviceName.search.windows.net/indexes('global-index')/docs?api-version=2015-02-28-Preview&search=%28test%7C%7Ctest%7C%7Ctest%7C%7Ctest%7C%7Ctest%7C%7Ctest%7C%7Ctest%29%2B%20Contacts%2BCampaigns%2BCompanies%2BTargets%2BComplanits%2BClaims%2BActivities%2BOpportunities%2BCompleted%20Activities&$skip=50"

But, when I run the same query through my Java service, I get following link for next document :

"@odata.nextLink": "https://serviceName.search.windows.net/indexes('global-index')/docs/search.post.search?api-version=2016-09-01"

By this second link, I can't get the next document. Why the link in JSON object differs from actual required link?

Another thing is , when I put condition for "top" through my code, result never return me link for next document.What may be the reason for this?

How may I get the correct link to see next documents in my JSON output?

Abelmosk

Answer 1

It looks like you're using different HTTP verbs and that's why you're getting different links. The first link is for a GET request, while the second link is for a POST request.

It is not recommended to rely on the @odata.nextLink to implement paging. That mechanism exists to protect your service from expensive requests, not to be a general mechanism for paging. The search service itself decides when to return this link and that behavior is subject to change, so you should not rely on it.

Instead, you should request the pages you want via $top and $skip. For example, if you want to present results in pages of 100 documents each, your first request would have $top=100&$skip=0, your second request would have $top=100&$skip=100, your third request would have $top=100&$skip=200, and so on. You can continue fetching like this until there are no more results.

Boxfish

Thanks. It would be easy to to implement JAVA logic after getting total count. Currently I need to retrieve 2123 documents(JSON objects in result). Is there any way so that we may get total count of results, after running search query? – Abelmosk

Use $count=true. All the Search API parameters are documented here: learn.microsoft.com/rest/api/searchservice/… – Boxfish