Solr Error Document is missing mandatory uniqueKey field id

Question

While Importing the data into Solr using DataImportHandler, I am getting the below error. Please someone provide your suggestion.

org.apache.solr.common.SolrException: Document is missing mandatory uniqueKey field: id
        at org.apache.solr.update.AddUpdateCommand.getIndexedId(AddUpdateCommand.java:92)
        at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:717)
        at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:557)
        at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
        at org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:70)
        at org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:235)
        at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:512)
        at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416)
        at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:331)
        at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:239)
        at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:411)
        at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
        at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:464)

Knotweed

Please provide a minimal example of your input and import code. – Whiz

Like it says: the id field is mandatory (but does not exist or might be empty). Add an id field before posting the document to the solr server. – Planography

Answer 1

In the schema.xml file you have mentioned id as required field = true.

Also the document that you are trying to index in SOLR do not contain this id field and hence SOLR is throwing this error.

Solution

1. Either add id to all your documents

OR

2. Remove required = true form schema file for id field.

Please share your schema.xml file and the documents that you are trying to index into SOLR.

Also keep in mind if you want quick response try to provide as much details as you can.

Deirdra

Answer 2

this is probably where the error reporting should be improved for solr

for my case, I've defined a string field not related with KEY, and I am trying to put a null value to that field. I should probably indicate the field is nullable.

Logia

Answer 3

I had this problem too and the reason was in my data import configuration. If you are using a hsqldb connection to import your data, pay attention to the way you map the database columns to your solr fields. Hsqldb is case sensitive. So, for example, the following was producing an error.

<field column="item_id" name="id" />

While this is working perfectly:

<field column="ITEM_ID" name="id" />

Pulverize

Answer 4

I got this issue whereby I either had nulls or duplicates in the id field.

Check that there are no duplicates in your id field with:

SELECT MAX(c) FROM (SELECT COUNT(x) as c, if FROM TABLE GROUP BY id)

And check the response is = 1. If not, you can find the fields with:

SELECT COUNT(*) as c, id FROM <table> GROUP BY id HAVING c > 1

Neomaneomah

Answer 5

I had the same issue, even if everything was working fine. I created new SolrDocument in each loop while indexing.

HttpSolrServer server =
      new HttpSolrServer("http://localhost:8983/solr/shard1");
Collection<SolrInputDocument> docs = new ArrayList<>();

for (list<String> data: datas) {
  
    SolrInputDocument doc = new SolrInputDocument();
    String id = data("id");
    String fileName = data("fileName");
    String column2 = data("column2");
    String column3 = data("column3");

    i++;
    doc.addField("id", id);
    doc.addField("title", fileName);
    doc.addField("size", getFileSizeInMb(fileEntry.length()));
    doc.addField("column2", column2);
    doc.addField("column3", column3);

    System.out.println("doc: " + doc);
    docs.add(doc);
    
    if (i % 100 == 0) {
        System.out.println("Committed :" + i);
        server.add(docs);
        docs.clear();
    }  
}
server.commit();

initializing this in each loop made my code work

SolrInputDocument doc = new SolrInputDocument();

Coccidioidomycosis