How to convert between Json and Map<String, AttributeValue> in

Question

I want to save an object that is encoded as a Json string to DynamoDB using the AWS Java SDK 2.0.

In the AWS Java SDK 1.n, it is possible to convert standard Json strings to DynamoDB AttributeValues using Item.fromJSON(myJsonString).toAttributeValues().

Though it is possible to use both SDKs at once, the AttributeValue defined by the two SDK versions (1.11, 2.0) are not the same and cannot be used interchangeably.

Is there any AWS-provided or community-standard way to go from a json string/blob to a Map<String, AttributeValue> for the AWS Java SDK 2.0?

---

Please note—this question is asking about how to solve the problem for AWS Java SDK 2.0, not the dynamodbv2 model of AWS Java SDK 1.n. If you think this question is a duplicate, please double check the SDK version of the question/answer that it duplicates.

Grajeda

Were you able to figure this out? – Doyenne

My data model was fairly small, so I hand-wrote my own converter directly from my data model to AttributeValues. – Grajeda

Answer 1

This was a bit of a pain, but this should work for people:

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.MapperFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;

import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.ParameterizedType;
import java.lang.reflect.Type;
import java.util.Map;
import java.util.Objects;
import java.util.stream.Collectors;

public class DdbUtils {
    private static final ItemParameterizedType ITEM_PARAMETERIZED_TYPE = new ItemParameterizedType();
    private static final TypeReference<Map<String, AttributeValue.Builder>> ITEM_TYPEREF = new TypeReference<>() {
        @Override
        public Type getType() {
            return ITEM_PARAMETERIZED_TYPE;
        }
    };
    private final ObjectMapper objectMapper;

    public DdbUtils() {
        // This is necessary because the fields are commonly "S" or "N", but the AttributeValue builder class
        // expects "s" and "n".
        this(new ObjectMapper().configure(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES, true));
    }

    public DdbUtils(ObjectMapper objectMapper) {
        this.objectMapper = Objects.requireNonNull(objectMapper);
    }

    public Map<String, AttributeValue> readItem(String string) throws IOException {
        return buildItemMap(objectMapper.readValue(string, ITEM_TYPEREF));
    }

    public Map<String, AttributeValue> readItem(InputStream inputStream) throws IOException {
        return buildItemMap(objectMapper.readValue(inputStream, ITEM_TYPEREF));
    }

    public Map<String, AttributeValue> convertItem(Object object) {
        return buildItemMap(objectMapper.convertValue(object, ITEM_TYPEREF));
    }

    // ObjectMapper has a lot of methods; feel free to create more *Item methods right here.

    private Map<String, AttributeValue> buildItemMap(Map<String, AttributeValue.Builder> map) {
        return map.entrySet().stream()
            .collect(Collectors.toMap(
                Map.Entry::getKey,
                entry -> entry.getValue().build()
            ));
    }

    // Map<String, AttributeValue.Builder>
    private static class ItemParameterizedType implements ParameterizedType {
        @Override
        public Type getRawType() {
            return Map.class;
        }

        @Override
        public Type[] getActualTypeArguments() {
            return new Type[] { String.class, AttributeValue.serializableBuilderClass() };
        }

        @Override
        public Type getOwnerType() {
            return null;
        }
    }
}

Usage:

DdbUtils ddbUtils = new DdbUtils();
Map<String, AttributeValue> item = ddbUtils.readItem(/* whatever */);

Using AWS Java SDK v2, Jackson 2.12, and Java 17.

Syllogism

Answer 2

The core of problem is that the both AttributeValue and one of its non standart members, SdkBytes are not standard pojos and so don't have suitable Serializer/Deserializer.

Let's make this happen.

Serializer:

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.ser.std.StdSerializer;
import software.amazon.awssdk.core.SdkBytes;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.Map;

public class AttributeValueCustomSerializer extends StdSerializer<AttributeValue> {

    protected AttributeValueCustomSerializer(Class<AttributeValue> attributeValueClass) {
        super(attributeValueClass);
    }
    public AttributeValueCustomSerializer(){
        this(null);
    }
    @Override
    public void serialize(AttributeValue av, JsonGenerator jGen, SerializerProvider serializerProvider) throws IOException {
        jGen.writeStartObject();
        if(av.s() != null) {
            jGen.writeStringField("S", av.s());
        }
        if(av.n() != null) {
            jGen.writeStringField("N", av.n());
        }
        if(av.b() != null) {
            jGen.writeStringField("B", av.b().asString(StandardCharsets.UTF_8));
        }
        if(av.hasSs()) {
           jGen.writeArrayFieldStart("SS");
            for (String arg: av.ss()) {
                jGen.writeString(arg);
            }
           jGen.writeEndArray();
        }
        if(av.hasNs()) {
            jGen.writeArrayFieldStart("NS");
            for (String arg: av.ns()) {
                jGen.writeString(arg);
            }
            jGen.writeEndArray();
        }
        if(av.hasBs()) {
            jGen.writeArrayFieldStart("BS");
            for (SdkBytes arg: av.bs()) {
                jGen.writeString(arg.asString(StandardCharsets.UTF_8));
            }
            jGen.writeEndArray();
        }
        if(av.hasM()) {
            jGen.writeObjectFieldStart("M");
            for (Map.Entry<String,AttributeValue> e: av.m().entrySet()) {
                jGen.writeObjectField(e.getKey(), e.getValue());
            }
            jGen.writeEndObject();
        }
        if(av.hasL()) {
            jGen.writeArrayFieldStart("L");
            for (AttributeValue arg: av.l()) {
                jGen.writeObject(arg);
            }
            jGen.writeEndArray();
        }
        if(av.bool() != null) {
            jGen.writeBooleanField("BOOL", av.bool());
        }
        if(av.nul() != null) {
            jGen.writeBooleanField("NUL", av.nul());
        }
        jGen.writeEndObject();
    }
}

Deserializer:

import com.fasterxml.jackson.core.JacksonException;
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.deser.std.StdDeserializer;
import software.amazon.awssdk.core.SdkBytes;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;

import java.io.IOException;
import java.util.Map;
import java.util.stream.Collectors;
import java.util.stream.StreamSupport;

public class AttributeValueCustomDeserializer extends StdDeserializer<AttributeValue> {

    public AttributeValueCustomDeserializer() {
        this(null);
    }

    public AttributeValueCustomDeserializer(Class<?> vc) {
        super(vc);
    }

    private AttributeValue deserialize(JsonNode node) {
        AttributeValue.Builder builder  = AttributeValue.builder();
        if(node.has("S")) {
            builder = builder.s(node.get("S").asText());
        }
        if(node.has("N")) {
            builder = builder.n(node.get("N").asText());
        }
        if(node.has("B")) {
            builder = builder.b(SdkBytes.fromUtf8String(node.get("B").asText()));
        }
        if(node.has("SS")) {
            builder = builder.ss(StreamSupport.stream(node.get("SS").spliterator(),false).map(JsonNode::asText)
                    .collect(Collectors.toList()));
        }
        if(node.has("NS")) {
            builder = builder.ns(StreamSupport.stream(node.get("NS").spliterator(),false).map(JsonNode::asText)
                    .collect(Collectors.toList()));
        }
        if(node.has("BS")) {
            builder = builder.bs(StreamSupport.stream(node.get("BS").spliterator(),false)
                    .map(JsonNode::asText).map(SdkBytes::fromUtf8String).collect(Collectors.toList()));
        }
        if(node.has("M")) {
            builder = builder.m(node.get("M").properties().stream().collect(Collectors.toMap(Map.Entry::getKey,
                    e -> deserialize(e.getValue()))));
        }
        if(node.has("L")) {
            builder = builder.l(StreamSupport.stream(node.get("L").spliterator(),false).map(this::deserialize).collect(Collectors.toList()));
        }
        if(node.has("BOOL")) {
            builder = builder.bool(node.get("BOOL").asBoolean());
        }
        if(node.has("NUL")) {
            builder = builder.nul(node.get("NUL").asBoolean());
        }
        return builder.build();
    }

    @Override
    public AttributeValue deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException, JacksonException {
        JsonNode node = jp.getCodec().readTree(jp);
        return deserialize(node);
    }
}

Test/Usage:

  void serializeDeserializeTest() throws JsonProcessingException {
    // Mapper initialization
    ObjectMapper mapper = new ObjectMapper();
    //Custom Serializer/Deserializer registration
    SimpleModule simpleModule = new SimpleModule();
    simpleModule.addSerializer(AttributeValue.class, new AttributeValueCustomSerializer());
    simpleModule.addDeserializer(AttributeValue.class, new AttributeValueCustomDeserializer());
    mapper.registerModule(simpleModule);

    AttributeValue originalAttributeValue = generateAttributeValue();
    String s = mapper.writeValueAsString(originalAttributeValue);
    assertNotNull(s);
    assertEquals(originalAttributeValue, mapper.readValue(s,AttributeValue.class));
}

You should only generate the original AttributeValue. I'm using podam, for this porpoise, but it is out of the scope of this post. Used those two baedung articles about custom serialization and deserialization.

Astroid

Answer 3

An example for converting JSON to AttributeValue of DynamoDB

    String productString = readFileFromResources("data/categoryFix.json");
    HashMap<String,Object> result = new ObjectMapper().readValue(productString, HashMap.class);
    Map<String, AttributeValue> attributes = InternalUtils.fromSimpleMap(result);

Above code worked well, even though InternalUtils is deprecated.

Lisettelisha

This answer is not for SDK 2 as requested. – Tonguing

Answer 4

The InternalUtils is deprecated but the code below works fine and also shows, how you can convert that to PutItemResult:

HashMap responseMap = objectMapper.readValue(responseBody, HashMap.class);
            Map<String, AttributeValue> attributes = InternalUtils.fromSimpleMap(responseMap);

            PutItemResult result = new PutItemResult();
            result.setAttributes(attributes);

Lanark

Answer 5

Here are a couple ways to do this (in Kotlin) depending on whether you have simple JSON or the DynamoDB json:

import com.amazonaws.services.dynamodbv2.document.ItemUtils
import com.amazonaws.services.dynamodbv2.model.AttributeValue
import com.fasterxml.jackson.core.type.TypeReference

...
val objectMapper = ObjectMapper().apply {
    configure(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES, true)
}

...

// Method 1 - simple JSON
val obj1: MutableMap<String, AttributeValue> = ItemUtils.fromSimpleMap(
    objectMapper.readValue(
        """
                {
                 "fruits": ["apple", "pear"],
                 "user": { "name": "Bob" }
                }
            """,
        object : TypeReference<Map<String, Any>>() {})
)

// Method 2 - DynamoDB JSON
val obj2: Map<String, AttributeValue> =
    objectMapper.readValue("""
        {
         "fruits": {
          "L": [
             { 
              "S": "apple"
             },
             { 
              "S": "pear"
             }
          ]
         },
         "user": {
            "M": {
             "name": {
              "S": "Bob"
             }
            }
           }
        }
    """.trimIndent(),
    object : TypeReference<Map<String, AttributeValue>>() {}
)

Goldengoldenberg