I am connecting to a MongoDB sharding server via mongodb java driver in Spring MVC. I am using the following versions: spring-webmvc-3.2.1.RELEASE mongo-java-driver/2.10.0/mongo-java-driver-2.10....

Assume there is a worker service that receives messages from a queue, reads the product with the specified Id from a document database, applies some manipulation logic based on the message, and fin...

I have been reading about scalable architectures recently. In that context, two words that keep on showing up with regards to databases are sharding and partitioning. I looked up descriptions but s...

Mongo DB supports zoned sharding (as described here) by associating each shard with zone sh.addShardTag(<shard name>, "ZONENAME") and applying zone chunk ranges based on sharding key values w...

I read you can have multiple consumer apps per kinesis stream. http://docs.aws.amazon.com/kinesis/latest/dev/developing-consumers-with-kcl.html however, I heard you can only have one consumer per s...

What is the best approach for Sharding MySQL tables. The approaches I can think of are : Application Level sharding? Sharding at MySQL proxy layer? Central lookup server for sharding? Do you kn...

I have a collection "documentDev" present in the database with sharding key as 'dNumber' Sample Document : { "_id" : "12831221wadaee23", "dNumber" : "115", "processed": false } If I try to upda...

I have an index on AWS Elasticsearch which were unassighed due to NODE_LEFT. Here's an output of _cat/shards rawindex-2017.07.04 1 p STARTED rawindex-2017.07.04 3 p UNASSIGNED NODE_LEFT rawindex-...

I have an ES cluster with 4 nodes: number_of_replicas: 1 search01 - master: false, data: false search02 - master: true, data: true search03 - master: false, data: true search04 - master: false, da...

I'm having an issue with some of my elastic search indices in the cluster: I have 5 regular shards for an example index logs-2021.08, so when I'm running _cat/shards elastic API I'm getting good re...

What's the best way to deal with a sharded database in Rails? Should the sharding be handled at the application layer, the active record layer, the database driver layer, a proxy layer, or somethin...

I have a Django project based on multiple PostgreSQL servers. I want users to be sharded across those database servers using the same sharding logic used by Instagram: User ID => logical shard ID...

For an online marketplace product under construction, I have a situation which requires implementing a database sharding solution. I am new to sharding and after reading the posts in this forum I f...

I'm currently in the process of converting my 6 years old C# application to .NET Core v3 and EF Core (and also using Blazor). Most of it is working except for the Sharding part. Our application cre...

I'd like to try a project I found on GitHub, so I installed MongoDB on MacOS and now I'm trying to understand how to set up it correctly through the docker compose file in the directory. This is th...

With Elasticsearch 2.3.3, is there a way to get shard sizes using the GET API which returns JSON? Currently I have found the following methods to get shard size, both of which are problematic: /_r...

This might be a dummy question, but I cannot find any clue in all online doc. For a already-built-up mongodb cluster, How can I find which sharding key(s) is used for given collecton?

$lookup is new in MongoDB 3.2. It performs a left outer join to an unsharded collection in the same database to filter in documents from the “joined” collection for processing. To use $lookup, the...

Here's the problem I am trying to solve: I have recently completed a data layer re-design that allows me to load-balance my database across multiple shards. In order to keep shards balanced, I need...

I am using Hibernate/JPA. When i do an entity.save() or session.update(entity), hibernate generates a query like this :- update TABLE1 set COL_1=? , COL_2=? , COL_3=? where COL_PK=? Can I include ...

I recently had a SNAFU cause my cluster to end up with split-brain (despite having many controls in place) resulting in shards that are basically busted. I've got all the nodes back in play properl...

I am trying to shard MongoDB. I am done with Sharding configuration, but I am not sure how to verify if sharding is functional. How do i check whether my data is get sharded? Is there a query to v...

I am running an elastic search version 4.1 on windows 8. I tried to index a document through java. When running a JUNIT test the error appears as below. org.elasticsearch.action.UnavailableShards...

When I read about sharding, looks like authors don't take into account other tables the sharded table has to be joined to (even though they describe a shard as a "subset of an original database"). ...

In a non-sharded DB, I could just use auto-increment to generate a unique ID to reference a specific row. I want to shard my DB, say into 12 shards. Now when I insert into a specific shard, the au...