What is sharding in system design?

Sharding is a database architecture pattern that involves horizontally partitioning data across multiple databases. Each partition is referred to as a shard, and each shard can be hosted on a separate database server. Sharding is used to scale databases to handle large volumes of data and high throughput.

What are the different sharding strategies?

Common sharding strategies include Algorithmic/Hashed Sharding, Dynamic/Range-Based Sharding, and Directory-Based Sharding. The choice of strategy depends on factors like the data distribution, query patterns, and the need for rebalancing.

System Design Interview

Database Sharding

Learn how to partition your database for massive scale. Explore different sharding keys, the challenges of rebalancing, and how to handle cross-shard transactions.

Example: Hash-Based Sharding Logic in Python

A simple function to determine which shard a given key belongs to using a hash-based approach. This is a common way to distribute data evenly.


                                class HashSharding:
                                    def __init__(self, num_shards):
                                        self.num_shards = num_shards
                                
                                    def get_shard(self, key):
                                        # Use a simple hash function (in a real system, a more robust hash like MD5 or SHA-1 would be used)
                                        hashed_key = hash(key)
                                        shard_id = hashed_key % self.num_shards
                                        return shard_id

Our AI Coach can help you explore the pros and cons of different sharding keys and discuss complex topics like consistent hashing to minimize data movement when re-sharding.