Memcached and Sharding in Font

Encode Quick Response Code in Font Memcached and Sharding

Memcached and Sharding
QR Creator In None
Using Barcode encoder for Font Control to generate, create Quick Response Code image in Font applications.
www.OnBarcode.com
PDF-417 2d Barcode Printer In None
Using Barcode generator for Font Control to generate, create PDF 417 image in Font applications.
www.OnBarcode.com
The design of Memcached illustrates an important principle that is used in several other kinds of databases, and which you might want to employ in architectures of your own: the clients shard the database by hashing the keys string values and letting the hash determine which member of the cluster is consulted for each key. To understand why this is effective, consider a particular key/value pair like the key sq:42 and the value 1764 that might be stored by Listing 8 1. To make the best use of the RAM it has available, the Memcached cluster wants to store this key and value exactly once. But to make the service fast, it wants to avoid duplication without requiring any coordination between the different servers or communication between all of the clients. This means that all of the clients, without any other information to go on than (a) the key and (b) the list of Memcached servers with which they are configured, need some scheme for working out where that piece of information belongs. If they fail to make the same decision, then not only might the key and value be copied on to several servers and reduce the overall memory available, but also a client s attempt to remove an invalid entry could leave other invalid copies elsewhere. The solution is that the clients all implement a single, stable algorithm that can turn a key into an integer n that selects one of the servers from their list. They do this by using a hash algorithm, which mixes the bits of a string when forming a number so that any pattern in the string is, hopefully, obliterated. To see why patterns in key values must be obliterated, consider Listing 8 2. It loads a dictionary of English words (you might have to download a dictionary of your own or adjust the path to make the script run on your own machine), and explores how those words would be distributed across four servers if they were used as keys. The first algorithm tries to divide the alphabet into four roughly equal sections and distributes the keys using their first letter; the other two algorithms use hash functions. Listing 8 2. Two Schemes for Assigning Data to Servers #!/usr/bin/env python # Foundations of Python Network Programming - 8 - hashing.py # Hashes are a great way to divide work. import hashlib
Print Code 3 Of 9 In None
Using Barcode creator for Font Control to generate, create Code-39 image in Font applications.
www.OnBarcode.com
Code 128 Generation In None
Using Barcode creator for Font Control to generate, create Code 128 image in Font applications.
www.OnBarcode.com
CHAPTER 8 CACHES, MESSAGE QUEUES, AND MAP-REDUCE
UPC Code Creation In None
Using Barcode printer for Font Control to generate, create UPCA image in Font applications.
www.OnBarcode.com
Generate Data Matrix 2d Barcode In None
Using Barcode drawer for Font Control to generate, create DataMatrix image in Font applications.
www.OnBarcode.com
def
European Article Number 13 Maker In None
Using Barcode maker for Font Control to generate, create EAN-13 image in Font applications.
www.OnBarcode.com
Making MSI Plessey In None
Using Barcode printer for Font Control to generate, create MSI Plessey image in Font applications.
www.OnBarcode.com
alpha_shard(word): """Do a poor job of assigning data to servers by using first letters.""" if word[0] in 'abcdef': return 'server0' elif word[0] in 'ghijklm': return 'server1' elif word[0] in 'nopqrs': return 'server2' else: return 'server3'
QR Code ISO/IEC18004 Creation In None
Using Barcode creation for Office Excel Control to generate, create QR Code image in Microsoft Excel applications.
www.OnBarcode.com
QR-Code Printer In Java
Using Barcode encoder for Android Control to generate, create QR-Code image in Android applications.
www.OnBarcode.com
def hash_shard(word): """Do a great job of assigning data to servers using a hash value.""" return 'server%d' % (hash(word) % 4) def md5_shard(word): """Do a great job of assigning data to servers using a hash value.""" # digest() is a byte string, so we ord() its last character return 'server%d' % (ord(hashlib.md5(word).digest()[-1]) % 4)
Reading GS1-128 In C#
Using Barcode reader for VS .NET Control to read, scan read, scan image in VS .NET applications.
www.OnBarcode.com
UCC - 12 Generator In None
Using Barcode generation for Software Control to generate, create GS1-128 image in Software applications.
www.OnBarcode.com
words = open('/usr/share/dict/words').read().split() for function in alpha_shard, hash_shard, md5_shard: d = {'server0': 0, 'server1': 0, 'server2': 0, 'server3': 0} for word in words: d[function(word.lower())] += 1 print function.__name__[:-6], d
Barcode Maker In Java
Using Barcode drawer for Android Control to generate, create Barcode image in Android applications.
www.OnBarcode.com
GS1 - 13 Printer In Java
Using Barcode printer for BIRT reports Control to generate, create EAN / UCC - 13 image in BIRT reports applications.
www.OnBarcode.com
The hash() function is Python s own built-in hash routine, which is designed to be blazingly fast because it is used internally to implement Python dictionary lookup. The MD5 algorithm is much more sophisticated because it was actually designed as a cryptographic hash; although it is now considered too weak for security use, using it to distribute load across servers is fine (though slow). The results show quite plainly the danger of trying to distribute load using any method that could directly expose the patterns in your data: $ python hashing.py alpha {'server0': 35203, 'server1': 22816, 'server2': 28615, 'server3': 11934} hash {'server0': 24739, 'server1': 24622, 'server2': 24577, 'server3': 24630} md5 {'server0': 24671, 'server1': 24726, 'server2': 24536, 'server3': 24635} You can see that distributing load by first letters results in server 0 getting more than three times the load of server 3, even though it was assigned only six letters instead of seven! The hash routines, however, both performed like champions: despite all of the strong patterns that characterize not only the first letters but also the entire structure and endings of English words, the hash functions scattered the words very evenly across the four buckets. Though many data sets are not as skewed as the letter distributions of English words, sharded databases like Memcached always have to contend with the appearance of patterns in their input data. Listing 8 1, for example, was not unusual in its use of keys that always began with a common prefix (and that were followed by characters from a very restricted alphabet: the decimal digits). These kinds of obvious patterns are why sharding should always be performed through a hash function. Of course, this is an implementation detail that you can often ignore when you use a database system like Memcached that supports sharding internally. But if you ever need to design a service of your own that automatically assigns work or data to nodes in a cluster in a way that needs to be reproducible, then you will find the same technique useful in your own code.
Create EAN / UCC - 13 In Objective-C
Using Barcode generator for iPad Control to generate, create European Article Number 13 image in iPad applications.
www.OnBarcode.com
USS Code 128 Scanner In C#.NET
Using Barcode decoder for .NET framework Control to read, scan read, scan image in .NET framework applications.
www.OnBarcode.com
Painting PDF 417 In VS .NET
Using Barcode creator for Reporting Service Control to generate, create PDF417 image in Reporting Service applications.
www.OnBarcode.com
Code 128A Reader In .NET Framework
Using Barcode recognizer for VS .NET Control to read, scan read, scan image in VS .NET applications.
www.OnBarcode.com
Recognizing GTIN - 13 In VB.NET
Using Barcode reader for Visual Studio .NET Control to read, scan read, scan image in .NET framework applications.
www.OnBarcode.com
GTIN - 12 Drawer In Visual Basic .NET
Using Barcode printer for .NET Control to generate, create UPC-A Supplement 2 image in .NET framework applications.
www.OnBarcode.com
Copyright © OnBarcode.com . All rights reserved.