Compare commits

...
52 Commits
Author SHA1 Message Date
antirez 0155386608 Redis 2.4.14 2012-05-23 11:32:58 +02:00
jokea c6ee2fd595 Set fd to writable when poll(2) detects POLLERR or POLLHUP event. 2012-05-23 11:29:21 +02:00
ThePicard 8a599d36c0 Fixed a typo in install_server.sh 2012-05-23 11:12:22 +02:00
antirez 496e8133c9 No longer used TODO file removed. 2012-05-23 11:10:17 +02:00
antirez b0a2093956 Fixed issue #516 (ZINTERSTORE mixing sets and zsets).
Weeks ago trying to fix an harmless GCC warning I introduced a bug in
the ziplist-encoded implementations of sorted sets.

The bug completely broke zuiNext() iterator, that is used in the
ZINTERSTORE and ZUNIONSTORE implementation, so those two commands are no
longer reliable starting from Redis version 2.4.12 and latest 2.6.0-RC
releases.

This commit fixes the problem and adds a regression test.
2012-05-23 11:08:33 +02:00
antirez cd2f985a5e Add an implementation of mstime() to 2.4.
Commit 9419eb0 back ported a better expired keys collection algorithm
that is more incremental but for it to compile and work we need an
implementation of the mstime() function that was missing in 2.4.
2012-05-12 09:40:38 +02:00
antirez 47d8dab2b1 redis-cli pipe mode: handle EINTR properly as well so that SIGSTOP/SIGCONT are handled correctly. 2012-05-12 09:37:14 +02:00
antirez b074461732 redis-cli pipe mode: handle EAGAIN while writing to socket. 2012-05-12 09:37:09 +02:00
antirez f1af84d714 redis-cli --pipe for mass import. 2012-05-12 09:37:01 +02:00
antirez 59b084b7cf If the computer running the Redis test is slow, we revert to --clients 1 to avoid false positives. 2012-05-12 09:34:12 +02:00
antirez 735b0e0142 Comment improved so that the code goal is more clear. Thx to @agladysh. 2012-05-12 09:33:51 +02:00
antirez 9419eb0ffd More incremental active expired keys collection process.
If a large amonut of keys are all expiring about at the same time, the
"active" expired keys collection cycle used to block as far as the
percentage of already expired keys was >= 25% of the total population of
keys with an expire set.

This could block the server even for many seconds in order to reclaim
memory ASAP. The new algorithm uses at max a small amount of
milliseconds per cycle, even if this means reclaiming the memory less
promptly it also means a more responsive server.
2012-05-12 09:33:47 +02:00
antirez 6375e5dc0e memtest.c: integer overflow fixed. 2012-05-11 16:05:08 +02:00
antirez d8769edf15 Macros ULONG_ONEZERO / ULONG_ZEROONE were inverted in #ifdef to test 32/64 bit arch. 2012-05-11 16:04:27 +02:00
antirez e2c0197d98 Fix PREFIX typo in Makefile. 2012-05-09 20:45:00 +02:00
antirez 2cf55e3a47 Allow PREFIX to be overrided in Makefile. 2012-05-09 10:35:01 +02:00
antirez e74fe10b00 Redis 2.4.13 2012-05-02 12:14:48 +02:00
Pieter Noordhuis f2ffb59294 Use safe dictionary iterator from KEYS
Every matched key in a KEYS call is checked for expiration. When the key
is set to expire, the call to `getExpire` will assert that the key also
exists in the main dictionary. This in turn causes a rehashing step to
be executed. Rehashing a dictionary when there is an iterator active may
result in the iterator emitting duplicate entries, or not emitting some
entries at all. By using a safe iterator, the rehash step is omitted.
2012-05-01 10:52:27 +02:00
antirez 9cb09c4d2c Redis 2.4.12 2012-04-30 12:27:50 +02:00
antirez fc7ea2f7cd Properly wait the slave to sync with master in BRPOPLPUSH test. 2012-04-30 11:50:42 +02:00
antirez fb38fd3c89 A more lightweight implementation of issue 141 regression test. 2012-04-30 11:48:32 +02:00
antirez 9e066ae051 Redis test: More reliable BRPOPLPUSH replication test.
Now it uses the new wait_for_condition testing primitive.
Also wait_for_condition implementation was fixed in this commit to properly
escape the expr command and its argument.
2012-04-30 11:48:16 +02:00
antirez 28500d193f Testing framework fixes and improvements backported from 2.6. 2012-04-30 11:47:47 +02:00
antirez 63bae7c553 A few compiler warnings suppressed. 2012-04-29 17:20:56 +02:00
antirez 379ac85eed memtest.c fixed to actually use v1 and v2 in memtest_fill_value(). 2012-04-29 17:17:44 +02:00
antirez df10c797e0 Limit memory used by big SLOWLOG entries.
Two limits are added:

1) Up to SLOWLOG_ENTRY_MAX_ARGV arguments are logged.
2) Up to SLOWLOG_ENTRY_MAX_STRING bytes per argument are logged.
3) slowlog-max-len is set to 128 by default (was 1024).

The number of remaining arguments / bytes is logged in the entry
so that the user can understand better the nature of the logged command.
2012-04-21 21:50:25 +02:00
antirez f743cacc85 Redis 2.4.11 2012-04-19 14:56:16 +02:00
antirez 922283b553 New redis-cli backported from unstable. 2012-04-18 21:20:35 +02:00
antirez 32fef1566a Marginally cleaner lookupKeyByPattern() implementation.
just fieldobj itself as sentinel of the fact a field object is used or
not, instead of using the filed length, that may be confusing both for
people and for the compiler emitting a warning.
2012-04-18 11:40:48 +02:00
antirez 8e5e8f0eca lookupKeyByPattern() used by SORT GET/BY rewritten. Fixes issue #460.
lookupKeyByPattern() was implemented with a trick to speedup the lookup
process allocating two fake Redis obejcts on the stack. However now that
we propagate expires to the slave as DEL operations the lookup of the
key may result into a call to expireIfNeeded() having the stack
allocated object as argument, that may in turn use it to create the
protocol to send to the slave. But since this fake obejcts are
inherently read-only this is a problem.

As a side effect of this fix there are no longer size limits in the
pattern to be used with GET/BY option of SORT.

See https://github.com/antirez/redis/issues/460 for bug details.
2012-04-17 18:27:18 +02:00
jokea 7b4dc9b979 implement aeWait using poll(2). Fixes issue #267. 2012-04-06 11:49:40 +02:00
antirez 95c6a1a5b8 Structure field controlling the INFO field master_link_down_since_seconds initialized correctly to avoid strange INFO output at startup when a slave has yet to connect to its master. 2012-04-04 18:31:37 +02:00
antirez 1fa954a558 Redis 2.4.10 2012-03-30 13:07:18 +02:00
antirez 06665b7191 Useless GCC warning suppressed. 2012-03-30 13:02:08 +02:00
Salvatore Sanfilippo 3c38b0876e Merge pull request #417 from josephjang/e95ca1db21b2dbe82f39a3f7f6390cb5f58dafae
a memory leak with replication on two or more dbs including >= db10
2012-03-30 01:28:58 -07:00
Joseph Jang e95ca1db21 Fixed a memory leak with replication
occurs when two or more dbs are replicated and at least one of them is >db10
2012-03-30 02:17:21 +09:00
antirez 6938960b66 Fix for slaves chains. Force resync of slaves (simply disconnecting them) when SLAVEOF turns a master into a slave. 2012-03-29 09:31:39 +02:00
antirez 9a5cbf9f7e Memory addressing test implemented. 2012-03-19 14:03:00 +01:00
antirez 1dfe75a09b Redis 2.4.9 2012-03-18 22:14:28 +01:00
antirez 8122954b9a Memory test feature backported into 2.4. 2012-03-18 22:08:44 +01:00
antirez f5f95b35d5 More memory tests implemented. Default number of iterations lowered to a more acceptable value of 50. 2012-03-18 22:00:08 +01:00
antirez 015dba6adb Fixed typo. 2012-03-18 22:00:04 +01:00
antirez 83f0c677f8 Number of iteration of --test-memory is now 300 (several minutes per gigabyte). Memtest86 and Memtester links are also displayed while running the test. 2012-03-18 21:59:59 +01:00
antirez a7e66f8048 On crash suggest to give --test-memory a try. 2012-03-18 21:59:52 +01:00
antirez ae92e6dc15 Memory test function now less boring thanks to screen-wide progress bar. 2012-03-18 21:58:40 +01:00
antirez e306e99029 Hem... actual memtest.c file added. 2012-03-18 21:58:35 +01:00
antirez 9c3bc7529d First implementation of --test-memory. Still a work in progress. 2012-03-18 21:58:24 +01:00
antirez 7961d7ea0d redis-benchmark: backported the more capable version from the unstable branch (pipelining,run selected tests,fixes) 2012-02-26 11:41:44 +01:00
antirez a3b580c0c5 The universe is 80 columns wide, accept it. 2012-02-22 17:23:13 +01:00
Brad Voth 660a1277be updated utils/install_server.sh to support chkconfig boxes (redhat/
centos) as well as debian/ubuntu
2012-02-22 17:23:07 +01:00
antirez b805430a9e Fix for issue #306, thanks to tchajed (on github) for the pull request. The original patch was reworked a bit. 2012-02-22 16:09:01 +01:00
antirez a8a3275566 Redis-cli should accept help command even if there is no connection to the server. 2012-02-22 15:16:26 +01:00
30 changed files with 1215 additions and 298 deletions
+67
View File
@@ -18,6 +18,73 @@ to modify your program in order to use Redis 2.4.
CHANGELOG
---------
What's new in Redis 2.4.13
=========================
UPGRADE URGENCY: high, many non trivial bugs fixed in this release.
* [BUGFIX] Fixed issue #518 (Redis 99% CPU when master down).
* [BUGFIX] Fixed issue #516 (ZINTERSTORE mixing sets and zsets).
* [BUGFIX] Fixed a bug in install_server.sh when using chkconfig
* [BUGFIX] Fixes to --test-memory implementation.
* [BUGFIX] Allow PREFIX to be overridden in Makefile.
* [BUGFIX] The test is now more reliable on slow computers.
* redis-cli --pipe mode, see http://redis.io/topics/mass-insert
* Much better expired keys collection algorithm that makes the server much
more responsive when a lot of keys are expiring at the same time.
What's new in Redis 2.4.13
=========================
UPGRADE URGENCY: high for all the users of the KEYS command, otherwise low.
* [BUGFIX] Fix for KEYS command: if the DB contains keys with expires the KEYS
command may return the wrong output, having duplicated or missing
keys. See issue #487 and #488 on github for details.
What's new in Redis 2.4.12
=========================
UPGRADE URGENCY: low if you don't experience any of the fixed problems.
* [BUGFIX] Limit the amount of memory consumed by the slow log.
* [BUGFIX] --test-memory option fixes.
* [BUGFIX] Less false positives in tests.
What's new in Redis 2.4.11
=========================
UPGRADE URGENCY: moderate if you don't experience any of the fixed problems.
* [BUGFIX] Fixed a problem with aeWait() implementation. May cause a crash
under non easy to replicate condiitons. See issue #267 on github.
* [BUGFIX] SORT with GET/BY option fetching expiring keys fixed. Issue #460.
* [BUGFIX] INFO field master_link_down_since_seconds initialized correctly.
* [FEATURE] redis-cli back ported from Redis unstable. Now has support for
--bigkeys (to sample the DB for very large keys), --slave to
simulate a slave instance.
What's new in Redis 2.4.10
==========================
UPGRADE URGENCY: High for users using replication with a DB identifier higher than 9, or chaining replication so that slaves are attached to other slaves. Low for all the other users.
* [FEATURE] Redis --test-meory improved with new test testing addressing.
* [BUGFIX] Fixed a memory leak: happens when replication is used with DB > 9.
* [BUGFIX] Fixed a replication bug that happens in chained replicaiton when a slave is attached to a master that is later turned into a slave. Now the chained slave reconnection is properly forced.
What's new in Redis 2.4.9
=========================
UPGRADE URGENCY: low. Mostly new features and minor bug fixing.
* [FEATURE] Redis server is now able to test your memory for broken RAM.
Usage: ./redis-server --test-memory <megabytes>.
* [FEATURE] redis-benchmark backported from unstable. Pipelining, run selected
tests, and a few more features.
* [BUGFIX] utils/install_server.sh script now works on Redhat / Centos.
* [BUGFIX] Minor fix to redis-cli (github issue #306).
What's new in Redis 2.4.8
=========================
-4
View File
@@ -1,4 +0,0 @@
This is a stable release! No TODO file here.
Please check the TODO file in the master branch on github.
https://github.com/antirez/redis/raw/master/TODO
+1 -1
View File
@@ -356,7 +356,7 @@ slowlog-log-slower-than 10000
# There is no limit to this length. Just be aware that it will consume memory.
# You can reclaim memory used by the slow log with SLOWLOG RESET.
slowlog-max-len 1024
slowlog-max-len 128
################################ VIRTUAL MEMORY ###############################
+3 -3
View File
@@ -54,13 +54,13 @@ endif
CCOPT= $(CFLAGS) $(ARCH) $(PROF)
PREFIX= /usr/local
PREFIX?=/usr/local
INSTALL_BIN= $(PREFIX)/bin
INSTALL= cp -pf
OBJ = adlist.o ae.o anet.o dict.o redis.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o vm.o pubsub.o multi.o debug.o sort.o intset.o syncio.o slowlog.o bio.o
OBJ = adlist.o ae.o anet.o dict.o redis.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o vm.o pubsub.o multi.o debug.o sort.o intset.o syncio.o slowlog.o bio.o memtest.o
BENCHOBJ = ae.o anet.o redis-benchmark.o sds.o adlist.o zmalloc.o
CLIOBJ = anet.o sds.o adlist.o redis-cli.o zmalloc.o release.o
CLIOBJ = anet.o sds.o adlist.o redis-cli.o zmalloc.o release.o ae.o
CHECKDUMPOBJ = redis-check-dump.o lzf_c.o lzf_d.o
CHECKAOFOBJ = redis-check-aof.o
+12 -12
View File
@@ -35,6 +35,8 @@
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#include <poll.h>
#include <string.h>
#include "ae.h"
#include "zmalloc.h"
@@ -358,21 +360,19 @@ int aeProcessEvents(aeEventLoop *eventLoop, int flags)
/* Wait for millseconds until the given file descriptor becomes
* writable/readable/exception */
int aeWait(int fd, int mask, long long milliseconds) {
struct timeval tv;
fd_set rfds, wfds, efds;
struct pollfd pfd;
int retmask = 0, retval;
tv.tv_sec = milliseconds/1000;
tv.tv_usec = (milliseconds%1000)*1000;
FD_ZERO(&rfds);
FD_ZERO(&wfds);
FD_ZERO(&efds);
memset(&pfd, 0, sizeof(pfd));
pfd.fd = fd;
if (mask & AE_READABLE) pfd.events |= POLLIN;
if (mask & AE_WRITABLE) pfd.events |= POLLOUT;
if (mask & AE_READABLE) FD_SET(fd,&rfds);
if (mask & AE_WRITABLE) FD_SET(fd,&wfds);
if ((retval = select(fd+1, &rfds, &wfds, &efds, &tv)) > 0) {
if (FD_ISSET(fd,&rfds)) retmask |= AE_READABLE;
if (FD_ISSET(fd,&wfds)) retmask |= AE_WRITABLE;
if ((retval = poll(&pfd, 1, milliseconds))== 1) {
if (pfd.revents & POLLIN) retmask |= AE_READABLE;
if (pfd.revents & POLLOUT) retmask |= AE_WRITABLE;
if (pfd.revents & POLLERR) retmask |= AE_WRITABLE;
if (pfd.revents & POLLHUP) retmask |= AE_WRITABLE;
return retmask;
} else {
return retval;
+1 -1
View File
@@ -273,7 +273,7 @@ void keysCommand(redisClient *c) {
unsigned long numkeys = 0;
void *replylen = addDeferredMultiBulkLength(c);
di = dictGetIterator(c->db->dict);
di = dictGetSafeIterator(c->db->dict);
allkeys = (pattern[0] == '*' && pattern[1] == '\0');
while((de = dictNext(di)) != NULL) {
sds key = dictGetEntryKey(de);
+225
View File
@@ -0,0 +1,225 @@
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <assert.h>
#include <limits.h>
#include <errno.h>
#include <termios.h>
#include <sys/ioctl.h>
#if (ULONG_MAX == 4294967295UL)
#define MEMTEST_32BIT
#elif (ULONG_MAX == 18446744073709551615ULL)
#define MEMTEST_64BIT
#else
#error "ULONG_MAX value not supported."
#endif
#ifdef MEMTEST_32BIT
#define ULONG_ONEZERO 0xaaaaaaaaUL
#define ULONG_ZEROONE 0x55555555UL
#else
#define ULONG_ONEZERO 0xaaaaaaaaaaaaaaaaUL
#define ULONG_ZEROONE 0x5555555555555555UL
#endif
static struct winsize ws;
size_t progress_printed; /* Printed chars in screen-wide progress bar. */
size_t progress_full; /* How many chars to write to fill the progress bar. */
void memtest_progress_start(char *title, int pass) {
int j;
printf("\x1b[H\x1b[2J"); /* Cursor home, clear screen. */
/* Fill with dots. */
for (j = 0; j < ws.ws_col*(ws.ws_row-2); j++) printf(".");
printf("Please keep the test running several minutes per GB of memory.\n");
printf("Also check http://www.memtest86.com/ and http://pyropus.ca/software/memtester/");
printf("\x1b[H\x1b[2K"); /* Cursor home, clear current line. */
printf("%s [%d]\n", title, pass); /* Print title. */
progress_printed = 0;
progress_full = ws.ws_col*(ws.ws_row-3);
fflush(stdout);
}
void memtest_progress_end(void) {
printf("\x1b[H\x1b[2J"); /* Cursor home, clear screen. */
}
void memtest_progress_step(size_t curr, size_t size, char c) {
size_t chars = ((unsigned long long)curr*progress_full)/size, j;
for (j = 0; j < chars-progress_printed; j++) {
printf("%c",c);
progress_printed++;
}
fflush(stdout);
}
/* Test that addressing is fine. Every location is populated with its own
* address, and finally verified. This test is very fast but may detect
* ASAP big issues with the memory subsystem. */
void memtest_addressing(unsigned long *l, size_t bytes) {
unsigned long words = bytes/sizeof(unsigned long);
unsigned long j, *p;
/* Fill */
p = l;
for (j = 0; j < words; j++) {
*p = (unsigned long)p;
p++;
if ((j & 0xffff) == 0) memtest_progress_step(j,words*2,'A');
}
/* Test */
p = l;
for (j = 0; j < words; j++) {
if (*p != (unsigned long)p) {
printf("\n*** MEMORY ADDRESSING ERROR: %p contains %lu\n",
(void*) p, *p);
exit(1);
}
p++;
if ((j & 0xffff) == 0) memtest_progress_step(j+words,words*2,'A');
}
}
/* Fill words stepping a single page at every write, so we continue to
* touch all the pages in the smallest amount of time reducing the
* effectiveness of caches, and making it hard for the OS to transfer
* pages on the swap. */
void memtest_fill_random(unsigned long *l, size_t bytes) {
unsigned long step = 4096/sizeof(unsigned long);
unsigned long words = bytes/sizeof(unsigned long)/2;
unsigned long iwords = words/step; /* words per iteration */
unsigned long off, w, *l1, *l2;
assert((bytes & 4095) == 0);
for (off = 0; off < step; off++) {
l1 = l+off;
l2 = l1+words;
for (w = 0; w < iwords; w++) {
#ifdef MEMTEST_32BIT
*l1 = *l2 = ((unsigned long) (rand()&0xffff)) |
(((unsigned long) (rand()&0xffff)) << 16);
#else
*l1 = *l2 = ((unsigned long) (rand()&0xffff)) |
(((unsigned long) (rand()&0xffff)) << 16) |
(((unsigned long) (rand()&0xffff)) << 32) |
(((unsigned long) (rand()&0xffff)) << 48);
#endif
l1 += step;
l2 += step;
if ((w & 0xffff) == 0)
memtest_progress_step(w+iwords*off,words,'R');
}
}
}
/* Like memtest_fill_random() but uses the two specified values to fill
* memory, in an alternated way (v1|v2|v1|v2|...) */
void memtest_fill_value(unsigned long *l, size_t bytes, unsigned long v1,
unsigned long v2, char sym)
{
unsigned long step = 4096/sizeof(unsigned long);
unsigned long words = bytes/sizeof(unsigned long)/2;
unsigned long iwords = words/step; /* words per iteration */
unsigned long off, w, *l1, *l2, v;
assert((bytes & 4095) == 0);
for (off = 0; off < step; off++) {
l1 = l+off;
l2 = l1+words;
v = (off & 1) ? v2 : v1;
for (w = 0; w < iwords; w++) {
#ifdef MEMTEST_32BIT
*l1 = *l2 = ((unsigned long) v) |
(((unsigned long) v) << 16);
#else
*l1 = *l2 = ((unsigned long) v) |
(((unsigned long) v) << 16) |
(((unsigned long) v) << 32) |
(((unsigned long) v) << 48);
#endif
l1 += step;
l2 += step;
if ((w & 0xffff) == 0)
memtest_progress_step(w+iwords*off,words,sym);
}
}
}
void memtest_compare(unsigned long *l, size_t bytes) {
unsigned long words = bytes/sizeof(unsigned long)/2;
unsigned long w, *l1, *l2;
assert((bytes & 4095) == 0);
l1 = l;
l2 = l1+words;
for (w = 0; w < words; w++) {
if (*l1 != *l2) {
printf("\n*** MEMORY ERROR DETECTED: %p != %p (%lu vs %lu)\n",
(void*)l1, (void*)l2, *l1, *l2);
exit(1);
}
l1 ++;
l2 ++;
if ((w & 0xffff) == 0) memtest_progress_step(w,words,'=');
}
}
void memtest_compare_times(unsigned long *m, size_t bytes, int pass, int times) {
int j;
for (j = 0; j < times; j++) {
memtest_progress_start("Compare",pass);
memtest_compare(m,bytes);
memtest_progress_end();
}
}
void memtest_test(size_t megabytes, int passes) {
size_t bytes = megabytes*1024*1024;
unsigned long *m = malloc(bytes);
int pass = 0;
if (m == NULL) {
fprintf(stderr,"Unable to allocate %zu megabytes: %s",
megabytes, strerror(errno));
exit(1);
}
while (pass != passes) {
pass++;
memtest_progress_start("Addressing test",pass);
memtest_addressing(m,bytes);
memtest_progress_end();
memtest_progress_start("Random fill",pass);
memtest_fill_random(m,bytes);
memtest_progress_end();
memtest_compare_times(m,bytes,pass,4);
memtest_progress_start("Solid fill",pass);
memtest_fill_value(m,bytes,0,(unsigned long)-1,'S');
memtest_progress_end();
memtest_compare_times(m,bytes,pass,4);
memtest_progress_start("Checkerboard fill",pass);
memtest_fill_value(m,bytes,ULONG_ONEZERO,ULONG_ZEROONE,'C');
memtest_progress_end();
memtest_compare_times(m,bytes,pass,4);
}
}
void memtest(size_t megabytes, int passes) {
if (ioctl(1, TIOCGWINSZ, &ws) == -1) {
ws.ws_col = 80;
ws.ws_row = 20;
}
memtest_test(megabytes,passes);
printf("\nYour memory passed this test.\n");
printf("Please if you are still in doubt use the following two tools:\n");
printf("1) memtest86: http://www.memtest86.com/\n");
printf("2) memtester: http://pyropus.ca/software/memtester/\n");
exit(0);
}
+15 -14
View File
@@ -491,6 +491,16 @@ static void freeClientArgv(redisClient *c) {
c->cmd = NULL;
}
/* Close all the slaves connections. This is useful in chained replication
* when we resync with our own master and want to force all our slaves to
* resync with us as well. */
void disconnectSlaves(void) {
while (listLength(server.slaves)) {
listNode *ln = listFirst(server.slaves);
freeClient((redisClient*)ln->value);
}
}
void freeClient(redisClient *c) {
listNode *ln;
@@ -568,21 +578,12 @@ void freeClient(redisClient *c) {
server.master = NULL;
server.replstate = REDIS_REPL_CONNECT;
server.repl_down_since = time(NULL);
/* Since we lost the connection with the master, we should also
* close the connection with all our slaves if we have any, so
* when we'll resync with the master the other slaves will sync again
* with us as well. Note that also when the slave is not connected
* to the master it will keep refusing connections by other slaves.
/* We lost connection with our master, force our slaves to resync
* with us as well to load the new data set.
*
* We do this only if server.masterhost != NULL. If it is NULL this
* means the user called SLAVEOF NO ONE and we are freeing our
* link with the master, so no need to close link with slaves. */
if (server.masterhost != NULL) {
while (listLength(server.slaves)) {
ln = listFirst(server.slaves);
freeClient((redisClient*)ln->value);
}
}
* If server.masterhost is NULL the user called SLAVEOF NO ONE so
* slave resync is not needed. */
if (server.masterhost != NULL) disconnectSlaves();
}
/* Release memory */
zfree(c->argv);
+194 -89
View File
@@ -62,24 +62,28 @@ static struct config {
int randomkeys;
int randomkeys_keyspacelen;
int keepalive;
int pipeline;
long long start;
long long totlatency;
long long *latency;
const char *title;
list *clients;
int quiet;
int csv;
int loop;
int idlemode;
char *tests;
} config;
typedef struct _client {
redisContext *context;
sds obuf;
char *randptr[10]; /* needed for MSET against 10 keys */
char *randptr[32]; /* needed for MSET against 10 keys */
size_t randlen;
unsigned int written; /* bytes of 'obuf' already written */
long long start; /* start time of a request */
long long latency; /* request latency */
int pending; /* Number of pending requests (sent but no reply received) */
} *client;
/* Prototypes */
@@ -135,6 +139,7 @@ static void resetClient(client c) {
aeDeleteFileEvent(config.el,c->context->fd,AE_READABLE);
aeCreateFileEvent(config.el,c->context->fd,AE_WRITABLE,writeHandler,c);
c->written = 0;
c->pending = config.pipeline;
}
static void randomizeClientKey(client c) {
@@ -180,19 +185,26 @@ static void readHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
fprintf(stderr,"Error: %s\n",c->context->errstr);
exit(1);
} else {
if (redisGetReply(c->context,&reply) != REDIS_OK) {
fprintf(stderr,"Error: %s\n",c->context->errstr);
exit(1);
}
if (reply != NULL) {
if (reply == (void*)REDIS_REPLY_ERROR) {
fprintf(stderr,"Unexpected error reply, exiting...\n");
while(c->pending) {
if (redisGetReply(c->context,&reply) != REDIS_OK) {
fprintf(stderr,"Error: %s\n",c->context->errstr);
exit(1);
}
if (reply != NULL) {
if (reply == (void*)REDIS_REPLY_ERROR) {
fprintf(stderr,"Unexpected error reply, exiting...\n");
exit(1);
}
if (config.requests_finished < config.requests)
config.latency[config.requests_finished++] = c->latency;
clientDone(c);
freeReplyObject(reply);
if (config.requests_finished < config.requests)
config.latency[config.requests_finished++] = c->latency;
c->pending--;
if (c->pending == 0) clientDone(c);
} else {
break;
}
}
}
}
@@ -234,8 +246,10 @@ static void writeHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
}
}
static client createClient(const char *cmd, size_t len) {
static client createClient(char *cmd, size_t len) {
int j;
client c = zmalloc(sizeof(struct _client));
if (config.hostsocket == NULL) {
c->context = redisConnectNonBlock(config.hostip,config.hostport);
} else {
@@ -249,19 +263,21 @@ static client createClient(const char *cmd, size_t len) {
fprintf(stderr,"%s: %s\n",config.hostsocket,c->context->errstr);
exit(1);
}
c->obuf = sdsnewlen(cmd,len);
/* Queue N requests accordingly to the pipeline size. */
c->obuf = sdsempty();
for (j = 0; j < config.pipeline; j++)
c->obuf = sdscatlen(c->obuf,cmd,len);
c->randlen = 0;
c->written = 0;
c->pending = config.pipeline;
/* Find substrings in the output buffer that need to be randomized. */
if (config.randomkeys) {
char *p = c->obuf, *newline;
char *p = c->obuf;
while ((p = strstr(p,":rand:")) != NULL) {
newline = strstr(p,"\r\n");
assert(newline-(p+6) == 12); /* 12 chars for randomness */
assert(c->randlen < (signed)(sizeof(c->randptr)/sizeof(char*)));
c->randptr[c->randlen++] = p+6;
p = newline+2;
p += 6;
}
}
@@ -276,7 +292,7 @@ static void createMissingClients(client c) {
int n = 0;
while(config.liveclients < config.numclients) {
createClient(c->obuf,sdslen(c->obuf));
createClient(c->obuf,sdslen(c->obuf)/config.pipeline);
/* Listen backlog is quite limited on most systems */
if (++n > 64) {
@@ -295,7 +311,7 @@ static void showLatencyReport(void) {
float perc, reqpersec;
reqpersec = (float)config.requests_finished/((float)config.totlatency/1000);
if (!config.quiet) {
if (!config.quiet && !config.csv) {
printf("====== %s ======\n", config.title);
printf(" %d requests completed in %.2f seconds\n", config.requests_finished,
(float)config.totlatency/1000);
@@ -313,12 +329,14 @@ static void showLatencyReport(void) {
}
}
printf("%.2f requests per second\n\n", reqpersec);
} else if (config.csv) {
printf("\"%s\",\"%.2f\"\n", config.title, reqpersec);
} else {
printf("%s: %.2f requests per second\n", config.title, reqpersec);
}
}
static void benchmark(const char *title, const char *cmd, int len) {
static void benchmark(char *title, char *cmd, int len) {
client c;
config.title = title;
@@ -367,7 +385,11 @@ int parseOptions(int argc, const char **argv) {
if (lastarg) goto invalid;
config.datasize = atoi(argv[++i]);
if (config.datasize < 1) config.datasize=1;
if (config.datasize > 1024*1024) config.datasize = 1024*1024;
if (config.datasize > 1024*1024*1024) config.datasize = 1024*1024*1024;
} else if (!strcmp(argv[i],"-P")) {
if (lastarg) goto invalid;
config.pipeline = atoi(argv[++i]);
if (config.pipeline <= 0) config.pipeline=1;
} else if (!strcmp(argv[i],"-r")) {
if (lastarg) goto invalid;
config.randomkeys = 1;
@@ -376,10 +398,23 @@ int parseOptions(int argc, const char **argv) {
config.randomkeys_keyspacelen = 0;
} else if (!strcmp(argv[i],"-q")) {
config.quiet = 1;
} else if (!strcmp(argv[i],"--csv")) {
config.csv = 1;
} else if (!strcmp(argv[i],"-l")) {
config.loop = 1;
} else if (!strcmp(argv[i],"-I")) {
config.idlemode = 1;
} else if (!strcmp(argv[i],"-t")) {
if (lastarg) goto invalid;
/* We get the list of tests to run as a string in the form
* get,set,lrange,...,test_N. Then we add a comma before and
* after the string in order to make sure that searching
* for ",testname," will always get a match if the test is
* enabled. */
config.tests = sdsnew(",");
config.tests = sdscat(config.tests,(char*)argv[++i]);
config.tests = sdscat(config.tests,",");
sdstolower(config.tests);
} else if (!strcmp(argv[i],"--help")) {
exit_status = 0;
goto usage;
@@ -398,24 +433,41 @@ invalid:
printf("Invalid option \"%s\" or option argument missing\n\n",argv[i]);
usage:
printf("Usage: redis-benchmark [-h <host>] [-p <port>] [-c <clients>] [-n <requests]> [-k <boolean>]\n\n");
printf(" -h <hostname> Server hostname (default 127.0.0.1)\n");
printf(" -p <port> Server port (default 6379)\n");
printf(" -s <socket> Server socket (overrides host and port)\n");
printf(" -c <clients> Number of parallel connections (default 50)\n");
printf(" -n <requests> Total number of requests (default 10000)\n");
printf(" -d <size> Data size of SET/GET value in bytes (default 2)\n");
printf(" -k <boolean> 1=keep alive 0=reconnect (default 1)\n");
printf(" -r <keyspacelen> Use random keys for SET/GET/INCR, random values for SADD\n");
printf(" Using this option the benchmark will get/set keys\n");
printf(" in the form mykey_rand000000012456 instead of constant\n");
printf(" keys, the <keyspacelen> argument determines the max\n");
printf(" number of values for the random number. For instance\n");
printf(" if set to 10 only rand000000000000 - rand000000000009\n");
printf(" range will be allowed.\n");
printf(" -q Quiet. Just show query/sec values\n");
printf(" -l Loop. Run the tests forever\n");
printf(" -I Idle mode. Just open N idle connections and wait.\n");
printf(
"Usage: redis-benchmark [-h <host>] [-p <port>] [-c <clients>] [-n <requests]> [-k <boolean>]\n\n"
" -h <hostname> Server hostname (default 127.0.0.1)\n"
" -p <port> Server port (default 6379)\n"
" -s <socket> Server socket (overrides host and port)\n"
" -c <clients> Number of parallel connections (default 50)\n"
" -n <requests> Total number of requests (default 10000)\n"
" -d <size> Data size of SET/GET value in bytes (default 2)\n"
" -k <boolean> 1=keep alive 0=reconnect (default 1)\n"
" -r <keyspacelen> Use random keys for SET/GET/INCR, random values for SADD\n"
" Using this option the benchmark will get/set keys\n"
" in the form mykey_rand:000000012456 instead of constant\n"
" keys, the <keyspacelen> argument determines the max\n"
" number of values for the random number. For instance\n"
" if set to 10 only rand:000000000000 - rand:000000000009\n"
" range will be allowed.\n"
" -P <numreq> Pipeline <numreq> requests. Default 1 (no pipeline).\n"
" -q Quiet. Just show query/sec values\n"
" --csv Output in CSV format\n"
" -l Loop. Run the tests forever\n"
" -t <tests> Only run the comma separated list of tests. The test\n"
" names are the same as the ones produced as output.\n"
" -I Idle mode. Just open N idle connections and wait.\n\n"
"Examples:\n\n"
" Run the benchmark with the default configuration against 127.0.0.1:6379:\n"
" $ redis-benchmark\n\n"
" Use 20 parallel clients, for a total of 100k requests, against 192.168.1.1:\n"
" $ redis-benchmark -h 192.168.1.1 -p 6379 -n 100000 -c 20\n\n"
" Fill 127.0.0.1:6379 with about 1 million keys only using the SET test:\n"
" $ redis-benchmark -t set -n 1000000 -r 100000000\n\n"
" Benchmark 127.0.0.1:6379 for a few commands producing CSV output:\n"
" $ redis-benchmark -t ping,set,get -n 100000 --csv\n\n"
" Fill a list with 10000 random elements:\n"
" $ redis-benchmark -r 10000 -n 10000 lpush mylist ele:rand:000000000000\n\n"
);
exit(exit_status);
}
@@ -424,6 +476,7 @@ int showThroughput(struct aeEventLoop *eventLoop, long long id, void *clientData
REDIS_NOTUSED(id);
REDIS_NOTUSED(clientData);
if (config.csv) return 250;
float dt = (float)(mstime()-config.start)/1000.0;
float rps = (float)config.requests_finished/dt;
printf("%s: %.2f\r", config.title, rps);
@@ -431,6 +484,20 @@ int showThroughput(struct aeEventLoop *eventLoop, long long id, void *clientData
return 250; /* every 250ms */
}
/* Return true if the named test was selected using the -t command line
* switch, or if all the tests are selected (no -t passed by user). */
int test_is_selected(char *name) {
char buf[256];
int l = strlen(name);
if (config.tests == NULL) return 1;
buf[0] = ',';
memcpy(buf+1,name,l);
buf[l+1] = ',';
buf[l+2] = '\0';
return strstr(config.tests,buf) != NULL;
}
int main(int argc, const char **argv) {
int i;
char *data, *cmd;
@@ -448,9 +515,11 @@ int main(int argc, const char **argv) {
aeCreateTimeEvent(config.el,1,showThroughput,NULL,NULL);
config.keepalive = 1;
config.datasize = 3;
config.pipeline = 1;
config.randomkeys = 0;
config.randomkeys_keyspacelen = 0;
config.quiet = 0;
config.csv = 0;
config.loop = 0;
config.idlemode = 0;
config.latency = NULL;
@@ -458,6 +527,7 @@ int main(int argc, const char **argv) {
config.hostip = "127.0.0.1";
config.hostport = 6379;
config.hostsocket = NULL;
config.tests = NULL;
i = parseOptions(argc,argv);
argc -= i;
@@ -500,71 +570,106 @@ int main(int argc, const char **argv) {
memset(data,'x',config.datasize);
data[config.datasize] = '\0';
benchmark("PING (inline)","PING\r\n",6);
if (test_is_selected("ping_inline") || test_is_selected("ping"))
benchmark("PING_INLINE","PING\r\n",6);
len = redisFormatCommand(&cmd,"PING");
benchmark("PING",cmd,len);
free(cmd);
const char *argv[21];
argv[0] = "MSET";
for (i = 1; i < 21; i += 2) {
argv[i] = "foo:rand:000000000000";
argv[i+1] = data;
if (test_is_selected("ping_mbulk") || test_is_selected("ping")) {
len = redisFormatCommand(&cmd,"PING");
benchmark("PING_BULK",cmd,len);
free(cmd);
}
len = redisFormatCommandArgv(&cmd,21,argv,NULL);
benchmark("MSET (10 keys)",cmd,len);
free(cmd);
len = redisFormatCommand(&cmd,"SET foo:rand:000000000000 %s",data);
benchmark("SET",cmd,len);
free(cmd);
if (test_is_selected("set")) {
len = redisFormatCommand(&cmd,"SET foo:rand:000000000000 %s",data);
benchmark("SET",cmd,len);
free(cmd);
}
len = redisFormatCommand(&cmd,"GET foo:rand:000000000000");
benchmark("GET",cmd,len);
free(cmd);
if (test_is_selected("get")) {
len = redisFormatCommand(&cmd,"GET foo:rand:000000000000");
benchmark("GET",cmd,len);
free(cmd);
}
len = redisFormatCommand(&cmd,"INCR counter:rand:000000000000");
benchmark("INCR",cmd,len);
free(cmd);
if (test_is_selected("incr")) {
len = redisFormatCommand(&cmd,"INCR counter:rand:000000000000");
benchmark("INCR",cmd,len);
free(cmd);
}
len = redisFormatCommand(&cmd,"LPUSH mylist %s",data);
benchmark("LPUSH",cmd,len);
free(cmd);
if (test_is_selected("lpush")) {
len = redisFormatCommand(&cmd,"LPUSH mylist %s",data);
benchmark("LPUSH",cmd,len);
free(cmd);
}
len = redisFormatCommand(&cmd,"LPOP mylist");
benchmark("LPOP",cmd,len);
free(cmd);
if (test_is_selected("lpop")) {
len = redisFormatCommand(&cmd,"LPOP mylist");
benchmark("LPOP",cmd,len);
free(cmd);
}
len = redisFormatCommand(&cmd,"SADD myset counter:rand:000000000000");
benchmark("SADD",cmd,len);
free(cmd);
if (test_is_selected("sadd")) {
len = redisFormatCommand(&cmd,
"SADD myset counter:rand:000000000000");
benchmark("SADD",cmd,len);
free(cmd);
}
len = redisFormatCommand(&cmd,"SPOP myset");
benchmark("SPOP",cmd,len);
free(cmd);
if (test_is_selected("spop")) {
len = redisFormatCommand(&cmd,"SPOP myset");
benchmark("SPOP",cmd,len);
free(cmd);
}
len = redisFormatCommand(&cmd,"LPUSH mylist %s",data);
benchmark("LPUSH (again, in order to bench LRANGE)",cmd,len);
free(cmd);
if (test_is_selected("lrange") ||
test_is_selected("lrange_100") ||
test_is_selected("lrange_300") ||
test_is_selected("lrange_500") ||
test_is_selected("lrange_600"))
{
len = redisFormatCommand(&cmd,"LPUSH mylist %s",data);
benchmark("LPUSH (needed to benchmark LRANGE)",cmd,len);
free(cmd);
}
len = redisFormatCommand(&cmd,"LRANGE mylist 0 99");
benchmark("LRANGE (first 100 elements)",cmd,len);
free(cmd);
if (test_is_selected("lrange") || test_is_selected("lrange_100")) {
len = redisFormatCommand(&cmd,"LRANGE mylist 0 99");
benchmark("LRANGE_100 (first 100 elements)",cmd,len);
free(cmd);
}
len = redisFormatCommand(&cmd,"LRANGE mylist 0 299");
benchmark("LRANGE (first 300 elements)",cmd,len);
free(cmd);
if (test_is_selected("lrange") || test_is_selected("lrange_300")) {
len = redisFormatCommand(&cmd,"LRANGE mylist 0 299");
benchmark("LRANGE_300 (first 300 elements)",cmd,len);
free(cmd);
}
len = redisFormatCommand(&cmd,"LRANGE mylist 0 449");
benchmark("LRANGE (first 450 elements)",cmd,len);
free(cmd);
if (test_is_selected("lrange") || test_is_selected("lrange_500")) {
len = redisFormatCommand(&cmd,"LRANGE mylist 0 449");
benchmark("LRANGE_500 (first 450 elements)",cmd,len);
free(cmd);
}
len = redisFormatCommand(&cmd,"LRANGE mylist 0 599");
benchmark("LRANGE (first 600 elements)",cmd,len);
free(cmd);
if (test_is_selected("lrange") || test_is_selected("lrange_600")) {
len = redisFormatCommand(&cmd,"LRANGE mylist 0 599");
benchmark("LRANGE_600 (first 600 elements)",cmd,len);
free(cmd);
}
printf("\n");
if (test_is_selected("mset")) {
const char *argv[21];
argv[0] = "MSET";
for (i = 1; i < 21; i += 2) {
argv[i] = "foo:rand:000000000000";
argv[i+1] = data;
}
len = redisFormatCommandArgv(&cmd,21,argv,NULL);
benchmark("MSET (10 keys)",cmd,len);
free(cmd);
}
if (!config.csv) printf("\n");
} while(config.loop);
return 0;
+388 -41
View File
@@ -46,9 +46,15 @@
#include "zmalloc.h"
#include "linenoise.h"
#include "help.h"
#include "anet.h"
#include "ae.h"
#define REDIS_NOTUSED(V) ((void) V)
#define OUTPUT_STANDARD 0
#define OUTPUT_RAW 1
#define OUTPUT_CSV 2
static redisContext *context;
static struct config {
char *hostip;
@@ -62,11 +68,14 @@ static struct config {
int monitor_mode;
int pubsub_mode;
int latency_mode;
int slave_mode;
int pipe_mode;
int bigkeys;
int stdinarg; /* get last arg from stdin. (-x option) */
char *auth;
int raw_output; /* output mode per command */
int output; /* output mode, see OUTPUT_* defines */
sds mb_delim;
char prompt[32];
char prompt[128];
} config;
static void usage();
@@ -88,12 +97,19 @@ static long long mstime(void) {
}
static void cliRefreshPrompt(void) {
if (config.dbnum == 0)
snprintf(config.prompt,sizeof(config.prompt),"redis %s:%d> ",
config.hostip, config.hostport);
int len;
if (config.hostsocket != NULL)
len = snprintf(config.prompt,sizeof(config.prompt),"redis %s",
config.hostsocket);
else
snprintf(config.prompt,sizeof(config.prompt),"redis %s:%d[%d]> ",
config.hostip, config.hostport, config.dbnum);
len = snprintf(config.prompt,sizeof(config.prompt),"redis %s:%d",
config.hostip, config.hostport);
/* Add [dbnum] if needed */
if (config.dbnum != 0)
len += snprintf(config.prompt+len,sizeof(config.prompt)-len,"[%d]",
config.dbnum);
snprintf(config.prompt+len,sizeof(config.prompt)-len,"> ");
}
/*------------------------------------------------------------------------------
@@ -424,10 +440,47 @@ static sds cliFormatReplyRaw(redisReply *r) {
return out;
}
static sds cliFormatReplyCSV(redisReply *r) {
unsigned int i;
sds out = sdsempty();
switch (r->type) {
case REDIS_REPLY_ERROR:
out = sdscat(out,"ERROR,");
out = sdscatrepr(out,r->str,strlen(r->str));
break;
case REDIS_REPLY_STATUS:
out = sdscatrepr(out,r->str,r->len);
break;
case REDIS_REPLY_INTEGER:
out = sdscatprintf(out,"%lld",r->integer);
break;
case REDIS_REPLY_STRING:
out = sdscatrepr(out,r->str,r->len);
break;
case REDIS_REPLY_NIL:
out = sdscat(out,"NIL\n");
break;
case REDIS_REPLY_ARRAY:
for (i = 0; i < r->elements; i++) {
sds tmp = cliFormatReplyCSV(r->element[i]);
out = sdscatlen(out,tmp,sdslen(tmp));
if (i != r->elements-1) out = sdscat(out,",");
sdsfree(tmp);
}
break;
default:
fprintf(stderr,"Unknown reply type: %d\n", r->type);
exit(1);
}
return out;
}
static int cliReadReply(int output_raw_strings) {
void *_reply;
redisReply *reply;
sds out;
sds out = NULL;
int output = 1;
if (redisGetReply(context,&_reply) != REDIS_OK) {
if (config.shutdown)
@@ -445,18 +498,24 @@ static int cliReadReply(int output_raw_strings) {
}
reply = (redisReply*)_reply;
if (output_raw_strings) {
out = cliFormatReplyRaw(reply);
} else {
if (config.raw_output) {
if (output) {
if (output_raw_strings) {
out = cliFormatReplyRaw(reply);
out = sdscat(out,"\n");
} else {
out = cliFormatReplyTTY(reply,"");
if (config.output == OUTPUT_RAW) {
out = cliFormatReplyRaw(reply);
out = sdscat(out,"\n");
} else if (config.output == OUTPUT_STANDARD) {
out = cliFormatReplyTTY(reply,"");
} else if (config.output == OUTPUT_CSV) {
out = cliFormatReplyCSV(reply);
out = sdscat(out,"\n");
}
}
fwrite(out,sdslen(out),1,stdout);
sdsfree(out);
}
fwrite(out,sdslen(out),1,stdout);
sdsfree(out);
freeReplyObject(reply);
return REDIS_OK;
}
@@ -466,6 +525,11 @@ static int cliSendCommand(int argc, char **argv, int repeat) {
size_t *argvlen;
int j, output_raw;
if (!strcasecmp(command,"help") || !strcasecmp(command,"?")) {
cliOutputHelp(--argc, ++argv);
return REDIS_OK;
}
if (context == NULL) return REDIS_ERR;
output_raw = 0;
@@ -477,10 +541,6 @@ static int cliSendCommand(int argc, char **argv, int repeat) {
output_raw = 1;
}
if (!strcasecmp(command,"help") || !strcasecmp(command,"?")) {
cliOutputHelp(--argc, ++argv);
return REDIS_OK;
}
if (!strcasecmp(command,"shutdown")) config.shutdown = 1;
if (!strcasecmp(command,"monitor")) config.monitor_mode = 1;
if (!strcasecmp(command,"subscribe") ||
@@ -499,7 +559,7 @@ static int cliSendCommand(int argc, char **argv, int repeat) {
}
if (config.pubsub_mode) {
if (!config.raw_output)
if (config.output != OUTPUT_RAW)
printf("Reading messages... (press Ctrl-C to quit)\n");
while (1) {
if (cliReadReply(output_raw) != REDIS_OK) exit(1);
@@ -536,8 +596,7 @@ static int parseOptions(int argc, char **argv) {
if (!strcmp(argv[i],"-h") && !lastarg) {
sdsfree(config.hostip);
config.hostip = sdsnew(argv[i+1]);
i++;
config.hostip = sdsnew(argv[++i]);
} else if (!strcmp(argv[i],"-h") && lastarg) {
usage();
} else if (!strcmp(argv[i],"--help")) {
@@ -545,32 +604,33 @@ static int parseOptions(int argc, char **argv) {
} else if (!strcmp(argv[i],"-x")) {
config.stdinarg = 1;
} else if (!strcmp(argv[i],"-p") && !lastarg) {
config.hostport = atoi(argv[i+1]);
i++;
config.hostport = atoi(argv[++i]);
} else if (!strcmp(argv[i],"-s") && !lastarg) {
config.hostsocket = argv[i+1];
i++;
config.hostsocket = argv[++i];
} else if (!strcmp(argv[i],"-r") && !lastarg) {
config.repeat = strtoll(argv[i+1],NULL,10);
i++;
config.repeat = strtoll(argv[++i],NULL,10);
} else if (!strcmp(argv[i],"-i") && !lastarg) {
double seconds = atof(argv[i+1]);
double seconds = atof(argv[++i]);
config.interval = seconds*1000000;
i++;
} else if (!strcmp(argv[i],"-n") && !lastarg) {
config.dbnum = atoi(argv[i+1]);
i++;
config.dbnum = atoi(argv[++i]);
} else if (!strcmp(argv[i],"-a") && !lastarg) {
config.auth = argv[i+1];
i++;
config.auth = argv[++i];
} else if (!strcmp(argv[i],"--raw")) {
config.raw_output = 1;
config.output = OUTPUT_RAW;
} else if (!strcmp(argv[i],"--csv")) {
config.output = OUTPUT_CSV;
} else if (!strcmp(argv[i],"--latency")) {
config.latency_mode = 1;
} else if (!strcmp(argv[i],"--slave")) {
config.slave_mode = 1;
} else if (!strcmp(argv[i],"--pipe")) {
config.pipe_mode = 1;
} else if (!strcmp(argv[i],"--bigkeys")) {
config.bigkeys = 1;
} else if (!strcmp(argv[i],"-d") && !lastarg) {
sdsfree(config.mb_delim);
config.mb_delim = sdsnew(argv[i+1]);
i++;
config.mb_delim = sdsnew(argv[++i]);
} else if (!strcmp(argv[i],"-v") || !strcmp(argv[i], "--version")) {
sds version = cliVersion();
printf("redis-cli %s\n", version);
@@ -618,6 +678,9 @@ static void usage() {
" -d <delimiter> Multi-bulk delimiter in for raw formatting (default: \\n)\n"
" --raw Use raw formatting for replies (default when STDOUT is not a tty)\n"
" --latency Enter a special mode continuously sampling latency.\n"
" --slave Simulate a slave showing commands received from the master.\n"
" --pipe Transfer raw Redis protocol from stdin to server.\n"
" --bigkeys Sample Redis keys looking for big keys.\n"
" --help Output this help and exit\n"
" --version Output version and exit\n"
"\n"
@@ -743,7 +806,7 @@ static int noninteractive(int argc, char **argv) {
static void latencyMode(void) {
redisReply *reply;
long long start, latency, min, max, tot, count = 0;
long long start, latency, min = 0, max = 0, tot = 0, count = 0;
double avg;
if (!context) exit(1);
@@ -773,6 +836,266 @@ static void latencyMode(void) {
}
}
static void slaveMode(void) {
/* To start we need to send the SYNC command and return the payload.
* The hiredis client lib does not understand this part of the protocol
* and we don't want to mess with its buffers, so everything is performed
* using direct low-level I/O. */
int fd = context->fd;
char buf[1024], *p;
ssize_t nread;
unsigned long long payload;
/* Send the SYNC command. */
if (write(fd,"SYNC\r\n",6) != 6) {
fprintf(stderr,"Error writing to master\n");
exit(1);
}
/* Read $<payload>\r\n, making sure to read just up to "\n" */
p = buf;
while(1) {
nread = read(fd,p,1);
if (nread <= 0) {
fprintf(stderr,"Error reading bulk length while SYNCing\n");
exit(1);
}
if (*p == '\n') break;
p++;
}
*p = '\0';
payload = strtoull(buf+1,NULL,10);
fprintf(stderr,"SYNC with master, discarding %lld bytes of bulk tranfer...\n",
payload);
/* Discard the payload. */
while(payload) {
nread = read(fd,buf,(payload > sizeof(buf)) ? sizeof(buf) : payload);
if (nread <= 0) {
fprintf(stderr,"Error reading RDB payload while SYNCing\n");
exit(1);
}
payload -= nread;
}
fprintf(stderr,"SYNC done. Logging commands from master.\n");
/* Now we can use the hiredis to read the incoming protocol. */
config.output = OUTPUT_CSV;
while (cliReadReply(0) == REDIS_OK);
}
static void pipeMode(void) {
int fd = context->fd;
long long errors = 0, replies = 0, obuf_len = 0, obuf_pos = 0;
char ibuf[1024*16], obuf[1024*16]; /* Input and output buffers */
char aneterr[ANET_ERR_LEN];
redisReader *reader = redisReaderCreate();
redisReply *reply;
int eof = 0; /* True once we consumed all the standard input. */
int done = 0;
char magic[20]; /* Special reply we recognize. */
srand(time(NULL));
/* Use non blocking I/O. */
if (anetNonBlock(aneterr,fd) == ANET_ERR) {
fprintf(stderr, "Can't set the socket in non blocking mode: %s\n",
aneterr);
exit(1);
}
/* Transfer raw protocol and read replies from the server at the same
* time. */
while(!done) {
int mask = AE_READABLE;
if (!eof || obuf_len != 0) mask |= AE_WRITABLE;
mask = aeWait(fd,mask,1000);
/* Handle the readable state: we can read replies from the server. */
if (mask & AE_READABLE) {
ssize_t nread;
/* Read from socket and feed the hiredis reader. */
do {
nread = read(fd,ibuf,sizeof(ibuf));
if (nread == -1 && errno != EAGAIN && errno != EINTR) {
fprintf(stderr, "Error reading from the server: %s\n",
strerror(errno));
exit(1);
}
if (nread > 0) redisReaderFeed(reader,ibuf,nread);
} while(nread > 0);
/* Consume replies. */
do {
if (redisReaderGetReply(reader,(void**)&reply) == REDIS_ERR) {
fprintf(stderr, "Error reading replies from server\n");
exit(1);
}
if (reply) {
if (reply->type == REDIS_REPLY_ERROR) {
fprintf(stderr,"%s\n", reply->str);
errors++;
} else if (eof && reply->type == REDIS_REPLY_STRING &&
reply->len == 20) {
/* Check if this is the reply to our final ECHO
* command. If so everything was received
* from the server. */
if (memcmp(reply->str,magic,20) == 0) {
printf("Last reply received from server.\n");
done = 1;
replies--;
}
}
replies++;
freeReplyObject(reply);
}
} while(reply);
}
/* Handle the writable state: we can send protocol to the server. */
if (mask & AE_WRITABLE) {
while(1) {
/* Transfer current buffer to server. */
if (obuf_len != 0) {
ssize_t nwritten = write(fd,obuf+obuf_pos,obuf_len);
if (nwritten == -1) {
if (errno != EAGAIN && errno != EINTR) {
fprintf(stderr, "Error writing to the server: %s\n",
strerror(errno));
exit(1);
} else {
nwritten = 0;
}
}
obuf_len -= nwritten;
obuf_pos += nwritten;
if (obuf_len != 0) break; /* Can't accept more data. */
}
/* If buffer is empty, load from stdin. */
if (obuf_len == 0 && !eof) {
ssize_t nread = read(STDIN_FILENO,obuf,sizeof(obuf));
if (nread == 0) {
char echo[] =
"*2\r\n$4\r\nECHO\r\n$20\r\n01234567890123456789\r\n";
int j;
eof = 1;
/* Everything transfered, so we queue a special
* ECHO command that we can match in the replies
* to make sure everything was read from the server. */
for (j = 0; j < 20; j++)
magic[j] = rand() & 0xff;
memcpy(echo+19,magic,20);
memcpy(obuf,echo,sizeof(echo)-1);
obuf_len = sizeof(echo)-1;
obuf_pos = 0;
printf("All data transferred. Waiting for the last reply...\n");
} else if (nread == -1) {
fprintf(stderr, "Error reading from stdin: %s\n",
strerror(errno));
exit(1);
} else {
obuf_len = nread;
obuf_pos = 0;
}
}
if (obuf_len == 0 && eof) break;
}
}
}
redisReaderFree(reader);
printf("errors: %lld, replies: %lld\n", errors, replies);
if (errors)
exit(1);
else
exit(0);
}
#define TYPE_STRING 0
#define TYPE_LIST 1
#define TYPE_SET 2
#define TYPE_HASH 3
#define TYPE_ZSET 4
static void findBigKeys(void) {
unsigned long long biggest[5] = {0,0,0,0,0};
unsigned long long samples = 0;
redisReply *reply1, *reply2, *reply3 = NULL;
char *sizecmd, *typename[] = {"string","list","set","hash","zset"};
int type;
printf("\n# Press ctrl+c when you have had enough of it... :)\n");
printf("# You can use -i 0.1 to sleep 0.1 sec every 100 sampled keys\n");
printf("# in order to reduce server load (usually not needed).\n\n");
while(1) {
/* Sample with RANDOMKEY */
reply1 = redisCommand(context,"RANDOMKEY");
if (reply1 == NULL) {
fprintf(stderr,"\nI/O error\n");
exit(1);
} else if (reply1->type == REDIS_REPLY_ERROR) {
fprintf(stderr, "RANDOMKEY error: %s\n",
reply1->str);
exit(1);
}
/* Get the key type */
reply2 = redisCommand(context,"TYPE %s",reply1->str);
assert(reply2 && reply2->type == REDIS_REPLY_STATUS);
samples++;
/* Get the key "size" */
if (!strcmp(reply2->str,"string")) {
sizecmd = "STRLEN";
type = TYPE_STRING;
} else if (!strcmp(reply2->str,"list")) {
sizecmd = "LLEN";
type = TYPE_LIST;
} else if (!strcmp(reply2->str,"set")) {
sizecmd = "SCARD";
type = TYPE_SET;
} else if (!strcmp(reply2->str,"hash")) {
sizecmd = "HLEN";
type = TYPE_HASH;
} else if (!strcmp(reply2->str,"zset")) {
sizecmd = "ZCARD";
type = TYPE_ZSET;
} else if (!strcmp(reply2->str,"none")) {
freeReplyObject(reply1);
freeReplyObject(reply2);
freeReplyObject(reply3);
continue;
} else {
fprintf(stderr, "Unknown key type '%s' for key '%s'\n",
reply2->str, reply1->str);
exit(1);
}
reply3 = redisCommand(context,"%s %s", sizecmd, reply1->str);
if (reply3 && reply3->type == REDIS_REPLY_INTEGER) {
if (biggest[type] < (unsigned)reply3->integer) {
printf("[%6s] %s | biggest so far with size %llu\n",
typename[type], reply1->str,
(unsigned long long) reply3->integer);
biggest[type] = reply3->integer;
}
}
if ((samples % 1000000) == 0)
printf("(%llu keys sampled)\n", samples);
if ((samples % 100) == 0 && config.interval)
usleep(config.interval);
freeReplyObject(reply1);
freeReplyObject(reply2);
if (reply3) freeReplyObject(reply3);
}
}
int main(int argc, char **argv) {
int firstarg;
@@ -787,9 +1110,15 @@ int main(int argc, char **argv) {
config.monitor_mode = 0;
config.pubsub_mode = 0;
config.latency_mode = 0;
config.slave_mode = 0;
config.pipe_mode = 0;
config.bigkeys = 0;
config.stdinarg = 0;
config.auth = NULL;
config.raw_output = !isatty(fileno(stdout)) && (getenv("FAKETTY") == NULL);
if (!isatty(fileno(stdout)) && (getenv("FAKETTY") == NULL))
config.output = OUTPUT_RAW;
else
config.output = OUTPUT_STANDARD;
config.mb_delim = sdsnew("\n");
cliInitHelp();
@@ -797,12 +1126,30 @@ int main(int argc, char **argv) {
argc -= firstarg;
argv += firstarg;
/* Start in latency mode if appropriate */
/* Latency mode */
if (config.latency_mode) {
cliConnect(0);
latencyMode();
}
/* Slave mode */
if (config.slave_mode) {
cliConnect(0);
slaveMode();
}
/* Pipe mode */
if (config.pipe_mode) {
cliConnect(0);
pipeMode();
}
/* Find big keys */
if (config.bigkeys) {
cliConnect(0);
findBigKeys();
}
/* Start interactive mode when no command is provided */
if (argc == 0) {
/* Note that in repl mode we don't abort on connection error.
+30 -14
View File
@@ -480,9 +480,10 @@ void updateDictResizePolicy(void) {
* keys that can be removed from the keyspace. */
void activeExpireCycle(void) {
int j;
long long start = mstime();
for (j = 0; j < server.dbnum; j++) {
int expired;
int expired, iteration = 0;
redisDb *db = server.db+j;
/* Continue to expire if at the end of the cycle more than 25%
@@ -511,6 +512,12 @@ void activeExpireCycle(void) {
server.stat_expiredkeys++;
}
}
/* We can't block forever here even if there are many keys to
* expire. So after a given amount of milliseconds return to the
* caller waiting for the other active expire cycle. */
iteration++;
if ((iteration & 0xff) == 0 && /* Check once every 255 iterations */
(mstime()-start) > REDIS_EXPIRELOOKUPS_TIME_LIMIT) return;
} while (expired > REDIS_EXPIRELOOKUPS_PER_CRON/4);
}
}
@@ -773,16 +780,11 @@ void createSharedObjects(void) {
shared.space = createObject(REDIS_STRING,sdsnew(" "));
shared.colon = createObject(REDIS_STRING,sdsnew(":"));
shared.plus = createObject(REDIS_STRING,sdsnew("+"));
shared.select0 = createStringObject("select 0\r\n",10);
shared.select1 = createStringObject("select 1\r\n",10);
shared.select2 = createStringObject("select 2\r\n",10);
shared.select3 = createStringObject("select 3\r\n",10);
shared.select4 = createStringObject("select 4\r\n",10);
shared.select5 = createStringObject("select 5\r\n",10);
shared.select6 = createStringObject("select 6\r\n",10);
shared.select7 = createStringObject("select 7\r\n",10);
shared.select8 = createStringObject("select 8\r\n",10);
shared.select9 = createStringObject("select 9\r\n",10);
for (j = 0; j < REDIS_SHARED_SELECT_CMDS; j++) {
shared.select[j] = createObject(REDIS_STRING,
sdscatprintf(sdsempty(),"select %d\r\n", j));
}
shared.messagebulk = createStringObject("$7\r\nmessage\r\n",13);
shared.pmessagebulk = createStringObject("$8\r\npmessage\r\n",14);
shared.subscribebulk = createStringObject("$9\r\nsubscribe\r\n",15);
@@ -871,7 +873,7 @@ void initServerConfig() {
server.replstate = REDIS_REPL_NONE;
server.repl_syncio_timeout = REDIS_REPL_SYNCIO_TIMEOUT;
server.repl_serve_stale_data = 1;
server.repl_down_since = -1;
server.repl_down_since = time(NULL);
/* Double constants initialization */
R_Zero = 0.0;
@@ -1743,13 +1745,26 @@ void version() {
void usage() {
fprintf(stderr,"Usage: ./redis-server [/path/to/redis.conf]\n");
fprintf(stderr," ./redis-server - (read config from stdin)\n");
fprintf(stderr," ./redis-server --test-memory <megabytes>\n\n");
exit(1);
}
void memtest(size_t megabytes, int passes);
int main(int argc, char **argv) {
time_t start;
initServerConfig();
if (argc >= 2 && strcmp(argv[1], "--test-memory") == 0) {
if (argc == 3) {
memtest(atoi(argv[2]),50);
exit(0);
} else {
fprintf(stderr,"Please specify the amount of memory to test in megabytes.\n");
fprintf(stderr,"Example: ./redis-server --test-memory 4096\n\n");
exit(1);
}
}
if (argc == 2) {
if (strcmp(argv[1], "-v") == 0 ||
strcmp(argv[1], "--version") == 0) version();
@@ -1903,8 +1918,9 @@ static void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
redisLog(REDIS_WARNING,
"=== REDIS BUG REPORT END. Make sure to include from START to END. ===\n\n"
" Please report the crash opening an issue on github:\n\n"
" http://github.com/antirez/redis/issues\n\n"
" Please report the crash opening an issue on github:\n\n"
" http://github.com/antirez/redis/issues\n\n"
" Suspect RAM error? Use redis-server --test-memory to veryfy it.\n\n"
);
/* free(messages); Don't call free() with possibly corrupted memory. */
if (server.daemonize) unlink(server.pidfile);
+5 -3
View File
@@ -45,8 +45,10 @@
#define REDIS_CONFIGLINE_MAX 1024
#define REDIS_MAX_SYNC_TIME 60 /* Slave can't take more to sync */
#define REDIS_EXPIRELOOKUPS_PER_CRON 10 /* lookup 10 expires per loop */
#define REDIS_EXPIRELOOKUPS_TIME_LIMIT 25 /* Time limit in milliseconds */
#define REDIS_MAX_WRITE_PER_EVENT (1024*64)
#define REDIS_REQUEST_MAX_SIZE (1024*1024*256) /* max bytes in inline command */
#define REDIS_SHARED_SELECT_CMDS 10
#define REDIS_SHARED_INTEGERS 10000
#define REDIS_REPLY_CHUNK_BYTES (5*1500) /* 5 TCP packets with default MTU */
#define REDIS_INLINE_MAX_SIZE (1024*64) /* Max size of inline reads */
@@ -54,7 +56,7 @@
#define REDIS_AUTO_AOFREWRITE_PERC 100
#define REDIS_AUTO_AOFREWRITE_MIN_SIZE (1024*1024)
#define REDIS_SLOWLOG_LOG_SLOWER_THAN 10000
#define REDIS_SLOWLOG_MAX_LEN 64
#define REDIS_SLOWLOG_MAX_LEN 128
#define REDIS_REPL_TIMEOUT 60
#define REDIS_REPL_PING_SLAVE_PERIOD 10
@@ -372,8 +374,7 @@ struct sharedObjectsStruct {
*colon, *nullbulk, *nullmultibulk, *queued,
*emptymultibulk, *wrongtypeerr, *nokeyerr, *syntaxerr, *sameobjecterr,
*outofrangeerr, *loadingerr, *plus,
*select0, *select1, *select2, *select3, *select4,
*select5, *select6, *select7, *select8, *select9,
*select[REDIS_SHARED_SELECT_CMDS],
*messagebulk, *pmessagebulk, *subscribebulk, *unsubscribebulk, *mbulk3,
*mbulk4, *psubscribebulk, *punsubscribebulk,
*integers[REDIS_SHARED_INTEGERS];
@@ -718,6 +719,7 @@ sds getAllClientsInfoString(void);
void rewriteClientCommandVector(redisClient *c, int argc, ...);
unsigned long getClientOutputBufferMemoryUsage(redisClient *c);
void flushSlavesOutputBuffers(void);
void disconnectSlaves(void);
#ifdef __GNUC__
void addReplyErrorFormat(redisClient *c, const char *fmt, ...)
+6 -14
View File
@@ -25,24 +25,15 @@ void replicationFeedSlaves(list *slaves, int dictid, robj **argv, int argc) {
if (slave->slaveseldb != dictid) {
robj *selectcmd;
switch(dictid) {
case 0: selectcmd = shared.select0; break;
case 1: selectcmd = shared.select1; break;
case 2: selectcmd = shared.select2; break;
case 3: selectcmd = shared.select3; break;
case 4: selectcmd = shared.select4; break;
case 5: selectcmd = shared.select5; break;
case 6: selectcmd = shared.select6; break;
case 7: selectcmd = shared.select7; break;
case 8: selectcmd = shared.select8; break;
case 9: selectcmd = shared.select9; break;
default:
if (dictid >= 0 && dictid < REDIS_SHARED_SELECT_CMDS) {
incrRefCount(shared.select[dictid]);
selectcmd = shared.select[dictid];
} else {
selectcmd = createObject(REDIS_STRING,
sdscatprintf(sdsempty(),"select %d\r\n",dictid));
selectcmd->refcount = 0;
break;
}
addReply(slave,selectcmd);
decrRefCount(selectcmd);
slave->slaveseldb = dictid;
}
addReplyMultiBulkLen(slave,argc);
@@ -507,6 +498,7 @@ void slaveofCommand(redisClient *c) {
server.masterhost = sdsdup(c->argv[1]->ptr);
server.masterport = atoi(c->argv[2]->ptr);
if (server.master) freeClient(server.master);
disconnectSlaves(); /* Force our slaves to resync with us as well. */
if (server.replstate == REDIS_REPL_TRANSFER)
replicationAbortSyncTransfer();
server.replstate = REDIS_REPL_CONNECT;
+29 -6
View File
@@ -16,13 +16,36 @@
* this function. */
slowlogEntry *slowlogCreateEntry(robj **argv, int argc, long long duration) {
slowlogEntry *se = zmalloc(sizeof(*se));
int j;
int j, slargc = argc;
se->argc = argc;
se->argv = zmalloc(sizeof(robj*)*argc);
for (j = 0; j < argc; j++) {
se->argv[j] = argv[j];
incrRefCount(argv[j]);
if (slargc > SLOWLOG_ENTRY_MAX_ARGC) slargc = SLOWLOG_ENTRY_MAX_ARGC;
se->argc = slargc;
se->argv = zmalloc(sizeof(robj*)*slargc);
for (j = 0; j < slargc; j++) {
/* Logging too many arguments is a useless memory waste, so we stop
* at SLOWLOG_ENTRY_MAX_ARGC, but use the last argument to specify
* how many remaining arguments there were in the original command. */
if (slargc != argc && j == slargc-1) {
se->argv[j] = createObject(REDIS_STRING,
sdscatprintf(sdsempty(),"... (%d more arguments)",
argc-slargc+1));
} else {
/* Trim too long strings as well... */
if (argv[j]->type == REDIS_STRING &&
argv[j]->encoding == REDIS_ENCODING_RAW &&
sdslen(argv[j]->ptr) > SLOWLOG_ENTRY_MAX_STRING)
{
sds s = sdsnewlen(argv[j]->ptr, SLOWLOG_ENTRY_MAX_STRING);
s = sdscatprintf(s,"... (%lu more bytes)",
(unsigned long)
sdslen(argv[j]->ptr) - SLOWLOG_ENTRY_MAX_STRING);
se->argv[j] = createObject(REDIS_STRING,s);
} else {
se->argv[j] = argv[j];
incrRefCount(argv[j]);
}
}
}
se->time = time(NULL);
se->duration = duration;
+3
View File
@@ -1,3 +1,6 @@
#define SLOWLOG_ENTRY_MAX_ARGC 32
#define SLOWLOG_ENTRY_MAX_STRING 128
/* This structure defines an entry inside the slow log list */
typedef struct slowlogEntry {
robj **argv;
+43 -33
View File
@@ -8,21 +8,27 @@ redisSortOperation *createSortOperation(int type, robj *pattern) {
return so;
}
/* Return the value associated to the key with a name obtained
* substituting the first occurence of '*' in 'pattern' with 'subst'.
/* Return the value associated to the key with a name obtained using
* the following rules:
*
* 1) The first occurence of '*' in 'pattern' is substituted with 'subst'.
*
* 2) If 'pattern' matches the "->" string, everything on the left of
* the arrow is treated as the name of an hash field, and the part on the
* left as the key name containing an hash. The value of the specified
* field is returned.
*
* 3) If 'pattern' equals "#", the function simply returns 'subst' itself so
* that the SORT command can be used like: SORT key GET # to retrieve
* the Set/List elements directly.
*
* The returned object will always have its refcount increased by 1
* when it is non-NULL. */
robj *lookupKeyByPattern(redisDb *db, robj *pattern, robj *subst) {
char *p, *f;
char *p, *f, *k;
sds spat, ssub;
robj keyobj, fieldobj, *o;
robj *keyobj, *fieldobj = NULL, *o;
int prefixlen, sublen, postfixlen, fieldlen;
/* Expoit the internal sds representation to create a sds string allocated on the stack in order to make this function faster */
struct {
int len;
int free;
char buf[REDIS_SORTKEY_MAX+1];
} keyname, fieldname;
/* If the pattern is "#" return the substitution object itself in order
* to implement the "SORT ... GET #" feature. */
@@ -36,9 +42,10 @@ robj *lookupKeyByPattern(redisDb *db, robj *pattern, robj *subst) {
* a decoded object on the fly. Otherwise getDecodedObject will just
* increment the ref count, that we'll decrement later. */
subst = getDecodedObject(subst);
ssub = subst->ptr;
if (sdslen(spat)+sdslen(ssub)-1 > REDIS_SORTKEY_MAX) return NULL;
/* If we can't find '*' in the pattern we return NULL as to GET a
* fixed key does not make sense. */
p = strchr(spat,'*');
if (!p) {
decrRefCount(subst);
@@ -46,46 +53,49 @@ robj *lookupKeyByPattern(redisDb *db, robj *pattern, robj *subst) {
}
/* Find out if we're dealing with a hash dereference. */
if ((f = strstr(p+1, "->")) != NULL) {
fieldlen = sdslen(spat)-(f-spat);
/* this also copies \0 character */
memcpy(fieldname.buf,f+2,fieldlen-1);
fieldname.len = fieldlen-2;
if ((f = strstr(p+1, "->")) != NULL && *(f+2) != '\0') {
fieldlen = sdslen(spat)-(f-spat)-2;
fieldobj = createStringObject(f+2,fieldlen);
} else {
fieldlen = 0;
}
/* Perform the '*' substitution. */
prefixlen = p-spat;
sublen = sdslen(ssub);
postfixlen = sdslen(spat)-(prefixlen+1)-fieldlen;
memcpy(keyname.buf,spat,prefixlen);
memcpy(keyname.buf+prefixlen,ssub,sublen);
memcpy(keyname.buf+prefixlen+sublen,p+1,postfixlen);
keyname.buf[prefixlen+sublen+postfixlen] = '\0';
keyname.len = prefixlen+sublen+postfixlen;
decrRefCount(subst);
postfixlen = sdslen(spat)-(prefixlen+1)-(fieldlen ? fieldlen+2 : 0);
keyobj = createStringObject(NULL,prefixlen+sublen+postfixlen);
k = keyobj->ptr;
memcpy(k,spat,prefixlen);
memcpy(k+prefixlen,ssub,sublen);
memcpy(k+prefixlen+sublen,p+1,postfixlen);
decrRefCount(subst); /* Incremented by decodeObject() */
/* Lookup substituted key */
initStaticStringObject(keyobj,((char*)&keyname)+(sizeof(struct sdshdr)));
o = lookupKeyRead(db,&keyobj);
if (o == NULL) return NULL;
o = lookupKeyRead(db,keyobj);
if (o == NULL) goto noobj;
if (fieldlen > 0) {
if (o->type != REDIS_HASH || fieldname.len < 1) return NULL;
if (fieldobj) {
if (o->type != REDIS_HASH) goto noobj;
/* Retrieve value from hash by the field name. This operation
* already increases the refcount of the returned object. */
initStaticStringObject(fieldobj,((char*)&fieldname)+(sizeof(struct sdshdr)));
o = hashTypeGetObject(o, &fieldobj);
o = hashTypeGetObject(o, fieldobj);
} else {
if (o->type != REDIS_STRING) return NULL;
if (o->type != REDIS_STRING) goto noobj;
/* Every object that this function returns needs to have its refcount
* increased. sortCommand decreases it again. */
incrRefCount(o);
}
decrRefCount(keyobj);
if (fieldobj) decrRefCount(fieldobj);
return o;
noobj:
decrRefCount(keyobj);
if (fieldlen) decrRefCount(fieldobj);
return NULL;
}
/* sortCompare() is used by qsort in sortCommand(). Given that qsort_r with
+4 -1
View File
@@ -1261,8 +1261,11 @@ int zuiNext(zsetopsrc *op, zsetopval *val) {
if (op->type == REDIS_SET) {
iterset *it = &op->iter.set;
if (op->encoding == REDIS_ENCODING_INTSET) {
if (!intsetGet(it->is.is,it->is.ii,(int64_t*)&val->ell))
int64_t ell;
if (!intsetGet(it->is.is,it->is.ii,&ell))
return 0;
val->ell = ell;
val->score = 1.0;
/* Move to next element. */
+5
View File
@@ -339,6 +339,11 @@ long long ustime(void) {
return ust;
}
/* Return the UNIX time in milliseconds */
long long mstime(void) {
return ustime()/1000;
}
#ifdef UTIL_TEST_MAIN
#include <assert.h>
+1
View File
@@ -9,5 +9,6 @@ int string2ll(char *s, size_t slen, long long *value);
int string2l(char *s, size_t slen, long *value);
int d2string(char *buf, size_t len, double value);
long long ustime(void);
long long mstime(void);
#endif
+1 -1
View File
@@ -1 +1 @@
#define REDIS_VERSION "2.4.8"
#define REDIS_VERSION "2.4.14"
+3 -1
View File
@@ -489,7 +489,9 @@ static unsigned char *__ziplistInsert(unsigned char *zl, unsigned char *p, unsig
size_t offset;
int nextdiff = 0;
unsigned char encoding = 0;
long long value;
long long value = 123456789; /* initialized to avoid warning. Using a value
that is easy to see if for some reason
we use it uninitialized. */
zlentry entry, tail;
/* Find out prevlen for the entry that is inserted. */
+38 -18
View File
@@ -2,16 +2,23 @@ start_server {tags {"repl"}} {
start_server {} {
test {First server should have role slave after SLAVEOF} {
r -1 slaveof [srv 0 host] [srv 0 port]
after 1000
s -1 role
} {slave}
wait_for_condition 50 100 {
[s -1 role] eq {slave} &&
[string match {*master_link_status:up*} [r -1 info]]
} else {
fail "Can't turn the instance into a slave"
}
}
test {BRPOPLPUSH replication, when blocking against empty list} {
set rd [redis_deferring_client]
$rd brpoplpush a b 5
r lpush a foo
after 1000
assert_equal [r debug digest] [r -1 debug digest]
wait_for_condition 50 100 {
[r debug digest] eq [r -1 debug digest]
} else {
fail "Master and slave have different digest: [r debug digest] VS [r -1 debug digest]"
}
}
test {BRPOPLPUSH replication, list exists} {
@@ -79,12 +86,11 @@ start_server {tags {"repl"}} {
set master_host [srv 0 host]
set master_port [srv 0 port]
set slaves {}
set load_handle0 [start_write_load $master_host $master_port 20]
set load_handle1 [start_write_load $master_host $master_port 20]
set load_handle0 [start_write_load $master_host $master_port 3]
set load_handle1 [start_write_load $master_host $master_port 5]
set load_handle2 [start_write_load $master_host $master_port 20]
set load_handle3 [start_write_load $master_host $master_port 20]
set load_handle4 [start_write_load $master_host $master_port 20]
after 2000
set load_handle3 [start_write_load $master_host $master_port 8]
set load_handle4 [start_write_load $master_host $master_port 4]
start_server {} {
lappend slaves [srv 0 client]
start_server {} {
@@ -92,6 +98,7 @@ start_server {tags {"repl"}} {
start_server {} {
lappend slaves [srv 0 client]
test "Connect multiple slaves at the same time (issue #141)" {
# Send SALVEOF commands to slaves
[lindex $slaves 0] slaveof $master_host $master_port
[lindex $slaves 1] slaveof $master_host $master_port
[lindex $slaves 2] slaveof $master_host $master_port
@@ -110,16 +117,33 @@ start_server {tags {"repl"}} {
if {$retry == 0} {
error "assertion:Slaves not correctly synchronized"
}
# Stop the write load
stop_write_load $load_handle0
stop_write_load $load_handle1
stop_write_load $load_handle2
stop_write_load $load_handle3
stop_write_load $load_handle4
set retry 10
while {$retry && ([$master debug digest] ne [[lindex $slaves 0] debug digest])} {
after 1000
incr retry -1
# Wait that slaves exit the "loading" state
wait_for_condition 500 100 {
![string match {*loading:1*} [[lindex $slaves 0] info]] &&
![string match {*loading:1*} [[lindex $slaves 1] info]] &&
![string match {*loading:1*} [[lindex $slaves 2] info]]
} else {
fail "Slaves still loading data after too much time"
}
# Make sure that slaves and master have same number of keys
wait_for_condition 500 100 {
[$master dbsize] == [[lindex $slaves 0] dbsize] &&
[$master dbsize] == [[lindex $slaves 1] dbsize] &&
[$master dbsize] == [[lindex $slaves 2] dbsize]
} else {
fail "Different number of keys between masted and slave after too long time."
}
# Check digests
set digest [$master debug digest]
set digest0 [[lindex $slaves 0] debug digest]
set digest1 [[lindex $slaves 1] debug digest]
@@ -128,10 +152,6 @@ start_server {tags {"repl"}} {
assert {$digest eq $digest0}
assert {$digest eq $digest1}
assert {$digest eq $digest2}
#puts [$master dbsize]
#puts [[lindex $slaves 0] dbsize]
#puts [[lindex $slaves 1] dbsize]
#puts [[lindex $slaves 2] dbsize]
}
}
}
+8 -2
View File
@@ -142,9 +142,15 @@ proc ::redis::redis_multi_bulk_read fd {
set count [redis_read_line $fd]
if {$count == -1} return {}
set l {}
set err {}
for {set i 0} {$i < $count} {incr i} {
lappend l [redis_read_reply $fd]
if {[catch {
lappend l [redis_read_reply $fd]
} e] && $err eq {}} {
set err $e
}
}
if {$err ne {}} {return -code error $err}
return $l
}
@@ -160,7 +166,7 @@ proc ::redis::redis_read_reply fd {
- {return -code error [redis_read_line $fd]}
$ {redis_bulk_read $fd}
* {redis_multi_bulk_read $fd}
default {return -code error "Bad protocol, $type as reply type byte"}
default {return -code error "Bad protocol, '$type' as reply type byte"}
}
}
+23 -14
View File
@@ -2,13 +2,14 @@ set ::global_overrides {}
set ::tags {}
set ::valgrind_errors {}
proc error_and_quit {config_file error} {
puts "!!COULD NOT START REDIS-SERVER\n"
puts "CONFIGURATION:"
puts [exec cat $config_file]
puts "\nERROR:"
puts [string trim $error]
exit 1
proc start_server_error {config_file error} {
set err {}
append err "Cant' start the Redis server\n"
append err "CONFIGURATION:"
append err [exec cat $config_file]
append err "\nERROR:"
append err [string trim $error]
send_data_packet $::test_server_fd err $err
}
proc check_valgrind_errors stderr {
@@ -16,7 +17,7 @@ proc check_valgrind_errors stderr {
set buf [read $fd]
close $fd
if {![regexp -- {ERROR SUMMARY: 0 errors} $buf] ||
if {[regexp -- { at 0x} $buf] ||
(![regexp -- {definitely lost: 0 bytes} $buf] &&
![regexp -- {no leaks are possible} $buf])} {
send_data_packet $::test_server_fd err "Valgrind error: $buf\n"
@@ -45,11 +46,16 @@ proc kill_server config {
}
# kill server and wait for the process to be totally exited
catch {exec kill $pid}
while {[is_alive $config]} {
if {[incr wait 10] % 1000 == 0} {
incr wait 10
if {$wait >= 5000} {
puts "Forcing process $pid to exit..."
catch {exec kill -KILL $pid}
} elseif {$wait % 1000 == 0} {
puts "Waiting for process $pid to exit..."
}
catch {exec kill $pid}
after 10
}
@@ -175,14 +181,14 @@ proc start_server {options {code undefined}} {
set stderr [format "%s/%s" [dict get $config "dir"] "stderr"]
if {$::valgrind} {
exec valgrind --suppressions=src/valgrind.sup src/redis-server $config_file > $stdout 2> $stderr &
exec valgrind --suppressions=src/valgrind.sup --show-reachable=no --show-possibly-lost=no --leak-check=full src/redis-server $config_file > $stdout 2> $stderr &
} else {
exec src/redis-server $config_file > $stdout 2> $stderr &
}
# check that the server actually started
# ugly but tries to be as fast as possible...
set retrynum 100
if {$::valgrind} {set retrynum 1000} else {set retrynum 100}
set serverisup 0
if {$::verbose} {
@@ -209,7 +215,10 @@ proc start_server {options {code undefined}} {
}
if {!$serverisup} {
error_and_quit $config_file [exec cat $stderr]
set err {}
append err [exec cat $stdout] "\n" [exec cat $stderr]
start_server_error $config_file $err
return
}
# find out the pid
@@ -243,7 +252,7 @@ proc start_server {options {code undefined}} {
while 1 {
# check that the server actually started and is ready for connections
if {[exec cat $stdout | grep "ready to accept" | wc -l] > 0} {
if {[exec grep "ready to accept" | wc -l < $stdout] > 0} {
break
}
after 10
+17
View File
@@ -3,6 +3,10 @@ set ::num_passed 0
set ::num_failed 0
set ::tests_failed {}
proc fail {msg} {
error "assertion:$msg"
}
proc assert {condition} {
if {![uplevel 1 [list expr $condition]]} {
error "assertion:Expected condition '$condition' to be true ([uplevel 1 [list subst -nocommands $condition]])"
@@ -44,6 +48,19 @@ proc assert_type {type key} {
assert_equal $type [r type $key]
}
# Wait for the specified condition to be true, with the specified number of
# max retries and delay between retries. Otherwise the 'elsescript' is
# executed.
proc wait_for_condition {maxtries delay e _else_ elsescript} {
while {[incr maxtries -1] >= 0} {
if {[uplevel 1 [list expr $e]]} break
after $delay
}
if {$maxtries == -1} {
uplevel 1 $elsescript
}
}
# Test if TERM looks like to support colors
proc color_term {} {
expr {[info exists ::env(TERM)] && [string match *xterm* $::env(TERM)]}
+4
View File
@@ -294,3 +294,7 @@ proc csvdump r {
proc csvstring s {
return "\"$s\""
}
proc roundFloat f {
format "%.10g" $f
}
+17
View File
@@ -392,6 +392,18 @@ for {set j 0} {$j < [llength $argv]} {incr j} {
}
}
# With the parallel test running multiple Redis instances at the same time
# we need a fast enough computer, otherwise a lot of tests may generate
# false positives.
# If the computer is too slow we revert the sequetial test without any
# parallelism, that is, clients == 1.
proc is_a_slow_computer {} {
set start [clock milliseconds]
for {set j 0} {$j < 1000000} {incr j} {}
set elapsed [expr [clock milliseconds]-$start]
expr {$elapsed > 200}
}
if {$::client} {
if {[catch { test_client_main $::test_server_port } err]} {
set estr "Executing test client: $err.\n$::errorInfo"
@@ -401,6 +413,11 @@ if {$::client} {
exit 1
}
} else {
if {[is_a_slow_computer]} {
puts "** SLOW COMPUTER ** Using a single client to avoid false positives."
set ::numclients 1
}
if {[catch { test_server_main } err]} {
if {[string length $err] > 0} {
# only display error when not generated by the test suite
+17
View File
@@ -38,4 +38,21 @@ start_server {tags {"slowlog"} overrides {slowlog-log-slower-than 1000000}} {
assert_equal [expr {[lindex $e 2] > 100000}] 1
assert_equal [lindex $e 3] {debug sleep 0.2}
}
test {SLOWLOG - commands with too many arguments are trimmed} {
r config set slowlog-log-slower-than 0
r slowlog reset
r sadd set 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
set e [lindex [r slowlog get] 0]
lindex $e 3
} {sadd set 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 {... (2 more arguments)}}
test {SLOWLOG - too long arguments are trimmed} {
r config set slowlog-log-slower-than 0
r slowlog reset
set arg [string repeat A 129]
r sadd set foo $arg
set e [lindex [r slowlog get] 0]
lindex $e 3
} {sadd set foo {AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA... (1 more bytes)}}
}
+8
View File
@@ -524,6 +524,14 @@ start_server {tags {"zset"}} {
r zrange out 0 -1 withscores
} {neginf 0}
test {ZINTERSTORE #516 regression, mixed sets and ziplist zsets} {
r sadd one 100 101 102 103
r sadd two 100 200 201 202
r zadd three 1 500 1 501 1 502 1 503 1 100
r zinterstore to_here 3 one two three WEIGHTS 0 0 1
r zrange to_here 0 -1
} {100}
proc stressers {encoding} {
if {$encoding == "ziplist"} {
# Little extra to allow proper fuzzing in the sorting stresser
+47 -26
View File
@@ -2,41 +2,38 @@
# Copyright 2011 Dvir Volk <dvirsk at gmail dot com>. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification, are
# permitted provided that the following conditions are met:
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice, this list of
# conditions and the following disclaimer.
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice, this list
# of conditions and the following disclaimer in the documentation and/or other materials
# provided with the distribution.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL Dvir Volk OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
# EVENT SHALL Dvir Volk OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
# OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
# EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
#
# # # # # # # # # # # # # # # # # # # # # # # # # #
################################################################################
#
# Interactive service installer for redis server
# this generates a redis config file and an /etc/init.d script, and installs them
# this scripts should be run as root
#
die () {
echo "ERROR: $1. Aborting!"
exit 1
}
#Initial defaults
_REDIS_PORT=6379
@@ -135,8 +132,27 @@ CONF=\"$REDIS_CONFIG_FILE\"\n\n
REDISPORT=\"$REDIS_PORT\"\n\n
###############\n\n"
#combine the header and the template (which is actually a static footer)
echo $REDIS_INIT_HEADER > $TMP_FILE && cat $INIT_TPL_FILE >> $TMP_FILE || die "Could not write init script to $TMP_FILE"
REDIS_CHKCONFIG_INFO=\
"# REDHAT chkconfig header\n\n
# chkconfig: - 58 74\n
# description: redis_6379 is the redis daemon.\n
### BEGIN INIT INFO\n
# Provides: redis_6379\n
# Required-Start: $network $local_fs $remote_fs\n
# Required-Stop: $network $local_fs $remote_fs\n
# Should-Start: $syslog $named\n
# Should-Stop: $syslog $named\n
# Short-Description: start and stop redis_6379\n
# Description: Redis daemon\n
### END INIT INFO\n\n"
if [[ ! `which chkconfig` ]] ; then
#combine the header and the template (which is actually a static footer)
echo -e $REDIS_INIT_HEADER > $TMP_FILE && cat $INIT_TPL_FILE >> $TMP_FILE || die "Could not write init script to $TMP_FILE"
else
#if we're a box with chkconfig on it we want to include info for chkconfig
echo -e $REDIS_INIT_HEADER $REDIS_CHKCONFIG_INFO > $TMP_FILE && cat $INIT_TPL_FILE >> $TMP_FILE || die "Could not write init script to $TMP_FILE"
fi
#copy to /etc/init.d
cp -f $TMP_FILE $INIT_SCRIPT_DEST && chmod +x $INIT_SCRIPT_DEST || die "Could not copy redis init script to $INIT_SCRIPT_DEST"
@@ -144,13 +160,18 @@ echo "Copied $TMP_FILE => $INIT_SCRIPT_DEST"
#Install the service
echo "Installing service..."
update-rc.d redis_$REDIS_PORT defaults && echo "Success!"
if [[ ! `which chkconfig` ]] ; then
#if we're not a chkconfig box assume we're able to use update-rc.d
update-rc.d redis_$REDIS_PORT defaults && echo "Success!"
else
# we're chkconfig, so lets add to chkconfig and put in runlevel 345
chkconfig --add redis_$REDIS_PORT && echo "Successfully added to chkconfig!"
chkconfig --level 345 redis_$REDIS_PORT on && echo "Successfully added to runlevels 345!"
fi
/etc/init.d/redis_$REDIS_PORT start || die "Failed starting service..."
#tada
echo "Installation successful!"
exit 0