Redis/redis-For-Windows
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Compare commits

base: Redis/redis-For-Windows:2.4.0-rc3
Branches Tags
Redis/redis-For-Windows:develop Redis/redis-For-Windows:win-5.0 Redis/redis-For-Windows:win-6.2
Redis/redis-For-Windows:5.0.14.1 Redis/redis-For-Windows:v5.0.14.1 Redis/redis-For-Windows:v5.0.14 Redis/redis-For-Windows:v5.0.10 Redis/redis-For-Windows:v5.0.9 Redis/redis-For-Windows:v4.0.14.2 Redis/redis-For-Windows:v4.0.14.1 Redis/redis-For-Windows:v4.0.14 Redis/redis-For-Windows:v4.0.2.3-alpha Redis/redis-For-Windows:v4.0.2.2-alpha Redis/redis-For-Windows:v4.0.2.1-alpha Redis/redis-For-Windows:v4.0.2-alpha Redis/redis-For-Windows:win-3.2.100 Redis/redis-For-Windows:win-3.0.504 Redis/redis-For-Windows:win-3.0.503 Redis/redis-For-Windows:win-2.8.2402 Redis/redis-For-Windows:win-3.0.502 Redis/redis-For-Windows:win-2.8.2401 Redis/redis-For-Windows:3.2.1 Redis/redis-For-Windows:win-3.2.0000-preview Redis/redis-For-Windows:3.2.0 Redis/redis-For-Windows:3.2.0-rc3 Redis/redis-For-Windows:3.2.0-rc2 Redis/redis-For-Windows:win-2.8.2400 Redis/redis-For-Windows:win-3.0.501 Redis/redis-For-Windows:3.2-rc1 Redis/redis-For-Windows:3.0.6 Redis/redis-For-Windows:2.8.24 Redis/redis-For-Windows:win-3.0.500 Redis/redis-For-Windows:win-3.0.500-rc2 Redis/redis-For-Windows:win-3.0.500-rc1 Redis/redis-For-Windows:win-2.8.2104 Redis/redis-For-Windows:3.0.5 Redis/redis-For-Windows:2.8.23 Redis/redis-For-Windows:win-3.0.300-beta1 Redis/redis-For-Windows:win-2.8.2103 Redis/redis-For-Windows:2.8.22 Redis/redis-For-Windows:3.0.4 Redis/redis-For-Windows:win-3.0.300-alpha3 Redis/redis-For-Windows:win-2.8.2102 Redis/redis-For-Windows:win-3.0.300-alpha2 Redis/redis-For-Windows:win-3.0.100-alpha1 Redis/redis-For-Windows:3.0.3 Redis/redis-For-Windows:win-2.8.2101 Redis/redis-For-Windows:win-2.8.21 Redis/redis-For-Windows:3.0.2 Redis/redis-For-Windows:2.8.21 Redis/redis-For-Windows:2.8.20 Redis/redis-For-Windows:3.0.1 Redis/redis-For-Windows:win-2.8.19.1 Redis/redis-For-Windows:3.0.0 Redis/redis-For-Windows:3.0.0-rc6 Redis/redis-For-Windows:3.0.0-rc5 Redis/redis-For-Windows:win-2.8.19-rc1 Redis/redis-For-Windows:win-2.8.19 Redis/redis-For-Windows:3.0.0-rc4 Redis/redis-For-Windows:win-2.8.19-pre2 Redis/redis-For-Windows:win-2.8.17.4 Redis/redis-For-Windows:3.0.0-rc3 Redis/redis-For-Windows:3.0.0-rc2 Redis/redis-For-Windows:win-2.8.17.3 Redis/redis-For-Windows:win-2.8.19-pre1 Redis/redis-For-Windows:win-2.8.17.2 Redis/redis-For-Windows:win-2.8.17.1 Redis/redis-For-Windows:2.8.19 Redis/redis-For-Windows:2.8.18 Redis/redis-For-Windows:3.0.0-rc1 Redis/redis-For-Windows:2.8.17 Redis/redis-For-Windows:2.8.16 Redis/redis-For-Windows:2.8.15 Redis/redis-For-Windows:win2.8.12 Redis/redis-For-Windows:2.8.14 Redis/redis-For-Windows:3.0.0-beta8 Redis/redis-For-Windows:2.8.13 Redis/redis-For-Windows:3.0.0-beta7 Redis/redis-For-Windows:win_2.8.9 Redis/redis-For-Windows:2.8.12 Redis/redis-For-Windows:2.8.11 Redis/redis-For-Windows:3.0.0-beta6 Redis/redis-For-Windows:2.8.10 Redis/redis-For-Windows:3.0.0-beta5 Redis/redis-For-Windows:3.0.0-beta4 Redis/redis-For-Windows:2.6.14_msopentech Redis/redis-For-Windows:2.8.4_msopentech Redis/redis-For-Windows:msopentech_2.8.4 Redis/redis-For-Windows:3.0.0-beta3 Redis/redis-For-Windows:2.8.9 Redis/redis-For-Windows:2.8.8 Redis/redis-For-Windows:3.0.0-beta2 Redis/redis-For-Windows:2.8.7 Redis/redis-For-Windows:2.8.6 Redis/redis-For-Windows:3.0.0-beta1 Redis/redis-For-Windows:2.8.5 Redis/redis-For-Windows:2.8.4 Redis/redis-For-Windows:2.8.3 Redis/redis-For-Windows:2.8.2 Redis/redis-For-Windows:2.8.1 Redis/redis-For-Windows:2.8.0 Redis/redis-For-Windows:2.8.0-rc6 Redis/redis-For-Windows:2.8.0-rc5 Redis/redis-For-Windows:2.8.0-rc4 Redis/redis-For-Windows:2.8.0-rc3 Redis/redis-For-Windows:2.8.0-rc2 Redis/redis-For-Windows:2.8.0-rc1 Redis/redis-For-Windows:2.6.14 Redis/redis-For-Windows:Win2.6.8 Redis/redis-For-Windows:2.6.13 Redis/redis-For-Windows:2.6.12 Redis/redis-For-Windows:2.6.11 Redis/redis-For-Windows:2.6.10 Redis/redis-For-Windows:2.6.9 Redis/redis-For-Windows:2.6.8 Redis/redis-For-Windows:2.6.7 Redis/redis-For-Windows:2.6.6 Redis/redis-For-Windows:2.6.5 Redis/redis-For-Windows:2.6.4 Redis/redis-For-Windows:2.6.3 Redis/redis-For-Windows:2.6.2 Redis/redis-For-Windows:2.6.1 Redis/redis-For-Windows:2.6.0 Redis/redis-For-Windows:2.6.0-rc8 Redis/redis-For-Windows:2.6.0-rc7 Redis/redis-For-Windows:2.6.0-rc6 Redis/redis-For-Windows:2.6.0-rc5 Redis/redis-For-Windows:2.6.0-rc4 Redis/redis-For-Windows:2.6.0-rc3 Redis/redis-For-Windows:2.6.0-rc2 Redis/redis-For-Windows:2.6.0-rc1 Redis/redis-For-Windows:2.4.11 Redis/redis-For-Windows:2.4.10 Redis/redis-For-Windows:2.4.9 Redis/redis-For-Windows:2.4.8 Redis/redis-For-Windows:2.4.6_win32 Redis/redis-For-Windows:2.4.7 Redis/redis-For-Windows:2.4.6 Redis/redis-For-Windows:2.4.5 Redis/redis-For-Windows:2.4.4 Redis/redis-For-Windows:2.4.3 Redis/redis-For-Windows:2.4.2 Redis/redis-For-Windows:2.4.1 Redis/redis-For-Windows:2.4.0 Redis/redis-For-Windows:2.4.0-rc8 Redis/redis-For-Windows:2.4.0-rc7 Redis/redis-For-Windows:2.4.0-rc6 Redis/redis-For-Windows:2.4.0-rc5 Redis/redis-For-Windows:2.4.0-rc4 Redis/redis-For-Windows:2.4.0-rc3 Redis/redis-For-Windows:with-deprecated-diskstore Redis/redis-For-Windows:2.4.0-rc2 Redis/redis-For-Windows:2.4.0-rc1 Redis/redis-For-Windows:2.2.4 Redis/redis-For-Windows:2.2.3 Redis/redis-For-Windows:3.0-alpha0 Redis/redis-For-Windows:2.2.2 Redis/redis-For-Windows:2.2.1 Redis/redis-For-Windows:2.2.0 Redis/redis-For-Windows:2.2.0-rc4 Redis/redis-For-Windows:2.3-alpha0 Redis/redis-For-Windows:2.2.0-rc3 Redis/redis-For-Windows:2.2.0-rc2 Redis/redis-For-Windows:2.2.0-rc1 Redis/redis-For-Windows:2.2-alpha6 Redis/redis-For-Windows:2.2-alpha5 Redis/redis-For-Windows:2.2-alpha4 Redis/redis-For-Windows:2.2-alpha3 Redis/redis-For-Windows:2.2-alpha2 Redis/redis-For-Windows:2.2-alpha1 Redis/redis-For-Windows:2.2-alpha0 Redis/redis-For-Windows:v2.1.1-watch Redis/redis-For-Windows:v2.0.0-rc1 Redis/redis-For-Windows:v1.3.12 Redis/redis-For-Windows:v1.3.11 Redis/redis-For-Windows:v1.3.10 Redis/redis-For-Windows:v1.3.9 Redis/redis-For-Windows:v1.3.8 Redis/redis-For-Windows:v1.3.7 Redis/redis-For-Windows:1.3.6 Redis/redis-For-Windows:vm-playpen
..
compare: Redis/redis-For-Windows:2.4.2
Branches Tags
Redis/redis-For-Windows:win-5.0 Redis/redis-For-Windows:develop Redis/redis-For-Windows:win-6.2
Redis/redis-For-Windows:5.0.14.1 Redis/redis-For-Windows:v5.0.14.1 Redis/redis-For-Windows:v5.0.14 Redis/redis-For-Windows:v5.0.10 Redis/redis-For-Windows:v5.0.9 Redis/redis-For-Windows:v4.0.14.2 Redis/redis-For-Windows:v4.0.14.1 Redis/redis-For-Windows:v4.0.14 Redis/redis-For-Windows:v4.0.2.3-alpha Redis/redis-For-Windows:v4.0.2.2-alpha Redis/redis-For-Windows:v4.0.2.1-alpha Redis/redis-For-Windows:v4.0.2-alpha Redis/redis-For-Windows:win-3.2.100 Redis/redis-For-Windows:win-3.0.504 Redis/redis-For-Windows:win-3.0.503 Redis/redis-For-Windows:win-2.8.2402 Redis/redis-For-Windows:win-3.0.502 Redis/redis-For-Windows:win-2.8.2401 Redis/redis-For-Windows:3.2.1 Redis/redis-For-Windows:win-3.2.0000-preview Redis/redis-For-Windows:3.2.0 Redis/redis-For-Windows:3.2.0-rc3 Redis/redis-For-Windows:3.2.0-rc2 Redis/redis-For-Windows:win-2.8.2400 Redis/redis-For-Windows:win-3.0.501 Redis/redis-For-Windows:3.2-rc1 Redis/redis-For-Windows:3.0.6 Redis/redis-For-Windows:2.8.24 Redis/redis-For-Windows:win-3.0.500 Redis/redis-For-Windows:win-3.0.500-rc2 Redis/redis-For-Windows:win-3.0.500-rc1 Redis/redis-For-Windows:win-2.8.2104 Redis/redis-For-Windows:3.0.5 Redis/redis-For-Windows:2.8.23 Redis/redis-For-Windows:win-3.0.300-beta1 Redis/redis-For-Windows:win-2.8.2103 Redis/redis-For-Windows:2.8.22 Redis/redis-For-Windows:3.0.4 Redis/redis-For-Windows:win-3.0.300-alpha3 Redis/redis-For-Windows:win-2.8.2102 Redis/redis-For-Windows:win-3.0.300-alpha2 Redis/redis-For-Windows:win-3.0.100-alpha1 Redis/redis-For-Windows:3.0.3 Redis/redis-For-Windows:win-2.8.2101 Redis/redis-For-Windows:win-2.8.21 Redis/redis-For-Windows:3.0.2 Redis/redis-For-Windows:2.8.21 Redis/redis-For-Windows:2.8.20 Redis/redis-For-Windows:3.0.1 Redis/redis-For-Windows:win-2.8.19.1 Redis/redis-For-Windows:3.0.0 Redis/redis-For-Windows:3.0.0-rc6 Redis/redis-For-Windows:3.0.0-rc5 Redis/redis-For-Windows:win-2.8.19-rc1 Redis/redis-For-Windows:win-2.8.19 Redis/redis-For-Windows:3.0.0-rc4 Redis/redis-For-Windows:win-2.8.19-pre2 Redis/redis-For-Windows:win-2.8.17.4 Redis/redis-For-Windows:3.0.0-rc3 Redis/redis-For-Windows:3.0.0-rc2 Redis/redis-For-Windows:win-2.8.17.3 Redis/redis-For-Windows:win-2.8.19-pre1 Redis/redis-For-Windows:win-2.8.17.2 Redis/redis-For-Windows:win-2.8.17.1 Redis/redis-For-Windows:2.8.19 Redis/redis-For-Windows:2.8.18 Redis/redis-For-Windows:3.0.0-rc1 Redis/redis-For-Windows:2.8.17 Redis/redis-For-Windows:2.8.16 Redis/redis-For-Windows:2.8.15 Redis/redis-For-Windows:win2.8.12 Redis/redis-For-Windows:2.8.14 Redis/redis-For-Windows:3.0.0-beta8 Redis/redis-For-Windows:2.8.13 Redis/redis-For-Windows:3.0.0-beta7 Redis/redis-For-Windows:win_2.8.9 Redis/redis-For-Windows:2.8.12 Redis/redis-For-Windows:2.8.11 Redis/redis-For-Windows:3.0.0-beta6 Redis/redis-For-Windows:2.8.10 Redis/redis-For-Windows:3.0.0-beta5 Redis/redis-For-Windows:3.0.0-beta4 Redis/redis-For-Windows:2.6.14_msopentech Redis/redis-For-Windows:2.8.4_msopentech Redis/redis-For-Windows:msopentech_2.8.4 Redis/redis-For-Windows:3.0.0-beta3 Redis/redis-For-Windows:2.8.9 Redis/redis-For-Windows:2.8.8 Redis/redis-For-Windows:3.0.0-beta2 Redis/redis-For-Windows:2.8.7 Redis/redis-For-Windows:2.8.6 Redis/redis-For-Windows:3.0.0-beta1 Redis/redis-For-Windows:2.8.5 Redis/redis-For-Windows:2.8.4 Redis/redis-For-Windows:2.8.3 Redis/redis-For-Windows:2.8.2 Redis/redis-For-Windows:2.8.1 Redis/redis-For-Windows:2.8.0 Redis/redis-For-Windows:2.8.0-rc6 Redis/redis-For-Windows:2.8.0-rc5 Redis/redis-For-Windows:2.8.0-rc4 Redis/redis-For-Windows:2.8.0-rc3 Redis/redis-For-Windows:2.8.0-rc2 Redis/redis-For-Windows:2.8.0-rc1 Redis/redis-For-Windows:2.6.14 Redis/redis-For-Windows:Win2.6.8 Redis/redis-For-Windows:2.6.13 Redis/redis-For-Windows:2.6.12 Redis/redis-For-Windows:2.6.11 Redis/redis-For-Windows:2.6.10 Redis/redis-For-Windows:2.6.9 Redis/redis-For-Windows:2.6.8 Redis/redis-For-Windows:2.6.7 Redis/redis-For-Windows:2.6.6 Redis/redis-For-Windows:2.6.5 Redis/redis-For-Windows:2.6.4 Redis/redis-For-Windows:2.6.3 Redis/redis-For-Windows:2.6.2 Redis/redis-For-Windows:2.6.1 Redis/redis-For-Windows:2.6.0 Redis/redis-For-Windows:2.6.0-rc8 Redis/redis-For-Windows:2.6.0-rc7 Redis/redis-For-Windows:2.6.0-rc6 Redis/redis-For-Windows:2.6.0-rc5 Redis/redis-For-Windows:2.6.0-rc4 Redis/redis-For-Windows:2.6.0-rc3 Redis/redis-For-Windows:2.6.0-rc2 Redis/redis-For-Windows:2.6.0-rc1 Redis/redis-For-Windows:2.4.11 Redis/redis-For-Windows:2.4.10 Redis/redis-For-Windows:2.4.9 Redis/redis-For-Windows:2.4.8 Redis/redis-For-Windows:2.4.6_win32 Redis/redis-For-Windows:2.4.7 Redis/redis-For-Windows:2.4.6 Redis/redis-For-Windows:2.4.5 Redis/redis-For-Windows:2.4.4 Redis/redis-For-Windows:2.4.3 Redis/redis-For-Windows:2.4.2 Redis/redis-For-Windows:2.4.1 Redis/redis-For-Windows:2.4.0 Redis/redis-For-Windows:2.4.0-rc8 Redis/redis-For-Windows:2.4.0-rc7 Redis/redis-For-Windows:2.4.0-rc6 Redis/redis-For-Windows:2.4.0-rc5 Redis/redis-For-Windows:2.4.0-rc4 Redis/redis-For-Windows:2.4.0-rc3 Redis/redis-For-Windows:with-deprecated-diskstore Redis/redis-For-Windows:2.4.0-rc2 Redis/redis-For-Windows:2.4.0-rc1 Redis/redis-For-Windows:2.2.4 Redis/redis-For-Windows:2.2.3 Redis/redis-For-Windows:3.0-alpha0 Redis/redis-For-Windows:2.2.2 Redis/redis-For-Windows:2.2.1 Redis/redis-For-Windows:2.2.0 Redis/redis-For-Windows:2.2.0-rc4 Redis/redis-For-Windows:2.3-alpha0 Redis/redis-For-Windows:2.2.0-rc3 Redis/redis-For-Windows:2.2.0-rc2 Redis/redis-For-Windows:2.2.0-rc1 Redis/redis-For-Windows:2.2-alpha6 Redis/redis-For-Windows:2.2-alpha5 Redis/redis-For-Windows:2.2-alpha4 Redis/redis-For-Windows:2.2-alpha3 Redis/redis-For-Windows:2.2-alpha2 Redis/redis-For-Windows:2.2-alpha1 Redis/redis-For-Windows:2.2-alpha0 Redis/redis-For-Windows:v2.1.1-watch Redis/redis-For-Windows:v2.0.0-rc1 Redis/redis-For-Windows:v1.3.12 Redis/redis-For-Windows:v1.3.11 Redis/redis-For-Windows:v1.3.10 Redis/redis-For-Windows:v1.3.9 Redis/redis-For-Windows:v1.3.8 Redis/redis-For-Windows:v1.3.7 Redis/redis-For-Windows:1.3.6 Redis/redis-For-Windows:vm-playpen
94 Commits
2.4.0-rc3 .. 2.4.2
Author SHA1 Message Date
antirez 072a9052c7 Redis 2.4.2 2011-10-26 16:16:33 +02:00
Salvatore Sanfilippo 00af23bcbf Merge pull request #154 from florean/2.4 
2.4
 2011-10-26 04:01:01 -07:00
antirez 15c9497280 Clear the AOF rewrite scheduled flag once an AOF rewrite is triggered. Fix for issue #161, probably fixing 159 as well. 2011-10-26 12:55:09 +02:00
antirez 45bd4a1853 Better installation info inside README file. 2011-10-23 12:28:31 +02:00
antirez 0327d9b8e9 removed a few entries for gitignore 2011-10-23 12:16:30 +02:00
antirez 1e81d971dc template files for installation script added 2011-10-23 12:15:02 +02:00
antirez 9a3a84bfdf Script to install the server in Ubuntu/Debian added to 2.4 as well. 2011-10-23 11:49:58 +02:00
Nathan Florea 1126d5aa49 Set errno to 0 before calling strtol. 2011-10-20 18:20:58 -07:00
antirez b6fe152b83 Fix to the previous fix for issue #145, make sure to return only after the event was unregistered. 2011-10-18 17:12:16 +02:00
smly 1270a13667 Fixed a small memory leak I found with valgrind 2011-10-18 14:13:47 +02:00
antirez 98f5abb6e3 Return from syncWithMaster() ASAP if the event fired but the instance is no longer a slave. This should fix Issue #145. 2011-10-18 11:09:32 +02:00
antirez ab648d973d 2.4.1 release notes. 2011-10-17 10:46:00 +02:00
antirez 5ab72bedb7 version bumped to 2.4.1 2011-10-17 10:43:02 +02:00
antirez 9a83d13fb5 revert version bumping to 2.4.1, put in the previous commit for an error. 2011-10-17 10:42:47 +02:00
antirez 6662806a26 Regression test for issue #142 added 2011-10-17 10:40:11 +02:00
antirez 6e3f1a9e47 FLUSHALL now prevents rdbSave() from resetting the dirty counter, so that the command will get replicated and put inside the AOF. This fixes issue #142 2011-10-17 10:28:57 +02:00
antirez d12f26c231 FLUSHALL will only perform a blocking SAVE if RDB persistence is configured. 2011-10-17 10:25:35 +02:00
antirez a502f48320 2.4.0 Release notes. 2011-10-14 17:54:25 +02:00
antirez 0c71939c05 Version bumped to 2.4.0 2011-10-14 17:49:20 +02:00
antirez a4f46f211c Exit with Fatal error at startup if the RDB file signature or version is wrong. 
Ref: issue #103
 2011-10-14 17:04:24 +02:00
antirez 8185c2e9f1 redis-check-dump: RDB version 2 now supported. 2011-10-14 16:52:11 +02:00
antirez 71bb31bf16 More informative error when DEBUG RELOAD fails. 2011-10-14 14:30:41 +02:00
antirez c7d23f3ed1 Fix for issue #132. Now AUTH raises an error if no server password is configured. 2011-10-10 22:22:27 +02:00
Nathan Florea 09cefcfbb3 Added a config directive for a Unix socket mask 
Added a configuration directive to allow a user to specify the
permissions to be granted to the Unix socket file.  I followed
the format Pieter and Salvatore discusses in issue #85 (
https://github.com/antirez/redis/issues/85).
 2011-10-10 22:08:11 +02:00
Nathan Florea 1ca8deebef Unlink Unix socket file on shutdown 2011-10-10 22:08:03 +02:00
antirez b7f667b553 Issue #131. stime/utime reported in INFO was inverted. Fixed thanks to Didier Spezia. 2011-10-10 15:29:36 +02:00
antirez 067ad37f96 Fix for bug #128 about the RENAME command. 2011-10-10 15:25:28 +02:00
antirez a6e312bf9a Added two new tests for RENAME, currently both will fail because of bug #128. 2011-10-10 15:25:22 +02:00
antirez 0e841b998a CONFIG SET/GET support for loglevel 2011-10-06 15:25:53 +02:00
antirez cd0458470e Redis test ports selection made more robust. This prevents the test from hanging if an already bound port is selected but the TCP server listening to it does not cause a protocol error with a Redis client PING. Also base port moved away from the range near to the Redis Cluster gossip ports. 2011-10-04 10:07:29 +02:00
antirez 678dbc48fd Compilation fixed on OpenBSD making sure that _XOPEN_SOURCE is set to 700 in fmacros.h 2011-09-29 10:20:26 +02:00
Juri M. Vainonen 3a91aca033 fixed a small bug that caused redis-cli to segfault when given single numeric parameter greater that zero. 2011-09-21 22:33:01 +02:00
antirez 9286ecd767 Release notes for Redis 2.4 RC8 added 2011-09-21 14:05:36 +02:00
antirez 902a656f7a Version bumped to 2.3.11 2011-09-21 13:56:03 +02:00
antirez a58e486d79 no longer valid design documents removed 2011-09-20 11:15:18 +02:00
antirez b0fcf23304 hash zipmap max entries and entry size information removed from INFO command since we have this info and the equivalent info for sets, hashes, zset accessible via the CONFIG GET command. 2011-09-20 11:08:08 +02:00
vambo 57f611d953 Corrected typo in documentation for R_Zero, R_PosInf, R_NegInf, R_Nan; 2011-09-20 11:01:20 +02:00
Konstantin Tokarev 20c26321c2 Fixed compilation on Mac/PPC 
Applied patch from Issue 119 provided by
pahowes@gmail.com
 2011-09-20 10:59:47 +02:00
Konstantin Merenkov 2017d8baf7 [issue 620] don't segfault if number of arguments is less than 1 2011-09-20 10:56:43 +02:00
antirez a511ce1b49 Include port number in error when can't bind lisetning port on startup. 2011-09-20 10:48:33 +02:00
antirez 1f164f4e1d Fixed comment typo 2011-09-19 17:49:50 +02:00
antirez 9f775040c8 Comment out things in bio.c that are currently not useful but that may be useful in the future. 2011-09-19 17:44:21 +02:00
antirez 6ca793fc21 Emit a log message when AOF fsync is still in progress but we are forced to write from the main thread since two seconds already elapsed. 2011-09-19 17:44:17 +02:00
antirez 01ac844ba2 use aof_fsync wrapper instead of fsync(2) in bio.c 2011-09-19 17:44:11 +02:00
antirez 79b9ec53d1 fixed flushAppendOnlyFile() function prototype to reflect the new implementation, fixing compilation. 2011-09-19 17:43:56 +02:00
antirez 4c2d5f0980 postpone the AOF fsync if policy is everysec and there is a background fsync already going. 2011-09-19 17:43:49 +02:00
antirez 056c2acaea Fixed typo preventing compilation 2011-09-19 17:43:44 +02:00
antirez f85c383b6c Added aof_backgronud_fsync() function, and use it in the bacground rewrite done handler when the fsync policy is everysec. 2011-09-19 17:43:40 +02:00
antirez 3ae56d8458 REDIS_BIO_AOF_FSYNC implemented 2011-09-19 17:43:34 +02:00
antirez 0807f5b9ff fixed a problem in bioOlderJobType() when there are no jobs of the specified type. Now the function returns 0 when this happens. 2011-09-19 17:43:28 +02:00
antirez 2eae7cbc7d some more comment about bio.c design 2011-09-19 17:43:23 +02:00
antirez 693618f1c1 Remove backgroud jobs from the queue only when the processing was completed, and not just started. 2011-09-19 17:43:19 +02:00
antirez c67e8d3bf6 Use a different thread for every different type of background job 2011-09-19 17:43:13 +02:00
antirez 411cb75bcd Better comments for bioWaitPendingJobsLE 2011-09-19 17:43:07 +02:00
antirez 794419192e Background I/O library enhanced so that the main thread can query for the number of pending jobs of the specified type. 2011-09-19 17:43:01 +02:00
antirez 481a05cd23 commenting out aof-race test for now, waiting for a fix to issue #80 2011-09-16 15:43:27 +02:00
Pieter Noordhuis b9a65682fb Failing test related to AOF rewrite buffers 2011-09-16 11:24:08 +02:00
Pieter Noordhuis e3d1c8b080 Enforce upper bound for number of requests in redis-benchmark 2011-09-16 11:23:53 +02:00
Pieter Noordhuis 682b63312e Fix AOF race that may duplicate commands 2011-09-16 11:14:00 +02:00
antirez 6fd1a72712 Fixed a bug in the average latency metering of redis-cli --latency 2011-09-15 19:32:49 +02:00
antirez 2e48ad5422 Implemented --latency in redis-cli 2011-09-15 19:28:24 +02:00
antirez 67f594f9b5 Optimize LRANGE to scan the list starting from the head or the tail in order to traverse the minimal number of elements. Thanks to Didier Spezia for noticing the problem and providing a patch. 2011-09-14 15:17:04 +02:00
antirez b7bf29059e Don't replicate/AOF SAVE 2011-09-14 13:08:19 +02:00
antirez 889d468e51 use bio.c instead of libeio for closing the AOF file in background. Some comment added and other minor changes. 2011-09-14 10:52:07 +02:00
Pieter Noordhuis 84f48cc401 Asynchronously close old file after BGREWRITEAOF 2011-09-14 10:51:52 +02:00
antirez 44690ab15f make bio_list static as well 2011-09-14 10:50:29 +02:00
antirez a72c0423b4 debug message removed 2011-09-14 10:50:22 +02:00
antirez c8d0013f38 use pthread_cond_signal() to make sure the background thread will process the job. 2011-09-14 10:47:53 +02:00
antirez 5ca75751ed typo fixed in bio.c 2011-09-14 10:47:48 +02:00
antirez af14408e5c add background jobs initialization to Redis main init function 2011-09-14 10:47:31 +02:00
antirez ed56ad5689 bio.c typos fixes to make it compile 2011-09-14 10:47:26 +02:00
antirez c427f5d3c1 bio.o added to redis-server objects 2011-09-14 10:47:19 +02:00
Pieter Noordhuis 325d2eb8b5 Speed up protocol synthesis for AOF 2011-09-13 12:26:53 +02:00
Pieter Noordhuis 0fb6212cf4 Re-use AOF buffer when it is small enough 2011-09-13 12:26:49 +02:00
Pieter Noordhuis f602318349 Fix indent 2011-09-13 12:26:43 +02:00
antirez 2401c3eccb added a newline for code readability 2011-09-13 12:26:38 +02:00
Pieter Noordhuis 8d9804bf40 Avoid unnecessary calls to time(3) 2011-09-13 12:26:33 +02:00
antirez da30c9e1e3 Version bumped to 2.3.10, changelog updated 2011-09-12 12:13:44 +02:00
antirez 3c51d3b36b Remove the write handler only if there are no longer objects in the output queue AND if the static buffer is empty. This bug was the cause of a possible server-stop-responding-to-client bug under some specific work load. Thanks to Pieter Noordhuis for spotting and fixing it. 2011-09-12 11:06:11 +02:00
antirez 00c7f312cc regression test for bug 593 added 2011-09-12 10:54:43 +02:00
antirez b10913b93d fixed typos in the comments of rpoplpushHandlePush() 2011-09-12 10:05:38 +02:00
Hampus Wessman cbf01c0e01 Fix crash when chaining brpoplpush with other blocking commands. 2011-09-12 10:05:32 +02:00
antirez 214cf63730 Version bumped to 2.3.9. Changelog modified accordingly. 2011-08-09 12:04:26 +02:00
antirez 4a4f4cb5ff Auto rewrite integer overflow fixed. Thanks to Gaurav for finding it (see issue 602 on google code), and Pieter Noordhuis for tracing it. 2011-08-09 11:45:57 +02:00
antirez a57bd791e9 243b783fdc 2011-07-29 15:50:15 +02:00
antirez ed9f997ce5 fixed assert proc on Redis test 2011-07-29 15:49:31 +02:00
antirez 55eea93cae Release notes for 2.4.0-RC5 2011-07-29 15:44:21 +02:00
antirez 753e91b6c1 Version bumped to 2.3.8 2011-07-29 15:39:31 +02:00
Pieter Noordhuis a830d941a7 Fix adding bulk reply when getcwd fails 2011-07-27 16:22:47 +02:00
Pieter Noordhuis 9c3a89853c HDEL: Abort deleting fields when hash is removed 2011-07-27 12:30:35 +02:00
antirez 7ca6672338 version bumped to 2.3.7 2011-07-22 16:26:24 +02:00
antirez 6c68df4f32 Close the listening sockets. Apparently this allows faster restarts. 2011-07-22 12:12:45 +02:00
antirez 8b58723eb2 prepareForShutdown() fixed for correctness regarding what to do with AOF and RDB persistence on exit. 2011-07-22 11:55:34 +02:00
Salvatore Sanfilippo abd4cd7827 Added a 'runtest' script that is responsible to check if Tcl is available and run the test. This is invoked from Makefile as well. 2011-07-15 17:24:44 +02:00
43 changed files with 1604 additions and 728 deletions
Expand all files Collapse all files
.gitignore
-2
View File
@@ -9,12 +9,10 @@ redis-check-aof
doc-tools
mkrelease.sh
release
myredis.conf
misc/*
src/release.h
appendonly.aof
SHORT_TERM_TODO
redis.conf.*
release.h
src/transfer.sh
src/configs
00-RELEASENOTES
+71 -6
View File
@@ -7,7 +7,8 @@ Redis 2.2 is mostly a strict subset of 2.4.
The only thing you should be aware is that you can't use .rdb and AOF files
generated with 2.4 into a 2.2 instance.
2.4 slaves can be attached to 2.2 masters, but not the contrary.
2.4 slaves can be attached to 2.2 masters, but not the contrary, and only for
the time needed to perform the version upgrade.
From the point of view of the API Redis 2.4 only adds new commands
(other commands now accepts a variable number of arguments) so you don't need
@@ -17,13 +18,77 @@ to modify your program in order to use Redis 2.4.
CHANGELOG
---------
What's new in Redis 2.3.5 (2.4 Release Candidate 1)
What's new in Redis 2.4.2
=========================
* [BUGFIX] Unix socket creation mask config directive fixed.
* [BUGFIX] Fixed a bug that forced Redis to continously rewrite the AOF file
when a manual BGREWRITEAOF was issued during a BGSAVE was
still in progress.
* [BUGFIX] Fixed a server crash resulting from bad handling of SLAVEOF NO ONE.
* [BUGFIX] Fixed a memory leak in redis-cli.
* Debian/Ubuntu Redis system-wide installation script added under /utils.
What's new in Redis 2.4.1
=========================
* [BUGFIX] FLUSHALL was not replicated nor written into the Append Only File.
* [BUGFIX] FLUSHALL now only performs a sync SAVE if there is at least
one save point configured.
What's new in Redis 2.4.0
=========================
* [BUGFIX] redis-cli segfault with single numerical argument fixed.
* [BUGFIX] OpenBSD compilation problem fixed.
* [BUGFIX] More robust Redis test, with better random port selection.
* [BUGFIX] Fix for bug #128 about the RENAME command.
* [BUGFIX] Fixed Issue #131. stime/utime reported in INFO was inverted.
* [BUGFIX] Unlink Unix socket file on shutdown.
* [BUGFIX] AUTH now returns error if no password is set on the server.
* [BUGFIX] Exit with Fatal error at startup on RDB loading errors.
* redis-check-dump: RDB version 2 now supported.
* More informative error when DEBUG RELOAD fails.
* Added a config directive for a Unix socket mask.
* CONFIG SET/GET for loglevel.
What's new in Redis 2.3.11 (2.4 Release Candidate 8)
====================================================
* [BUGFIX] Fixed a rare but possible AOF race condition that could result into
duplicated commands inside the AOF.
* [BUGFIX] Fixed issue 620, don't segfault on corrupted (by hand) AOF.
* [BUGFIX] Fixed compilation on Mac/PPC.
* [BUGFIX] Don't replicate SAVE.
* LRANGE optimization may drastically improve performances when querying the
final part of a long list.
* redis-cli now implements a --latency mode to monitory Redis delay.
* Hash type settings removed from INFO (same info is available via config GET)
* Include port number on error when can't bind.
* AOF fsync is now performed in background when fsync policy is 'everysec'.
* AOF performances improved moving in background a possibly slow close(2) call.
* AOF protocol synthesis speedup.
What's new in Redis 2.3.10 (2.4 Release Candidate 7)
====================================================
* [BUGFIX] Fixed issue 593 (BRPOPLPUSH related crash).
* [BUGFIX] Fixed an issue with the networking layer that may prevent Redis from sending the whole reply back to client under extreme conditions.
What's new in Redis 2.3.9 (2.4 Release Candidate 6)
===================================================
This is the first Release Candidate of Redis 2.4, in our experience the
server is very stable, however it is always better to closely monitor your
server if you switch to 2.4 RC1. Also make sure to perform a backup of your
old data set before switching from 2.2 to 2.4.
* [BUGFIX] Fixed a bug with the automatic AOF rewrite causing continuous
rewrites for AOF files bigger than 4 GB.
* New maxmemory tests.
What's new in Redis 2.3.8 (2.4 Release Candidate 5)
===================================================
Compared to Redis 2.3.7 (RC4) there are the following changes:
* [BUGFIX] HDEL: Abort deleting fields when hash is removed.
* [BUGFIX] Fix adding bulk reply when getcwd fails.
WHAT'S NEW IN REDIS 2.4 compared to the 2.2 version?
====================================================
README
+29 -9
View File
@@ -11,11 +11,6 @@ It is as simple as:
% make
Redis is just a single binary, but if you want to install it you can use
the "make install" target that will copy the binary in /usr/local/bin
for default. You can also use "make PREFIX=/some/other/directory install"
if you wish to use a different destination.
You can run a 32 bit Redis binary using:
% make 32bit
@@ -36,13 +31,13 @@ glibc malloc() has memory fragmentation problems.
To force a libc malloc() build use:
make FORCE_LIBC_MALLOC=yes
% make FORCE_LIBC_MALLOC=yes
In all the other non Linux systems the libc malloc() is used by default.
On Mac OS X you can force a jemalloc based build using the following:
make USE_JEMALLOC=yes
% make USE_JEMALLOC=yes
Verbose build
-------------
@@ -50,7 +45,7 @@ Verbose build
Redis will build with a user friendly colorized output by default.
If you want to see a more verbose output use the following:
make V=1
% make V=1
Running Redis
-------------
@@ -90,5 +85,30 @@ You can find the list of all the available commands here:
http://redis.io/commands
Enjoy!
Installing Redis
-----------------
In order to install Redis binaries into /usr/local/bin just use:
% make install
You can use "make PREFIX=/some/other/directory install" if you wish to use a
different destination.
Make install will just install binaries in your system, but will not configure
init scripts and configuration files in the appropriate place. This is not
needed if you want just to play a bit with Redis, but if you are installing
it the proper way for a production system, we have a script doing this
for Ubuntu and Debian systems:
% cd utils
% ./install_server
The script will ask you a few questions and will setup everything you need
to run Redis properly as a background daemon that will start again on
system reboots.
You'll be able to stop and start Redis using the script named
/etc/init.d/redis_<portnumber>, for instance /etc/init.d/redis_6379.
Enjoy!
design-documents/REDIS-CLUSTER
-214
View File
@@ -1,214 +0,0 @@
Redis Cluster Design Proposal (work in progress)
28 Nov 2010: Ver 1.0 - initial version
22 APr 2010: Ver 1.1 - more details and rationales
Overview
========
Redis is a fast key-value store supporting complex aggregate data types as
values. For instance keys can be bound to lists with many elements, sets,
sub-dictionaries (hashes) and so forth.
While Redis is very fast, currently it lacks scalability in the form of ability
to transparently run across different nodes. This is desirable mainly for the
following three rasons:
A) Fault tolerance. Some node may go off line without affecting the operations.
B) Holding bigger datasets without using a single box with a lot of RAM.
C) Scaling writes.
Since a single Redis instance supports 140,000 operations per second in a good
Linux box costing less than $1000, the need for Redis Cluster arises more
from "A" and "B". Scaling writes can also be useful in very high load
environments. Scaling reads is already easily accomplished using Redis built-in
replication.
Design goals
============
Designing a DHT in 2010 is hard as there is too much bias towards good designs
that are already well tested in practice, like the Amazon Dynamo design.
Still a Dynamo alike DHT may not be the best fit for Redis.
Redis is very simple and fast at its core, so Redis cluster should try to
follow the same guidelines. The first problem with a Dynamo-alike DHT is that
Redis supports complex data types. Merging complex values like lsits, where
in the case of a netsplit may diverge in very complex ways, is not going to
be easy. The "most recent data" wins is not applicable and all the resolution
business should be in the application.
Even a simple application can end up with complex schema of keys and complex
values. Writing code in order to resolve conflicts is not going to be
programmer friendly.
So the author of this document claims that Redis does not need to resist to
netsplits, but it is enough to resist to M-1 nodes going offline, where
M is the number of nodes storing every key-value pair.
For instance in a three nodes cluster I may configure the cluster in order to
store every key into two instances (M=2). Such a cluster can resist to a single
node going offline without interruption of the service.
When more than M-1 nodes are off line the cluster should detect such a condition
and refusing any further query. The system administrator should check why
M-1 nodes are offline and bring them back again if possible.
Once resisting to big net splits is no longer a requirement as there is no
conflict resolution stage, since at least an original node responsible of
holding every possible key must be online for the cluster to work, there is
also no need for a design where every node can act as an independent entity
receiving queries and forwarding this queries to other nodes as needed.
Instead a more decoupled approach can be used, in the form of a Redis Proxy
node (or multiple Proxy nodes) that is contacted by clients, and
is responsible of forwarding queries and replies back and forth from data nodes.
Data nodes can be just vanilla redis-server instances.
Network layout
==============
- One ore more Data Nodes. Every node is identified by ip:port.
- A single Configuration Node.
- One more more Proxy Nodes (redis-cluster nodes).
- A single Handling Node.
Data Nodes and the Configuration Node are just vanilla redis-server instances.
Configuration Node
==================
- Contains information about all the Data nodes in the cluster.
- Contains information about all the Proxy nodes in the cluster.
- Contains information about what Data Node holds a given sub-space of keys.
The keyspace is divided into 1024 different "hashing slots".
(1024 is just an example, this value should be configurable)
Given a key perform SHA1(key) and use the last 10 bits of the result to get a 10 bit number representing the "key slot" (from 0 to 1023).
The Configuration node maps every slot of the keyspace to M different Data Nodes (every key is stored into M nodes, configurable).
The Configuration node can be modified by a single client at a time. Locking is performed using SETNX.
The Configuration node should be replicated as there is a single configuration node for the whole network. It is the only single point of failure of the system.
When a Configuration node fails the cluster does not stop operating, but is not
able to recover if there is some exceptional condition to handle, like a Data
Node going off line or the addition of a new Data Node to the cluster.
The Configuration node is a standard Redis server, like every other Data node.
Data Nodes
==========
Data nodes just hold data, and are normal Redis processes. There is no configuration stored on nodes, nor the nodes are "active" in the cluster, they just receive normal Redis commands.
Proxy Nodes
===========
Proxy nodes get requests from clients and route this requests to the right Redis nodes.
Proxy nodes take persistent connections to all the Data Nodes and the
Configuration Node. This connections are keep alive with PING requests from time
to time if there is no traffic. This way Proxy Nodes can understand asap if
there is a problem in some Data Node or in the Configuration Node.
When a Proxy Node is started it needs to know the Configuration node address in order to load the infomration about the Data nodes and the mapping between the key space and the nodes.
On startup a Proxy Node will also register itself in the Configuration node, and will make sure to refresh it's configuration every N seconds (via an EXPIREing key) so that it's possible to detect when a Proxy node fails.
Clients can submit queries to any Proxy Node, so well designed clients may ask
at startup the list of Proxy Nodes querying the Configuration Node. Then if
a query fails against a given Proxy Node it can be retried against the next.
The Proxy Node is also in charge of signaling failing Data nodes to the Configuration node, so that the Handling Node can take appropriate actions.
When a new Data node joins or leaves the cluster, and in general when the cluster configuration changes, all the Proxy nodes will receive a notification and will reload the configuration from the Configuration node.
Proxy Nodes - how queries are submited
======================================
This is how a query is processed:
1) A client sends a query to a Proxy Node, using the Redis protocol like if it was a plain Redis Node.
2) The Proxy Node inspects the command arguments to detect the key. The key is hashed. The Proxy Node has the table mapping a given key to M nodes, and persistent connections to all the nodes.
At this point the process is different in case of read or write queries:
WRITE QUERY:
3a) The Proxy Node forwards the query to M Data Nodes at the same time, waiting for replies.
3b) Once all the replies are received the Proxy Node checks that the replies are consistent. For instance all the M nodes need to reply with OK and so forth. If the query fails in a subset of nodes but succeeds in other nodes, the failing nodes are considered unreliable and are put off line notifying the configuration node.
3c) The reply is transfered back to the client.
READ QUERY:
3d) The Proxy Node forwards the query to a single random client, passing the reply back to the client.
Handling Node
=============
The handling node is a special Redis client with the following role:
- Handles the cluster configuration stored in the Config node.
- Is in charge for adding and removing nodes dynamically from the net.
- Relocates keys on nodes additions / removal.
- Signal a configuration change to Proxy nodes.
More details on hashing slots
============================
The Configuration node holds 1024 keys in the following form:
hashingslot:0
hashingslot:1
...
hashingslot:1023
Every hashing slot is actually a Redis list, containing a single or more ip:port pairs. For instance:
hashingslot:10 => 192.168.1.19:6379, 192.168.1.200:6379
This mean that keys hashing to slot 10 will be saved in the two Data nodes 192.168.1.19:6379 and 192.168.1.200:6379.
When a client performs a read operation (via a proxy node), the proxy will contact a random Data node among the data nodes in charge for the given slot.
For instance a client can ask for the following operation to a given Proxy node:
GET mykey
"mykey" hashes to (for instance) slot 10, so the Proxy will forward the request to either Data node 192.168.1.19:6379 or 192.168.1.200:6379, and then forward back the reply to the client.
When a write operation is performed, it is forwarded to both the Data nodes in the example (and in general to all the data nodes).
Adding or removing a node
=========================
When a Data node is added to the cluster, it is added via an LPUSH operation into a Redis list representing a queue of Data nodes that are ready to enter the cluster. This list is hold by the Configuration node of course, and can be added manually or via a configuration utility.
LPUSH newnodes 192.168.1.55:6379
The Handling node will check from time to time for this new elements in the "newode" list. If there are new nodes pending to enter the cluster, they are processed one after the other in this way:
For instance let's assume there are already two Data nodes in the cluster:
192.168.1.1:6379
192.168.1.2:6379
We add a new node 192.168.1.3:6379 via the LPUSH operation.
We can imagine that the 1024 hash slots are assigned equally among the two inital nodes. In order to add the new (third) node what we have to do is to move incrementally 341 slots form the two old servers to the new one.
For now we can think that every hash slot is only stored in a single server, to generalize the idea later.
In order to simplify the implementation every slot can be moved from one Data node to another one in a blocking way, that is, read operations will continue to all the 1024 slots, but a single slot at a time will delay write operations until the moving from one Data node to another is completed.
In order to do so the Handler node, before to move a given node, marks it as "write-locked" in the Configuration server, than asks all the Proxy nodes to refresh the configuration.
Then the slot is moved (1/1024 of all the keys). The Configuration server is modified to reflect the new hashing slots configuration, the slot is unlocked, the Proxy nodes notified.
Implementation details
======================
To run the Handling node and the Configuration node in the same physical computer is probably a good idea.
design-documents/REDIS-CLUSTER-2
-343
View File
@@ -1,343 +0,0 @@
Redis Cluster - Alternative 1
28 Apr 2010: Ver 1.0 - initial version
Overview
========
The motivations and design goals of Redis Cluster are already outlined in the
first design document of Redis Cluster. This document is just an attempt to
provide a completely alternative approach in order to explore more ideas.
In this document the alternative explored is a cluster where communication is
performed directly from client to the target node, without intermediate layer.
The intermediate layer can be used, in the form of a proxy, in order to provide
the same functionality to clients not able to directly use the cluster protocol.
So in a first stage clients can use a proxy to implement the hash ring, but
later this clients can switch to a native implementation, following a
specification that the Redis project will provide.
In this new design fault tolerance is achieved by replicating M-1 times every
data node instead of storing the same key M times across nodes.
From the point of view of CAP our biggest sacrifice is about "P", that is
resistance to partitioning. Only M-1 nodes can go down for the cluster still
be functional. Also when possible "A" is somewhat sacrificed for "L", that
is, Latency. Not really in the CAP equation but a very important parameter.
Network layout
==============
In this alternative design the network layout is simple as there are only
clients talking directly to N data nodes. So we can imagine to have:
- K Redis clients, directly talking to the data nodes.
- N Redis data nodes, that are, normal Redis instances.
Data nodes are replicate M-1 times (so there are a total of M copies for
every node). If M is one, the system is not fault tolerant. If M is 2 one
data node can go off line without affecting the operations. And so forth.
Hash slots
==========
The key space is divided into 1024 slots.
Given a key, the SHA1 function is applied to it.
The first 10 bytes of the SHA1 digest are interpreted as an unsigned integer
from 0 to 1023. This is the hash slot of the key.
Data nodes
==========
Data nodes are normal Redis instances, but a few additional commands are
provided.
HASHRING ADD ... list of hash slots ...
HASHRING DEL ... list of hash slots ...
HASHRING REHASHING slot
HASHRING SLOTS => returns the list of configured slots
HSAHRING KEYS ... list of hash slots ...
By default Redis instances are configured to accept operations about all
the hash slots. With this commands it's possible to configure a Redis instance
to accept only a subset of the key space.
If an operation is performed against a key hashing to a slot that is not
configured to be accepted, the Redis instance will reply with:
"-ERR wrong hash slot"
More details on the HASHRING command and sub commands will be showed later
in this document.
Additionally three other commands are added:
DUMP key
RESTORE key <dump data>
MIGRATE key host port
DUMP is used to output a very compact binary representation of the data stored at key.
RESTORE re-creates a value (storing it at key) starting from the output produced by DUMP.
MIGRATE is like a server-side DUMP+RESTORE command. This atomic command moves one key from the connected instance to another instance, returning the status code of the operation (+OK or an error).
The protocol described in this draft only uses the MIGRATE command, but this in turn will use RESTORE internally when connecting to another server, and DUMP is provided for symmetry.
Querying the cluster
====================
1) Reading the cluster config
-----------------------------
Clients of the cluster are required to have the cluster configuration loaded
into memory. The cluster configuration is the sum of the following info:
- Number of data nodes in the cluster, for instance, 10
- A map between hash slots and nodes, so for instnace:
hash slot 1 -> node 0
hash slot 2 -> node 5
hash slot 3 -> node 3
... and so forth ...
- Physical address of nodes, and their replicas.
node 0 addr -> 192.168.1.100
node 0 replicas -> 192.168.1.101, 192.168.1.105
- Configuration version: the SHA1 of the whole configuration
The configuration is stored in every single data node of the cluster.
A client without the configuration in memory is require, as a first step, to
read the config. In order to do so the client requires to have a list of IPs
that are with good probability data nodes of the cluster.
The client will try to get the config from all this nodes. If no node is found
responding, an error is reported to the user.
2) Caching and refreshing the configuration
-------------------------------------------
A node is allowed to cache the configuration in memory or in a different way
(for instance storing the configuration into a file), but every client is
required to check if the configuration changed at max every 10 seconds, asking
for the configuration version key with a single GET call, and checking if the
configuration version matches the one loaded in memory.
Also a client is required to refresh the configuration every time a node
replies with:
"-ERR wrong hash slot"
As this means that hash slots were reassigned in some way.
Checking the configuration every 10 seconds is not required in theory but is
a good protection against errors and failures that may happen in real world
environments. It is also very cheap to perform, as a GET operation from time
to time is going to have no impact in the overall performance.
3) Read query
-------------
To perform a read query the client hashes the key argument from the command
(in the intiial version of Redis Cluster only single-key commands are
allowed). Using the in memory configuration it maps the hash key to the
node ID.
If the client is configured to support read-after-write consistency, then
the "master" node for this hash slot is queried.
Otherwise the client picks a random node from the master and the replicas
available.
4) Write query
--------------
A write query is exactly like a read query, with the difference that the
write always targets the master node, instead of the replicas.
Creating a cluster
==================
In order to create a new cluster, the redis-cluster command line utility is
used. It gets a list of available nodes and replicas, in order to write the
initial configuration in all the nodes.
At this point the cluster is usable by clients.
Adding nodes to the cluster
===========================
The command line utility redis-cluster is used in order to add a node to the
cluster:
1) The cluster configuration is loaded.
2) A fair number of hash slots are assigned to the new data node.
3) Hash slots moved to the new node are marked as "REHASHING" in the old
nodes, using the HASHRING command:
HASHRING SETREHASHING 1 192.168.1.103 6380
The above command set the hash slot "1" in rehashing state, with the
"forwarding address" to 192.168.1.103:6380. As a result if this node receives
a query about a key hashing to hash slot 1, that *is not present* in the
current data set, it replies with:
"-MIGRATED 192.168.1.103:6380"
The client can then reissue the query against the new node.
Instead even if the hash slot is marked as rehashing but the requested key
is still there, the query is processed. This allows for non blocking
rehashing.
Note that no additional memory is used by Redis in order to provide such a
feature.
4) While the Hash slot is marked as "REHASHING", redis-cluster asks this node
the list of all the keys matching the specified hash slot. Then all the keys
are moved to the new node using the MIGRATE command.
5) Once all the keys are migrated, the hash slot is deleted from the old
node configuration with "HASHRING DEL 1". And the configuration is update.
Using this algorithm all the hash slots are migrated one after the other to the new node. In practical implementation before to start the migration the
redis-cluster utility should write a log into the configuration so that
in case of crash or any other problem the utility is able to recover from
were it left.
Fault tolerance
===============
Fault tolerance is reached replicating every data node M-1 times, so that we
have one master and M-1 replicas for a total of M nodes holding the same
hash slots. Up to M-1 nodes can go down without affecting the cluster.
The tricky part about fault tolerance is detecting when a node is failing and
signaling it to all the other clients.
When a master node is failing in a permanent way, promoting the first slave
is easy:
1) At some point a client will notice there are problems accessing a given node. It will try to refresh the config, but will notice that the config is already up to date.
2) In order to make sure the problem is not about the client connectivity itself, it will try to reach other nodes as well. If more than M-1 nodes appear to be down, it's either a client networking problem or alternatively the cluster can't be fixed as too many nodes are down anyway. So no action is taken, but an error is reported.
3) If instead only 1 or at max M-1 nodes appear to be down, the client promotes a slave as master and writes the new configuration to all the data nodes.
All the other clients will see the data node not working, and as a first step will try to refresh the configuration. They will successful refresh the configuration and the cluster will work again.
Every time a slave is promoted, the information is written in a log that is actually a Redis list, in all the data nodes, so that system administration tools can detect what happened in order to send notifications to the admin.
Intermittent problems
---------------------
In the above scenario a master was failing in a permanent way. Now instead
let's think to a case where a network cable is not working well so a node
appears to be a few seconds up and a few seconds down.
When this happens recovering can be much harder, as a client may notice the
problem and will promote a slave to master as a result, but then the host
will be up again and the other clients will not see the problem, writing to
the old master for at max 10 seconds (after 10 seconds all the clients are
required to perform a few GETs to check the configuration version of the
cluster and update if needed).
One way to fix this problem is to delegate the fail over mechanism to a
failover agent. When clients notice problems will not take any active action
but will just log the problem into a redis list in all the reachable nodes,
wait, check for configuration change, and retry.
The failover agent constantly monitor this logs: if some client is reporting
a failing node, it can take appropriate actions, checking if the failure is
permanent or not. If it's not he can send a SHUTDOWN command to the failing
master if possible. The failover agent can also consider better the problem
checking if the failing mode is advertised by all the clients or just a single
one, and can check itself if there is a real problem before to proceed with
the fail over.
Redis proxy
===========
In order to make the switch to the clustered version of Redis simpler, and
because the client-side protocol is non trivial to implement compared to the
usual Redis client lib protocol (where a minimal lib can be as small as
100 lines of code), a proxy will be provided to implement the cluster protocol
as a proxy.
Every client will talk to a redis-proxy node that is responsible of using
the new protocol and forwarding back the replies.
In the long run the aim is to switch all the major client libraries to the
new protocol in a native way.
Supported commands
==================
Because with this design we talk directly to data nodes and there is a single
"master" version of every value (that's the big gain dropping "P" from CAP!)
almost all the redis commands can be supported by the clustered version
including MULTI/EXEC and multi key commands as long as all the keys will hash
to the same hash slot. In order to guarantee this, key tags can be used,
where when a specific pattern is present in the key name, only that part is
hashed in order to obtain the hash index.
Random remarks
==============
- It's still not clear how to perform an atomic election of a slave to master.
- In normal conditions (all the nodes working) this new design is just
K clients talking to N nodes without intermediate layers, no routes:
this means it is horizontally scalable with O(1) lookups.
- The cluster should optionally be able to work with manual fail over
for environments where it's desirable to do so. For instance it's possible
to setup periodic checks on all the nodes, and switch IPs when needed
or other advanced configurations that can not be the default as they
are too environment dependent.
A few ideas about client-side slave election
============================================
Detecting failures in a collaborative way
-----------------------------------------
In order to take the node failure detection and slave election a distributed
effort, without any "control program" that is in some way a single point
of failure (the cluster will not stop when it stops, but errors are not
corrected without it running), it's possible to use a few consensus-alike
algorithms.
For instance all the nodes may take a list of errors detected by clients.
If Client-1 detects some failure accessing Node-3, for instance a connection
refused error or a timeout, it logs what happened with LPUSH commands against
all the other nodes. This "error messages" will have a timestamp and the Node
id. Something like:
LPUSH __cluster__:errors 3:1272545939
So if the error is reported many times in a small amount of time, at some
point a client can have enough hints about the need of performing a
slave election.
Atomic slave election
---------------------
In order to avoid races when electing a slave to master (that is in order to
avoid that some client can still contact the old master for that node in
the 10 seconds timeframe), the client performing the election may write
some hint in the configuration, change the configuration SHA1 accordingly and
wait for more than 10 seconds, in order to be sure all the clients will
refresh the configuration before a new access.
The config hint may be something like:
"we are switching to a new master, that is x.y.z.k:port, in a few seconds"
When a client updates the config and finds such a flag set, it starts to
continuously refresh the config until a change is noticed (this will take
at max 10-15 seconds).
The client performing the election will wait that famous 10 seconds time frame
and finally will update the config in a definitive way setting the new
slave as mater. All the clients at this point are guaranteed to have the new
config either because they refreshed or because in the next query their config
is already expired and they'll update the configuration.
EOF
redis.conf
+1
View File
@@ -34,6 +34,7 @@ port 6379
# on a unix socket when not specified.
#
# unixsocket /tmp/redis.sock
# unixsocketperm 755
# Close the connection after a client is idle for N seconds (0 to disable)
timeout 300
runtest
Executable
+9
View File
@@ -0,0 +1,9 @@
#!/bin/bash
TCL=tclsh8.5
which $TCL
if [ "$?" != "0" ]
then
echo "You need '$TCL' in order to run the Redis test"
exit 1
fi
$TCL tests/test_helper.tcl $*
src/Makefile
+7 -4
View File
@@ -58,7 +58,7 @@ PREFIX= /usr/local
INSTALL_BIN= $(PREFIX)/bin
INSTALL= cp -p
OBJ = adlist.o ae.o anet.o dict.o redis.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o vm.o pubsub.o multi.o debug.o sort.o intset.o syncio.o slowlog.o
OBJ = adlist.o ae.o anet.o dict.o redis.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o vm.o pubsub.o multi.o debug.o sort.o intset.o syncio.o slowlog.o bio.o
BENCHOBJ = ae.o anet.o redis-benchmark.o sds.o adlist.o zmalloc.o
CLIOBJ = anet.o sds.o adlist.o redis-cli.o zmalloc.o release.o
CHECKDUMPOBJ = redis-check-dump.o lzf_c.o lzf_d.o
@@ -84,6 +84,8 @@ ae_select.o: ae_select.c
anet.o: anet.c fmacros.h anet.h
aof.o: aof.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
bio.o: bio.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h bio.h
config.o: config.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
db.o: db.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
@@ -113,7 +115,8 @@ redis-check-dump.o: redis-check-dump.c lzf.h
redis-cli.o: redis-cli.c fmacros.h version.h ../deps/hiredis/hiredis.h \
sds.h zmalloc.h ../deps/linenoise/linenoise.h help.h
redis.o: redis.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h slowlog.h
zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h slowlog.h \
bio.h
release.o: release.c release.h
replication.o: replication.c redis.h fmacros.h config.h ae.h sds.h dict.h \
adlist.h zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
@@ -195,8 +198,8 @@ clean:
dep:
$(CC) -MM *.c -I ../deps/hiredis -I ../deps/linenoise
test: redis-server
(cd ..; tclsh8.5 tests/test_helper.tcl --tags "${TAGS}")
test: redis-server redis-check-aof
@(cd ..; ./runtest)
bench:
./redis-benchmark
src/anet.c
+4 -1
View File
@@ -32,6 +32,7 @@
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <sys/un.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
@@ -291,7 +292,7 @@ int anetTcpServer(char *err, int port, char *bindaddr)
return s;
}
int anetUnixServer(char *err, char *path)
int anetUnixServer(char *err, char *path, mode_t perm)
{
int s;
struct sockaddr_un sa;
@@ -304,6 +305,8 @@ int anetUnixServer(char *err, char *path)
strncpy(sa.sun_path,path,sizeof(sa.sun_path)-1);
if (anetListen(err,s,(struct sockaddr*)&sa,sizeof(sa)) == ANET_ERR)
return ANET_ERR;
if (perm)
chmod(sa.sun_path, perm);
return s;
}
src/anet.h
+1 -1
View File
@@ -46,7 +46,7 @@ int anetUnixNonBlockConnect(char *err, char *path);
int anetRead(int fd, char *buf, int count);
int anetResolve(char *err, char *host, char *ipbuf);
int anetTcpServer(char *err, int port, char *bindaddr);
int anetUnixServer(char *err, char *path);
int anetUnixServer(char *err, char *path, mode_t perm);
int anetTcpAccept(char *err, int serversock, char *ip, int *port);
int anetUnixAccept(char *err, int serversock);
int anetWrite(int fd, char *buf, int count);
src/aof.c
+199 -61
View File
@@ -1,4 +1,5 @@
#include "redis.h"
#include "bio.h"
#include <signal.h>
#include <fcntl.h>
@@ -10,10 +11,14 @@
void aofUpdateCurrentSize(void);
void aof_background_fsync(int fd) {
bioCreateBackgroundJob(REDIS_BIO_AOF_FSYNC,(void*)(long)fd,NULL,NULL);
}
/* Called when the user switches from "appendonly yes" to "appendonly no"
* at runtime using the CONFIG command. */
void stopAppendOnly(void) {
flushAppendOnlyFile();
flushAppendOnlyFile(1);
aof_fsync(server.appendfd);
close(server.appendfd);
@@ -58,63 +63,121 @@ int startAppendOnly(void) {
* and the only way the client socket can get a write is entering when the
* the event loop, we accumulate all the AOF writes in a memory
* buffer and write it on disk using this function just before entering
* the event loop again. */
void flushAppendOnlyFile(void) {
time_t now;
* the event loop again.
*
* About the 'force' argument:
*
* When the fsync policy is set to 'everysec' we may delay the flush if there
* is still an fsync() going on in the background thread, since for instance
* on Linux write(2) will be blocked by the background fsync anyway.
* When this happens we remember that there is some aof buffer to be
* flushed ASAP, and will try to do that in the serverCron() function.
*
* However if force is set to 1 we'll write regardless of the background
* fsync. */
void flushAppendOnlyFile(int force) {
ssize_t nwritten;
int sync_in_progress = 0;
if (sdslen(server.aofbuf) == 0) return;
if (server.appendfsync == APPENDFSYNC_EVERYSEC)
sync_in_progress = bioPendingJobsOfType(REDIS_BIO_AOF_FSYNC) != 0;
if (server.appendfsync == APPENDFSYNC_EVERYSEC && !force) {
/* With this append fsync policy we do background fsyncing.
* If the fsync is still in progress we can try to delay
* the write for a couple of seconds. */
if (sync_in_progress) {
if (server.aof_flush_postponed_start == 0) {
/* No previous write postponinig, remember that we are
* postponing the flush and return. */
server.aof_flush_postponed_start = server.unixtime;
return;
} else if (server.unixtime - server.aof_flush_postponed_start < 2) {
/* We were already waiting for fsync to finish, but for less
* than two seconds this is still ok. Postpone again. */
return;
}
/* Otherwise fall trough, and go write since we can't wait
* over two seconds. */
redisLog(REDIS_NOTICE,"Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down Redis.");
}
}
/* If you are following this code path, then we are going to write so
* set reset the postponed flush sentinel to zero. */
server.aof_flush_postponed_start = 0;
/* We want to perform a single write. This should be guaranteed atomic
* at least if the filesystem we are writing is a real physical one.
* While this will save us against the server being killed I don't think
* there is much to do about the whole server stopping for power problems
* or alike */
nwritten = write(server.appendfd,server.aofbuf,sdslen(server.aofbuf));
if (nwritten != (signed)sdslen(server.aofbuf)) {
nwritten = write(server.appendfd,server.aofbuf,sdslen(server.aofbuf));
if (nwritten != (signed)sdslen(server.aofbuf)) {
/* Ooops, we are in troubles. The best thing to do for now is
* aborting instead of giving the illusion that everything is
* working as expected. */
if (nwritten == -1) {
if (nwritten == -1) {
redisLog(REDIS_WARNING,"Exiting on error writing to the append-only file: %s",strerror(errno));
} else {
} else {
redisLog(REDIS_WARNING,"Exiting on short write while writing to the append-only file: %s",strerror(errno));
}
exit(1);
}
exit(1);
}
sdsfree(server.aofbuf);
server.aofbuf = sdsempty();
server.appendonly_current_size += nwritten;
/* Don't Fsync if no-appendfsync-on-rewrite is set to yes and we have
* childs performing heavy I/O on disk. */
/* Re-use AOF buffer when it is small enough. The maximum comes from the
* arena size of 4k minus some overhead (but is otherwise arbitrary). */
if ((sdslen(server.aofbuf)+sdsavail(server.aofbuf)) < 4000) {
sdsclear(server.aofbuf);
} else {
sdsfree(server.aofbuf);
server.aofbuf = sdsempty();
}
/* Don't fsync if no-appendfsync-on-rewrite is set to yes and there are
* children doing I/O in the background. */
if (server.no_appendfsync_on_rewrite &&
(server.bgrewritechildpid != -1 || server.bgsavechildpid != -1))
return;
/* Fsync if needed */
now = time(NULL);
if (server.appendfsync == APPENDFSYNC_ALWAYS ||
(server.appendfsync == APPENDFSYNC_EVERYSEC &&
now-server.lastfsync > 1))
{
/* Perform the fsync if needed. */
if (server.appendfsync == APPENDFSYNC_ALWAYS) {
/* aof_fsync is defined as fdatasync() for Linux in order to avoid
* flushing metadata. */
aof_fsync(server.appendfd); /* Let's try to get this data on the disk */
server.lastfsync = now;
server.lastfsync = server.unixtime;
} else if ((server.appendfsync == APPENDFSYNC_EVERYSEC &&
server.unixtime > server.lastfsync)) {
if (!sync_in_progress) aof_background_fsync(server.appendfd);
server.lastfsync = server.unixtime;
}
}
sds catAppendOnlyGenericCommand(sds buf, int argc, robj **argv) {
int j;
buf = sdscatprintf(buf,"*%d\r\n",argc);
sds catAppendOnlyGenericCommand(sds dst, int argc, robj **argv) {
char buf[32];
int len, j;
robj *o;
buf[0] = '*';
len = 1+ll2string(buf+1,sizeof(buf)-1,argc);
buf[len++] = '\r';
buf[len++] = '\n';
dst = sdscatlen(dst,buf,len);
for (j = 0; j < argc; j++) {
robj *o = getDecodedObject(argv[j]);
buf = sdscatprintf(buf,"$%lu\r\n",(unsigned long)sdslen(o->ptr));
buf = sdscatlen(buf,o->ptr,sdslen(o->ptr));
buf = sdscatlen(buf,"\r\n",2);
o = getDecodedObject(argv[j]);
buf[0] = '$';
len = 1+ll2string(buf+1,sizeof(buf)-1,sdslen(o->ptr));
buf[len++] = '\r';
buf[len++] = '\n';
dst = sdscatlen(dst,buf,len);
dst = sdscatlen(dst,o->ptr,sdslen(o->ptr));
dst = sdscatlen(dst,"\r\n",2);
decrRefCount(o);
}
return buf;
return dst;
}
sds catAppendOnlyExpireAtCommand(sds buf, robj *key, robj *seconds) {
@@ -264,6 +327,8 @@ int loadAppendOnlyFile(char *filename) {
}
if (buf[0] != '*') goto fmterr;
argc = atoi(buf+1);
if (argc < 1) goto fmterr;
argv = zmalloc(sizeof(robj*)*argc);
for (j = 0; j < argc; j++) {
if (fgets(buf,sizeof(buf),fp) == NULL) goto readerr;
@@ -626,6 +691,7 @@ int rewriteAppendOnlyFileBackground(void) {
}
redisLog(REDIS_NOTICE,
"Background append only file rewriting started by pid %d",childpid);
server.aofrewrite_scheduled = 0;
server.bgrewritechildpid = childpid;
updateDictResizePolicy();
/* We set appendseldb to -1 in order to force the next call to the
@@ -680,56 +746,128 @@ void backgroundRewriteDoneHandler(int statloc) {
int bysignal = WIFSIGNALED(statloc);
if (!bysignal && exitcode == 0) {
int fd;
int newfd, oldfd;
int nwritten;
char tmpfile[256];
long long now = ustime();
redisLog(REDIS_NOTICE,
"Background append only file rewriting terminated with success");
/* Now it's time to flush the differences accumulated by the parent */
snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", (int) server.bgrewritechildpid);
fd = open(tmpfile,O_WRONLY|O_APPEND);
if (fd == -1) {
redisLog(REDIS_WARNING, "Not able to open the temp append only file produced by the child: %s", strerror(errno));
"Background AOF rewrite terminated with success");
/* Flush the differences accumulated by the parent to the
* rewritten AOF. */
snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof",
(int)server.bgrewritechildpid);
newfd = open(tmpfile,O_WRONLY|O_APPEND);
if (newfd == -1) {
redisLog(REDIS_WARNING,
"Unable to open the temporary AOF produced by the child: %s", strerror(errno));
goto cleanup;
}
/* Flush our data... */
if (write(fd,server.bgrewritebuf,sdslen(server.bgrewritebuf)) !=
(signed) sdslen(server.bgrewritebuf)) {
redisLog(REDIS_WARNING, "Error or short write trying to flush the parent diff of the append log file in the child temp file: %s", strerror(errno));
close(fd);
nwritten = write(newfd,server.bgrewritebuf,sdslen(server.bgrewritebuf));
if (nwritten != (signed)sdslen(server.bgrewritebuf)) {
if (nwritten == -1) {
redisLog(REDIS_WARNING,
"Error trying to flush the parent diff to the rewritten AOF: %s", strerror(errno));
} else {
redisLog(REDIS_WARNING,
"Short write trying to flush the parent diff to the rewritten AOF: %s", strerror(errno));
}
close(newfd);
goto cleanup;
}
redisLog(REDIS_NOTICE,"Parent diff flushed into the new append log file with success (%lu bytes)",sdslen(server.bgrewritebuf));
/* Now our work is to rename the temp file into the stable file. And
* switch the file descriptor used by the server for append only. */
redisLog(REDIS_NOTICE,
"Parent diff successfully flushed to the rewritten AOF (%lu bytes)", nwritten);
/* The only remaining thing to do is to rename the temporary file to
* the configured file and switch the file descriptor used to do AOF
* writes. We don't want close(2) or rename(2) calls to block the
* server on old file deletion.
*
* There are two possible scenarios:
*
* 1) AOF is DISABLED and this was a one time rewrite. The temporary
* file will be renamed to the configured file. When this file already
* exists, it will be unlinked, which may block the server.
*
* 2) AOF is ENABLED and the rewritten AOF will immediately start
* receiving writes. After the temporary file is renamed to the
* configured file, the original AOF file descriptor will be closed.
* Since this will be the last reference to that file, closing it
* causes the underlying file to be unlinked, which may block the
* server.
*
* To mitigate the blocking effect of the unlink operation (either
* caused by rename(2) in scenario 1, or by close(2) in scenario 2), we
* use a background thread to take care of this. First, we
* make scenario 1 identical to scenario 2 by opening the target file
* when it exists. The unlink operation after the rename(2) will then
* be executed upon calling close(2) for its descriptor. Everything to
* guarantee atomicity for this switch has already happened by then, so
* we don't care what the outcome or duration of that close operation
* is, as long as the file descriptor is released again. */
if (server.appendfd == -1) {
/* AOF disabled */
/* Don't care if this fails: oldfd will be -1 and we handle that.
* One notable case of -1 return is if the old file does
* not exist. */
oldfd = open(server.appendfilename,O_RDONLY|O_NONBLOCK);
} else {
/* AOF enabled */
oldfd = -1; /* We'll set this to the current AOF filedes later. */
}
/* Rename the temporary file. This will not unlink the target file if
* it exists, because we reference it with "oldfd". */
if (rename(tmpfile,server.appendfilename) == -1) {
redisLog(REDIS_WARNING,"Can't rename the temp append only file into the stable one: %s", strerror(errno));
close(fd);
redisLog(REDIS_WARNING,
"Error trying to rename the temporary AOF: %s", strerror(errno));
close(newfd);
if (oldfd != -1) close(oldfd);
goto cleanup;
}
/* Mission completed... almost */
redisLog(REDIS_NOTICE,"Append only file successfully rewritten.");
if (server.appendfd != -1) {
/* If append only is actually enabled... */
close(server.appendfd);
server.appendfd = fd;
if (server.appendfsync != APPENDFSYNC_NO) aof_fsync(fd);
server.appendseldb = -1; /* Make sure it will issue SELECT */
redisLog(REDIS_NOTICE,"The new append only file was selected for future appends.");
if (server.appendfd == -1) {
/* AOF disabled, we don't need to set the AOF file descriptor
* to this new file, so we can close it. */
close(newfd);
} else {
/* AOF enabled, replace the old fd with the new one. */
oldfd = server.appendfd;
server.appendfd = newfd;
if (server.appendfsync == APPENDFSYNC_ALWAYS)
aof_fsync(newfd);
else if (server.appendfsync == APPENDFSYNC_EVERYSEC)
aof_background_fsync(newfd);
server.appendseldb = -1; /* Make sure SELECT is re-issued */
aofUpdateCurrentSize();
server.auto_aofrewrite_base_size = server.appendonly_current_size;
} else {
/* If append only is disabled we just generate a dump in this
* format. Why not? */
close(fd);
/* Clear regular AOF buffer since its contents was just written to
* the new AOF from the background rewrite buffer. */
sdsfree(server.aofbuf);
server.aofbuf = sdsempty();
}
redisLog(REDIS_NOTICE, "Background AOF rewrite successful");
/* Asynchronously close the overwritten AOF. */
if (oldfd != -1) bioCreateBackgroundJob(REDIS_BIO_CLOSE_FILE,(void*)(long)oldfd,NULL,NULL);
redisLog(REDIS_VERBOSE,
"Background AOF rewrite signal handler took %lldus", ustime()-now);
} else if (!bysignal && exitcode != 0) {
redisLog(REDIS_WARNING, "Background append only file rewriting error");
redisLog(REDIS_WARNING,
"Background AOF rewrite terminated with error");
} else {
redisLog(REDIS_WARNING,
"Background append only file rewriting terminated by signal %d",
"Background AOF rewrite terminated by signal %d",
WTERMSIG(statloc));
}
cleanup:
sdsfree(server.bgrewritebuf);
server.bgrewritebuf = sdsempty();
src/bio.c
+208
View File
@@ -0,0 +1,208 @@
/* Background I/O service for Redis.
*
* This file implements operations that we need to perform in the background.
* Currently there is only a single operation, that is a background close(2)
* system call. This is needed as when the process is the last owner of a
* reference to a file closing it means unlinking it, and the deletion of the
* file is slow, blocking the server.
*
* In the future we'll either continue implementing new things we need or
* we'll switch to libeio. However there are probably long term uses for this
* file as we may want to put here Redis specific background tasks (for instance
* it is not impossible that we'll need a non blocking FLUSHDB/FLUSHALL
* implementation).
*
* DESIGN
* ------
*
* The design is trivial, we have a structure representing a job to perform
* and a different thread and job queue for every job type.
* Every thread wait for new jobs in its queue, and process every job
* sequentially.
*
* Jobs of the same type are guaranteed to be processed from the least
* recently inserted to the most recently inserted (older jobs processed
* first).
*
* Currently there is no way for the creator of the job to be notified about
* the completion of the operation, this will only be added when/if needed.
*/
#include "redis.h"
#include "bio.h"
static pthread_mutex_t bio_mutex[REDIS_BIO_NUM_OPS];
static pthread_cond_t bio_condvar[REDIS_BIO_NUM_OPS];
static list *bio_jobs[REDIS_BIO_NUM_OPS];
/* The following array is used to hold the number of pending jobs for every
* OP type. This allows us to export the bioPendingJobsOfType() API that is
* useful when the main thread wants to perform some operation that may involve
* objects shared with the background thread. The main thread will just wait
* that there are no longer jobs of this type to be executed before performing
* the sensible operation. This data is also useful for reporting. */
static unsigned long long bio_pending[REDIS_BIO_NUM_OPS];
/* This structure represents a background Job. It is only used locally to this
* file as the API deos not expose the internals at all. */
struct bio_job {
time_t time; /* Time at which the job was created. */
/* Job specific arguments pointers. If we need to pass more than three
* arguments we can just pass a pointer to a structure or alike. */
void *arg1, *arg2, *arg3;
};
void *bioProcessBackgroundJobs(void *arg);
/* Make sure we have enough stack to perform all the things we do in the
* main thread. */
#define REDIS_THREAD_STACK_SIZE (1024*1024*4)
/* Initialize the background system, spawning the thread. */
void bioInit(void) {
pthread_attr_t attr;
pthread_t thread;
size_t stacksize;
int j;
/* Initialization of state vars and objects */
for (j = 0; j < REDIS_BIO_NUM_OPS; j++) {
pthread_mutex_init(&bio_mutex[j],NULL);
pthread_cond_init(&bio_condvar[j],NULL);
bio_jobs[j] = listCreate();
bio_pending[j] = 0;
}
/* Set the stack size as by default it may be small in some system */
pthread_attr_init(&attr);
pthread_attr_getstacksize(&attr,&stacksize);
if (!stacksize) stacksize = 1; /* The world is full of Solaris Fixes */
while (stacksize < REDIS_THREAD_STACK_SIZE) stacksize *= 2;
pthread_attr_setstacksize(&attr, stacksize);
/* Ready to spawn our threads. We use the single argument the thread
* function accepts in order to pass the job ID the thread is
* responsible of. */
for (j = 0; j < REDIS_BIO_NUM_OPS; j++) {
void *arg = (void*)(unsigned long) j;
if (pthread_create(&thread,&attr,bioProcessBackgroundJobs,arg) != 0) {
redisLog(REDIS_WARNING,"Fatal: Can't initialize Background Jobs.");
exit(1);
}
}
}
void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3) {
struct bio_job *job = zmalloc(sizeof(*job));
job->time = time(NULL);
job->arg1 = arg1;
job->arg2 = arg2;
job->arg3 = arg3;
pthread_mutex_lock(&bio_mutex[type]);
listAddNodeTail(bio_jobs[type],job);
bio_pending[type]++;
pthread_cond_signal(&bio_condvar[type]);
pthread_mutex_unlock(&bio_mutex[type]);
}
void *bioProcessBackgroundJobs(void *arg) {
struct bio_job *job;
unsigned long type = (unsigned long) arg;
pthread_detach(pthread_self());
pthread_mutex_lock(&bio_mutex[type]);
while(1) {
listNode *ln;
/* The loop always starts with the lock hold. */
if (listLength(bio_jobs[type]) == 0) {
pthread_cond_wait(&bio_condvar[type],&bio_mutex[type]);
continue;
}
/* Pop the job from the queue. */
ln = listFirst(bio_jobs[type]);
job = ln->value;
/* It is now possible to unlock the background system as we know have
* a stand alone job structure to process.*/
pthread_mutex_unlock(&bio_mutex[type]);
/* Process the job accordingly to its type. */
if (type == REDIS_BIO_CLOSE_FILE) {
close((long)job->arg1);
} else if (type == REDIS_BIO_AOF_FSYNC) {
aof_fsync((long)job->arg1);
} else {
redisPanic("Wrong job type in bioProcessBackgroundJobs().");
}
zfree(job);
/* Lock again before reiterating the loop, if there are no longer
* jobs to process we'll block again in pthread_cond_wait(). */
pthread_mutex_lock(&bio_mutex[type]);
listDelNode(bio_jobs[type],ln);
bio_pending[type]--;
}
}
/* Return the number of pending jobs of the specified type. */
unsigned long long bioPendingJobsOfType(int type) {
unsigned long long val;
pthread_mutex_lock(&bio_mutex[type]);
val = bio_pending[type];
pthread_mutex_unlock(&bio_mutex[type]);
return val;
}
#if 0 /* We don't use the following code for now, and bioWaitPendingJobsLE
probably needs a rewrite using conditional variables instead of the
current implementation. */
/* Wait until the number of pending jobs of the specified type are
* less or equal to the specified number.
*
* This function may block for long time, it should only be used to perform
* the following tasks:
*
* 1) To avoid that the main thread is pushing jobs of a given time so fast
* that the background thread can't process them at the same speed.
* So before creating a new job of a given type the main thread should
* call something like: bioWaitPendingJobsLE(job_type,10000);
* 2) In order to perform special operations that make it necessary to be sure
* no one is touching shared resourced in the background.
*/
void bioWaitPendingJobsLE(int type, unsigned long long num) {
unsigned long long iteration = 0;
/* We poll the jobs queue aggressively to start, and gradually relax
* the polling speed if it is going to take too much time. */
while(1) {
iteration++;
if (iteration > 1000 && iteration <= 10000) {
usleep(100);
} else if (iteration > 10000) {
usleep(1000);
}
if (bioPendingJobsOfType(type) <= num) break;
}
}
/* Return the older job of the specified type. */
time_t bioOlderJobOfType(int type) {
time_t time;
listNode *ln;
struct bio_job *job;
pthread_mutex_lock(&bio_mutex[type]);
ln = listFirst(bio_jobs[type]);
if (ln == NULL) {
pthread_mutex_unlock(&bio_mutex[type]);
return 0;
}
job = ln->value;
time = job->time;
pthread_mutex_unlock(&bio_mutex[type]);
return time;
}
#endif
src/bio.h
+11
View File
@@ -0,0 +1,11 @@
/* Exported API */
void bioInit(void);
void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3);
unsigned long long bioPendingJobsOfType(int type);
void bioWaitPendingJobsLE(int type, unsigned long long num);
time_t bioOlderJobOfType(int type);
/* Background job opcodes */
#define REDIS_BIO_CLOSE_FILE 0 /* Deferred close(2) syscall. */
#define REDIS_BIO_AOF_FSYNC 1 /* Deferred AOF fsync. */
#define REDIS_BIO_NUM_OPS 2
src/config.c
+36 -5
View File
@@ -74,6 +74,12 @@ void loadServerConfig(char *filename) {
server.bindaddr = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"unixsocket") && argc == 2) {
server.unixsocket = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"unixsocketperm") && argc == 2) {
errno = 0;
server.unixsocketperm = (mode_t)strtol(argv[1], NULL, 8);
if (errno || server.unixsocketperm > 0777) {
err = "Invalid socket file permissions"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"save") && argc == 3) {
int seconds = atoi(argv[1]);
int changes = atoi(argv[2]);
@@ -501,6 +507,18 @@ void configSetCommand(redisClient *c) {
} else if (!strcasecmp(c->argv[2]->ptr,"slowlog-max-len")) {
if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
server.slowlog_max_len = (unsigned)ll;
} else if (!strcasecmp(c->argv[2]->ptr,"loglevel")) {
if (!strcasecmp(o->ptr,"warning")) {
server.verbosity = REDIS_WARNING;
} else if (!strcasecmp(o->ptr,"notice")) {
server.verbosity = REDIS_NOTICE;
} else if (!strcasecmp(o->ptr,"verbose")) {
server.verbosity = REDIS_VERBOSE;
} else if (!strcasecmp(o->ptr,"debug")) {
server.verbosity = REDIS_DEBUG;
} else {
goto badfmt;
}
} else {
addReplyErrorFormat(c,"Unsupported CONFIG parameter: %s",
(char*)c->argv[2]->ptr);
@@ -526,12 +544,11 @@ void configGetCommand(redisClient *c) {
if (stringmatch(pattern,"dir",0)) {
char buf[1024];
addReplyBulkCString(c,"dir");
if (getcwd(buf,sizeof(buf)) == NULL) {
if (getcwd(buf,sizeof(buf)) == NULL)
buf[0] = '\0';
} else {
addReplyBulkCString(c,buf);
}
addReplyBulkCString(c,"dir");
addReplyBulkCString(c,buf);
matches++;
}
if (stringmatch(pattern,"dbfilename",0)) {
@@ -682,6 +699,20 @@ void configGetCommand(redisClient *c) {
addReplyBulkLongLong(c,server.slowlog_max_len);
matches++;
}
if (stringmatch(pattern,"loglevel",0)) {
char *s;
switch(server.verbosity) {
case REDIS_WARNING: s = "warning"; break;
case REDIS_VERBOSE: s = "verbose"; break;
case REDIS_NOTICE: s = "notice"; break;
case REDIS_DEBUG: s = "debug"; break;
default: s = "unknown"; break; /* too harmless to panic */
}
addReplyBulkCString(c,"loglevel");
addReplyBulkCString(c,s);
matches++;
}
setDeferredMultiBulkLength(c,replylen,matches*2);
}
src/db.c
+13 -4
View File
@@ -206,7 +206,13 @@ void flushallCommand(redisClient *c) {
kill(server.bgsavechildpid,SIGKILL);
rdbRemoveTempFile(server.bgsavechildpid);
}
rdbSave(server.dbfilename);
if (server.saveparamslen > 0) {
/* Normally rdbSave() will reset dirty, but we don't want this here
* as otherwise FLUSHALL will not be replicated nor put into the AOF. */
int saved_dirty = server.dirty;
rdbSave(server.dbfilename);
server.dirty = saved_dirty;
}
server.dirty++;
}
@@ -341,6 +347,7 @@ void shutdownCommand(redisClient *c) {
void renameGenericCommand(redisClient *c, int nx) {
robj *o;
time_t expire;
/* To use the same key as src and dst is probably an error */
if (sdscmp(c->argv[1]->ptr,c->argv[2]->ptr) == 0) {
@@ -352,16 +359,18 @@ void renameGenericCommand(redisClient *c, int nx) {
return;
incrRefCount(o);
expire = getExpire(c->db,c->argv[1]);
if (lookupKeyWrite(c->db,c->argv[2]) != NULL) {
if (nx) {
decrRefCount(o);
addReply(c,shared.czero);
return;
}
dbOverwrite(c->db,c->argv[2],o);
} else {
dbAdd(c->db,c->argv[2],o);
/* Overwrite: delete the old key before creating the new one with the same name. */
dbDelete(c->db,c->argv[2]);
}
dbAdd(c->db,c->argv[2],o);
if (expire != -1) setExpire(c->db,c->argv[2],expire);
dbDelete(c->db,c->argv[1]);
signalModifiedKey(c->db,c->argv[1]);
signalModifiedKey(c->db,c->argv[2]);
src/debug.c
+1 -1
View File
@@ -224,7 +224,7 @@ void debugCommand(redisClient *c) {
}
emptyDb();
if (rdbLoad(server.dbfilename) != REDIS_OK) {
addReply(c,shared.err);
addReplyError(c,"Error trying to load the RDB dump");
return;
}
redisLog(REDIS_WARNING,"DB reloaded by DEBUG RELOAD");
src/fmacros.h
+1 -1
View File
@@ -3,7 +3,7 @@
#define _BSD_SOURCE
#ifdef __linux__
#if defined(__linux__) || defined(__OpenBSD__)
#define _XOPEN_SOURCE 700
#else
#define _XOPEN_SOURCE
src/networking.c
+1 -1
View File
@@ -616,7 +616,7 @@ void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask) {
}
}
if (totwritten > 0) c->lastinteraction = time(NULL);
if (listLength(c->reply) == 0) {
if (c->bufpos == 0 && listLength(c->reply) == 0) {
c->sentlen = 0;
aeDeleteFileEvent(server.el,c->fd,AE_WRITABLE);
src/rdb.c
+6 -1
View File
@@ -937,18 +937,23 @@ int rdbLoad(char *filename) {
long loops = 0;
fp = fopen(filename,"r");
if (!fp) return REDIS_ERR;
if (!fp) {
errno = ENOENT;
return REDIS_ERR;
}
if (fread(buf,9,1,fp) == 0) goto eoferr;
buf[9] = '\0';
if (memcmp(buf,"REDIS",5) != 0) {
fclose(fp);
redisLog(REDIS_WARNING,"Wrong signature trying to load DB from file");
errno = EINVAL;
return REDIS_ERR;
}
rdbver = atoi(buf+5);
if (rdbver < 1 || rdbver > 2) {
fclose(fp);
redisLog(REDIS_WARNING,"Can't handle RDB format version %d",rdbver);
errno = EINVAL;
return REDIS_ERR;
}
src/redis-benchmark.c
+19 -11
View File
@@ -53,9 +53,10 @@ static struct config {
int hostport;
const char *hostsocket;
int numclients;
int requests;
int liveclients;
int donerequests;
int requests;
int requests_issued;
int requests_finished;
int keysize;
int datasize;
int randomkeys;
@@ -148,7 +149,7 @@ static void randomizeClientKey(client c) {
}
static void clientDone(client c) {
if (config.donerequests == config.requests) {
if (config.requests_finished == config.requests) {
freeClient(c);
aeStop(config.el);
return;
@@ -189,8 +190,8 @@ static void readHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
exit(1);
}
if (config.donerequests < config.requests)
config.latency[config.donerequests++] = c->latency;
if (config.requests_finished < config.requests)
config.latency[config.requests_finished++] = c->latency;
clientDone(c);
}
}
@@ -202,8 +203,15 @@ static void writeHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
REDIS_NOTUSED(fd);
REDIS_NOTUSED(mask);
/* When nothing was written yet, randomize keys and set start time. */
/* Initialize request when nothing was written. */
if (c->written == 0) {
/* Enforce upper bound to number of requests. */
if (config.requests_issued++ >= config.requests) {
freeClient(c);
return;
}
/* Really initialize: randomize keys and set start time. */
if (config.randomkeys) randomizeClientKey(c);
c->start = ustime();
c->latency = -1;
@@ -286,10 +294,10 @@ static void showLatencyReport(void) {
int i, curlat = 0;
float perc, reqpersec;
reqpersec = (float)config.donerequests/((float)config.totlatency/1000);
reqpersec = (float)config.requests_finished/((float)config.totlatency/1000);
if (!config.quiet) {
printf("====== %s ======\n", config.title);
printf(" %d requests completed in %.2f seconds\n", config.donerequests,
printf(" %d requests completed in %.2f seconds\n", config.requests_finished,
(float)config.totlatency/1000);
printf(" %d parallel clients\n", config.numclients);
printf(" %d bytes payload\n", config.datasize);
@@ -314,7 +322,8 @@ static void benchmark(const char *title, const char *cmd, int len) {
client c;
config.title = title;
config.donerequests = 0;
config.requests_issued = 0;
config.requests_finished = 0;
c = createClient(cmd,len);
createMissingClients(c);
@@ -416,7 +425,7 @@ int showThroughput(struct aeEventLoop *eventLoop, long long id, void *clientData
REDIS_NOTUSED(clientData);
float dt = (float)(mstime()-config.start)/1000.0;
float rps = (float)config.donerequests/dt;
float rps = (float)config.requests_finished/dt;
printf("%s: %.2f\r", config.title, rps);
fflush(stdout);
return 250; /* every 250ms */
@@ -438,7 +447,6 @@ int main(int argc, const char **argv) {
config.el = aeCreateEventLoop();
aeCreateTimeEvent(config.el,1,showThroughput,NULL,NULL);
config.keepalive = 1;
config.donerequests = 0;
config.datasize = 3;
config.randomkeys = 0;
config.randomkeys_keyspacelen = 0;
src/redis-check-dump.c
+13 -3
View File
@@ -16,6 +16,10 @@
#define REDIS_SET 2
#define REDIS_ZSET 3
#define REDIS_HASH 4
#define REDIS_HASH_ZIPMAP 9
#define REDIS_LIST_ZIPLIST 10
#define REDIS_SET_INTSET 11
#define REDIS_ZSET_ZIPLIST 12
/* Objects encoding. Some kind of objects like Strings and Hashes can be
* internally represented in multiple ways. The 'encoding' field of the object
@@ -132,7 +136,7 @@ int processHeader() {
}
dump_version = (int)strtol(buf + 5, NULL, 10);
if (dump_version != 1) {
if (dump_version < 1 || dump_version > 2) {
ERROR("Unknown RDB format version: %d\n", dump_version);
}
return 1;
@@ -144,7 +148,7 @@ int loadType(entry *e) {
/* this byte needs to qualify as type */
unsigned char t;
if (readBytes(&t, 1)) {
if (t <= 4 || t >= 253) {
if (t <= 4 || (t >=9 && t <= 12) || t >= 253) {
e->type = t;
return 1;
} else {
@@ -160,7 +164,8 @@ int loadType(entry *e) {
int peekType() {
unsigned char t;
if (readBytes(&t, -1) && (t <= 4 || t >= 253)) return t;
if (readBytes(&t, -1) && (t <= 4 || (t >=9 && t <= 12) || t >= 253))
return t;
return -1;
}
@@ -375,6 +380,10 @@ int loadPair(entry *e) {
switch(e->type) {
case REDIS_STRING:
case REDIS_HASH_ZIPMAP:
case REDIS_LIST_ZIPLIST:
case REDIS_SET_INTSET:
case REDIS_ZSET_ZIPLIST:
if (!processStringObject(NULL)) {
SHIFT_ERROR(offset, "Error reading entry value");
return 0;
@@ -591,6 +600,7 @@ void process() {
/* advance position */
positions[0] = positions[1];
}
free(entry.key);
}
/* because there is another potential error,
src/redis-cli.c
+44 -1
View File
@@ -61,6 +61,7 @@ static struct config {
int shutdown;
int monitor_mode;
int pubsub_mode;
int latency_mode;
int stdinarg; /* get last arg from stdin. (-x option) */
char *auth;
int raw_output; /* output mode per command */
@@ -564,6 +565,8 @@ static int parseOptions(int argc, char **argv) {
i++;
} else if (!strcmp(argv[i],"--raw")) {
config.raw_output = 1;
} else if (!strcmp(argv[i],"--latency")) {
config.latency_mode = 1;
} else if (!strcmp(argv[i],"-d") && !lastarg) {
sdsfree(config.mb_delim);
config.mb_delim = sdsnew(argv[i+1]);
@@ -614,6 +617,7 @@ static void usage() {
" -x Read last argument from STDIN\n"
" -d <delimiter> Multi-bulk delimiter in for raw formatting (default: \\n)\n"
" --raw Use raw formatting for replies (default when STDOUT is not a tty)\n"
" --latency Enter a special mode continuously sampling latency.\n"
" --help Output this help and exit\n"
" --version Output version and exit\n"
"\n"
@@ -690,7 +694,7 @@ static void repl() {
int repeat, skipargs = 0;
repeat = atoi(argv[0]);
if (repeat) {
if (argc > 1 && repeat) {
skipargs = 1;
} else {
repeat = 1;
@@ -736,6 +740,38 @@ static int noninteractive(int argc, char **argv) {
return retval;
}
static void latencyMode(void) {
redisReply *reply;
long long start, latency, min, max, tot, count = 0;
double avg;
if (!context) exit(1);
while(1) {
start = mstime();
reply = redisCommand(context,"PING");
if (reply == NULL) {
fprintf(stderr,"\nI/O error\n");
exit(1);
}
latency = mstime()-start;
freeReplyObject(reply);
count++;
if (count == 1) {
min = max = tot = latency;
avg = (double) latency;
} else {
if (latency < min) min = latency;
if (latency > max) max = latency;
tot += latency;
avg = (double) tot/count;
}
printf("\x1b[0G\x1b[2Kmin: %lld, max: %lld, avg: %.2f (%lld samples)",
min, max, avg, count);
fflush(stdout);
usleep(10000);
}
}
int main(int argc, char **argv) {
int firstarg;
@@ -749,6 +785,7 @@ int main(int argc, char **argv) {
config.shutdown = 0;
config.monitor_mode = 0;
config.pubsub_mode = 0;
config.latency_mode = 0;
config.stdinarg = 0;
config.auth = NULL;
config.raw_output = !isatty(fileno(stdout)) && (getenv("FAKETTY") == NULL);
@@ -759,6 +796,12 @@ int main(int argc, char **argv) {
argc -= firstarg;
argv += firstarg;
/* Start in latency mode if appropriate */
if (config.latency_mode) {
cliConnect(0);
latencyMode();
}
/* Start interactive mode when no command is provided */
if (argc == 0) {
/* Note that in repl mode we don't abort on connection error.
src/redis.c
+64 -26
View File
@@ -29,6 +29,7 @@
#include "redis.h"
#include "slowlog.h"
#include "bio.h"
#ifdef HAVE_BACKTRACE
#include <execinfo.h>
@@ -58,7 +59,7 @@
struct sharedObjectsStruct shared;
/* Global vars that are actally used as constants. The following double
/* Global vars that are actually used as constants. The following double
* values are used for double on-disk serialization, and are initialized
* at runtime to avoid strange compiler optimizations. */
@@ -530,6 +531,7 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
* in objects at every object access, and accuracy is not needed.
* To access a global var is faster than calling time(NULL) */
server.unixtime = time(NULL);
/* We have just 22 bits per object for LRU information.
* So we use an (eventually wrapping) LRU clock with 10 seconds resolution.
* 2^22 bits with 10 seconds resoluton is more or less 1.5 years.
@@ -635,7 +637,7 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
server.auto_aofrewrite_perc &&
server.appendonly_current_size > server.auto_aofrewrite_min_size)
{
int base = server.auto_aofrewrite_base_size ?
long long base = server.auto_aofrewrite_base_size ?
server.auto_aofrewrite_base_size : 1;
long long growth = (server.appendonly_current_size*100/base) - 100;
if (growth >= server.auto_aofrewrite_perc) {
@@ -645,6 +647,11 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
}
}
/* If we postponed an AOF buffer flush, let's try to do it every time the
* cron function is called. */
if (server.aof_flush_postponed_start) flushAppendOnlyFile(0);
/* Expire a few keys per cycle, only if this is a master.
* On slaves we wait for DEL operations synthesized by the master
* in order to guarantee a strict consistency. */
@@ -728,7 +735,7 @@ void beforeSleep(struct aeEventLoop *eventLoop) {
}
/* Write the AOF buffer on disk */
flushAppendOnlyFile();
flushAppendOnlyFile(0);
}
/* =========================== Server initialization ======================== */
@@ -791,6 +798,7 @@ void initServerConfig() {
server.port = REDIS_SERVERPORT;
server.bindaddr = NULL;
server.unixsocket = NULL;
server.unixsocketperm = 0;
server.ipfd = -1;
server.sofd = -1;
server.dbnum = REDIS_DEFAULT_DBNUM;
@@ -813,6 +821,7 @@ void initServerConfig() {
server.lastfsync = time(NULL);
server.appendfd = -1;
server.appendseldb = -1; /* Make sure the first time will not match */
server.aof_flush_postponed_start = 0;
server.pidfile = zstrdup("/var/run/redis.pid");
server.dbfilename = zstrdup("dump.rdb");
server.appendfilename = zstrdup("appendonly.aof");
@@ -900,13 +909,14 @@ void initServer() {
if (server.port != 0) {
server.ipfd = anetTcpServer(server.neterr,server.port,server.bindaddr);
if (server.ipfd == ANET_ERR) {
redisLog(REDIS_WARNING, "Opening port: %s", server.neterr);
redisLog(REDIS_WARNING, "Opening port %d: %s",
server.port, server.neterr);
exit(1);
}
}
if (server.unixsocket != NULL) {
unlink(server.unixsocket); /* don't care if this fails */
server.sofd = anetUnixServer(server.neterr,server.unixsocket);
server.sofd = anetUnixServer(server.neterr,server.unixsocket,server.unixsocketperm);
if (server.sofd == ANET_ERR) {
redisLog(REDIS_WARNING, "Opening socket: %s", server.neterr);
exit(1);
@@ -963,6 +973,7 @@ void initServer() {
if (server.vm_enabled) vmInit();
slowlogInit();
bioInit();
srand(time(NULL)^getpid());
}
@@ -1006,9 +1017,9 @@ void call(redisClient *c) {
duration = ustime()-start;
slowlogPushEntryIfNeeded(c->argv,c->argc,duration);
if (server.appendonly && dirty)
if (server.appendonly && dirty > 0)
feedAppendOnlyFile(c->cmd,c->db->id,c->argv,c->argc);
if ((dirty || c->cmd->flags & REDIS_CMD_FORCE_REPLICATION) &&
if ((dirty > 0 || c->cmd->flags & REDIS_CMD_FORCE_REPLICATION) &&
listLength(server.slaves))
replicationFeedSlaves(server.slaves,c->db->id,c->argv,c->argc);
if (listLength(server.monitors))
@@ -1115,20 +1126,29 @@ int processCommand(redisClient *c) {
/*================================== Shutdown =============================== */
int prepareForShutdown() {
redisLog(REDIS_WARNING,"User requested shutdown, saving DB...");
redisLog(REDIS_WARNING,"User requested shutdown...");
/* Kill the saving child if there is a background saving in progress.
We want to avoid race conditions, for instance our saving child may
overwrite the synchronous saving did by SHUTDOWN. */
if (server.bgsavechildpid != -1) {
redisLog(REDIS_WARNING,"There is a live saving child. Killing it!");
redisLog(REDIS_WARNING,"There is a child saving an .rdb. Killing it!");
kill(server.bgsavechildpid,SIGKILL);
rdbRemoveTempFile(server.bgsavechildpid);
}
if (server.appendonly) {
/* Kill the AOF saving child as the AOF we already have may be longer
* but contains the full dataset anyway. */
if (server.bgrewritechildpid != -1) {
redisLog(REDIS_WARNING,
"There is a child rewriting the AOF. Killing it!");
kill(server.bgrewritechildpid,SIGKILL);
}
/* Append only file: fsync() the AOF and exit */
redisLog(REDIS_NOTICE,"Calling fsync() on the AOF file.");
aof_fsync(server.appendfd);
if (server.vm_enabled) unlink(server.vm_swap_file);
} else if (server.saveparamslen > 0) {
}
if (server.saveparamslen > 0) {
redisLog(REDIS_NOTICE,"Saving the final RDB snapshot before exiting.");
/* Snapshotting. Perform a SYNC SAVE and exit */
if (rdbSave(server.dbfilename) != REDIS_OK) {
/* Ooops.. error saving! The best we can do is to continue
@@ -1136,21 +1156,36 @@ int prepareForShutdown() {
* in the next cron() Redis will be notified that the background
* saving aborted, handling special stuff like slaves pending for
* synchronization... */
redisLog(REDIS_WARNING,"Error trying to save the DB, can't exit");
redisLog(REDIS_WARNING,"Error trying to save the DB, can't exit.");
return REDIS_ERR;
}
} else {
redisLog(REDIS_WARNING,"Not saving DB.");
}
if (server.daemonize) unlink(server.pidfile);
redisLog(REDIS_WARNING,"Server exit now, bye bye...");
if (server.vm_enabled) {
redisLog(REDIS_NOTICE,"Removing the swap file.");
unlink(server.vm_swap_file);
}
if (server.daemonize) {
redisLog(REDIS_NOTICE,"Removing the pid file.");
unlink(server.pidfile);
}
/* Close the listening sockets. Apparently this allows faster restarts. */
if (server.ipfd != -1) close(server.ipfd);
if (server.sofd != -1) close(server.sofd);
if (server.unixsocket) {
redisLog(REDIS_NOTICE,"Removing the unix socket file.");
unlink(server.unixsocket); /* don't care if this fails */
}
redisLog(REDIS_WARNING,"Redis is now ready to exit, bye bye...");
return REDIS_OK;
}
/*================================== Commands =============================== */
void authCommand(redisClient *c) {
if (!server.requirepass || !strcmp(c->argv[1]->ptr, server.requirepass)) {
if (!server.requirepass) {
addReplyError(c,"Client sent AUTH, but no password is set");
} else if (!strcmp(c->argv[1]->ptr, server.requirepass)) {
c->authenticated = 1;
addReply(c,shared.ok);
} else {
@@ -1243,8 +1278,6 @@ sds genRedisInfoString(void) {
"evicted_keys:%lld\r\n"
"keyspace_hits:%lld\r\n"
"keyspace_misses:%lld\r\n"
"hash_max_zipmap_entries:%zu\r\n"
"hash_max_zipmap_value:%zu\r\n"
"pubsub_channels:%ld\r\n"
"pubsub_patterns:%u\r\n"
"latest_fork_usec:%lld\r\n"
@@ -1259,10 +1292,10 @@ sds genRedisInfoString(void) {
uptime,
uptime/(3600*24),
(unsigned long) server.lruclock,
(float)self_ru.ru_utime.tv_sec+(float)self_ru.ru_utime.tv_usec/1000000,
(float)self_ru.ru_stime.tv_sec+(float)self_ru.ru_stime.tv_usec/1000000,
(float)c_ru.ru_utime.tv_sec+(float)c_ru.ru_utime.tv_usec/1000000,
(float)self_ru.ru_utime.tv_sec+(float)self_ru.ru_utime.tv_usec/1000000,
(float)c_ru.ru_stime.tv_sec+(float)c_ru.ru_stime.tv_usec/1000000,
(float)c_ru.ru_utime.tv_sec+(float)c_ru.ru_utime.tv_usec/1000000,
listLength(server.clients)-listLength(server.slaves),
listLength(server.slaves),
lol, bib,
@@ -1286,8 +1319,6 @@ sds genRedisInfoString(void) {
server.stat_evictedkeys,
server.stat_keyspace_hits,
server.stat_keyspace_misses,
server.hash_max_zipmap_entries,
server.hash_max_zipmap_value,
dictSize(server.pubsub_channels),
listLength(server.pubsub_patterns),
server.stat_fork_time,
@@ -1620,8 +1651,13 @@ int main(int argc, char **argv) {
if (loadAppendOnlyFile(server.appendfilename) == REDIS_OK)
redisLog(REDIS_NOTICE,"DB loaded from append only file: %ld seconds",time(NULL)-start);
} else {
if (rdbLoad(server.dbfilename) == REDIS_OK)
redisLog(REDIS_NOTICE,"DB loaded from disk: %ld seconds",time(NULL)-start);
if (rdbLoad(server.dbfilename) == REDIS_OK) {
redisLog(REDIS_NOTICE,"DB loaded from disk: %ld seconds",
time(NULL)-start);
} else if (errno != ENOENT) {
redisLog(REDIS_WARNING,"Fatal error loading the DB. Exiting.");
exit(1);
}
}
if (server.ipfd > 0)
redisLog(REDIS_NOTICE,"The server is now ready to accept connections on port %d", server.port);
@@ -1642,8 +1678,10 @@ static void *getMcontextEip(ucontext_t *uc) {
#elif defined(__APPLE__) && !defined(MAC_OS_X_VERSION_10_6)
#if __x86_64__
return (void*) uc->uc_mcontext->__ss.__rip;
#else
#elif __i386__
return (void*) uc->uc_mcontext->__ss.__eip;
#else
return (void*) uc->uc_mcontext->__ss.__srr0;
#endif
#elif defined(__APPLE__) && defined(MAC_OS_X_VERSION_10_6)
#if defined(_STRUCT_X86_THREAD_STATE64) && !defined(__i386__)
src/redis.h
+3 -1
View File
@@ -378,6 +378,7 @@ struct redisServer {
int port;
char *bindaddr;
char *unixsocket;
mode_t unixsocketperm;
int ipfd;
int sofd;
redisDb *db;
@@ -428,6 +429,7 @@ struct redisServer {
time_t lastfsync;
int appendfd;
int appendseldb;
time_t aof_flush_postponed_start;
char *pidfile;
pid_t bgsavechildpid;
pid_t bgrewritechildpid;
@@ -796,7 +798,7 @@ void backgroundSaveDoneHandler(int statloc);
int getObjectSaveType(robj *o);
/* AOF persistence */
void flushAppendOnlyFile(void);
void flushAppendOnlyFile(int force);
void feedAppendOnlyFile(struct redisCommand *cmd, int dictid, robj **argv, int argc);
void aofRemoveTempFile(pid_t childpid);
int rewriteAppendOnlyFileBackground(void);
src/replication.c
+7
View File
@@ -382,6 +382,13 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
* function is called, so we can delete it. */
aeDeleteFileEvent(server.el,fd,AE_READABLE|AE_WRITABLE);
/* If this event fired after the user turned the instance into a master
* with SLAVEOF NO ONE we must just return ASAP. */
if (server.replstate == REDIS_REPL_NONE) {
close(fd);
return;
}
/* AUTH with the master if required. */
if(server.masterauth) {
char authcmd[1024];
src/sds.c
+7
View File
@@ -92,6 +92,13 @@ void sdsupdatelen(sds s) {
sh->len = reallen;
}
void sdsclear(sds s) {
struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));
sh->free += sh->len;
sh->len = 0;
sh->buf[0] = '\0';
}
static sds sdsMakeRoomFor(sds s, size_t addlen) {
struct sdshdr *sh, *newsh;
size_t free = sdsavail(s);
src/sds.h
+1
View File
@@ -76,6 +76,7 @@ sds sdscatprintf(sds s, const char *fmt, ...);
sds sdstrim(sds s, const char *cset);
sds sdsrange(sds s, int start, int end);
void sdsupdatelen(sds s);
void sdsclear(sds s);
int sdscmp(sds s1, sds s2);
sds *sdssplitlen(char *s, int len, char *sep, int seplen, int *count);
void sdsfreesplitres(sds *tokens, int count);
src/t_hash.c
+4 -1
View File
@@ -403,8 +403,11 @@ void hdelCommand(redisClient *c) {
for (j = 2; j < c->argc; j++) {
if (hashTypeDelete(o,c->argv[j])) {
if (hashTypeLength(o) == 0) dbDelete(c->db,c->argv[1]);
deleted++;
if (hashTypeLength(o) == 0) {
dbDelete(c->db,c->argv[1]);
break;
}
}
}
if (deleted) {
src/t_list.c
+12 -5
View File
@@ -519,7 +519,12 @@ void lrangeCommand(redisClient *c) {
p = ziplistNext(o->ptr,p);
}
} else if (o->encoding == REDIS_ENCODING_LINKEDLIST) {
listNode *ln = listIndex(o->ptr,start);
listNode *ln;
/* If we are nearest to the end of the list, reach the element
* starting from tail and going backward, as it is faster. */
if (start > llen/2) start -= llen;
ln = listIndex(o->ptr,start);
while(rangelen--) {
addReplyBulk(c,ln->value);
@@ -643,7 +648,7 @@ void lremCommand(redisClient *c) {
void rpoplpushHandlePush(redisClient *origclient, redisClient *c, robj *dstkey, robj *dstobj, robj *value) {
robj *aux;
if (!handleClientsWaitingListPush(c,dstkey,value)) {
if (!handleClientsWaitingListPush(origclient,dstkey,value)) {
/* Create the list if the key does not exist */
if (!dstobj) {
dstobj = createZiplistObject();
@@ -653,10 +658,12 @@ void rpoplpushHandlePush(redisClient *origclient, redisClient *c, robj *dstkey,
}
listTypePush(dstobj,value,REDIS_HEAD);
/* If we are pushing as a result of LPUSH against a key
* watched by BLPOPLPUSH, we need to rewrite the command vector.
* But if this is called directly by RPOPLPUSH (either directly
* watched by BRPOPLPUSH, we need to rewrite the command vector
* as an LPUSH.
*
* If this is called directly by RPOPLPUSH (either directly
* or via a BRPOPLPUSH where the popped list exists)
* we should replicate the BRPOPLPUSH command itself. */
* we should replicate the RPOPLPUSH command itself. */
if (c != origclient) {
aux = createStringObject("LPUSH",5);
rewriteClientCommandVector(origclient,3,aux,dstkey,value);
src/version.h
+1 -1
View File
@@ -1 +1 @@
#define REDIS_VERSION "2.3.6"
#define REDIS_VERSION "2.4.2"
tests/integration/aof-race.tcl
+35
View File
@@ -0,0 +1,35 @@
set defaults { appendonly {yes} appendfilename {appendonly.aof} }
set server_path [tmpdir server.aof]
set aof_path "$server_path/appendonly.aof"
proc start_server_aof {overrides code} {
upvar defaults defaults srv srv server_path server_path
set config [concat $defaults $overrides]
start_server [list overrides $config] $code
}
tags {"aof"} {
# Specific test for a regression where internal buffers were not properly
# cleaned after a child responsible for an AOF rewrite exited. This buffer
# was subsequently appended to the new AOF, resulting in duplicate commands.
start_server_aof [list dir $server_path] {
set client [redis [srv host] [srv port]]
set bench [open "|src/redis-benchmark -q -p [srv port] -c 20 -n 20000 incr foo" "r+"]
after 100
# Benchmark should be running by now: start background rewrite
$client bgrewriteaof
# Read until benchmark pipe reaches EOF
while {[string length [read $bench]] > 0} {}
# Check contents of foo
assert_equal 20000 [$client get foo]
}
# Restart server to replay AOF
start_server_aof [list dir $server_path] {
set client [redis [srv host] [srv port]]
assert_equal 20000 [$client get foo]
}
}
tests/integration/replication.tcl
+6
View File
@@ -57,5 +57,11 @@ start_server {tags {"repl"}} {
if {$::valgrind} {after 2000}
r 0 get mykey
} {bar}
test {FLUSHALL should replicate} {
r -1 flushall
if {$::valgrind} {after 2000}
list [r -1 dbsize] [r 0 dbsize]
} {0 0}
}
}
tests/support/server.tcl
+2 -1
View File
@@ -154,7 +154,8 @@ proc start_server {options {code undefined}} {
dict set config dir [tmpdir server]
# start every server on a different port
dict set config port [incr ::port]
set ::port [find_available_port [expr {$::port+1}]]
dict set config port $::port
# apply overrides from global space and arguments
foreach {directive arguments} [concat $::global_overrides $overrides] {
tests/support/test.tcl
+1 -1
View File
@@ -5,7 +5,7 @@ set ::tests_failed {}
proc assert {condition} {
if {![uplevel 1 expr $condition]} {
error "assertion:Expected '$value' to be true"
error "assertion:Expected condition '$condition' to be true"
}
}
tests/test_helper.tcl
+24 -21
View File
@@ -31,12 +31,13 @@ set ::all_tests {
integration/aof
unit/pubsub
unit/slowlog
unit/maxmemory
}
# Index to the next test to run in the ::all_tests list.
set ::next_test 0
set ::host 127.0.0.1
set ::port 16379
set ::port 21111
set ::traceleaks 0
set ::valgrind 0
set ::verbose 0
@@ -147,36 +148,38 @@ proc cleanup {} {
puts "OK"
}
proc find_available_port start {
for {set j $start} {$j < $start+1024} {incr j} {
if {[catch {
set fd [socket 127.0.0.1 $start]
}]} {
return $start
} else {
close $fd
}
}
if {$j == $start+1024} {
error "Can't find a non busy port in the $start-[expr {$start+1023}] range."
}
}
proc test_server_main {} {
cleanup
# Open a listening socket, trying different ports in order to find a
# non busy one.
set port 11111
while 1 {
puts "Starting test server at port $port"
if {[catch {socket -server accept_test_clients $port} e]} {
if {[string match {*address already in use*} $e]} {
if {$port == 20000} {
puts "Can't find an available TCP port for test server."
exit 1
} else {
incr port
}
} else {
puts "Fatal error starting test server: $e"
exit 1
}
} else {
break
}
}
set port [find_available_port 11111]
puts "Starting test server at port $port"
socket -server accept_test_clients $port
# Start the client instances
set ::clients_pids {}
set start_port [expr {$::port+100}]
for {set j 0} {$j < $::numclients} {incr j} {
set start_port [find_available_port $start_port]
set p [exec tclsh8.5 [info script] {*}$::argv \
--client $port --port [expr {$::port+($j*10)}] &]
--client $port --port $start_port &]
lappend ::clients_pids $p
incr start_port 10
}
# Setup global state for the test server
tests/unit/auth.tcl
+14 -2
View File
@@ -1,15 +1,27 @@
start_server {tags {"auth"}} {
test {AUTH fails if there is no password configured server side} {
catch {r auth foo} err
set _ $err
} {ERR*no password*}
}
start_server {tags {"auth"} overrides {requirepass foobar}} {
test {AUTH fails when a wrong password is given} {
catch {r auth wrong!} err
format $err
set _ $err
} {ERR*invalid password}
test {Arbitrary command gives an error when AUTH is required} {
catch {r set foo bar} err
format $err
set _ $err
} {ERR*operation not permitted}
test {AUTH succeeds when the right password is given} {
r auth foobar
} {OK}
test {Once AUTH succeeded we can actually send commands to the server} {
r set foo 100
r incr foo
} {101}
}
tests/unit/basic.tcl
+19
View File
@@ -262,6 +262,25 @@ start_server {tags {"basic"}} {
format $err
} {ERR*}
test {RENAME with volatile key, should move the TTL as well} {
r del mykey mykey2
r set mykey foo
r expire mykey 100
assert {[r ttl mykey] > 95 && [r ttl mykey] <= 100}
r rename mykey mykey2
assert {[r ttl mykey2] > 95 && [r ttl mykey2] <= 100}
}
test {RENAME with volatile key, should not inherit TTL of target key} {
r del mykey mykey2
r set mykey foo
r set mykey2 bar
r expire mykey2 100
assert {[r ttl mykey] == -1 && [r ttl mykey2] > 0}
r rename mykey mykey2
r ttl mykey2
} {-1}
test {DEL all keys again (DB 0)} {
foreach key [r keys *] {
r del $key
tests/unit/maxmemory.tcl
+120
View File
@@ -0,0 +1,120 @@
start_server {tags {"maxmemory"}} {
foreach policy {
allkeys-random allkeys-lru volatile-lru volatile-random volatile-ttl
} {
test "maxmemory - is the memory limit honoured? (policy $policy)" {
# make sure to start with a blank instance
r flushall
# Get the current memory limit and calculate a new limit.
# We just add 100k to the current memory size so that it is
# fast for us to reach that limit.
set used [s used_memory]
set limit [expr {$used+100*1024}]
r config set maxmemory $limit
r config set maxmemory-policy $policy
# Now add keys until the limit is almost reached.
set numkeys 0
while 1 {
r setex [randomKey] 10000 x
incr numkeys
if {[s used_memory]+4096 > $limit} {
assert {$numkeys > 10}
break
}
}
# If we add the same number of keys already added again, we
# should still be under the limit.
for {set j 0} {$j < $numkeys} {incr j} {
r setex [randomKey] 10000 x
}
assert {[s used_memory] < ($limit+4096)}
}
}
foreach policy {
allkeys-random allkeys-lru volatile-lru volatile-random volatile-ttl
} {
test "maxmemory - only allkeys-* should remove non-volatile keys ($policy)" {
# make sure to start with a blank instance
r flushall
# Get the current memory limit and calculate a new limit.
# We just add 100k to the current memory size so that it is
# fast for us to reach that limit.
set used [s used_memory]
set limit [expr {$used+100*1024}]
r config set maxmemory $limit
r config set maxmemory-policy $policy
# Now add keys until the limit is almost reached.
set numkeys 0
while 1 {
r set [randomKey] x
incr numkeys
if {[s used_memory]+4096 > $limit} {
assert {$numkeys > 10}
break
}
}
# If we add the same number of keys already added again and
# the policy is allkeys-* we should still be under the limit.
# Otherwise we should see an error reported by Redis.
set err 0
for {set j 0} {$j < $numkeys} {incr j} {
if {[catch {r set [randomKey] x} e]} {
if {[string match {*used memory*} $e]} {
set err 1
}
}
}
if {[string match allkeys-* $policy]} {
assert {[s used_memory] < ($limit+4096)}
} else {
assert {$err == 1}
}
}
}
foreach policy {
volatile-lru volatile-random volatile-ttl
} {
test "maxmemory - policy $policy should only remove volatile keys." {
# make sure to start with a blank instance
r flushall
# Get the current memory limit and calculate a new limit.
# We just add 100k to the current memory size so that it is
# fast for us to reach that limit.
set used [s used_memory]
set limit [expr {$used+100*1024}]
r config set maxmemory $limit
r config set maxmemory-policy $policy
# Now add keys until the limit is almost reached.
set numkeys 0
while 1 {
# Odd keys are volatile
# Even keys are non volatile
if {$numkeys % 2} {
r setex "key:$numkeys" 10000 x
} else {
r set "key:$numkeys" x
}
if {[s used_memory]+4096 > $limit} {
assert {$numkeys > 10}
break
}
incr numkeys
}
# Now we add the same number of volatile keys already added.
# We expect Redis to evict only volatile keys in order to make
# space.
set err 0
for {set j 0} {$j < $numkeys} {incr j} {
catch {r setex "foo:$j" 10000 x}
}
# We should still be under the limit.
assert {[s used_memory] < ($limit+4096)}
# However all our non volatile keys should be here.
for {set j 0} {$j < $numkeys} {incr j 2} {
assert {[r exists "key:$j"]}
}
}
}
}
tests/unit/type/hash.tcl
+7
View File
@@ -235,6 +235,13 @@ start_server {tags {"hash"}} {
r hgetall myhash
} {b 2}
test {HDEL - hash becomes empty before deleting all specified fields} {
r del myhash
r hmset myhash a 1 b 2 c 3
assert_equal 3 [r hdel myhash a b c d e]
assert_equal 0 [r exists myhash]
}
test {HEXISTS} {
set rv {}
set k [lindex [array names smallhash *] 0]
tests/unit/type/list.tcl
+14
View File
@@ -728,4 +728,18 @@ start_server {
assert_equal 3 [r llen myotherlist]
}
}
test "Regression for bug 593 - chaining BRPOPLPUSH with other blocking cmds" {
set rd1 [redis_deferring_client]
set rd2 [redis_deferring_client]
$rd1 brpoplpush a b 0
$rd1 brpoplpush a b 0
$rd2 brpoplpush b c 0
after 1000
r lpush a data
$rd1 close
$rd2 close
r ping
} {PONG}
}
utils/install_server.sh
Executable
+156
View File
@@ -0,0 +1,156 @@
#! /bin/sh
# Copyright 2011 Dvir Volk <dvirsk at gmail dot com>. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification, are
# permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice, this list of
# conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice, this list
# of conditions and the following disclaimer in the documentation and/or other materials
# provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL Dvir Volk OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
#
# # # # # # # # # # # # # # # # # # # # # # # # # #
#
# Interactive service installer for redis server
# this generates a redis config file and an /etc/init.d script, and installs them
# this scripts should be run as root
#
die () {
echo "ERROR: $1. Aborting!"
exit 1
}
#Initial defaults
_REDIS_PORT=6379
echo "Welcome to the redis service installer"
echo "This script will help you easily set up a running redis server
"
#check for root user TODO: replace this with a call to "id"
if [ `whoami` != "root" ] ; then
echo "You must run this script as root. Sorry!"
exit 1
fi
#Read the redis port
read -p "Please select the redis port for this instance: [$_REDIS_PORT] " REDIS_PORT
if [ ! `echo $REDIS_PORT | egrep "^[0-9]+\$"` ] ; then
echo "Selecting default: $_REDIS_PORT"
REDIS_PORT=$_REDIS_PORT
fi
#read the redis config file
_REDIS_CONFIG_FILE="/etc/redis/$REDIS_PORT.conf"
read -p "Please select the redis config file name [$_REDIS_CONFIG_FILE] " REDIS_CONFIG_FILE
if [ !"$REDIS_CONFIG_FILE" ] ; then
REDIS_CONFIG_FILE=$_REDIS_CONFIG_FILE
echo "Selected default - $REDIS_CONFIG_FILE"
fi
#try and create it
mkdir -p `dirname "$REDIS_CONFIG_FILE"` || die "Could not create redis config directory"
#read the redis log file path
_REDIS_LOG_FILE="/var/log/redis_$REDIS_PORT.log"
read -p "Please select the redis log file name [$_REDIS_LOG_FILE] " REDIS_LOG_FILE
if [ !"$REDIS_LOG_FILE" ] ; then
REDIS_LOG_FILE=$_REDIS_LOG_FILE
echo "Selected default - $REDIS_LOG_FILE"
fi
#get the redis data directory
_REDIS_DATA_DIR="/var/lib/redis/$REDIS_PORT"
read -p "Please select the data directory for this instance [$_REDIS_DATA_DIR] " REDIS_DATA_DIR
if [ !"$REDIS_DATA_DIR" ] ; then
REDIS_DATA_DIR=$_REDIS_DATA_DIR
echo "Selected default - $REDIS_DATA_DIR"
fi
mkdir -p $REDIS_DATA_DIR || die "Could not create redis data directory"
#get the redis executable path
_REDIS_EXECUTABLE=`which redis-server`
read -p "Please select the redis executable path [$_REDIS_EXECUTABLE] " REDIS_EXECUTABLE
if [ ! -f "$REDIS_EXECUTABLE" ] ; then
REDIS_EXECUTABLE=$_REDIS_EXECUTABLE
if [ ! -f "$REDIS_EXECUTABLE" ] ; then
echo "Mmmmm... it seems like you don't have a redis executable. Did you run make install yet?"
exit 1
fi
fi
#render the tmplates
TMP_FILE="/tmp/$REDIS_PORT.conf"
TPL_FILE="./redis.conf.tpl"
INIT_TPL_FILE="./redis_init_script.tpl"
INIT_SCRIPT_DEST="/etc/init.d/redis_$REDIS_PORT"
PIDFILE="/var/run/redis_$REDIS_PORT.pid"
#check the default for redis cli
CLI_EXEC=`which redis-cli`
if [ ! "$CLI_EXEC" ] ; then
CLI_EXEC=`dirname $REDIS_EXECUTABLE`"/redis-cli"
fi
#Generate config file from template
echo "## Generated by install_server.sh ##" > $TMP_FILE
cat $TPL_FILE | while read line; do eval "echo \"$line\"" >> $TMP_FILE; done
cp -f $TMP_FILE $REDIS_CONFIG_FILE || exit 1
#Generate sample script from template file
rm -f $TMP_FILE
#we hard code the configs here to avoid issues with templates containing env vars
#kinda lame but works!
REDIS_INIT_HEADER=\
"#/bin/sh\n
#Configurations injected by install_server below....\n\n
EXEC=$REDIS_EXECUTABLE\n
CLIEXEC=$CLI_EXEC\n
PIDFILE=$PIDFILE\n
CONF=\"$REDIS_CONFIG_FILE\"\n\n
REDISPORT=\"$REDIS_PORT\"\n\n
###############\n\n"
#combine the header and the template (which is actually a static footer)
echo $REDIS_INIT_HEADER > $TMP_FILE && cat $INIT_TPL_FILE >> $TMP_FILE || die "Could not write init script to $TMP_FILE"
#copy to /etc/init.d
cp -f $TMP_FILE $INIT_SCRIPT_DEST && chmod +x $INIT_SCRIPT_DEST || die "Could not copy redis init script to $INIT_SCRIPT_DEST"
echo "Copied $TMP_FILE => $INIT_SCRIPT_DEST"
#Install the service
echo "Installing service..."
update-rc.d redis_$REDIS_PORT defaults && echo "Success!"
/etc/init.d/redis_$REDIS_PORT start || die "Failed starting service..."
#tada
echo "Installation successful!"
exit 0
utils/redis.conf.tpl
+402
View File
@@ -0,0 +1,402 @@
# Redis configuration file example
# Note on units: when memory size is needed, it is possible to specifiy
# it in the usual form of 1k 5GB 4M and so forth:
#
# 1k => 1000 bytes
# 1kb => 1024 bytes
# 1m => 1000000 bytes
# 1mb => 1024*1024 bytes
# 1g => 1000000000 bytes
# 1gb => 1024*1024*1024 bytes
#
# units are case insensitive so 1GB 1Gb 1gB are all the same.
# By default Redis does not run as a daemon. Use 'yes' if you need it.
# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
daemonize yes
# When running daemonized, Redis writes a pid file in /var/run/redis.pid by
# default. You can specify a custom pid file location here.
pidfile $PIDFILE
# Accept connections on the specified port, default is 6379.
# If port 0 is specified Redis will not listen on a TCP socket.
port $REDIS_PORT
# If you want you can bind a single interface, if the bind option is not
# specified all the interfaces will listen for incoming connections.
#
# bind 127.0.0.1
# Specify the path for the unix socket that will be used to listen for
# incoming connections. There is no default, so Redis will not listen
# on a unix socket when not specified.
#
# unixsocket /tmp/redis.sock
# Close the connection after a client is idle for N seconds (0 to disable)
timeout 300
# Set server verbosity to 'debug'
# it can be one of:
# debug (a lot of information, useful for development/testing)
# verbose (many rarely useful info, but not a mess like the debug level)
# notice (moderately verbose, what you want in production probably)
# warning (only very important / critical messages are logged)
loglevel verbose
# Specify the log file name. Also 'stdout' can be used to force
# Redis to log on the standard output. Note that if you use standard
# output for logging but daemonize, logs will be sent to /dev/null
logfile $REDIS_LOG_FILE
# To enable logging to the system logger, just set 'syslog-enabled' to yes,
# and optionally update the other syslog parameters to suit your needs.
# syslog-enabled no
# Specify the syslog identity.
# syslog-ident redis
# Specify the syslog facility. Must be USER or between LOCAL0-LOCAL7.
# syslog-facility local0
# Set the number of databases. The default database is DB 0, you can select
# a different one on a per-connection basis using SELECT <dbid> where
# dbid is a number between 0 and 'databases'-1
databases 16
################################ SNAPSHOTTING #################################
#
# Save the DB on disk:
#
# save <seconds> <changes>
#
# Will save the DB if both the given number of seconds and the given
# number of write operations against the DB occurred.
#
# In the example below the behaviour will be to save:
# after 900 sec (15 min) if at least 1 key changed
# after 300 sec (5 min) if at least 10 keys changed
# after 60 sec if at least 10000 keys changed
#
# Note: you can disable saving at all commenting all the "save" lines.
save 900 1
save 300 10
save 60 10000
# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes
# The filename where to dump the DB
dbfilename dump.rdb
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# Also the Append Only File will be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir $REDIS_DATA_DIR
################################# REPLICATION #################################
# Master-Slave replication. Use slaveof to make a Redis instance a copy of
# another Redis server. Note that the configuration is local to the slave
# so for example it is possible to configure the slave to save the DB with a
# different interval, or to listen to another port, and so on.
#
# slaveof <masterip> <masterport>
# If the master is password protected (using the "requirepass" configuration
# directive below) it is possible to tell the slave to authenticate before
# starting the replication synchronization process, otherwise the master will
# refuse the slave request.
#
# masterauth <master-password>
# When a slave lost the connection with the master, or when the replication
# is still in progress, the slave can act in two different ways:
#
# 1) if slave-serve-stale-data is set to 'yes' (the default) the slave will
# still reply to client requests, possibly with out of data data, or the
# data set may just be empty if this is the first synchronization.
#
# 2) if slave-serve-stale data is set to 'no' the slave will reply with
# an error "SYNC with master in progress" to all the kind of commands
# but to INFO and SLAVEOF.
#
slave-serve-stale-data yes
################################## SECURITY ###################################
# Require clients to issue AUTH <PASSWORD> before processing any other
# commands. This might be useful in environments in which you do not trust
# others with access to the host running redis-server.
#
# This should stay commented out for backward compatibility and because most
# people do not need auth (e.g. they run their own servers).
#
# Warning: since Redis is pretty fast an outside user can try up to
# 150k passwords per second against a good box. This means that you should
# use a very strong password otherwise it will be very easy to break.
#
# requirepass foobared
# Command renaming.
#
# It is possilbe to change the name of dangerous commands in a shared
# environment. For instance the CONFIG command may be renamed into something
# of hard to guess so that it will be still available for internal-use
# tools but not available for general clients.
#
# Example:
#
# rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52
#
# It is also possilbe to completely kill a command renaming it into
# an empty string:
#
# rename-command CONFIG ""
################################### LIMITS ####################################
# Set the max number of connected clients at the same time. By default there
# is no limit, and it's up to the number of file descriptors the Redis process
# is able to open. The special value '0' means no limits.
# Once the limit is reached Redis will close all the new connections sending
# an error 'max number of clients reached'.
#
# maxclients 128
# Don't use more memory than the specified amount of bytes.
# When the memory limit is reached Redis will try to remove keys with an
# EXPIRE set. It will try to start freeing keys that are going to expire
# in little time and preserve keys with a longer time to live.
# Redis will also try to remove objects from free lists if possible.
#
# If all this fails, Redis will start to reply with errors to commands
# that will use more memory, like SET, LPUSH, and so on, and will continue
# to reply to most read-only commands like GET.
#
# WARNING: maxmemory can be a good idea mainly if you want to use Redis as a
# 'state' server or cache, not as a real DB. When Redis is used as a real
# database the memory usage will grow over the weeks, it will be obvious if
# it is going to use too much memory in the long run, and you'll have the time
# to upgrade. With maxmemory after the limit is reached you'll start to get
# errors for write operations, and this may even lead to DB inconsistency.
#
# maxmemory <bytes>
# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
# is reached? You can select among five behavior:
#
# volatile-lru -> remove the key with an expire set using an LRU algorithm
# allkeys-lru -> remove any key accordingly to the LRU algorithm
# volatile-random -> remove a random key with an expire set
# allkeys->random -> remove a random key, any key
# volatile-ttl -> remove the key with the nearest expire time (minor TTL)
# noeviction -> don't expire at all, just return an error on write operations
#
# Note: with all the kind of policies, Redis will return an error on write
# operations, when there are not suitable keys for eviction.
#
# At the date of writing this commands are: set setnx setex append
# incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd
# sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby
# zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby
# getset mset msetnx exec sort
#
# The default is:
#
# maxmemory-policy volatile-lru
# LRU and minimal TTL algorithms are not precise algorithms but approximated
# algorithms (in order to save memory), so you can select as well the sample
# size to check. For instance for default Redis will check three keys and
# pick the one that was used less recently, you can change the sample size
# using the following configuration directive.
#
# maxmemory-samples 3
############################## APPEND ONLY MODE ###############################
# By default Redis asynchronously dumps the dataset on disk. If you can live
# with the idea that the latest records will be lost if something like a crash
# happens this is the preferred way to run Redis. If instead you care a lot
# about your data and don't want to that a single record can get lost you should
# enable the append only mode: when this mode is enabled Redis will append
# every write operation received in the file appendonly.aof. This file will
# be read on startup in order to rebuild the full dataset in memory.
#
# Note that you can have both the async dumps and the append only file if you
# like (you have to comment the "save" statements above to disable the dumps).
# Still if append only mode is enabled Redis will load the data from the
# log file at startup ignoring the dump.rdb file.
#
# IMPORTANT: Check the BGREWRITEAOF to check how to rewrite the append
# log file in background when it gets too big.
appendonly no
# The name of the append only file (default: "appendonly.aof")
# appendfilename appendonly.aof
# The fsync() call tells the Operating System to actually write data on disk
# instead to wait for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log . Slow, Safest.
# everysec: fsync only if one second passed since the last fsync. Compromise.
#
# The default is "everysec" that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# If unsure, use "everysec".
# appendfsync always
appendfsync everysec
# appendfsync no
# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving the durability of Redis is
# the same as "appendfsync none", that in pratical terms means that it is
# possible to lost up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.
no-appendfsync-on-rewrite no
# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size will growth by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (or if no rewrite happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a precentage of zero in order to disable the automatic AOF
# rewrite feature.
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
################################ LUA SCRIPTING ###############################
# Max execution time of a Lua script in milliseconds.
# This prevents that a programming error generating an infinite loop will block
# your server forever. Set it to 0 or a negative value for unlimited execution.
#lua-time-limit 60000
################################## SLOW LOG ###################################
# The Redis Slow Log is a system to log queries that exceeded a specified
# execution time. The execution time does not include the I/O operations
# like talking with the client, sending the reply and so forth,
# but just the time needed to actually execute the command (this is the only
# stage of command execution where the thread is blocked and can not serve
# other requests in the meantime).
#
# You can configure the slow log with two parameters: one tells Redis
# what is the execution time, in microseconds, to exceed in order for the
# command to get logged, and the other parameter is the length of the
# slow log. When a new command is logged the oldest one is removed from the
# queue of logged commands.
# The following time is expressed in microseconds, so 1000000 is equivalent
# to one second. Note that a negative number disables the slow log, while
# a value of zero forces the logging of every command.
slowlog-log-slower-than 10000
# There is no limit to this length. Just be aware that it will consume memory.
# You can reclaim memory used by the slow log with SLOWLOG RESET.
slowlog-max-len 1024
############################### ADVANCED CONFIG ###############################
# Hashes are encoded in a special way (much more memory efficient) when they
# have at max a given numer of elements, and the biggest element does not
# exceed a given threshold. You can configure this limits with the following
# configuration directives.
hash-max-zipmap-entries 512
hash-max-zipmap-value 64
# Similarly to hashes, small lists are also encoded in a special way in order
# to save a lot of space. The special representation is only used when
# you are under the following limits:
list-max-ziplist-entries 512
list-max-ziplist-value 64
# Sets have a special encoding in just one case: when a set is composed
# of just strings that happens to be integers in radix 10 in the range
# of 64 bit signed integers.
# The following configuration setting sets the limit in the size of the
# set in order to use this special memory saving encoding.
set-max-intset-entries 512
# Similarly to hashes and lists, sorted sets are also specially encoded in
# order to save a lot of space. This encoding is only used when the length and
# elements of a sorted set are below the following limits:
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
# Active rehashing uses 1 millisecond every 100 milliseconds of CPU time in
# order to help rehashing the main Redis hash table (the one mapping top-level
# keys to values). The hash table implementation redis uses (see dict.c)
# performs a lazy rehashing: the more operation you run into an hash table
# that is rhashing, the more rehashing "steps" are performed, so if the
# server is idle the rehashing is never complete and some more memory is used
# by the hash table.
#
# The default is to use this millisecond 10 times every second in order to
# active rehashing the main dictionaries, freeing memory when possible.
#
# If unsure:
# use "activerehashing no" if you have hard latency requirements and it is
# not a good thing in your environment that Redis can reply form time to time
# to queries with 2 milliseconds delay.
#
# use "activerehashing yes" if you don't have such hard requirements but
# want to free memory asap when possible.
activerehashing yes
################################## INCLUDES ###################################
# Include one or more other config files here. This is useful if you
# have a standard template that goes to all redis server but also need
# to customize a few per-server settings. Include files can include
# other files, so use this wisely.
#
# include /path/to/local.conf
# include /path/to/other.conf
utils/redis_init_script.tpl
Executable
+31
View File
@@ -0,0 +1,31 @@
case "$1" in
start)
if [ -f $$PIDFILE ]
then
echo "$PIDFILE exists, process is already running or crashed"
else
echo "Starting Redis server..."
$EXEC $CONF
fi
;;
stop)
if [ ! -f $PIDFILE ]
then
echo "$PIDFILE does not exist, process is not running"
else
PID=$(cat $PIDFILE)
echo "Stopping ..."
$CLIEXEC -p $REDISPORT shutdown
while [ -x /proc/${PID} ]
do
echo "Waiting for Redis to shutdown ..."
sleep 1
done
echo "Redis stopped"
fi
;;
*)
echo "Please use start or stop as first argument"
;;
esac