Redis/redis-For-Windows
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Compare commits

base: Redis/redis-For-Windows:2.4.0-rc3
Branches Tags
Redis/redis-For-Windows:develop Redis/redis-For-Windows:win-5.0 Redis/redis-For-Windows:win-6.2
Redis/redis-For-Windows:5.0.14.1 Redis/redis-For-Windows:v5.0.14.1 Redis/redis-For-Windows:v5.0.14 Redis/redis-For-Windows:v5.0.10 Redis/redis-For-Windows:v5.0.9 Redis/redis-For-Windows:v4.0.14.2 Redis/redis-For-Windows:v4.0.14.1 Redis/redis-For-Windows:v4.0.14 Redis/redis-For-Windows:v4.0.2.3-alpha Redis/redis-For-Windows:v4.0.2.2-alpha Redis/redis-For-Windows:v4.0.2.1-alpha Redis/redis-For-Windows:v4.0.2-alpha Redis/redis-For-Windows:win-3.2.100 Redis/redis-For-Windows:win-3.0.504 Redis/redis-For-Windows:win-3.0.503 Redis/redis-For-Windows:win-2.8.2402 Redis/redis-For-Windows:win-3.0.502 Redis/redis-For-Windows:win-2.8.2401 Redis/redis-For-Windows:3.2.1 Redis/redis-For-Windows:win-3.2.0000-preview Redis/redis-For-Windows:3.2.0 Redis/redis-For-Windows:3.2.0-rc3 Redis/redis-For-Windows:3.2.0-rc2 Redis/redis-For-Windows:win-2.8.2400 Redis/redis-For-Windows:win-3.0.501 Redis/redis-For-Windows:3.2-rc1 Redis/redis-For-Windows:3.0.6 Redis/redis-For-Windows:2.8.24 Redis/redis-For-Windows:win-3.0.500 Redis/redis-For-Windows:win-3.0.500-rc2 Redis/redis-For-Windows:win-3.0.500-rc1 Redis/redis-For-Windows:win-2.8.2104 Redis/redis-For-Windows:3.0.5 Redis/redis-For-Windows:2.8.23 Redis/redis-For-Windows:win-3.0.300-beta1 Redis/redis-For-Windows:win-2.8.2103 Redis/redis-For-Windows:2.8.22 Redis/redis-For-Windows:3.0.4 Redis/redis-For-Windows:win-3.0.300-alpha3 Redis/redis-For-Windows:win-2.8.2102 Redis/redis-For-Windows:win-3.0.300-alpha2 Redis/redis-For-Windows:win-3.0.100-alpha1 Redis/redis-For-Windows:3.0.3 Redis/redis-For-Windows:win-2.8.2101 Redis/redis-For-Windows:win-2.8.21 Redis/redis-For-Windows:3.0.2 Redis/redis-For-Windows:2.8.21 Redis/redis-For-Windows:2.8.20 Redis/redis-For-Windows:3.0.1 Redis/redis-For-Windows:win-2.8.19.1 Redis/redis-For-Windows:3.0.0 Redis/redis-For-Windows:3.0.0-rc6 Redis/redis-For-Windows:3.0.0-rc5 Redis/redis-For-Windows:win-2.8.19-rc1 Redis/redis-For-Windows:win-2.8.19 Redis/redis-For-Windows:3.0.0-rc4 Redis/redis-For-Windows:win-2.8.19-pre2 Redis/redis-For-Windows:win-2.8.17.4 Redis/redis-For-Windows:3.0.0-rc3 Redis/redis-For-Windows:3.0.0-rc2 Redis/redis-For-Windows:win-2.8.17.3 Redis/redis-For-Windows:win-2.8.19-pre1 Redis/redis-For-Windows:win-2.8.17.2 Redis/redis-For-Windows:win-2.8.17.1 Redis/redis-For-Windows:2.8.19 Redis/redis-For-Windows:2.8.18 Redis/redis-For-Windows:3.0.0-rc1 Redis/redis-For-Windows:2.8.17 Redis/redis-For-Windows:2.8.16 Redis/redis-For-Windows:2.8.15 Redis/redis-For-Windows:win2.8.12 Redis/redis-For-Windows:2.8.14 Redis/redis-For-Windows:3.0.0-beta8 Redis/redis-For-Windows:2.8.13 Redis/redis-For-Windows:3.0.0-beta7 Redis/redis-For-Windows:win_2.8.9 Redis/redis-For-Windows:2.8.12 Redis/redis-For-Windows:2.8.11 Redis/redis-For-Windows:3.0.0-beta6 Redis/redis-For-Windows:2.8.10 Redis/redis-For-Windows:3.0.0-beta5 Redis/redis-For-Windows:3.0.0-beta4 Redis/redis-For-Windows:2.6.14_msopentech Redis/redis-For-Windows:2.8.4_msopentech Redis/redis-For-Windows:msopentech_2.8.4 Redis/redis-For-Windows:3.0.0-beta3 Redis/redis-For-Windows:2.8.9 Redis/redis-For-Windows:2.8.8 Redis/redis-For-Windows:3.0.0-beta2 Redis/redis-For-Windows:2.8.7 Redis/redis-For-Windows:2.8.6 Redis/redis-For-Windows:3.0.0-beta1 Redis/redis-For-Windows:2.8.5 Redis/redis-For-Windows:2.8.4 Redis/redis-For-Windows:2.8.3 Redis/redis-For-Windows:2.8.2 Redis/redis-For-Windows:2.8.1 Redis/redis-For-Windows:2.8.0 Redis/redis-For-Windows:2.8.0-rc6 Redis/redis-For-Windows:2.8.0-rc5 Redis/redis-For-Windows:2.8.0-rc4 Redis/redis-For-Windows:2.8.0-rc3 Redis/redis-For-Windows:2.8.0-rc2 Redis/redis-For-Windows:2.8.0-rc1 Redis/redis-For-Windows:2.6.14 Redis/redis-For-Windows:Win2.6.8 Redis/redis-For-Windows:2.6.13 Redis/redis-For-Windows:2.6.12 Redis/redis-For-Windows:2.6.11 Redis/redis-For-Windows:2.6.10 Redis/redis-For-Windows:2.6.9 Redis/redis-For-Windows:2.6.8 Redis/redis-For-Windows:2.6.7 Redis/redis-For-Windows:2.6.6 Redis/redis-For-Windows:2.6.5 Redis/redis-For-Windows:2.6.4 Redis/redis-For-Windows:2.6.3 Redis/redis-For-Windows:2.6.2 Redis/redis-For-Windows:2.6.1 Redis/redis-For-Windows:2.6.0 Redis/redis-For-Windows:2.6.0-rc8 Redis/redis-For-Windows:2.6.0-rc7 Redis/redis-For-Windows:2.6.0-rc6 Redis/redis-For-Windows:2.6.0-rc5 Redis/redis-For-Windows:2.6.0-rc4 Redis/redis-For-Windows:2.6.0-rc3 Redis/redis-For-Windows:2.6.0-rc2 Redis/redis-For-Windows:2.6.0-rc1 Redis/redis-For-Windows:2.4.11 Redis/redis-For-Windows:2.4.10 Redis/redis-For-Windows:2.4.9 Redis/redis-For-Windows:2.4.8 Redis/redis-For-Windows:2.4.6_win32 Redis/redis-For-Windows:2.4.7 Redis/redis-For-Windows:2.4.6 Redis/redis-For-Windows:2.4.5 Redis/redis-For-Windows:2.4.4 Redis/redis-For-Windows:2.4.3 Redis/redis-For-Windows:2.4.2 Redis/redis-For-Windows:2.4.1 Redis/redis-For-Windows:2.4.0 Redis/redis-For-Windows:2.4.0-rc8 Redis/redis-For-Windows:2.4.0-rc7 Redis/redis-For-Windows:2.4.0-rc6 Redis/redis-For-Windows:2.4.0-rc5 Redis/redis-For-Windows:2.4.0-rc4 Redis/redis-For-Windows:2.4.0-rc3 Redis/redis-For-Windows:with-deprecated-diskstore Redis/redis-For-Windows:2.4.0-rc2 Redis/redis-For-Windows:2.4.0-rc1 Redis/redis-For-Windows:2.2.4 Redis/redis-For-Windows:2.2.3 Redis/redis-For-Windows:3.0-alpha0 Redis/redis-For-Windows:2.2.2 Redis/redis-For-Windows:2.2.1 Redis/redis-For-Windows:2.2.0 Redis/redis-For-Windows:2.2.0-rc4 Redis/redis-For-Windows:2.3-alpha0 Redis/redis-For-Windows:2.2.0-rc3 Redis/redis-For-Windows:2.2.0-rc2 Redis/redis-For-Windows:2.2.0-rc1 Redis/redis-For-Windows:2.2-alpha6 Redis/redis-For-Windows:2.2-alpha5 Redis/redis-For-Windows:2.2-alpha4 Redis/redis-For-Windows:2.2-alpha3 Redis/redis-For-Windows:2.2-alpha2 Redis/redis-For-Windows:2.2-alpha1 Redis/redis-For-Windows:2.2-alpha0 Redis/redis-For-Windows:v2.1.1-watch Redis/redis-For-Windows:v2.0.0-rc1 Redis/redis-For-Windows:v1.3.12 Redis/redis-For-Windows:v1.3.11 Redis/redis-For-Windows:v1.3.10 Redis/redis-For-Windows:v1.3.9 Redis/redis-For-Windows:v1.3.8 Redis/redis-For-Windows:v1.3.7 Redis/redis-For-Windows:1.3.6 Redis/redis-For-Windows:vm-playpen
..
compare: Redis/redis-For-Windows:2.4.6
Branches Tags
Redis/redis-For-Windows:win-5.0 Redis/redis-For-Windows:develop Redis/redis-For-Windows:win-6.2
Redis/redis-For-Windows:5.0.14.1 Redis/redis-For-Windows:v5.0.14.1 Redis/redis-For-Windows:v5.0.14 Redis/redis-For-Windows:v5.0.10 Redis/redis-For-Windows:v5.0.9 Redis/redis-For-Windows:v4.0.14.2 Redis/redis-For-Windows:v4.0.14.1 Redis/redis-For-Windows:v4.0.14 Redis/redis-For-Windows:v4.0.2.3-alpha Redis/redis-For-Windows:v4.0.2.2-alpha Redis/redis-For-Windows:v4.0.2.1-alpha Redis/redis-For-Windows:v4.0.2-alpha Redis/redis-For-Windows:win-3.2.100 Redis/redis-For-Windows:win-3.0.504 Redis/redis-For-Windows:win-3.0.503 Redis/redis-For-Windows:win-2.8.2402 Redis/redis-For-Windows:win-3.0.502 Redis/redis-For-Windows:win-2.8.2401 Redis/redis-For-Windows:3.2.1 Redis/redis-For-Windows:win-3.2.0000-preview Redis/redis-For-Windows:3.2.0 Redis/redis-For-Windows:3.2.0-rc3 Redis/redis-For-Windows:3.2.0-rc2 Redis/redis-For-Windows:win-2.8.2400 Redis/redis-For-Windows:win-3.0.501 Redis/redis-For-Windows:3.2-rc1 Redis/redis-For-Windows:3.0.6 Redis/redis-For-Windows:2.8.24 Redis/redis-For-Windows:win-3.0.500 Redis/redis-For-Windows:win-3.0.500-rc2 Redis/redis-For-Windows:win-3.0.500-rc1 Redis/redis-For-Windows:win-2.8.2104 Redis/redis-For-Windows:3.0.5 Redis/redis-For-Windows:2.8.23 Redis/redis-For-Windows:win-3.0.300-beta1 Redis/redis-For-Windows:win-2.8.2103 Redis/redis-For-Windows:2.8.22 Redis/redis-For-Windows:3.0.4 Redis/redis-For-Windows:win-3.0.300-alpha3 Redis/redis-For-Windows:win-2.8.2102 Redis/redis-For-Windows:win-3.0.300-alpha2 Redis/redis-For-Windows:win-3.0.100-alpha1 Redis/redis-For-Windows:3.0.3 Redis/redis-For-Windows:win-2.8.2101 Redis/redis-For-Windows:win-2.8.21 Redis/redis-For-Windows:3.0.2 Redis/redis-For-Windows:2.8.21 Redis/redis-For-Windows:2.8.20 Redis/redis-For-Windows:3.0.1 Redis/redis-For-Windows:win-2.8.19.1 Redis/redis-For-Windows:3.0.0 Redis/redis-For-Windows:3.0.0-rc6 Redis/redis-For-Windows:3.0.0-rc5 Redis/redis-For-Windows:win-2.8.19-rc1 Redis/redis-For-Windows:win-2.8.19 Redis/redis-For-Windows:3.0.0-rc4 Redis/redis-For-Windows:win-2.8.19-pre2 Redis/redis-For-Windows:win-2.8.17.4 Redis/redis-For-Windows:3.0.0-rc3 Redis/redis-For-Windows:3.0.0-rc2 Redis/redis-For-Windows:win-2.8.17.3 Redis/redis-For-Windows:win-2.8.19-pre1 Redis/redis-For-Windows:win-2.8.17.2 Redis/redis-For-Windows:win-2.8.17.1 Redis/redis-For-Windows:2.8.19 Redis/redis-For-Windows:2.8.18 Redis/redis-For-Windows:3.0.0-rc1 Redis/redis-For-Windows:2.8.17 Redis/redis-For-Windows:2.8.16 Redis/redis-For-Windows:2.8.15 Redis/redis-For-Windows:win2.8.12 Redis/redis-For-Windows:2.8.14 Redis/redis-For-Windows:3.0.0-beta8 Redis/redis-For-Windows:2.8.13 Redis/redis-For-Windows:3.0.0-beta7 Redis/redis-For-Windows:win_2.8.9 Redis/redis-For-Windows:2.8.12 Redis/redis-For-Windows:2.8.11 Redis/redis-For-Windows:3.0.0-beta6 Redis/redis-For-Windows:2.8.10 Redis/redis-For-Windows:3.0.0-beta5 Redis/redis-For-Windows:3.0.0-beta4 Redis/redis-For-Windows:2.6.14_msopentech Redis/redis-For-Windows:2.8.4_msopentech Redis/redis-For-Windows:msopentech_2.8.4 Redis/redis-For-Windows:3.0.0-beta3 Redis/redis-For-Windows:2.8.9 Redis/redis-For-Windows:2.8.8 Redis/redis-For-Windows:3.0.0-beta2 Redis/redis-For-Windows:2.8.7 Redis/redis-For-Windows:2.8.6 Redis/redis-For-Windows:3.0.0-beta1 Redis/redis-For-Windows:2.8.5 Redis/redis-For-Windows:2.8.4 Redis/redis-For-Windows:2.8.3 Redis/redis-For-Windows:2.8.2 Redis/redis-For-Windows:2.8.1 Redis/redis-For-Windows:2.8.0 Redis/redis-For-Windows:2.8.0-rc6 Redis/redis-For-Windows:2.8.0-rc5 Redis/redis-For-Windows:2.8.0-rc4 Redis/redis-For-Windows:2.8.0-rc3 Redis/redis-For-Windows:2.8.0-rc2 Redis/redis-For-Windows:2.8.0-rc1 Redis/redis-For-Windows:2.6.14 Redis/redis-For-Windows:Win2.6.8 Redis/redis-For-Windows:2.6.13 Redis/redis-For-Windows:2.6.12 Redis/redis-For-Windows:2.6.11 Redis/redis-For-Windows:2.6.10 Redis/redis-For-Windows:2.6.9 Redis/redis-For-Windows:2.6.8 Redis/redis-For-Windows:2.6.7 Redis/redis-For-Windows:2.6.6 Redis/redis-For-Windows:2.6.5 Redis/redis-For-Windows:2.6.4 Redis/redis-For-Windows:2.6.3 Redis/redis-For-Windows:2.6.2 Redis/redis-For-Windows:2.6.1 Redis/redis-For-Windows:2.6.0 Redis/redis-For-Windows:2.6.0-rc8 Redis/redis-For-Windows:2.6.0-rc7 Redis/redis-For-Windows:2.6.0-rc6 Redis/redis-For-Windows:2.6.0-rc5 Redis/redis-For-Windows:2.6.0-rc4 Redis/redis-For-Windows:2.6.0-rc3 Redis/redis-For-Windows:2.6.0-rc2 Redis/redis-For-Windows:2.6.0-rc1 Redis/redis-For-Windows:2.4.11 Redis/redis-For-Windows:2.4.10 Redis/redis-For-Windows:2.4.9 Redis/redis-For-Windows:2.4.8 Redis/redis-For-Windows:2.4.6_win32 Redis/redis-For-Windows:2.4.7 Redis/redis-For-Windows:2.4.6 Redis/redis-For-Windows:2.4.5 Redis/redis-For-Windows:2.4.4 Redis/redis-For-Windows:2.4.3 Redis/redis-For-Windows:2.4.2 Redis/redis-For-Windows:2.4.1 Redis/redis-For-Windows:2.4.0 Redis/redis-For-Windows:2.4.0-rc8 Redis/redis-For-Windows:2.4.0-rc7 Redis/redis-For-Windows:2.4.0-rc6 Redis/redis-For-Windows:2.4.0-rc5 Redis/redis-For-Windows:2.4.0-rc4 Redis/redis-For-Windows:2.4.0-rc3 Redis/redis-For-Windows:with-deprecated-diskstore Redis/redis-For-Windows:2.4.0-rc2 Redis/redis-For-Windows:2.4.0-rc1 Redis/redis-For-Windows:2.2.4 Redis/redis-For-Windows:2.2.3 Redis/redis-For-Windows:3.0-alpha0 Redis/redis-For-Windows:2.2.2 Redis/redis-For-Windows:2.2.1 Redis/redis-For-Windows:2.2.0 Redis/redis-For-Windows:2.2.0-rc4 Redis/redis-For-Windows:2.3-alpha0 Redis/redis-For-Windows:2.2.0-rc3 Redis/redis-For-Windows:2.2.0-rc2 Redis/redis-For-Windows:2.2.0-rc1 Redis/redis-For-Windows:2.2-alpha6 Redis/redis-For-Windows:2.2-alpha5 Redis/redis-For-Windows:2.2-alpha4 Redis/redis-For-Windows:2.2-alpha3 Redis/redis-For-Windows:2.2-alpha2 Redis/redis-For-Windows:2.2-alpha1 Redis/redis-For-Windows:2.2-alpha0 Redis/redis-For-Windows:v2.1.1-watch Redis/redis-For-Windows:v2.0.0-rc1 Redis/redis-For-Windows:v1.3.12 Redis/redis-For-Windows:v1.3.11 Redis/redis-For-Windows:v1.3.10 Redis/redis-For-Windows:v1.3.9 Redis/redis-For-Windows:v1.3.8 Redis/redis-For-Windows:v1.3.7 Redis/redis-For-Windows:1.3.6 Redis/redis-For-Windows:vm-playpen
149 Commits
2.4.0-rc3 .. 2.4.6
Author SHA1 Message Date
antirez e1849b6456 Redis 2.4.6 2012-01-11 20:30:55 +01:00
antirez d0f37e247c error in comment fixed 2012-01-11 20:25:41 +01:00
antirez 3b67a0f905 show GCC version in INFO output. 2012-01-10 18:39:32 +01:00
antirez 2c2b6159d3 Regression test for the main problem causing issue #141. Minor changes/fixes/additions to the test suite itself needed to write the test. 2012-01-07 12:52:42 +01:00
antirez d4945b253b Redis test: when assertion fails print not just the expression but also expanded values in the error message. 2012-01-07 12:52:32 +01:00
antirez 53785789a0 Regression tests for protocol desync bug related to Issue #141 2012-01-07 12:51:22 +01:00
antirez e7b85b3315 Protections against protocol desyncs, leading to infinite query buffer growing, due to nul-terms in specific bytes of the request or indefinitely long multi bulk or bulk count strings without newlines. This bug is related to Issue #141 as well. 2012-01-07 12:50:44 +01:00
antirez a5045d552c Fixed replication when multiple slaves are attaching at the same time. The output buffer was not copied correctly between slaves. This fixes issue #141. 2012-01-07 12:48:10 +01:00
antirez 8939ff1230 Do not propagate DEBUG LOADAOF 2012-01-07 12:47:10 +01:00
antirez 8b102e041a List connected slaves with ip,port,state information in INFO, as requested by github issue #219 2012-01-07 12:44:27 +01:00
antirez 273f41023d Redis 2.4.5 2011-12-23 09:43:24 +01:00
antirez 5bb2565959 Added regression test for ZUNIONSTORE creating NaN (github issue #264) 2011-12-23 09:34:41 +01:00
antirez 0692d060b3 Prevent NaN scores in sorted sets resulting from calls to ZUNIONSTORE and ZINTERSTORE. 2011-12-23 09:28:51 +01:00
antirez 2e689217d5 Redis test: vaoid two false positives while running under valgrind. 2011-12-20 10:35:33 +01:00
antirez b76088845c Redis test: two redundant tests removed as they tend to create issues when running the test with valgrind. 2011-12-20 10:35:23 +01:00
antirez 8a625029e7 Redis test port allocation fixed. 2011-12-20 10:34:51 +01:00
antirez 8a6b21da61 unit/introspection.tcl added 2011-12-19 10:22:04 +01:00
antirez c08eb8e74d unit/introspection added among tests executed by default 2011-12-19 10:19:12 +01:00
antirez 8b860b36b3 Fixed memleak in CLIENT INFO, added simple test that will work as regression test on mac os x and in the CI when running over valgrind. This fixes issue #256 2011-12-19 10:19:08 +01:00
antirez 61e44f614d added assertion in zslInsert() that ensures the inserted element score is not NaN 2011-12-18 11:13:13 +01:00
antirez bc62bc5eac Fixed SORT bugs (issue #224) with regression tests. 2011-12-01 16:08:30 +01:00
antirez 67d6b29404 Replication bug fixed: now non blocking connect is also forced to follow the configured replication timeout. 2011-11-30 17:13:15 +01:00
antirez 901120f8e6 --quiet option implemented in the Redis test. Output improved a bit. 2011-11-30 09:36:30 +01:00
antirez 54a2934f10 Redis 2.4.4 2011-11-28 11:22:08 +01:00
antirez a675e5b171 two new AOF related INFO fields that can be interesting information for debugging. 2011-11-28 11:11:13 +01:00
antirez 0f2199ec29 show initial querybuf bytes on querybuf overflow. 2011-11-25 17:08:25 +01:00
antirez 241e3cff83 jemalloc gitignore updated to jemalloc 2.2.5 2011-11-25 16:48:07 +01:00
jbergstroem a9c2f0f28a Update to jemalloc 2.2.5 2011-11-25 16:47:55 +01:00
antirez 7558b1fe4d log client protocol errors for log level >= verbose 2011-11-25 16:09:16 +01:00
antirez fcf5783632 Added regression tests for issue #209 2011-11-25 12:27:00 +01:00
woowenjie 6532df2e7e clean REDIS_DIRTY_CAS when discard Command runs. otherwise the next MULTI/EXEC may fail in the same RedisClient 2011-11-25 12:13:57 +01:00
antirez 8a82ee0f3c better bug report info on crash (backported from unstable) 2011-11-24 15:53:35 +01:00
antirez f4e2abfcd4 minor refactoring to networking.c adding a separated function to get a string representing the current state of all the connected clients. 2011-11-24 15:47:55 +01:00
antirez 3852e2a831 last executed command in CLIENT LIST output. 2011-11-24 14:56:43 +01:00
antirez 6a252e9498 Redis 2.4.3 2011-11-21 18:47:39 +01:00
antirez 5d5e68a486 Fixed bug in getClientInfoString() that was not rendering the N (no flags) special flag correctly. 2011-11-21 18:37:22 +01:00
antirez a6e4627438 Close client connection and log the event when the client input buffer reaches 1GB. 2011-11-21 18:37:17 +01:00
antirez 93babe4b41 show active events in client file descriptor in CLIENT LIST. 2011-11-21 18:37:05 +01:00
antirez 713b4c5401 new ae.c API to get current events by file descriptor. 2011-11-21 18:37:01 +01:00
antirez 51baa1922b added output list and buffer length, query buffer size, to CLIENT LIST output. 2011-11-21 18:36:56 +01:00
antirez f3ac12aac2 code generating the CLIENT LIST output refactored to have a function that is able to render a single client into a client info string. 2011-11-21 18:36:41 +01:00
antirez 0b1531e50b Added sdscatsds() to sds.c/h 2011-11-21 18:32:29 +01:00
antirez b1b98c64cc Max log message length set to 4k to avoid truncated INFO output in logs on Redis crash. This is just a workaround and it is handled better in the unstable branch. 2011-11-21 18:32:08 +01:00
Chris Lamb cec6b04842 Fix crash on SPARC due to improper alignment of allocated memory 
I believe that you should be able to drop 'defined(__sun)' completely
from this clause, as Solaris on x86 hardware probably does not have
strict alignment requirements, but I don't have a way to test that.

Thanks to Jurij Smakov <jurij@wooyd.org>.

Signed-off-by: Chris Lamb <lamby@debian.org>
 2011-11-16 18:24:06 +01:00
antirez beb4bacce6 I/O buffer length enlarged 2011-11-11 17:19:57 +01:00
antirez 3fac86ff1d set default client timeout to zero inside redis.h as well 2011-11-11 17:16:03 +01:00
antirez 18fe946f6a use timeout 0 by default, as this is a common source of problems. 2011-10-31 15:43:23 +01:00
antirez 9e68987f53 added more clarifications in redis.conf about ther right value to set as replication timeout. 2011-10-31 11:17:40 +01:00
Salvatore Sanfilippo 7e11ca0ae9 Merge pull request #149 from hgfischer/2.4 
repl_ping_slave_period and repl_timeout configs
 2011-10-31 03:11:17 -07:00
antirez d1387aa64d It is now possible to disable password authentication via CONFIG SET. Thanks to Jan Oberst. This closes issue #171. 2011-10-31 09:57:18 +01:00
antirez 7d31a020ea Fixed a memory leak in redis-cli.c 2011-10-28 17:43:43 +02:00
antirez cacab3b1c8 sds.c single quotes support 2011-10-28 17:43:39 +02:00
antirez 072a9052c7 Redis 2.4.2 2011-10-26 16:16:33 +02:00
Salvatore Sanfilippo 00af23bcbf Merge pull request #154 from florean/2.4 
2.4
 2011-10-26 04:01:01 -07:00
antirez 15c9497280 Clear the AOF rewrite scheduled flag once an AOF rewrite is triggered. Fix for issue #161, probably fixing 159 as well. 2011-10-26 12:55:09 +02:00
antirez 45bd4a1853 Better installation info inside README file. 2011-10-23 12:28:31 +02:00
antirez 0327d9b8e9 removed a few entries for gitignore 2011-10-23 12:16:30 +02:00
antirez 1e81d971dc template files for installation script added 2011-10-23 12:15:02 +02:00
antirez 9a3a84bfdf Script to install the server in Ubuntu/Debian added to 2.4 as well. 2011-10-23 11:49:58 +02:00
Nathan Florea 1126d5aa49 Set errno to 0 before calling strtol. 2011-10-20 18:20:58 -07:00
Herbert G. Fischer 1fe4cd5de9 fixed source indenting 2011-10-18 19:03:18 -02:00
Herbert G. Fischer 421f6afd3d fixed typo in comments 2011-10-18 19:01:30 -02:00
Herbert G. Fischer 7c6da7327f Added repl_ping_slave_period and repl_timeout options support 2011-10-18 18:56:10 -02:00
antirez b6fe152b83 Fix to the previous fix for issue #145, make sure to return only after the event was unregistered. 2011-10-18 17:12:16 +02:00
smly 1270a13667 Fixed a small memory leak I found with valgrind 2011-10-18 14:13:47 +02:00
antirez 98f5abb6e3 Return from syncWithMaster() ASAP if the event fired but the instance is no longer a slave. This should fix Issue #145. 2011-10-18 11:09:32 +02:00
antirez ab648d973d 2.4.1 release notes. 2011-10-17 10:46:00 +02:00
antirez 5ab72bedb7 version bumped to 2.4.1 2011-10-17 10:43:02 +02:00
antirez 9a83d13fb5 revert version bumping to 2.4.1, put in the previous commit for an error. 2011-10-17 10:42:47 +02:00
antirez 6662806a26 Regression test for issue #142 added 2011-10-17 10:40:11 +02:00
antirez 6e3f1a9e47 FLUSHALL now prevents rdbSave() from resetting the dirty counter, so that the command will get replicated and put inside the AOF. This fixes issue #142 2011-10-17 10:28:57 +02:00
antirez d12f26c231 FLUSHALL will only perform a blocking SAVE if RDB persistence is configured. 2011-10-17 10:25:35 +02:00
antirez a502f48320 2.4.0 Release notes. 2011-10-14 17:54:25 +02:00
antirez 0c71939c05 Version bumped to 2.4.0 2011-10-14 17:49:20 +02:00
antirez a4f46f211c Exit with Fatal error at startup if the RDB file signature or version is wrong. 
Ref: issue #103
 2011-10-14 17:04:24 +02:00
antirez 8185c2e9f1 redis-check-dump: RDB version 2 now supported. 2011-10-14 16:52:11 +02:00
antirez 71bb31bf16 More informative error when DEBUG RELOAD fails. 2011-10-14 14:30:41 +02:00
antirez c7d23f3ed1 Fix for issue #132. Now AUTH raises an error if no server password is configured. 2011-10-10 22:22:27 +02:00
Nathan Florea 09cefcfbb3 Added a config directive for a Unix socket mask 
Added a configuration directive to allow a user to specify the
permissions to be granted to the Unix socket file.  I followed
the format Pieter and Salvatore discusses in issue #85 (
https://github.com/antirez/redis/issues/85).
 2011-10-10 22:08:11 +02:00
Nathan Florea 1ca8deebef Unlink Unix socket file on shutdown 2011-10-10 22:08:03 +02:00
antirez b7f667b553 Issue #131. stime/utime reported in INFO was inverted. Fixed thanks to Didier Spezia. 2011-10-10 15:29:36 +02:00
antirez 067ad37f96 Fix for bug #128 about the RENAME command. 2011-10-10 15:25:28 +02:00
antirez a6e312bf9a Added two new tests for RENAME, currently both will fail because of bug #128. 2011-10-10 15:25:22 +02:00
antirez 0e841b998a CONFIG SET/GET support for loglevel 2011-10-06 15:25:53 +02:00
antirez cd0458470e Redis test ports selection made more robust. This prevents the test from hanging if an already bound port is selected but the TCP server listening to it does not cause a protocol error with a Redis client PING. Also base port moved away from the range near to the Redis Cluster gossip ports. 2011-10-04 10:07:29 +02:00
antirez 678dbc48fd Compilation fixed on OpenBSD making sure that _XOPEN_SOURCE is set to 700 in fmacros.h 2011-09-29 10:20:26 +02:00
Juri M. Vainonen 3a91aca033 fixed a small bug that caused redis-cli to segfault when given single numeric parameter greater that zero. 2011-09-21 22:33:01 +02:00
antirez 9286ecd767 Release notes for Redis 2.4 RC8 added 2011-09-21 14:05:36 +02:00
antirez 902a656f7a Version bumped to 2.3.11 2011-09-21 13:56:03 +02:00
antirez a58e486d79 no longer valid design documents removed 2011-09-20 11:15:18 +02:00
antirez b0fcf23304 hash zipmap max entries and entry size information removed from INFO command since we have this info and the equivalent info for sets, hashes, zset accessible via the CONFIG GET command. 2011-09-20 11:08:08 +02:00
vambo 57f611d953 Corrected typo in documentation for R_Zero, R_PosInf, R_NegInf, R_Nan; 2011-09-20 11:01:20 +02:00
Konstantin Tokarev 20c26321c2 Fixed compilation on Mac/PPC 
Applied patch from Issue 119 provided by
pahowes@gmail.com
 2011-09-20 10:59:47 +02:00
Konstantin Merenkov 2017d8baf7 [issue 620] don't segfault if number of arguments is less than 1 2011-09-20 10:56:43 +02:00
antirez a511ce1b49 Include port number in error when can't bind lisetning port on startup. 2011-09-20 10:48:33 +02:00
antirez 1f164f4e1d Fixed comment typo 2011-09-19 17:49:50 +02:00
antirez 9f775040c8 Comment out things in bio.c that are currently not useful but that may be useful in the future. 2011-09-19 17:44:21 +02:00
antirez 6ca793fc21 Emit a log message when AOF fsync is still in progress but we are forced to write from the main thread since two seconds already elapsed. 2011-09-19 17:44:17 +02:00
antirez 01ac844ba2 use aof_fsync wrapper instead of fsync(2) in bio.c 2011-09-19 17:44:11 +02:00
antirez 79b9ec53d1 fixed flushAppendOnlyFile() function prototype to reflect the new implementation, fixing compilation. 2011-09-19 17:43:56 +02:00
antirez 4c2d5f0980 postpone the AOF fsync if policy is everysec and there is a background fsync already going. 2011-09-19 17:43:49 +02:00
antirez 056c2acaea Fixed typo preventing compilation 2011-09-19 17:43:44 +02:00
antirez f85c383b6c Added aof_backgronud_fsync() function, and use it in the bacground rewrite done handler when the fsync policy is everysec. 2011-09-19 17:43:40 +02:00
antirez 3ae56d8458 REDIS_BIO_AOF_FSYNC implemented 2011-09-19 17:43:34 +02:00
antirez 0807f5b9ff fixed a problem in bioOlderJobType() when there are no jobs of the specified type. Now the function returns 0 when this happens. 2011-09-19 17:43:28 +02:00
antirez 2eae7cbc7d some more comment about bio.c design 2011-09-19 17:43:23 +02:00
antirez 693618f1c1 Remove backgroud jobs from the queue only when the processing was completed, and not just started. 2011-09-19 17:43:19 +02:00
antirez c67e8d3bf6 Use a different thread for every different type of background job 2011-09-19 17:43:13 +02:00
antirez 411cb75bcd Better comments for bioWaitPendingJobsLE 2011-09-19 17:43:07 +02:00
antirez 794419192e Background I/O library enhanced so that the main thread can query for the number of pending jobs of the specified type. 2011-09-19 17:43:01 +02:00
antirez 481a05cd23 commenting out aof-race test for now, waiting for a fix to issue #80 2011-09-16 15:43:27 +02:00
Pieter Noordhuis b9a65682fb Failing test related to AOF rewrite buffers 2011-09-16 11:24:08 +02:00
Pieter Noordhuis e3d1c8b080 Enforce upper bound for number of requests in redis-benchmark 2011-09-16 11:23:53 +02:00
Pieter Noordhuis 682b63312e Fix AOF race that may duplicate commands 2011-09-16 11:14:00 +02:00
antirez 6fd1a72712 Fixed a bug in the average latency metering of redis-cli --latency 2011-09-15 19:32:49 +02:00
antirez 2e48ad5422 Implemented --latency in redis-cli 2011-09-15 19:28:24 +02:00
antirez 67f594f9b5 Optimize LRANGE to scan the list starting from the head or the tail in order to traverse the minimal number of elements. Thanks to Didier Spezia for noticing the problem and providing a patch. 2011-09-14 15:17:04 +02:00
antirez b7bf29059e Don't replicate/AOF SAVE 2011-09-14 13:08:19 +02:00
antirez 889d468e51 use bio.c instead of libeio for closing the AOF file in background. Some comment added and other minor changes. 2011-09-14 10:52:07 +02:00
Pieter Noordhuis 84f48cc401 Asynchronously close old file after BGREWRITEAOF 2011-09-14 10:51:52 +02:00
antirez 44690ab15f make bio_list static as well 2011-09-14 10:50:29 +02:00
antirez a72c0423b4 debug message removed 2011-09-14 10:50:22 +02:00
antirez c8d0013f38 use pthread_cond_signal() to make sure the background thread will process the job. 2011-09-14 10:47:53 +02:00
antirez 5ca75751ed typo fixed in bio.c 2011-09-14 10:47:48 +02:00
antirez af14408e5c add background jobs initialization to Redis main init function 2011-09-14 10:47:31 +02:00
antirez ed56ad5689 bio.c typos fixes to make it compile 2011-09-14 10:47:26 +02:00
antirez c427f5d3c1 bio.o added to redis-server objects 2011-09-14 10:47:19 +02:00
Pieter Noordhuis 325d2eb8b5 Speed up protocol synthesis for AOF 2011-09-13 12:26:53 +02:00
Pieter Noordhuis 0fb6212cf4 Re-use AOF buffer when it is small enough 2011-09-13 12:26:49 +02:00
Pieter Noordhuis f602318349 Fix indent 2011-09-13 12:26:43 +02:00
antirez 2401c3eccb added a newline for code readability 2011-09-13 12:26:38 +02:00
Pieter Noordhuis 8d9804bf40 Avoid unnecessary calls to time(3) 2011-09-13 12:26:33 +02:00
antirez da30c9e1e3 Version bumped to 2.3.10, changelog updated 2011-09-12 12:13:44 +02:00
antirez 3c51d3b36b Remove the write handler only if there are no longer objects in the output queue AND if the static buffer is empty. This bug was the cause of a possible server-stop-responding-to-client bug under some specific work load. Thanks to Pieter Noordhuis for spotting and fixing it. 2011-09-12 11:06:11 +02:00
antirez 00c7f312cc regression test for bug 593 added 2011-09-12 10:54:43 +02:00
antirez b10913b93d fixed typos in the comments of rpoplpushHandlePush() 2011-09-12 10:05:38 +02:00
Hampus Wessman cbf01c0e01 Fix crash when chaining brpoplpush with other blocking commands. 2011-09-12 10:05:32 +02:00
antirez 214cf63730 Version bumped to 2.3.9. Changelog modified accordingly. 2011-08-09 12:04:26 +02:00
antirez 4a4f4cb5ff Auto rewrite integer overflow fixed. Thanks to Gaurav for finding it (see issue 602 on google code), and Pieter Noordhuis for tracing it. 2011-08-09 11:45:57 +02:00
antirez a57bd791e9 243b783fdc 2011-07-29 15:50:15 +02:00
antirez ed9f997ce5 fixed assert proc on Redis test 2011-07-29 15:49:31 +02:00
antirez 55eea93cae Release notes for 2.4.0-RC5 2011-07-29 15:44:21 +02:00
antirez 753e91b6c1 Version bumped to 2.3.8 2011-07-29 15:39:31 +02:00
Pieter Noordhuis a830d941a7 Fix adding bulk reply when getcwd fails 2011-07-27 16:22:47 +02:00
Pieter Noordhuis 9c3a89853c HDEL: Abort deleting fields when hash is removed 2011-07-27 12:30:35 +02:00
antirez 7ca6672338 version bumped to 2.3.7 2011-07-22 16:26:24 +02:00
antirez 6c68df4f32 Close the listening sockets. Apparently this allows faster restarts. 2011-07-22 12:12:45 +02:00
antirez 8b58723eb2 prepareForShutdown() fixed for correctness regarding what to do with AOF and RDB persistence on exit. 2011-07-22 11:55:34 +02:00
Salvatore Sanfilippo abd4cd7827 Added a 'runtest' script that is responsible to check if Tcl is available and run the test. This is invoked from Makefile as well. 2011-07-15 17:24:44 +02:00
82 changed files with 2765 additions and 2083 deletions
Expand all files Collapse all files
.gitignore
-2
View File
@@ -9,12 +9,10 @@ redis-check-aof
doc-tools
mkrelease.sh
release
myredis.conf
misc/*
src/release.h
appendonly.aof
SHORT_TERM_TODO
redis.conf.*
release.h
src/transfer.sh
src/configs
00-RELEASENOTES
+124 -6
View File
@@ -7,7 +7,8 @@ Redis 2.2 is mostly a strict subset of 2.4.
The only thing you should be aware is that you can't use .rdb and AOF files
generated with 2.4 into a 2.2 instance.
2.4 slaves can be attached to 2.2 masters, but not the contrary.
2.4 slaves can be attached to 2.2 masters, but not the contrary, and only for
the time needed to perform the version upgrade.
From the point of view of the API Redis 2.4 only adds new commands
(other commands now accepts a variable number of arguments) so you don't need
@@ -17,13 +18,130 @@ to modify your program in order to use Redis 2.4.
CHANGELOG
---------
What's new in Redis 2.3.5 (2.4 Release Candidate 1)
What's new in Redis 2.4.6
=========================
* [BUGFIX] Fixed issue #141 part 1: Possible protocol desyncs when clients send
wrong protocol is now fixed. (See issue 141 for more details)
* [BUGFIX] Fixed issue #141 part 2: Connection of multiple slaves used to result
from time to time into corrupted protocol send to slaves connected
after the first one. (See issue 141 for more details)
* [BUGFIX] Do not propagate DEBUG LOADAOF.
* New INFO contains information such as ip/port/state for every conneced slave.
* Show GCC version in INFO output.
What's new in Redis 2.4.5
=========================
* [BUGFIX] Fixed a ZUNIONSTORE/ZINTERSTORE bug that can cause a NaN to be
inserted as a sorted set element score. This happens when one of the
elements has +inf/-inf score and the weight used is 0.
* [BUGFIX] Fixed memory leak in CLIENT INFO.
* [BUGFIX] Fixed a non critical SORT bug (Issue 224).
* [BUGFIX] Fixed a replication bug: now the timeout configuration is respected
during the connection with the master.
* --quiet option implemented in the Redis test.
What's new in Redis 2.4.4
=========================
* [BUGFIX] jemalloc upgraded to version 2.2.5, previous versions had a
potentially serious issue when allocating big memory areas, something that
Redis actually does. However we never received bug reports that appear
to be caused by jemalloc.
* [BUGFIX] DISCARD now clears DIRTY_CAS flag in the client. Now the next
transaction will not fail if the previous transaction used WATCH and
the key was touched.
* CLIENT LIST output modified to include the last command executed by clients.
* Better bug report on crash.
* Protocol errors are now logged for loglevel >= verbose.
* Two new INFO fields related to AOF, that can be useful when investigating
Redis issues.
What's new in Redis 2.4.3
=========================
* redis-cli now supports 'single quotes' style strings.
* It is possible to disable password auth with CONFIG SET.
* Some section of redis.conf better documented.
* Default timeout for client connections now is 0 (no timeout).
* I/O buffer length modified for better performances with big payloads.
* Fixed crash on SPARC due to improper alighment due to bad assumptions about data types size.
* CLIENT LIST output improved, code refactored.
* [BUGFIX] Max log message length set to 4k to avoid truncation in INFO output written in the log file after a crash.
* [BUGFIX] Close client connection when the query buffer reaches 1GB and log informaiton about the offending client, instead of crashing the instance when the query buffer reaches 2GB (for sds.c string overflow). This is related to issue #141 (github issues) and should fix the effect of this rare bug related to replication, but the cause is still not perfectly clear.
What's new in Redis 2.4.2
=========================
* [BUGFIX] Unix socket creation mask config directive fixed.
* [BUGFIX] Fixed a bug that forced Redis to continously rewrite the AOF file
when a manual BGREWRITEAOF was issued during a BGSAVE was
still in progress.
* [BUGFIX] Fixed a server crash resulting from bad handling of SLAVEOF NO ONE.
* [BUGFIX] Fixed a memory leak in redis-cli.
* Debian/Ubuntu Redis system-wide installation script added under /utils.
What's new in Redis 2.4.1
=========================
* [BUGFIX] FLUSHALL was not replicated nor written into the Append Only File.
* [BUGFIX] FLUSHALL now only performs a sync SAVE if there is at least
one save point configured.
What's new in Redis 2.4.0
=========================
* [BUGFIX] redis-cli segfault with single numerical argument fixed.
* [BUGFIX] OpenBSD compilation problem fixed.
* [BUGFIX] More robust Redis test, with better random port selection.
* [BUGFIX] Fix for bug #128 about the RENAME command.
* [BUGFIX] Fixed Issue #131. stime/utime reported in INFO was inverted.
* [BUGFIX] Unlink Unix socket file on shutdown.
* [BUGFIX] AUTH now returns error if no password is set on the server.
* [BUGFIX] Exit with Fatal error at startup on RDB loading errors.
* redis-check-dump: RDB version 2 now supported.
* More informative error when DEBUG RELOAD fails.
* Added a config directive for a Unix socket mask.
* CONFIG SET/GET for loglevel.
What's new in Redis 2.3.11 (2.4 Release Candidate 8)
====================================================
* [BUGFIX] Fixed a rare but possible AOF race condition that could result into
duplicated commands inside the AOF.
* [BUGFIX] Fixed issue 620, don't segfault on corrupted (by hand) AOF.
* [BUGFIX] Fixed compilation on Mac/PPC.
* [BUGFIX] Don't replicate SAVE.
* LRANGE optimization may drastically improve performances when querying the
final part of a long list.
* redis-cli now implements a --latency mode to monitory Redis delay.
* Hash type settings removed from INFO (same info is available via config GET)
* Include port number on error when can't bind.
* AOF fsync is now performed in background when fsync policy is 'everysec'.
* AOF performances improved moving in background a possibly slow close(2) call.
* AOF protocol synthesis speedup.
What's new in Redis 2.3.10 (2.4 Release Candidate 7)
====================================================
* [BUGFIX] Fixed issue 593 (BRPOPLPUSH related crash).
* [BUGFIX] Fixed an issue with the networking layer that may prevent Redis from sending the whole reply back to client under extreme conditions.
What's new in Redis 2.3.9 (2.4 Release Candidate 6)
===================================================
This is the first Release Candidate of Redis 2.4, in our experience the
server is very stable, however it is always better to closely monitor your
server if you switch to 2.4 RC1. Also make sure to perform a backup of your
old data set before switching from 2.2 to 2.4.
* [BUGFIX] Fixed a bug with the automatic AOF rewrite causing continuous
rewrites for AOF files bigger than 4 GB.
* New maxmemory tests.
What's new in Redis 2.3.8 (2.4 Release Candidate 5)
===================================================
Compared to Redis 2.3.7 (RC4) there are the following changes:
* [BUGFIX] HDEL: Abort deleting fields when hash is removed.
* [BUGFIX] Fix adding bulk reply when getcwd fails.
WHAT'S NEW IN REDIS 2.4 compared to the 2.2 version?
====================================================
Changelog
-1032
View File
File diff suppressed because it is too large Load Diff
README
+29 -9
View File
@@ -11,11 +11,6 @@ It is as simple as:
% make
Redis is just a single binary, but if you want to install it you can use
the "make install" target that will copy the binary in /usr/local/bin
for default. You can also use "make PREFIX=/some/other/directory install"
if you wish to use a different destination.
You can run a 32 bit Redis binary using:
% make 32bit
@@ -36,13 +31,13 @@ glibc malloc() has memory fragmentation problems.
To force a libc malloc() build use:
make FORCE_LIBC_MALLOC=yes
% make FORCE_LIBC_MALLOC=yes
In all the other non Linux systems the libc malloc() is used by default.
On Mac OS X you can force a jemalloc based build using the following:
make USE_JEMALLOC=yes
% make USE_JEMALLOC=yes
Verbose build
-------------
@@ -50,7 +45,7 @@ Verbose build
Redis will build with a user friendly colorized output by default.
If you want to see a more verbose output use the following:
make V=1
% make V=1
Running Redis
-------------
@@ -90,5 +85,30 @@ You can find the list of all the available commands here:
http://redis.io/commands
Enjoy!
Installing Redis
-----------------
In order to install Redis binaries into /usr/local/bin just use:
% make install
You can use "make PREFIX=/some/other/directory install" if you wish to use a
different destination.
Make install will just install binaries in your system, but will not configure
init scripts and configuration files in the appropriate place. This is not
needed if you want just to play a bit with Redis, but if you are installing
it the proper way for a production system, we have a script doing this
for Ubuntu and Debian systems:
% cd utils
% ./install_server
The script will ask you a few questions and will setup everything you need
to run Redis properly as a background daemon that will start again on
system reboots.
You'll be able to stop and start Redis using the script named
/etc/init.d/redis_<portnumber>, for instance /etc/init.d/redis_6379.
Enjoy!
deps/jemalloc/.gitignore
+19 -11
View File
@@ -1,15 +1,23 @@
/Makefile
/autom4te.cache/
/config.stamp
/config.log
/config.status
/configure
/doc/html.xsl
/doc/jemalloc.xml
/doc/manpages.xsl
/src/*.d
/lib
# The following headers are generated
/include/jemalloc/*.h
/include/jemalloc/internal/jemalloc_internal.h
/test/jemalloc_test.h
/doc/jemalloc.xml
/doc/jemalloc.html
/doc/jemalloc.3
/lib/
/Makefile
/include/jemalloc/internal/jemalloc_internal\.h
/include/jemalloc/jemalloc\.h
/include/jemalloc/jemalloc_defs\.h
/test/jemalloc_test\.h
/src/*.[od]
/test/*.[od]
/test/*.out
/test/[a-z]*
!test/*.c
!test/*.exp
/VERSION
deps/jemalloc/ChangeLog
Vendored
+37
View File
@@ -6,6 +6,43 @@ found in the git revision history:
http://www.canonware.com/cgi-bin/gitweb.cgi?p=jemalloc.git
git://canonware.com/jemalloc.git
* 2.2.5 (November 14, 2011)
Bug fixes:
- Fix huge_ralloc() race when using mremap(2). This is a serious bug that
could cause memory corruption and/or crashes.
- Fix huge_ralloc() to maintain chunk statistics.
- Fix malloc_stats_print(..., "a") output.
* 2.2.4 (November 5, 2011)
Bug fixes:
- Initialize arenas_tsd before using it. This bug existed for 2.2.[0-3], as
well as for --disable-tls builds in earlier releases.
- Do not assume a 4 KiB page size in test/rallocm.c.
* 2.2.3 (August 31, 2011)
This version fixes numerous bugs related to heap profiling.
Bug fixes:
- Fix a prof-related race condition. This bug could cause memory corruption,
but only occurred in non-default configurations (prof_accum:false).
- Fix off-by-one backtracing issues (make sure that prof_alloc_prep() is
excluded from backtraces).
- Fix a prof-related bug in realloc() (only triggered by OOM errors).
- Fix prof-related bugs in allocm() and rallocm().
- Fix prof_tdata_cleanup() for --disable-tls builds.
- Fix a relative include path, to fix objdir builds.
* 2.2.2 (July 30, 2011)
Bug fixes:
- Fix a build error for --disable-tcache.
- Fix assertions in arena_purge() (for real this time).
- Add the --with-private-namespace option. This is a workaround for symbol
conflicts that can inadvertently arise when using static libraries.
* 2.2.1 (March 30, 2011)
Bug fixes:
deps/jemalloc/INSTALL
Vendored
+6
View File
@@ -42,6 +42,12 @@ any of the following arguments (not a definitive list) to 'configure':
jemalloc overlays the default malloc zone, but makes no attempt to actually
replace the "malloc", "calloc", etc. symbols.
--with-private-namespace=<prefix>
Prefix all library-private APIs with <prefix>. For shared libraries,
symbol visibility mechanisms prevent these symbols from being exported, but
for static libraries, naming collisions are a real possibility. By
default, the prefix is "" (empty string).
--with-install-suffix=<suffix>
Append <suffix> to the base name of all installed files, such that multiple
versions of jemalloc can coexist in the same installation directory. For
deps/jemalloc/Makefile.in
Vendored
+2 -2
View File
@@ -136,9 +136,9 @@ doc: $(DOCS)
@objroot@lib/libjemalloc@install_suffix@.$(SO)
@mkdir -p $(@D)
ifneq (@RPATH@, )
$(CC) -o $@ $< @RPATH@@objroot@lib -L@objroot@lib -ljemalloc@install_suffix@
$(CC) -o $@ $< @RPATH@@objroot@lib -L@objroot@lib -ljemalloc@install_suffix@ -lpthread
else
$(CC) -o $@ $< -L@objroot@lib -ljemalloc@install_suffix@
$(CC) -o $@ $< -L@objroot@lib -ljemalloc@install_suffix@ -lpthread
endif
install_bin:
deps/jemalloc/VERSION
Vendored
+1 -1
View File
@@ -1 +1 @@
2.2.1-0-g5ef7abf6d846720fb3fb8c737861c99b5ad1d862
2.2.5-0-gfc1bb70e5f0d9a58b39efa39cc549b5af5104760
deps/jemalloc/configure
Vendored
+33 -7
View File
@@ -617,8 +617,6 @@ enable_prof
enable_stats
enable_debug
install_suffix
jemalloc_cprefix
jemalloc_prefix
AUTOCONF
LD
AR
@@ -705,6 +703,7 @@ with_xslroot
with_rpath
enable_autogen
with_jemalloc_prefix
with_private_namespace
with_install_suffix
enable_cc_silence
enable_debug
@@ -1375,6 +1374,8 @@ Optional Packages:
--with-rpath=<rpath> Colon-separated rpath (ELF systems only)
--with-jemalloc-prefix=<prefix>
Prefix to prepend to all public APIs
--with-private-namespace=<prefix>
Prefix to prepend to all library-private APIs
--with-install-suffix=<suffix>
Suffix to append to all installed files
--with-static-libunwind=<libunwind.a>
@@ -4800,10 +4801,6 @@ _ACEOF
#define JEMALLOC_CPREFIX "$JEMALLOC_CPREFIX"
_ACEOF
jemalloc_prefix="$JEMALLOC_PREFIX"
jemalloc_cprefix="$JEMALLOC_CPREFIX"
cat >>confdefs.h <<_ACEOF
#define JEMALLOC_P(string_that_no_one_should_want_to_use_as_a_jemalloc_API_prefix) ${JEMALLOC_PREFIX}##string_that_no_one_should_want_to_use_as_a_jemalloc_API_prefix
_ACEOF
@@ -4811,6 +4808,31 @@ _ACEOF
fi
# Check whether --with-private_namespace was given.
if test "${with_private_namespace+set}" = set; then :
withval=$with_private_namespace; JEMALLOC_PRIVATE_NAMESPACE="$with_private_namespace"
else
JEMALLOC_PRIVATE_NAMESPACE=""
fi
cat >>confdefs.h <<_ACEOF
#define JEMALLOC_PRIVATE_NAMESPACE "$JEMALLOC_PRIVATE_NAMESPACE"
_ACEOF
if test "x$JEMALLOC_PRIVATE_NAMESPACE" != "x" ; then
cat >>confdefs.h <<_ACEOF
#define JEMALLOC_N(string_that_no_one_should_want_to_use_as_a_jemalloc_private_namespace_prefix) ${JEMALLOC_PRIVATE_NAMESPACE}##string_that_no_one_should_want_to_use_as_a_jemalloc_private_namespace_prefix
_ACEOF
else
cat >>confdefs.h <<_ACEOF
#define JEMALLOC_N(string_that_no_one_should_want_to_use_as_a_jemalloc_private_namespace_prefix) string_that_no_one_should_want_to_use_as_a_jemalloc_private_namespace_prefix
_ACEOF
fi
# Check whether --with-install_suffix was given.
if test "${with_install_suffix+set}" = set; then :
withval=$with_install_suffix; INSTALL_SUFFIX="$with_install_suffix"
@@ -5508,7 +5530,7 @@ fi
if test -d "${srcroot}../.git" ; then
if test -d "${srcroot}.git" ; then
git describe --long --abbrev=40 > ${srcroot}VERSION
fi
jemalloc_version=`cat ${srcroot}VERSION`
@@ -7346,6 +7368,10 @@ $as_echo "abs_objroot : ${abs_objroot}" >&6; }
$as_echo "" >&6; }
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: JEMALLOC_PREFIX : ${JEMALLOC_PREFIX}" >&5
$as_echo "JEMALLOC_PREFIX : ${JEMALLOC_PREFIX}" >&6; }
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: JEMALLOC_PRIVATE_NAMESPACE" >&5
$as_echo "JEMALLOC_PRIVATE_NAMESPACE" >&6; }
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: : ${JEMALLOC_PRIVATE_NAMESPACE}" >&5
$as_echo " : ${JEMALLOC_PRIVATE_NAMESPACE}" >&6; }
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: install_suffix : ${install_suffix}" >&5
$as_echo "install_suffix : ${install_suffix}" >&6; }
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: autogen : ${enable_autogen}" >&5
deps/jemalloc/configure.ac
Vendored
+16 -5
View File
@@ -292,13 +292,22 @@ if test "x$JEMALLOC_PREFIX" != "x" ; then
JEMALLOC_CPREFIX=`echo ${JEMALLOC_PREFIX} | tr "a-z" "A-Z"`
AC_DEFINE_UNQUOTED([JEMALLOC_PREFIX], ["$JEMALLOC_PREFIX"])
AC_DEFINE_UNQUOTED([JEMALLOC_CPREFIX], ["$JEMALLOC_CPREFIX"])
jemalloc_prefix="$JEMALLOC_PREFIX"
jemalloc_cprefix="$JEMALLOC_CPREFIX"
AC_SUBST([jemalloc_prefix])
AC_SUBST([jemalloc_cprefix])
AC_DEFINE_UNQUOTED([JEMALLOC_P(string_that_no_one_should_want_to_use_as_a_jemalloc_API_prefix)], [${JEMALLOC_PREFIX}##string_that_no_one_should_want_to_use_as_a_jemalloc_API_prefix])
fi
dnl Do not mangle library-private APIs by default.
AC_ARG_WITH([private_namespace],
[AS_HELP_STRING([--with-private-namespace=<prefix>], [Prefix to prepend to all library-private APIs])],
[JEMALLOC_PRIVATE_NAMESPACE="$with_private_namespace"],
[JEMALLOC_PRIVATE_NAMESPACE=""]
)
AC_DEFINE_UNQUOTED([JEMALLOC_PRIVATE_NAMESPACE], ["$JEMALLOC_PRIVATE_NAMESPACE"])
if test "x$JEMALLOC_PRIVATE_NAMESPACE" != "x" ; then
AC_DEFINE_UNQUOTED([JEMALLOC_N(string_that_no_one_should_want_to_use_as_a_jemalloc_private_namespace_prefix)], [${JEMALLOC_PRIVATE_NAMESPACE}##string_that_no_one_should_want_to_use_as_a_jemalloc_private_namespace_prefix])
else
AC_DEFINE_UNQUOTED([JEMALLOC_N(string_that_no_one_should_want_to_use_as_a_jemalloc_private_namespace_prefix)], [string_that_no_one_should_want_to_use_as_a_jemalloc_private_namespace_prefix])
fi
dnl Do not add suffix to installed files by default.
AC_ARG_WITH([install_suffix],
[AS_HELP_STRING([--with-install-suffix=<suffix>], [Suffix to append to all installed files])],
@@ -688,7 +697,7 @@ dnl jemalloc configuration.
dnl
dnl Set VERSION if source directory has an embedded git repository.
if test -d "${srcroot}../.git" ; then
if test -d "${srcroot}.git" ; then
git describe --long --abbrev=40 > ${srcroot}VERSION
fi
jemalloc_version=`cat ${srcroot}VERSION`
@@ -905,6 +914,8 @@ AC_MSG_RESULT([objroot : ${objroot}])
AC_MSG_RESULT([abs_objroot : ${abs_objroot}])
AC_MSG_RESULT([])
AC_MSG_RESULT([JEMALLOC_PREFIX : ${JEMALLOC_PREFIX}])
AC_MSG_RESULT([JEMALLOC_PRIVATE_NAMESPACE])
AC_MSG_RESULT([ : ${JEMALLOC_PRIVATE_NAMESPACE}])
AC_MSG_RESULT([install_suffix : ${install_suffix}])
AC_MSG_RESULT([autogen : ${enable_autogen}])
AC_MSG_RESULT([cc-silence : ${enable_cc_silence}])
deps/jemalloc/doc/jemalloc.3
Vendored
+5 -5
View File
@@ -2,12 +2,12 @@
.\" Title: JEMALLOC
.\" Author: Jason Evans
.\" Generator: DocBook XSL Stylesheets v1.75.2 <http://docbook.sf.net/>
.\" Date: 03/30/2011
.\" Date: 11/14/2011
.\" Manual: User Manual
.\" Source: jemalloc 2.2.1-0-g5ef7abf6d846720fb3fb8c737861c99b5ad1d862
.\" Source: jemalloc 2.2.5-0-gfc1bb70e5f0d9a58b39efa39cc549b5af5104760
.\" Language: English
.\"
.TH "JEMALLOC" "3" "03/30/2011" "jemalloc 2.2.1-0-g5ef7abf6d846" "User Manual"
.TH "JEMALLOC" "3" "11/14/2011" "jemalloc 2.2.5-0-gfc1bb70e5f0d" "User Manual"
.\" -----------------------------------------------------------------
.\" * Define some portability stuff
.\" -----------------------------------------------------------------
@@ -31,7 +31,7 @@
jemalloc \- general purpose memory allocation functions
.SH "LIBRARY"
.PP
This manual describes jemalloc 2\&.2\&.1\-0\-g5ef7abf6d846720fb3fb8c737861c99b5ad1d862\&. More information can be found at the
This manual describes jemalloc 2\&.2\&.5\-0\-gfc1bb70e5f0d9a58b39efa39cc549b5af5104760\&. More information can be found at the
\m[blue]\fBjemalloc website\fR\m[]\&\s-2\u[1]\d\s+2\&.
.SH "SYNOPSIS"
.sp
@@ -1215,7 +1215,7 @@ is successfully written to\&.
Number of file descriptors in use for swap\&.
.RE
.PP
"swap\&.fds" (\fBint *\fR) r\- [\fB\-\-enable\-swap\fR]
"swap\&.fds" (\fBint *\fR) rw [\fB\-\-enable\-swap\fR]
.RS 4
When written to, the files associated with the specified file descriptors are contiguously mapped via
\fBmmap\fR(2)\&. The resulting virtual memory region is preferred over anonymous
deps/jemalloc/doc/jemalloc.html
Vendored
+9 -9
View File
@@ -1,8 +1,8 @@
<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>JEMALLOC</title><meta name="generator" content="DocBook XSL Stylesheets V1.75.2"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="refentry" title="JEMALLOC"><a name="id2783946"></a><div class="titlepage"></div><div class="refnamediv"><h2>Name</h2><p>jemalloc &#8212; general purpose memory allocation functions</p></div><div class="refsect1" title="LIBRARY"><a name="library"></a><h2>LIBRARY</h2><p>This manual describes jemalloc 2.2.1-0-g5ef7abf6d846720fb3fb8c737861c99b5ad1d862. More information
<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>JEMALLOC</title><meta name="generator" content="DocBook XSL Stylesheets V1.75.2"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="refentry" title="JEMALLOC"><a name="id2968890"></a><div class="titlepage"></div><div class="refnamediv"><h2>Name</h2><p>jemalloc &#8212; general purpose memory allocation functions</p></div><div class="refsect1" title="LIBRARY"><a name="library"></a><h2>LIBRARY</h2><p>This manual describes jemalloc 2.2.5-0-gfc1bb70e5f0d9a58b39efa39cc549b5af5104760. More information
can be found at the <a class="ulink" href="http://www.canonware.com/jemalloc/" target="_top">jemalloc website</a>.</p></div><div class="refsynopsisdiv" title="SYNOPSIS"><h2>SYNOPSIS</h2><div class="funcsynopsis"><pre class="funcsynopsisinfo">#include &lt;<code class="filename">stdlib.h</code>&gt;
#include &lt;<code class="filename">jemalloc/jemalloc.h</code>&gt;</pre><div class="refsect2" title="Standard API"><a name="id2830421"></a><h3>Standard API</h3><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">void *<b class="fsfunc">malloc</b>(</code></td><td>size_t <var class="pdparam">size</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">void *<b class="fsfunc">calloc</b>(</code></td><td>size_t <var class="pdparam">number</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">size</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">posix_memalign</b>(</code></td><td>void **<var class="pdparam">ptr</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">alignment</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">size</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">void *<b class="fsfunc">realloc</b>(</code></td><td>void *<var class="pdparam">ptr</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">size</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">void <b class="fsfunc">free</b>(</code></td><td>void *<var class="pdparam">ptr</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div></div><div class="refsect2" title="Non-standard API"><a name="id2837717"></a><h3>Non-standard API</h3><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">size_t <b class="fsfunc">malloc_usable_size</b>(</code></td><td>const void *<var class="pdparam">ptr</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">void <b class="fsfunc">malloc_stats_print</b>(</code></td><td>void <var class="pdparam">(*write_cb)</var>
#include &lt;<code class="filename">jemalloc/jemalloc.h</code>&gt;</pre><div class="refsect2" title="Standard API"><a name="id2992781"></a><h3>Standard API</h3><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">void *<b class="fsfunc">malloc</b>(</code></td><td>size_t <var class="pdparam">size</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">void *<b class="fsfunc">calloc</b>(</code></td><td>size_t <var class="pdparam">number</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">size</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">posix_memalign</b>(</code></td><td>void **<var class="pdparam">ptr</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">alignment</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">size</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">void *<b class="fsfunc">realloc</b>(</code></td><td>void *<var class="pdparam">ptr</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">size</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">void <b class="fsfunc">free</b>(</code></td><td>void *<var class="pdparam">ptr</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div></div><div class="refsect2" title="Non-standard API"><a name="id2998350"></a><h3>Non-standard API</h3><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">size_t <b class="fsfunc">malloc_usable_size</b>(</code></td><td>const void *<var class="pdparam">ptr</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">void <b class="fsfunc">malloc_stats_print</b>(</code></td><td>void <var class="pdparam">(*write_cb)</var>
<code>(</code>void *, const char *<code>)</code>
, </td></tr><tr><td> </td><td>void *<var class="pdparam">cbopaque</var>, </td></tr><tr><td> </td><td>const char *<var class="pdparam">opts</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">mallctl</b>(</code></td><td>const char *<var class="pdparam">name</var>, </td></tr><tr><td> </td><td>void *<var class="pdparam">oldp</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">oldlenp</var>, </td></tr><tr><td> </td><td>void *<var class="pdparam">newp</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">newlen</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">mallctlnametomib</b>(</code></td><td>const char *<var class="pdparam">name</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">mibp</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">miblenp</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">mallctlbymib</b>(</code></td><td>const size_t *<var class="pdparam">mib</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">miblen</var>, </td></tr><tr><td> </td><td>void *<var class="pdparam">oldp</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">oldlenp</var>, </td></tr><tr><td> </td><td>void *<var class="pdparam">newp</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">newlen</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">void <b class="fsfunc">(*malloc_message)</b>(</code></td><td>void *<var class="pdparam">cbopaque</var>, </td></tr><tr><td> </td><td>const char *<var class="pdparam">s</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><p><span class="type">const char *</span><code class="varname">malloc_conf</code>;</p></div><div class="refsect2" title="Experimental API"><a name="id2830042"></a><h3>Experimental API</h3><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">allocm</b>(</code></td><td>void **<var class="pdparam">ptr</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">rsize</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">size</var>, </td></tr><tr><td> </td><td>int <var class="pdparam">flags</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">rallocm</b>(</code></td><td>void **<var class="pdparam">ptr</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">rsize</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">size</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">extra</var>, </td></tr><tr><td> </td><td>int <var class="pdparam">flags</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">sallocm</b>(</code></td><td>const void *<var class="pdparam">ptr</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">rsize</var>, </td></tr><tr><td> </td><td>int <var class="pdparam">flags</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">dallocm</b>(</code></td><td>void *<var class="pdparam">ptr</var>, </td></tr><tr><td> </td><td>int <var class="pdparam">flags</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div></div></div></div><div class="refsect1" title="DESCRIPTION"><a name="description"></a><h2>DESCRIPTION</h2><div class="refsect2" title="Standard API"><a name="id2840968"></a><h3>Standard API</h3><p>The <code class="function">malloc</code>(<em class="parameter"><code></code></em>) function allocates
, </td></tr><tr><td> </td><td>void *<var class="pdparam">cbopaque</var>, </td></tr><tr><td> </td><td>const char *<var class="pdparam">opts</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">mallctl</b>(</code></td><td>const char *<var class="pdparam">name</var>, </td></tr><tr><td> </td><td>void *<var class="pdparam">oldp</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">oldlenp</var>, </td></tr><tr><td> </td><td>void *<var class="pdparam">newp</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">newlen</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">mallctlnametomib</b>(</code></td><td>const char *<var class="pdparam">name</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">mibp</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">miblenp</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">mallctlbymib</b>(</code></td><td>const size_t *<var class="pdparam">mib</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">miblen</var>, </td></tr><tr><td> </td><td>void *<var class="pdparam">oldp</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">oldlenp</var>, </td></tr><tr><td> </td><td>void *<var class="pdparam">newp</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">newlen</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">void <b class="fsfunc">(*malloc_message)</b>(</code></td><td>void *<var class="pdparam">cbopaque</var>, </td></tr><tr><td> </td><td>const char *<var class="pdparam">s</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><p><span class="type">const char *</span><code class="varname">malloc_conf</code>;</p></div><div class="refsect2" title="Experimental API"><a name="id3014125"></a><h3>Experimental API</h3><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">allocm</b>(</code></td><td>void **<var class="pdparam">ptr</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">rsize</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">size</var>, </td></tr><tr><td> </td><td>int <var class="pdparam">flags</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">rallocm</b>(</code></td><td>void **<var class="pdparam">ptr</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">rsize</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">size</var>, </td></tr><tr><td> </td><td>size_t <var class="pdparam">extra</var>, </td></tr><tr><td> </td><td>int <var class="pdparam">flags</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">sallocm</b>(</code></td><td>const void *<var class="pdparam">ptr</var>, </td></tr><tr><td> </td><td>size_t *<var class="pdparam">rsize</var>, </td></tr><tr><td> </td><td>int <var class="pdparam">flags</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div><table border="0" summary="Function synopsis" cellspacing="0" cellpadding="0" class="funcprototype-table"><tr><td><code class="funcdef">int <b class="fsfunc">dallocm</b>(</code></td><td>void *<var class="pdparam">ptr</var>, </td></tr><tr><td> </td><td>int <var class="pdparam">flags</var><code>)</code>;</td></tr></table><div class="funcprototype-spacer"> </div></div></div></div><div class="refsect1" title="DESCRIPTION"><a name="description"></a><h2>DESCRIPTION</h2><div class="refsect2" title="Standard API"><a name="id3014924"></a><h3>Standard API</h3><p>The <code class="function">malloc</code>(<em class="parameter"><code></code></em>) function allocates
<em class="parameter"><code>size</code></em> bytes of uninitialized memory. The allocated
space is suitably aligned (after possible pointer coercion) for storage
of any type of object.</p><p>The <code class="function">calloc</code>(<em class="parameter"><code></code></em>) function allocates
@@ -32,7 +32,7 @@
<code class="function">malloc</code>(<em class="parameter"><code></code></em>) for the specified size.</p><p>The <code class="function">free</code>(<em class="parameter"><code></code></em>) function causes the
allocated memory referenced by <em class="parameter"><code>ptr</code></em> to be made
available for future allocations. If <em class="parameter"><code>ptr</code></em> is
<code class="constant">NULL</code>, no action occurs.</p></div><div class="refsect2" title="Non-standard API"><a name="id2827225"></a><h3>Non-standard API</h3><p>The <code class="function">malloc_usable_size</code>(<em class="parameter"><code></code></em>) function
<code class="constant">NULL</code>, no action occurs.</p></div><div class="refsect2" title="Non-standard API"><a name="id3025603"></a><h3>Non-standard API</h3><p>The <code class="function">malloc_usable_size</code>(<em class="parameter"><code></code></em>) function
returns the usable size of the allocation pointed to by
<em class="parameter"><code>ptr</code></em>. The return value may be larger than the size
that was requested during allocation. The
@@ -112,7 +112,7 @@ for (i = 0; i &lt; nbins; i++) {
len = sizeof(bin_size);
mallctlbymib(mib, miblen, &amp;bin_size, &amp;len, NULL, 0);
/* Do something with bin_size... */
}</pre></div><div class="refsect2" title="Experimental API"><a name="id2807945"></a><h3>Experimental API</h3><p>The experimental API is subject to change or removal without regard
}</pre></div><div class="refsect2" title="Experimental API"><a name="id3013809"></a><h3>Experimental API</h3><p>The experimental API is subject to change or removal without regard
for backward compatibility.</p><p>The <code class="function">allocm</code>(<em class="parameter"><code></code></em>),
<code class="function">rallocm</code>(<em class="parameter"><code></code></em>),
<code class="function">sallocm</code>(<em class="parameter"><code></code></em>), and
@@ -1397,7 +1397,7 @@ malloc_conf = "xmalloc:true";</pre><p>
"<code class="mallctl">swap.fds</code>"
(<span class="type">int *</span>)
<code class="literal">r-</code>
<code class="literal">rw</code>
[<code class="option">--enable-swap</code>]
</span></dt><dd><p>When written to, the files associated with the
specified file descriptors are contiguously mapped via
@@ -1447,7 +1447,7 @@ malloc_conf = "xmalloc:true";</pre><p>
<code class="function">malloc_stats_print</code>(<em class="parameter"><code></code></em>), followed by a string
pointer. Please note that doing anything which tries to allocate memory in
this function is likely to result in a crash or deadlock.</p><p>All messages are prefixed by
&#8220;<code class="computeroutput">&lt;jemalloc&gt;: </code>&#8221;.</p></div><div class="refsect1" title="RETURN VALUES"><a name="return_values"></a><h2>RETURN VALUES</h2><div class="refsect2" title="Standard API"><a name="id2844722"></a><h3>Standard API</h3><p>The <code class="function">malloc</code>(<em class="parameter"><code></code></em>) and
&#8220;<code class="computeroutput">&lt;jemalloc&gt;: </code>&#8221;.</p></div><div class="refsect1" title="RETURN VALUES"><a name="return_values"></a><h2>RETURN VALUES</h2><div class="refsect2" title="Standard API"><a name="id3029250"></a><h3>Standard API</h3><p>The <code class="function">malloc</code>(<em class="parameter"><code></code></em>) and
<code class="function">calloc</code>(<em class="parameter"><code></code></em>) functions return a pointer to the
allocated memory if successful; otherwise a <code class="constant">NULL</code>
pointer is returned and <code class="varname">errno</code> is set to
@@ -1467,7 +1467,7 @@ malloc_conf = "xmalloc:true";</pre><p>
allocation failure. The <code class="function">realloc</code>(<em class="parameter"><code></code></em>)
function always leaves the original buffer intact when an error occurs.
</p><p>The <code class="function">free</code>(<em class="parameter"><code></code></em>) function returns no
value.</p></div><div class="refsect2" title="Non-standard API"><a name="id2844875"></a><h3>Non-standard API</h3><p>The <code class="function">malloc_usable_size</code>(<em class="parameter"><code></code></em>) function
value.</p></div><div class="refsect2" title="Non-standard API"><a name="id3029403"></a><h3>Non-standard API</h3><p>The <code class="function">malloc_usable_size</code>(<em class="parameter"><code></code></em>) function
returns the usable size of the allocation pointed to by
<em class="parameter"><code>ptr</code></em>. </p><p>The <code class="function">mallctl</code>(<em class="parameter"><code></code></em>),
<code class="function">mallctlnametomib</code>(<em class="parameter"><code></code></em>), and
@@ -1486,7 +1486,7 @@ malloc_conf = "xmalloc:true";</pre><p>
occurred.</p></dd><dt><span class="term"><span class="errorname">EFAULT</span></span></dt><dd><p>An interface with side effects failed in some way
not directly related to <code class="function">mallctl*</code>(<em class="parameter"><code></code></em>)
read/write processing.</p></dd></dl></div><p>
</p></div><div class="refsect2" title="Experimental API"><a name="id2845053"></a><h3>Experimental API</h3><p>The <code class="function">allocm</code>(<em class="parameter"><code></code></em>),
</p></div><div class="refsect2" title="Experimental API"><a name="id3029581"></a><h3>Experimental API</h3><p>The <code class="function">allocm</code>(<em class="parameter"><code></code></em>),
<code class="function">rallocm</code>(<em class="parameter"><code></code></em>),
<code class="function">sallocm</code>(<em class="parameter"><code></code></em>), and
<code class="function">dallocm</code>(<em class="parameter"><code></code></em>) functions return
deps/jemalloc/doc/jemalloc.xml.in
Vendored
+1 -1
View File
@@ -2025,7 +2025,7 @@ malloc_conf = "xmalloc:true";]]></programlisting>
<term>
<mallctl>swap.fds</mallctl>
(<type>int *</type>)
<literal>r-</literal>
<literal>rw</literal>
[<option>--enable-swap</option>]
</term>
<listitem><para>When written to, the files associated with the
deps/jemalloc/include/jemalloc/internal/chunk.h
Vendored
+1 -1
View File
@@ -50,7 +50,7 @@ extern size_t map_bias; /* Number of arena chunk header pages. */
extern size_t arena_maxclass; /* Max size class for arenas. */
void *chunk_alloc(size_t size, bool base, bool *zero);
void chunk_dealloc(void *chunk, size_t size);
void chunk_dealloc(void *chunk, size_t size, bool unmap);
bool chunk_boot(void);
#endif /* JEMALLOC_H_EXTERNS */
deps/jemalloc/include/jemalloc/internal/hash.h
Vendored
+1 -1
View File
@@ -26,7 +26,7 @@ uint64_t hash(const void *key, size_t len, uint64_t seed);
JEMALLOC_INLINE uint64_t
hash(const void *key, size_t len, uint64_t seed)
{
const uint64_t m = 0xc6a4a7935bd1e995;
const uint64_t m = 0xc6a4a7935bd1e995LLU;
const int r = 47;
uint64_t h = seed ^ (len * m);
const uint64_t *data = (const uint64_t *)key;
deps/jemalloc/include/jemalloc/internal/jemalloc_internal.h.in
Vendored
+7 -1
View File
@@ -33,6 +33,8 @@
#define JEMALLOC_MANGLE
#include "../jemalloc@install_suffix@.h"
#include "jemalloc/internal/private_namespace.h"
#if (defined(JEMALLOC_OSATOMIC) || defined(JEMALLOC_OSSPIN))
#include <libkern/OSAtomic.h>
#endif
@@ -633,7 +635,11 @@ ipalloc(size_t usize, size_t alignment, bool zero)
if (usize <= arena_maxclass && alignment <= PAGE_SIZE)
ret = arena_malloc(usize, zero);
else {
size_t run_size = 0;
size_t run_size
#ifdef JEMALLOC_CC_SILENCE
= 0
#endif
;
/*
* Ideally we would only ever call sa2u() once per aligned
deps/jemalloc/include/jemalloc/internal/private_namespace.h
Vendored
+195
View File
@@ -0,0 +1,195 @@
#define arena_bin_index JEMALLOC_N(arena_bin_index)
#define arena_boot JEMALLOC_N(arena_boot)
#define arena_dalloc JEMALLOC_N(arena_dalloc)
#define arena_dalloc_bin JEMALLOC_N(arena_dalloc_bin)
#define arena_dalloc_large JEMALLOC_N(arena_dalloc_large)
#define arena_malloc JEMALLOC_N(arena_malloc)
#define arena_malloc_large JEMALLOC_N(arena_malloc_large)
#define arena_malloc_small JEMALLOC_N(arena_malloc_small)
#define arena_new JEMALLOC_N(arena_new)
#define arena_palloc JEMALLOC_N(arena_palloc)
#define arena_prof_accum JEMALLOC_N(arena_prof_accum)
#define arena_prof_ctx_get JEMALLOC_N(arena_prof_ctx_get)
#define arena_prof_ctx_set JEMALLOC_N(arena_prof_ctx_set)
#define arena_prof_promoted JEMALLOC_N(arena_prof_promoted)
#define arena_purge_all JEMALLOC_N(arena_purge_all)
#define arena_ralloc JEMALLOC_N(arena_ralloc)
#define arena_ralloc_no_move JEMALLOC_N(arena_ralloc_no_move)
#define arena_run_regind JEMALLOC_N(arena_run_regind)
#define arena_salloc JEMALLOC_N(arena_salloc)
#define arena_salloc_demote JEMALLOC_N(arena_salloc_demote)
#define arena_stats_merge JEMALLOC_N(arena_stats_merge)
#define arena_tcache_fill_small JEMALLOC_N(arena_tcache_fill_small)
#define arenas_bin_i_index JEMALLOC_N(arenas_bin_i_index)
#define arenas_extend JEMALLOC_N(arenas_extend)
#define arenas_lrun_i_index JEMALLOC_N(arenas_lrun_i_index)
#define atomic_add_uint32 JEMALLOC_N(atomic_add_uint32)
#define atomic_add_uint64 JEMALLOC_N(atomic_add_uint64)
#define atomic_sub_uint32 JEMALLOC_N(atomic_sub_uint32)
#define atomic_sub_uint64 JEMALLOC_N(atomic_sub_uint64)
#define base_alloc JEMALLOC_N(base_alloc)
#define base_boot JEMALLOC_N(base_boot)
#define base_node_alloc JEMALLOC_N(base_node_alloc)
#define base_node_dealloc JEMALLOC_N(base_node_dealloc)
#define bitmap_full JEMALLOC_N(bitmap_full)
#define bitmap_get JEMALLOC_N(bitmap_get)
#define bitmap_info_init JEMALLOC_N(bitmap_info_init)
#define bitmap_info_ngroups JEMALLOC_N(bitmap_info_ngroups)
#define bitmap_init JEMALLOC_N(bitmap_init)
#define bitmap_set JEMALLOC_N(bitmap_set)
#define bitmap_sfu JEMALLOC_N(bitmap_sfu)
#define bitmap_size JEMALLOC_N(bitmap_size)
#define bitmap_unset JEMALLOC_N(bitmap_unset)
#define bt_init JEMALLOC_N(bt_init)
#define buferror JEMALLOC_N(buferror)
#define choose_arena JEMALLOC_N(choose_arena)
#define choose_arena_hard JEMALLOC_N(choose_arena_hard)
#define chunk_alloc JEMALLOC_N(chunk_alloc)
#define chunk_alloc_dss JEMALLOC_N(chunk_alloc_dss)
#define chunk_alloc_mmap JEMALLOC_N(chunk_alloc_mmap)
#define chunk_alloc_mmap_noreserve JEMALLOC_N(chunk_alloc_mmap_noreserve)
#define chunk_alloc_swap JEMALLOC_N(chunk_alloc_swap)
#define chunk_boot JEMALLOC_N(chunk_boot)
#define chunk_dealloc JEMALLOC_N(chunk_dealloc)
#define chunk_dealloc_dss JEMALLOC_N(chunk_dealloc_dss)
#define chunk_dealloc_mmap JEMALLOC_N(chunk_dealloc_mmap)
#define chunk_dealloc_swap JEMALLOC_N(chunk_dealloc_swap)
#define chunk_dss_boot JEMALLOC_N(chunk_dss_boot)
#define chunk_in_dss JEMALLOC_N(chunk_in_dss)
#define chunk_in_swap JEMALLOC_N(chunk_in_swap)
#define chunk_mmap_boot JEMALLOC_N(chunk_mmap_boot)
#define chunk_swap_boot JEMALLOC_N(chunk_swap_boot)
#define chunk_swap_enable JEMALLOC_N(chunk_swap_enable)
#define ckh_bucket_search JEMALLOC_N(ckh_bucket_search)
#define ckh_count JEMALLOC_N(ckh_count)
#define ckh_delete JEMALLOC_N(ckh_delete)
#define ckh_evict_reloc_insert JEMALLOC_N(ckh_evict_reloc_insert)
#define ckh_insert JEMALLOC_N(ckh_insert)
#define ckh_isearch JEMALLOC_N(ckh_isearch)
#define ckh_iter JEMALLOC_N(ckh_iter)
#define ckh_new JEMALLOC_N(ckh_new)
#define ckh_pointer_hash JEMALLOC_N(ckh_pointer_hash)
#define ckh_pointer_keycomp JEMALLOC_N(ckh_pointer_keycomp)
#define ckh_rebuild JEMALLOC_N(ckh_rebuild)
#define ckh_remove JEMALLOC_N(ckh_remove)
#define ckh_search JEMALLOC_N(ckh_search)
#define ckh_string_hash JEMALLOC_N(ckh_string_hash)
#define ckh_string_keycomp JEMALLOC_N(ckh_string_keycomp)
#define ckh_try_bucket_insert JEMALLOC_N(ckh_try_bucket_insert)
#define ckh_try_insert JEMALLOC_N(ckh_try_insert)
#define create_zone JEMALLOC_N(create_zone)
#define ctl_boot JEMALLOC_N(ctl_boot)
#define ctl_bymib JEMALLOC_N(ctl_bymib)
#define ctl_byname JEMALLOC_N(ctl_byname)
#define ctl_nametomib JEMALLOC_N(ctl_nametomib)
#define extent_tree_ad_first JEMALLOC_N(extent_tree_ad_first)
#define extent_tree_ad_insert JEMALLOC_N(extent_tree_ad_insert)
#define extent_tree_ad_iter JEMALLOC_N(extent_tree_ad_iter)
#define extent_tree_ad_iter_recurse JEMALLOC_N(extent_tree_ad_iter_recurse)
#define extent_tree_ad_iter_start JEMALLOC_N(extent_tree_ad_iter_start)
#define extent_tree_ad_last JEMALLOC_N(extent_tree_ad_last)
#define extent_tree_ad_new JEMALLOC_N(extent_tree_ad_new)
#define extent_tree_ad_next JEMALLOC_N(extent_tree_ad_next)
#define extent_tree_ad_nsearch JEMALLOC_N(extent_tree_ad_nsearch)
#define extent_tree_ad_prev JEMALLOC_N(extent_tree_ad_prev)
#define extent_tree_ad_psearch JEMALLOC_N(extent_tree_ad_psearch)
#define extent_tree_ad_remove JEMALLOC_N(extent_tree_ad_remove)
#define extent_tree_ad_reverse_iter JEMALLOC_N(extent_tree_ad_reverse_iter)
#define extent_tree_ad_reverse_iter_recurse JEMALLOC_N(extent_tree_ad_reverse_iter_recurse)
#define extent_tree_ad_reverse_iter_start JEMALLOC_N(extent_tree_ad_reverse_iter_start)
#define extent_tree_ad_search JEMALLOC_N(extent_tree_ad_search)
#define extent_tree_szad_first JEMALLOC_N(extent_tree_szad_first)
#define extent_tree_szad_insert JEMALLOC_N(extent_tree_szad_insert)
#define extent_tree_szad_iter JEMALLOC_N(extent_tree_szad_iter)
#define extent_tree_szad_iter_recurse JEMALLOC_N(extent_tree_szad_iter_recurse)
#define extent_tree_szad_iter_start JEMALLOC_N(extent_tree_szad_iter_start)
#define extent_tree_szad_last JEMALLOC_N(extent_tree_szad_last)
#define extent_tree_szad_new JEMALLOC_N(extent_tree_szad_new)
#define extent_tree_szad_next JEMALLOC_N(extent_tree_szad_next)
#define extent_tree_szad_nsearch JEMALLOC_N(extent_tree_szad_nsearch)
#define extent_tree_szad_prev JEMALLOC_N(extent_tree_szad_prev)
#define extent_tree_szad_psearch JEMALLOC_N(extent_tree_szad_psearch)
#define extent_tree_szad_remove JEMALLOC_N(extent_tree_szad_remove)
#define extent_tree_szad_reverse_iter JEMALLOC_N(extent_tree_szad_reverse_iter)
#define extent_tree_szad_reverse_iter_recurse JEMALLOC_N(extent_tree_szad_reverse_iter_recurse)
#define extent_tree_szad_reverse_iter_start JEMALLOC_N(extent_tree_szad_reverse_iter_start)
#define extent_tree_szad_search JEMALLOC_N(extent_tree_szad_search)
#define hash JEMALLOC_N(hash)
#define huge_boot JEMALLOC_N(huge_boot)
#define huge_dalloc JEMALLOC_N(huge_dalloc)
#define huge_malloc JEMALLOC_N(huge_malloc)
#define huge_palloc JEMALLOC_N(huge_palloc)
#define huge_prof_ctx_get JEMALLOC_N(huge_prof_ctx_get)
#define huge_prof_ctx_set JEMALLOC_N(huge_prof_ctx_set)
#define huge_ralloc JEMALLOC_N(huge_ralloc)
#define huge_ralloc_no_move JEMALLOC_N(huge_ralloc_no_move)
#define huge_salloc JEMALLOC_N(huge_salloc)
#define iallocm JEMALLOC_N(iallocm)
#define icalloc JEMALLOC_N(icalloc)
#define idalloc JEMALLOC_N(idalloc)
#define imalloc JEMALLOC_N(imalloc)
#define ipalloc JEMALLOC_N(ipalloc)
#define iralloc JEMALLOC_N(iralloc)
#define isalloc JEMALLOC_N(isalloc)
#define ivsalloc JEMALLOC_N(ivsalloc)
#define jemalloc_darwin_init JEMALLOC_N(jemalloc_darwin_init)
#define jemalloc_postfork JEMALLOC_N(jemalloc_postfork)
#define jemalloc_prefork JEMALLOC_N(jemalloc_prefork)
#define malloc_cprintf JEMALLOC_N(malloc_cprintf)
#define malloc_mutex_destroy JEMALLOC_N(malloc_mutex_destroy)
#define malloc_mutex_init JEMALLOC_N(malloc_mutex_init)
#define malloc_mutex_lock JEMALLOC_N(malloc_mutex_lock)
#define malloc_mutex_trylock JEMALLOC_N(malloc_mutex_trylock)
#define malloc_mutex_unlock JEMALLOC_N(malloc_mutex_unlock)
#define malloc_printf JEMALLOC_N(malloc_printf)
#define malloc_write JEMALLOC_N(malloc_write)
#define mb_write JEMALLOC_N(mb_write)
#define pow2_ceil JEMALLOC_N(pow2_ceil)
#define prof_backtrace JEMALLOC_N(prof_backtrace)
#define prof_boot0 JEMALLOC_N(prof_boot0)
#define prof_boot1 JEMALLOC_N(prof_boot1)
#define prof_boot2 JEMALLOC_N(prof_boot2)
#define prof_ctx_get JEMALLOC_N(prof_ctx_get)
#define prof_ctx_set JEMALLOC_N(prof_ctx_set)
#define prof_free JEMALLOC_N(prof_free)
#define prof_gdump JEMALLOC_N(prof_gdump)
#define prof_idump JEMALLOC_N(prof_idump)
#define prof_lookup JEMALLOC_N(prof_lookup)
#define prof_malloc JEMALLOC_N(prof_malloc)
#define prof_mdump JEMALLOC_N(prof_mdump)
#define prof_realloc JEMALLOC_N(prof_realloc)
#define prof_sample_accum_update JEMALLOC_N(prof_sample_accum_update)
#define prof_sample_threshold_update JEMALLOC_N(prof_sample_threshold_update)
#define prof_tdata_init JEMALLOC_N(prof_tdata_init)
#define pthread_create JEMALLOC_N(pthread_create)
#define rtree_get JEMALLOC_N(rtree_get)
#define rtree_get_locked JEMALLOC_N(rtree_get_locked)
#define rtree_new JEMALLOC_N(rtree_new)
#define rtree_set JEMALLOC_N(rtree_set)
#define s2u JEMALLOC_N(s2u)
#define sa2u JEMALLOC_N(sa2u)
#define stats_arenas_i_bins_j_index JEMALLOC_N(stats_arenas_i_bins_j_index)
#define stats_arenas_i_index JEMALLOC_N(stats_arenas_i_index)
#define stats_arenas_i_lruns_j_index JEMALLOC_N(stats_arenas_i_lruns_j_index)
#define stats_cactive_add JEMALLOC_N(stats_cactive_add)
#define stats_cactive_get JEMALLOC_N(stats_cactive_get)
#define stats_cactive_sub JEMALLOC_N(stats_cactive_sub)
#define stats_print JEMALLOC_N(stats_print)
#define szone2ozone JEMALLOC_N(szone2ozone)
#define tcache_alloc_easy JEMALLOC_N(tcache_alloc_easy)
#define tcache_alloc_large JEMALLOC_N(tcache_alloc_large)
#define tcache_alloc_small JEMALLOC_N(tcache_alloc_small)
#define tcache_alloc_small_hard JEMALLOC_N(tcache_alloc_small_hard)
#define tcache_bin_flush_large JEMALLOC_N(tcache_bin_flush_large)
#define tcache_bin_flush_small JEMALLOC_N(tcache_bin_flush_small)
#define tcache_boot JEMALLOC_N(tcache_boot)
#define tcache_create JEMALLOC_N(tcache_create)
#define tcache_dalloc_large JEMALLOC_N(tcache_dalloc_large)
#define tcache_dalloc_small JEMALLOC_N(tcache_dalloc_small)
#define tcache_destroy JEMALLOC_N(tcache_destroy)
#define tcache_event JEMALLOC_N(tcache_event)
#define tcache_get JEMALLOC_N(tcache_get)
#define tcache_stats_merge JEMALLOC_N(tcache_stats_merge)
#define thread_allocated_get JEMALLOC_N(thread_allocated_get)
#define thread_allocated_get_hard JEMALLOC_N(thread_allocated_get_hard)
#define u2s JEMALLOC_N(u2s)
deps/jemalloc/include/jemalloc/internal/prof.h
Vendored
+54 -68
View File
@@ -227,9 +227,60 @@ bool prof_boot2(void);
/******************************************************************************/
#ifdef JEMALLOC_H_INLINES
#define PROF_ALLOC_PREP(nignore, size, ret) do { \
prof_tdata_t *prof_tdata; \
prof_bt_t bt; \
\
assert(size == s2u(size)); \
\
prof_tdata = PROF_TCACHE_GET(); \
if (prof_tdata == NULL) { \
prof_tdata = prof_tdata_init(); \
if (prof_tdata == NULL) { \
ret = NULL; \
break; \
} \
} \
\
if (opt_prof_active == false) { \
/* Sampling is currently inactive, so avoid sampling. */\
ret = (prof_thr_cnt_t *)(uintptr_t)1U; \
} else if (opt_lg_prof_sample == 0) { \
/* Don't bother with sampling logic, since sampling */\
/* interval is 1. */\
bt_init(&bt, prof_tdata->vec); \
prof_backtrace(&bt, nignore, prof_bt_max); \
ret = prof_lookup(&bt); \
} else { \
if (prof_tdata->threshold == 0) { \
/* Initialize. Seed the prng differently for */\
/* each thread. */\
prof_tdata->prn_state = \
(uint64_t)(uintptr_t)&size; \
prof_sample_threshold_update(prof_tdata); \
} \
\
/* Determine whether to capture a backtrace based on */\
/* whether size is enough for prof_accum to reach */\
/* prof_tdata->threshold. However, delay updating */\
/* these variables until prof_{m,re}alloc(), because */\
/* we don't know for sure that the allocation will */\
/* succeed. */\
/* */\
/* Use subtraction rather than addition to avoid */\
/* potential integer overflow. */\
if (size >= prof_tdata->threshold - \
prof_tdata->accum) { \
bt_init(&bt, prof_tdata->vec); \
prof_backtrace(&bt, nignore, prof_bt_max); \
ret = prof_lookup(&bt); \
} else \
ret = (prof_thr_cnt_t *)(uintptr_t)1U; \
} \
} while (0)
#ifndef JEMALLOC_ENABLE_INLINE
void prof_sample_threshold_update(prof_tdata_t *prof_tdata);
prof_thr_cnt_t *prof_alloc_prep(size_t size);
prof_ctx_t *prof_ctx_get(const void *ptr);
void prof_ctx_set(const void *ptr, prof_ctx_t *ctx);
bool prof_sample_accum_update(size_t size);
@@ -272,71 +323,6 @@ prof_sample_threshold_update(prof_tdata_t *prof_tdata)
+ (uint64_t)1U;
}
JEMALLOC_INLINE prof_thr_cnt_t *
prof_alloc_prep(size_t size)
{
#ifdef JEMALLOC_ENABLE_INLINE
/* This function does not have its own stack frame, because it is inlined. */
# define NIGNORE 1
#else
# define NIGNORE 2
#endif
prof_thr_cnt_t *ret;
prof_tdata_t *prof_tdata;
prof_bt_t bt;
assert(size == s2u(size));
prof_tdata = PROF_TCACHE_GET();
if (prof_tdata == NULL) {
prof_tdata = prof_tdata_init();
if (prof_tdata == NULL)
return (NULL);
}
if (opt_prof_active == false) {
/* Sampling is currently inactive, so avoid sampling. */
ret = (prof_thr_cnt_t *)(uintptr_t)1U;
} else if (opt_lg_prof_sample == 0) {
/*
* Don't bother with sampling logic, since sampling interval is
* 1.
*/
bt_init(&bt, prof_tdata->vec);
prof_backtrace(&bt, NIGNORE, prof_bt_max);
ret = prof_lookup(&bt);
} else {
if (prof_tdata->threshold == 0) {
/*
* Initialize. Seed the prng differently for each
* thread.
*/
prof_tdata->prn_state = (uint64_t)(uintptr_t)&size;
prof_sample_threshold_update(prof_tdata);
}
/*
* Determine whether to capture a backtrace based on whether
* size is enough for prof_accum to reach
* prof_tdata->threshold. However, delay updating these
* variables until prof_{m,re}alloc(), because we don't know
* for sure that the allocation will succeed.
*
* Use subtraction rather than addition to avoid potential
* integer overflow.
*/
if (size >= prof_tdata->threshold - prof_tdata->accum) {
bt_init(&bt, prof_tdata->vec);
prof_backtrace(&bt, NIGNORE, prof_bt_max);
ret = prof_lookup(&bt);
} else
ret = (prof_thr_cnt_t *)(uintptr_t)1U;
}
return (ret);
#undef NIGNORE
}
JEMALLOC_INLINE prof_ctx_t *
prof_ctx_get(const void *ptr)
{
@@ -415,7 +401,7 @@ prof_malloc(const void *ptr, size_t size, prof_thr_cnt_t *cnt)
* always possible to tell in advance how large an
* object's usable size will be, so there should never
* be a difference between the size passed to
* prof_alloc_prep() and prof_malloc().
* PROF_ALLOC_PREP() and prof_malloc().
*/
assert((uintptr_t)cnt == (uintptr_t)1U);
}
@@ -459,7 +445,7 @@ prof_realloc(const void *ptr, size_t size, prof_thr_cnt_t *cnt,
if (prof_sample_accum_update(size)) {
/*
* Don't sample. The size passed to
* prof_alloc_prep() was larger than what
* PROF_ALLOC_PREP() was larger than what
* actually got allocated, so a backtrace was
* captured for this allocation, even though
* its actual size was insufficient to cross
deps/jemalloc/include/jemalloc/jemalloc_defs.h.in
Vendored
+9
View File
@@ -18,6 +18,15 @@
#undef JEMALLOC_P
#endif
/*
* JEMALLOC_PRIVATE_NAMESPACE is used as a prefix for all library-private APIs.
* For shared libraries, symbol visibility mechanisms prevent these symbols
* from being exported, but for static libraries, naming collisions are a real
* possibility.
*/
#undef JEMALLOC_PRIVATE_NAMESPACE
#undef JEMALLOC_N
/*
* Hyper-threaded CPUs may need a special instruction inside spin loops in
* order to yield to another virtual CPU.
deps/jemalloc/src/arena.c
Vendored
+4 -3
View File
@@ -569,7 +569,7 @@ arena_chunk_dealloc(arena_t *arena, arena_chunk_t *chunk)
arena->ndirty -= spare->ndirty;
}
malloc_mutex_unlock(&arena->lock);
chunk_dealloc((void *)spare, chunksize);
chunk_dealloc((void *)spare, chunksize, true);
malloc_mutex_lock(&arena->lock);
#ifdef JEMALLOC_STATS
arena->stats.mapped -= chunksize;
@@ -869,9 +869,9 @@ arena_purge(arena_t *arena, bool all)
assert(ndirty == arena->ndirty);
#endif
assert(arena->ndirty > arena->npurgatory || all);
assert(arena->ndirty > chunk_npages || all);
assert(arena->ndirty - arena->npurgatory > chunk_npages || all);
assert((arena->nactive >> opt_lg_dirty_mult) < (arena->ndirty -
npurgatory) || all);
arena->npurgatory) || all);
#ifdef JEMALLOC_STATS
arena->stats.npurge++;
@@ -1657,6 +1657,7 @@ arena_prof_promoted(const void *ptr, size_t size)
assert(ptr != NULL);
assert(CHUNK_ADDR2BASE(ptr) != ptr);
assert(isalloc(ptr) == PAGE_SIZE);
assert(size <= small_maxclass);
chunk = (arena_chunk_t *)CHUNK_ADDR2BASE(ptr);
pageind = ((uintptr_t)ptr - (uintptr_t)chunk) >> PAGE_SHIFT;
deps/jemalloc/src/chunk.c
Vendored
+9 -7
View File
@@ -70,7 +70,7 @@ RETURN:
#ifdef JEMALLOC_IVSALLOC
if (base == false && ret != NULL) {
if (rtree_set(chunks_rtree, (uintptr_t)ret, ret)) {
chunk_dealloc(ret, size);
chunk_dealloc(ret, size, true);
return (NULL);
}
}
@@ -108,7 +108,7 @@ RETURN:
}
void
chunk_dealloc(void *chunk, size_t size)
chunk_dealloc(void *chunk, size_t size, bool unmap)
{
assert(chunk != NULL);
@@ -125,15 +125,17 @@ chunk_dealloc(void *chunk, size_t size)
malloc_mutex_unlock(&chunks_mtx);
#endif
if (unmap) {
#ifdef JEMALLOC_SWAP
if (swap_enabled && chunk_dealloc_swap(chunk, size) == false)
return;
if (swap_enabled && chunk_dealloc_swap(chunk, size) == false)
return;
#endif
#ifdef JEMALLOC_DSS
if (chunk_dealloc_dss(chunk, size) == false)
return;
if (chunk_dealloc_dss(chunk, size) == false)
return;
#endif
chunk_dealloc_mmap(chunk, size);
chunk_dealloc_mmap(chunk, size);
}
}
bool
deps/jemalloc/src/ckh.c
Vendored
+1 -1
View File
@@ -556,7 +556,7 @@ ckh_string_hash(const void *key, unsigned minbits, size_t *hash1, size_t *hash2)
} else {
ret1 = h;
ret2 = hash(key, strlen((const char *)key),
0x8432a476666bbc13U);
0x8432a476666bbc13LLU);
}
*hash1 = ret1;
deps/jemalloc/src/ctl.c
Vendored
+2
View File
@@ -1151,11 +1151,13 @@ thread_arena_ctl(const size_t *mib, size_t miblen, void *oldp, size_t *oldlenp,
/* Set new arena association. */
ARENA_SET(arena);
#ifdef JEMALLOC_TCACHE
{
tcache_t *tcache = TCACHE_GET();
if (tcache != NULL)
tcache->arena = arena;
}
#endif
}
ret = 0;
deps/jemalloc/src/huge.c
Vendored
+14 -7
View File
@@ -110,12 +110,12 @@ huge_palloc(size_t size, size_t alignment, bool zero)
if (offset == 0) {
/* Trim trailing space. */
chunk_dealloc((void *)((uintptr_t)ret + chunk_size), alloc_size
- chunk_size);
- chunk_size, true);
} else {
size_t trailsize;
/* Trim leading space. */
chunk_dealloc(ret, alignment - offset);
chunk_dealloc(ret, alignment - offset, true);
ret = (void *)((uintptr_t)ret + (alignment - offset));
@@ -124,7 +124,7 @@ huge_palloc(size_t size, size_t alignment, bool zero)
/* Trim trailing space. */
assert(trailsize < alloc_size);
chunk_dealloc((void *)((uintptr_t)ret + chunk_size),
trailsize);
trailsize, true);
}
}
@@ -234,6 +234,13 @@ huge_ralloc(void *ptr, size_t oldsize, size_t size, size_t extra,
) {
size_t newsize = huge_salloc(ret);
/*
* Remove ptr from the tree of huge allocations before
* performing the remap operation, in order to avoid the
* possibility of another thread acquiring that mapping before
* this one removes it from the tree.
*/
huge_dalloc(ptr, false);
if (mremap(ptr, oldsize, newsize, MREMAP_MAYMOVE|MREMAP_FIXED,
ret) == MAP_FAILED) {
/*
@@ -253,9 +260,8 @@ huge_ralloc(void *ptr, size_t oldsize, size_t size, size_t extra,
if (opt_abort)
abort();
memcpy(ret, ptr, copysize);
idalloc(ptr);
} else
huge_dalloc(ptr, false);
chunk_dealloc_mmap(ptr, oldsize);
}
} else
#endif
{
@@ -295,9 +301,10 @@ huge_dalloc(void *ptr, bool unmap)
memset(node->addr, 0x5a, node->size);
#endif
#endif
chunk_dealloc(node->addr, node->size);
}
chunk_dealloc(node->addr, node->size, unmap);
base_node_dealloc(node);
}
deps/jemalloc/src/jemalloc.c
Vendored
+66 -32
View File
@@ -84,6 +84,7 @@ static void malloc_conf_error(const char *msg, const char *k, size_t klen,
const char *v, size_t vlen);
static void malloc_conf_init(void);
static bool malloc_init_hard(void);
static int imemalign(void **memptr, size_t alignment, size_t size);
/******************************************************************************/
/* malloc_message() setup. */
@@ -688,7 +689,7 @@ malloc_init_hard(void)
result = sysconf(_SC_PAGESIZE);
assert(result != -1);
pagesize = (unsigned)result;
pagesize = (size_t)result;
/*
* We assume that pagesize is a power of 2 when calculating
@@ -768,6 +769,14 @@ malloc_init_hard(void)
}
#endif
if (malloc_mutex_init(&arenas_lock))
return (true);
if (pthread_key_create(&arenas_tsd, arenas_cleanup) != 0) {
malloc_mutex_unlock(&init_lock);
return (true);
}
/*
* Create enough scaffolding to allow recursive allocation in
* malloc_ncpus().
@@ -794,14 +803,6 @@ malloc_init_hard(void)
ARENA_SET(arenas[0]);
arenas[0]->nthreads++;
if (malloc_mutex_init(&arenas_lock))
return (true);
if (pthread_key_create(&arenas_tsd, arenas_cleanup) != 0) {
malloc_mutex_unlock(&init_lock);
return (true);
}
#ifdef JEMALLOC_PROF
if (prof_boot2()) {
malloc_mutex_unlock(&init_lock);
@@ -939,7 +940,8 @@ JEMALLOC_P(malloc)(size_t size)
#ifdef JEMALLOC_PROF
if (opt_prof) {
usize = s2u(size);
if ((cnt = prof_alloc_prep(usize)) == NULL) {
PROF_ALLOC_PREP(1, usize, cnt);
if (cnt == NULL) {
ret = NULL;
goto OOM;
}
@@ -988,9 +990,15 @@ RETURN:
}
JEMALLOC_ATTR(nonnull(1))
JEMALLOC_ATTR(visibility("default"))
int
JEMALLOC_P(posix_memalign)(void **memptr, size_t alignment, size_t size)
#ifdef JEMALLOC_PROF
/*
* Avoid any uncertainty as to how many backtrace frames to ignore in
* PROF_ALLOC_PREP().
*/
JEMALLOC_ATTR(noinline)
#endif
static int
imemalign(void **memptr, size_t alignment, size_t size)
{
int ret;
size_t usize
@@ -1057,7 +1065,8 @@ JEMALLOC_P(posix_memalign)(void **memptr, size_t alignment, size_t size)
#ifdef JEMALLOC_PROF
if (opt_prof) {
if ((cnt = prof_alloc_prep(usize)) == NULL) {
PROF_ALLOC_PREP(2, usize, cnt);
if (cnt == NULL) {
result = NULL;
ret = EINVAL;
} else {
@@ -1110,6 +1119,15 @@ RETURN:
return (ret);
}
JEMALLOC_ATTR(nonnull(1))
JEMALLOC_ATTR(visibility("default"))
int
JEMALLOC_P(posix_memalign)(void **memptr, size_t alignment, size_t size)
{
return imemalign(memptr, alignment, size);
}
JEMALLOC_ATTR(malloc)
JEMALLOC_ATTR(visibility("default"))
void *
@@ -1165,7 +1183,8 @@ JEMALLOC_P(calloc)(size_t num, size_t size)
#ifdef JEMALLOC_PROF
if (opt_prof) {
usize = s2u(num_size);
if ((cnt = prof_alloc_prep(usize)) == NULL) {
PROF_ALLOC_PREP(1, usize, cnt);
if (cnt == NULL) {
ret = NULL;
goto RETURN;
}
@@ -1278,7 +1297,9 @@ JEMALLOC_P(realloc)(void *ptr, size_t size)
if (opt_prof) {
usize = s2u(size);
old_ctx = prof_ctx_get(ptr);
if ((cnt = prof_alloc_prep(usize)) == NULL) {
PROF_ALLOC_PREP(1, usize, cnt);
if (cnt == NULL) {
old_ctx = NULL;
ret = NULL;
goto OOM;
}
@@ -1288,8 +1309,13 @@ JEMALLOC_P(realloc)(void *ptr, size_t size)
false, false);
if (ret != NULL)
arena_prof_promoted(ret, usize);
} else
else
old_ctx = NULL;
} else {
ret = iralloc(ptr, size, 0, 0, false, false);
if (ret == NULL)
old_ctx = NULL;
}
} else
#endif
{
@@ -1327,7 +1353,8 @@ OOM:
#ifdef JEMALLOC_PROF
if (opt_prof) {
usize = s2u(size);
if ((cnt = prof_alloc_prep(usize)) == NULL)
PROF_ALLOC_PREP(1, usize, cnt);
if (cnt == NULL)
ret = NULL;
else {
if (prof_promote && (uintptr_t)cnt !=
@@ -1432,7 +1459,7 @@ JEMALLOC_P(memalign)(size_t alignment, size_t size)
#ifdef JEMALLOC_CC_SILENCE
int result =
#endif
JEMALLOC_P(posix_memalign)(&ret, alignment, size);
imemalign(&ret, alignment, size);
#ifdef JEMALLOC_CC_SILENCE
if (result != 0)
return (NULL);
@@ -1451,7 +1478,7 @@ JEMALLOC_P(valloc)(size_t size)
#ifdef JEMALLOC_CC_SILENCE
int result =
#endif
JEMALLOC_P(posix_memalign)(&ret, PAGE_SIZE, size);
imemalign(&ret, PAGE_SIZE, size);
#ifdef JEMALLOC_CC_SILENCE
if (result != 0)
return (NULL);
@@ -1566,14 +1593,14 @@ JEMALLOC_P(allocm)(void **ptr, size_t *rsize, size_t size, int flags)
if (malloc_init())
goto OOM;
usize = (alignment == 0) ? s2u(size) : sa2u(size, alignment,
NULL);
usize = (alignment == 0) ? s2u(size) : sa2u(size, alignment, NULL);
if (usize == 0)
goto OOM;
#ifdef JEMALLOC_PROF
if (opt_prof) {
if ((cnt = prof_alloc_prep(usize)) == NULL)
PROF_ALLOC_PREP(1, usize, cnt);
if (cnt == NULL)
goto OOM;
if (prof_promote && (uintptr_t)cnt != (uintptr_t)1U && usize <=
small_maxclass) {
@@ -1590,7 +1617,7 @@ JEMALLOC_P(allocm)(void **ptr, size_t *rsize, size_t size, int flags)
if (p == NULL)
goto OOM;
}
prof_malloc(p, usize, cnt);
if (rsize != NULL)
*rsize = usize;
} else
@@ -1645,7 +1672,6 @@ JEMALLOC_P(rallocm)(void **ptr, size_t *rsize, size_t size, size_t extra,
bool no_move = flags & ALLOCM_NO_MOVE;
#ifdef JEMALLOC_PROF
prof_thr_cnt_t *cnt;
prof_ctx_t *old_ctx;
#endif
assert(ptr != NULL);
@@ -1660,25 +1686,33 @@ JEMALLOC_P(rallocm)(void **ptr, size_t *rsize, size_t size, size_t extra,
/*
* usize isn't knowable before iralloc() returns when extra is
* non-zero. Therefore, compute its maximum possible value and
* use that in prof_alloc_prep() to decide whether to capture a
* use that in PROF_ALLOC_PREP() to decide whether to capture a
* backtrace. prof_realloc() will use the actual usize to
* decide whether to sample.
*/
size_t max_usize = (alignment == 0) ? s2u(size+extra) :
sa2u(size+extra, alignment, NULL);
prof_ctx_t *old_ctx = prof_ctx_get(p);
old_size = isalloc(p);
old_ctx = prof_ctx_get(p);
if ((cnt = prof_alloc_prep(max_usize)) == NULL)
PROF_ALLOC_PREP(1, max_usize, cnt);
if (cnt == NULL)
goto OOM;
if (prof_promote && (uintptr_t)cnt != (uintptr_t)1U && max_usize
<= small_maxclass) {
/*
* Use minimum usize to determine whether promotion may happen.
*/
if (prof_promote && (uintptr_t)cnt != (uintptr_t)1U
&& ((alignment == 0) ? s2u(size) : sa2u(size,
alignment, NULL)) <= small_maxclass) {
q = iralloc(p, small_maxclass+1, (small_maxclass+1 >=
size+extra) ? 0 : size+extra - (small_maxclass+1),
alignment, zero, no_move);
if (q == NULL)
goto ERR;
usize = isalloc(q);
arena_prof_promoted(q, usize);
if (max_usize < PAGE_SIZE) {
usize = max_usize;
arena_prof_promoted(q, usize);
} else
usize = isalloc(q);
} else {
q = iralloc(p, size, extra, alignment, zero, no_move);
if (q == NULL)
deps/jemalloc/src/prof.c
Vendored
+49 -48
View File
@@ -474,11 +474,23 @@ prof_lookup(prof_bt_t *bt)
/*
* Artificially raise curobjs, in order to avoid a race
* condition with prof_ctx_merge()/prof_ctx_destroy().
*
* No locking is necessary for ctx here because no other
* threads have had the opportunity to fetch it from
* bt2ctx yet.
*/
ctx.p->cnt_merged.curobjs++;
new_ctx = true;
} else
} else {
/*
* Artificially raise curobjs, in order to avoid a race
* condition with prof_ctx_merge()/prof_ctx_destroy().
*/
malloc_mutex_lock(&ctx.p->lock);
ctx.p->cnt_merged.curobjs++;
malloc_mutex_unlock(&ctx.p->lock);
new_ctx = false;
}
prof_leave();
/* Link a prof_thd_cnt_t into ctx for this thread. */
@@ -491,8 +503,9 @@ prof_lookup(prof_bt_t *bt)
*/
ret.p = ql_last(&prof_tdata->lru_ql, lru_link);
assert(ret.v != NULL);
ckh_remove(&prof_tdata->bt2cnt, ret.p->ctx->bt, NULL,
NULL);
if (ckh_remove(&prof_tdata->bt2cnt, ret.p->ctx->bt,
NULL, NULL))
assert(false);
ql_remove(&prof_tdata->lru_ql, ret.p, lru_link);
prof_ctx_merge(ret.p->ctx, ret.p);
/* ret can now be re-used. */
@@ -503,11 +516,8 @@ prof_lookup(prof_bt_t *bt)
/* Allocate and partially initialize a new cnt. */
ret.v = imalloc(sizeof(prof_thr_cnt_t));
if (ret.p == NULL) {
if (new_ctx) {
malloc_mutex_lock(&ctx.p->lock);
ctx.p->cnt_merged.curobjs--;
malloc_mutex_unlock(&ctx.p->lock);
}
if (new_ctx)
prof_ctx_destroy(ctx.p);
return (NULL);
}
ql_elm_new(ret.p, cnts_link);
@@ -518,19 +528,15 @@ prof_lookup(prof_bt_t *bt)
ret.p->epoch = 0;
memset(&ret.p->cnts, 0, sizeof(prof_cnt_t));
if (ckh_insert(&prof_tdata->bt2cnt, btkey.v, ret.v)) {
if (new_ctx) {
malloc_mutex_lock(&ctx.p->lock);
ctx.p->cnt_merged.curobjs--;
malloc_mutex_unlock(&ctx.p->lock);
}
if (new_ctx)
prof_ctx_destroy(ctx.p);
idalloc(ret.v);
return (NULL);
}
ql_head_insert(&prof_tdata->lru_ql, ret.p, lru_link);
malloc_mutex_lock(&ctx.p->lock);
ql_tail_insert(&ctx.p->cnts_ql, ret.p, cnts_link);
if (new_ctx)
ctx.p->cnt_merged.curobjs--;
ctx.p->cnt_merged.curobjs--;
malloc_mutex_unlock(&ctx.p->lock);
} else {
/* Move ret to the front of the LRU. */
@@ -644,11 +650,10 @@ prof_ctx_destroy(prof_ctx_t *ctx)
/*
* Check that ctx is still unused by any thread cache before destroying
* it. prof_lookup() interlocks bt2ctx_mtx and ctx->lock in order to
* avoid a race condition with this function, and prof_ctx_merge()
* artificially raises ctx->cnt_merged.curobjs in order to avoid a race
* between the main body of prof_ctx_merge() and entry into this
* function.
* it. prof_lookup() artificially raises ctx->cnt_merge.curobjs in
* order to avoid a race condition with this function, as does
* prof_ctx_merge() in order to avoid a race between the main body of
* prof_ctx_merge() and entry into this function.
*/
prof_enter();
malloc_mutex_lock(&ctx->lock);
@@ -657,7 +662,8 @@ prof_ctx_destroy(prof_ctx_t *ctx)
assert(ctx->cnt_merged.accumobjs == 0);
assert(ctx->cnt_merged.accumbytes == 0);
/* Remove ctx from bt2ctx. */
ckh_remove(&bt2ctx, ctx->bt, NULL, NULL);
if (ckh_remove(&bt2ctx, ctx->bt, NULL, NULL))
assert(false);
prof_leave();
/* Destroy ctx. */
malloc_mutex_unlock(&ctx->lock);
@@ -665,7 +671,10 @@ prof_ctx_destroy(prof_ctx_t *ctx)
malloc_mutex_destroy(&ctx->lock);
idalloc(ctx);
} else {
/* Compensate for increment in prof_ctx_merge(). */
/*
* Compensate for increment in prof_ctx_merge() or
* prof_lookup().
*/
ctx->cnt_merged.curobjs--;
malloc_mutex_unlock(&ctx->lock);
prof_leave();
@@ -1072,7 +1081,7 @@ prof_bt_hash(const void *key, unsigned minbits, size_t *hash1, size_t *hash2)
} else {
ret1 = h;
ret2 = hash(bt->vec, bt->len * sizeof(void *),
0x8432a476666bbc13U);
0x8432a476666bbc13LLU);
}
*hash1 = ret1;
@@ -1109,7 +1118,6 @@ prof_tdata_init(void)
prof_tdata->vec = imalloc(sizeof(void *) * prof_bt_max);
if (prof_tdata->vec == NULL) {
ckh_delete(&prof_tdata->bt2cnt);
idalloc(prof_tdata);
return (NULL);
@@ -1127,33 +1135,26 @@ prof_tdata_init(void)
static void
prof_tdata_cleanup(void *arg)
{
prof_tdata_t *prof_tdata;
prof_thr_cnt_t *cnt;
prof_tdata_t *prof_tdata = (prof_tdata_t *)arg;
prof_tdata = PROF_TCACHE_GET();
if (prof_tdata != NULL) {
prof_thr_cnt_t *cnt;
/*
* Delete the hash table. All of its contents can still be iterated
* over via the LRU.
*/
ckh_delete(&prof_tdata->bt2cnt);
/*
* Delete the hash table. All of its contents can still be
* iterated over via the LRU.
*/
ckh_delete(&prof_tdata->bt2cnt);
/*
* Iteratively merge cnt's into the global stats and delete
* them.
*/
while ((cnt = ql_last(&prof_tdata->lru_ql, lru_link)) != NULL) {
prof_ctx_merge(cnt->ctx, cnt);
ql_remove(&prof_tdata->lru_ql, cnt, lru_link);
idalloc(cnt);
}
idalloc(prof_tdata->vec);
idalloc(prof_tdata);
PROF_TCACHE_SET(NULL);
/* Iteratively merge cnt's into the global stats and delete them. */
while ((cnt = ql_last(&prof_tdata->lru_ql, lru_link)) != NULL) {
ql_remove(&prof_tdata->lru_ql, cnt, lru_link);
prof_ctx_merge(cnt->ctx, cnt);
idalloc(cnt);
}
idalloc(prof_tdata->vec);
idalloc(prof_tdata);
PROF_TCACHE_SET(NULL);
}
void
deps/jemalloc/src/stats.c
Vendored
+1 -1
View File
@@ -748,7 +748,7 @@ stats_print(void (*write_cb)(void *, const char *), void *cbopaque,
ninitialized++;
}
if (ninitialized > 1) {
if (ninitialized > 1 || unmerged == false) {
/* Print merged arena stats. */
malloc_cprintf(write_cb, cbopaque,
"\nMerged arenas stats:\n");
deps/jemalloc/test/rallocm.c
Vendored
+14 -4
View File
@@ -1,6 +1,8 @@
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <assert.h>
#define JEMALLOC_MANGLE
#include "jemalloc_test.h"
@@ -8,12 +10,20 @@
int
main(void)
{
size_t pagesize;
void *p, *q;
size_t sz, tsz;
int r;
fprintf(stderr, "Test begin\n");
/* Get page size. */
{
long result = sysconf(_SC_PAGESIZE);
assert(result != -1);
pagesize = (size_t)result;
}
r = JEMALLOC_P(allocm)(&p, &sz, 42, 0);
if (r != ALLOCM_SUCCESS) {
fprintf(stderr, "Unexpected allocm() error\n");
@@ -66,7 +76,7 @@ main(void)
p = q;
sz = tsz;
r = JEMALLOC_P(rallocm)(&q, &tsz, 8192, 0, 0);
r = JEMALLOC_P(rallocm)(&q, &tsz, pagesize*2, 0, 0);
if (r != ALLOCM_SUCCESS)
fprintf(stderr, "Unexpected rallocm() error\n");
if (q == p)
@@ -78,7 +88,7 @@ main(void)
p = q;
sz = tsz;
r = JEMALLOC_P(rallocm)(&q, &tsz, 16384, 0, 0);
r = JEMALLOC_P(rallocm)(&q, &tsz, pagesize*4, 0, 0);
if (r != ALLOCM_SUCCESS)
fprintf(stderr, "Unexpected rallocm() error\n");
if (tsz == sz) {
@@ -88,7 +98,7 @@ main(void)
p = q;
sz = tsz;
r = JEMALLOC_P(rallocm)(&q, &tsz, 8192, 0, ALLOCM_NO_MOVE);
r = JEMALLOC_P(rallocm)(&q, &tsz, pagesize*2, 0, ALLOCM_NO_MOVE);
if (r != ALLOCM_SUCCESS)
fprintf(stderr, "Unexpected rallocm() error\n");
if (q != p)
@@ -99,7 +109,7 @@ main(void)
}
sz = tsz;
r = JEMALLOC_P(rallocm)(&q, &tsz, 16384, 0, ALLOCM_NO_MOVE);
r = JEMALLOC_P(rallocm)(&q, &tsz, pagesize*4, 0, ALLOCM_NO_MOVE);
if (r != ALLOCM_SUCCESS)
fprintf(stderr, "Unexpected rallocm() error\n");
if (q != p)
design-documents/REDIS-CLUSTER
-214
View File
@@ -1,214 +0,0 @@
Redis Cluster Design Proposal (work in progress)
28 Nov 2010: Ver 1.0 - initial version
22 APr 2010: Ver 1.1 - more details and rationales
Overview
========
Redis is a fast key-value store supporting complex aggregate data types as
values. For instance keys can be bound to lists with many elements, sets,
sub-dictionaries (hashes) and so forth.
While Redis is very fast, currently it lacks scalability in the form of ability
to transparently run across different nodes. This is desirable mainly for the
following three rasons:
A) Fault tolerance. Some node may go off line without affecting the operations.
B) Holding bigger datasets without using a single box with a lot of RAM.
C) Scaling writes.
Since a single Redis instance supports 140,000 operations per second in a good
Linux box costing less than $1000, the need for Redis Cluster arises more
from "A" and "B". Scaling writes can also be useful in very high load
environments. Scaling reads is already easily accomplished using Redis built-in
replication.
Design goals
============
Designing a DHT in 2010 is hard as there is too much bias towards good designs
that are already well tested in practice, like the Amazon Dynamo design.
Still a Dynamo alike DHT may not be the best fit for Redis.
Redis is very simple and fast at its core, so Redis cluster should try to
follow the same guidelines. The first problem with a Dynamo-alike DHT is that
Redis supports complex data types. Merging complex values like lsits, where
in the case of a netsplit may diverge in very complex ways, is not going to
be easy. The "most recent data" wins is not applicable and all the resolution
business should be in the application.
Even a simple application can end up with complex schema of keys and complex
values. Writing code in order to resolve conflicts is not going to be
programmer friendly.
So the author of this document claims that Redis does not need to resist to
netsplits, but it is enough to resist to M-1 nodes going offline, where
M is the number of nodes storing every key-value pair.
For instance in a three nodes cluster I may configure the cluster in order to
store every key into two instances (M=2). Such a cluster can resist to a single
node going offline without interruption of the service.
When more than M-1 nodes are off line the cluster should detect such a condition
and refusing any further query. The system administrator should check why
M-1 nodes are offline and bring them back again if possible.
Once resisting to big net splits is no longer a requirement as there is no
conflict resolution stage, since at least an original node responsible of
holding every possible key must be online for the cluster to work, there is
also no need for a design where every node can act as an independent entity
receiving queries and forwarding this queries to other nodes as needed.
Instead a more decoupled approach can be used, in the form of a Redis Proxy
node (or multiple Proxy nodes) that is contacted by clients, and
is responsible of forwarding queries and replies back and forth from data nodes.
Data nodes can be just vanilla redis-server instances.
Network layout
==============
- One ore more Data Nodes. Every node is identified by ip:port.
- A single Configuration Node.
- One more more Proxy Nodes (redis-cluster nodes).
- A single Handling Node.
Data Nodes and the Configuration Node are just vanilla redis-server instances.
Configuration Node
==================
- Contains information about all the Data nodes in the cluster.
- Contains information about all the Proxy nodes in the cluster.
- Contains information about what Data Node holds a given sub-space of keys.
The keyspace is divided into 1024 different "hashing slots".
(1024 is just an example, this value should be configurable)
Given a key perform SHA1(key) and use the last 10 bits of the result to get a 10 bit number representing the "key slot" (from 0 to 1023).
The Configuration node maps every slot of the keyspace to M different Data Nodes (every key is stored into M nodes, configurable).
The Configuration node can be modified by a single client at a time. Locking is performed using SETNX.
The Configuration node should be replicated as there is a single configuration node for the whole network. It is the only single point of failure of the system.
When a Configuration node fails the cluster does not stop operating, but is not
able to recover if there is some exceptional condition to handle, like a Data
Node going off line or the addition of a new Data Node to the cluster.
The Configuration node is a standard Redis server, like every other Data node.
Data Nodes
==========
Data nodes just hold data, and are normal Redis processes. There is no configuration stored on nodes, nor the nodes are "active" in the cluster, they just receive normal Redis commands.
Proxy Nodes
===========
Proxy nodes get requests from clients and route this requests to the right Redis nodes.
Proxy nodes take persistent connections to all the Data Nodes and the
Configuration Node. This connections are keep alive with PING requests from time
to time if there is no traffic. This way Proxy Nodes can understand asap if
there is a problem in some Data Node or in the Configuration Node.
When a Proxy Node is started it needs to know the Configuration node address in order to load the infomration about the Data nodes and the mapping between the key space and the nodes.
On startup a Proxy Node will also register itself in the Configuration node, and will make sure to refresh it's configuration every N seconds (via an EXPIREing key) so that it's possible to detect when a Proxy node fails.
Clients can submit queries to any Proxy Node, so well designed clients may ask
at startup the list of Proxy Nodes querying the Configuration Node. Then if
a query fails against a given Proxy Node it can be retried against the next.
The Proxy Node is also in charge of signaling failing Data nodes to the Configuration node, so that the Handling Node can take appropriate actions.
When a new Data node joins or leaves the cluster, and in general when the cluster configuration changes, all the Proxy nodes will receive a notification and will reload the configuration from the Configuration node.
Proxy Nodes - how queries are submited
======================================
This is how a query is processed:
1) A client sends a query to a Proxy Node, using the Redis protocol like if it was a plain Redis Node.
2) The Proxy Node inspects the command arguments to detect the key. The key is hashed. The Proxy Node has the table mapping a given key to M nodes, and persistent connections to all the nodes.
At this point the process is different in case of read or write queries:
WRITE QUERY:
3a) The Proxy Node forwards the query to M Data Nodes at the same time, waiting for replies.
3b) Once all the replies are received the Proxy Node checks that the replies are consistent. For instance all the M nodes need to reply with OK and so forth. If the query fails in a subset of nodes but succeeds in other nodes, the failing nodes are considered unreliable and are put off line notifying the configuration node.
3c) The reply is transfered back to the client.
READ QUERY:
3d) The Proxy Node forwards the query to a single random client, passing the reply back to the client.
Handling Node
=============
The handling node is a special Redis client with the following role:
- Handles the cluster configuration stored in the Config node.
- Is in charge for adding and removing nodes dynamically from the net.
- Relocates keys on nodes additions / removal.
- Signal a configuration change to Proxy nodes.
More details on hashing slots
============================
The Configuration node holds 1024 keys in the following form:
hashingslot:0
hashingslot:1
...
hashingslot:1023
Every hashing slot is actually a Redis list, containing a single or more ip:port pairs. For instance:
hashingslot:10 => 192.168.1.19:6379, 192.168.1.200:6379
This mean that keys hashing to slot 10 will be saved in the two Data nodes 192.168.1.19:6379 and 192.168.1.200:6379.
When a client performs a read operation (via a proxy node), the proxy will contact a random Data node among the data nodes in charge for the given slot.
For instance a client can ask for the following operation to a given Proxy node:
GET mykey
"mykey" hashes to (for instance) slot 10, so the Proxy will forward the request to either Data node 192.168.1.19:6379 or 192.168.1.200:6379, and then forward back the reply to the client.
When a write operation is performed, it is forwarded to both the Data nodes in the example (and in general to all the data nodes).
Adding or removing a node
=========================
When a Data node is added to the cluster, it is added via an LPUSH operation into a Redis list representing a queue of Data nodes that are ready to enter the cluster. This list is hold by the Configuration node of course, and can be added manually or via a configuration utility.
LPUSH newnodes 192.168.1.55:6379
The Handling node will check from time to time for this new elements in the "newode" list. If there are new nodes pending to enter the cluster, they are processed one after the other in this way:
For instance let's assume there are already two Data nodes in the cluster:
192.168.1.1:6379
192.168.1.2:6379
We add a new node 192.168.1.3:6379 via the LPUSH operation.
We can imagine that the 1024 hash slots are assigned equally among the two inital nodes. In order to add the new (third) node what we have to do is to move incrementally 341 slots form the two old servers to the new one.
For now we can think that every hash slot is only stored in a single server, to generalize the idea later.
In order to simplify the implementation every slot can be moved from one Data node to another one in a blocking way, that is, read operations will continue to all the 1024 slots, but a single slot at a time will delay write operations until the moving from one Data node to another is completed.
In order to do so the Handler node, before to move a given node, marks it as "write-locked" in the Configuration server, than asks all the Proxy nodes to refresh the configuration.
Then the slot is moved (1/1024 of all the keys). The Configuration server is modified to reflect the new hashing slots configuration, the slot is unlocked, the Proxy nodes notified.
Implementation details
======================
To run the Handling node and the Configuration node in the same physical computer is probably a good idea.
design-documents/REDIS-CLUSTER-2
-343
View File
@@ -1,343 +0,0 @@
Redis Cluster - Alternative 1
28 Apr 2010: Ver 1.0 - initial version
Overview
========
The motivations and design goals of Redis Cluster are already outlined in the
first design document of Redis Cluster. This document is just an attempt to
provide a completely alternative approach in order to explore more ideas.
In this document the alternative explored is a cluster where communication is
performed directly from client to the target node, without intermediate layer.
The intermediate layer can be used, in the form of a proxy, in order to provide
the same functionality to clients not able to directly use the cluster protocol.
So in a first stage clients can use a proxy to implement the hash ring, but
later this clients can switch to a native implementation, following a
specification that the Redis project will provide.
In this new design fault tolerance is achieved by replicating M-1 times every
data node instead of storing the same key M times across nodes.
From the point of view of CAP our biggest sacrifice is about "P", that is
resistance to partitioning. Only M-1 nodes can go down for the cluster still
be functional. Also when possible "A" is somewhat sacrificed for "L", that
is, Latency. Not really in the CAP equation but a very important parameter.
Network layout
==============
In this alternative design the network layout is simple as there are only
clients talking directly to N data nodes. So we can imagine to have:
- K Redis clients, directly talking to the data nodes.
- N Redis data nodes, that are, normal Redis instances.
Data nodes are replicate M-1 times (so there are a total of M copies for
every node). If M is one, the system is not fault tolerant. If M is 2 one
data node can go off line without affecting the operations. And so forth.
Hash slots
==========
The key space is divided into 1024 slots.
Given a key, the SHA1 function is applied to it.
The first 10 bytes of the SHA1 digest are interpreted as an unsigned integer
from 0 to 1023. This is the hash slot of the key.
Data nodes
==========
Data nodes are normal Redis instances, but a few additional commands are
provided.
HASHRING ADD ... list of hash slots ...
HASHRING DEL ... list of hash slots ...
HASHRING REHASHING slot
HASHRING SLOTS => returns the list of configured slots
HSAHRING KEYS ... list of hash slots ...
By default Redis instances are configured to accept operations about all
the hash slots. With this commands it's possible to configure a Redis instance
to accept only a subset of the key space.
If an operation is performed against a key hashing to a slot that is not
configured to be accepted, the Redis instance will reply with:
"-ERR wrong hash slot"
More details on the HASHRING command and sub commands will be showed later
in this document.
Additionally three other commands are added:
DUMP key
RESTORE key <dump data>
MIGRATE key host port
DUMP is used to output a very compact binary representation of the data stored at key.
RESTORE re-creates a value (storing it at key) starting from the output produced by DUMP.
MIGRATE is like a server-side DUMP+RESTORE command. This atomic command moves one key from the connected instance to another instance, returning the status code of the operation (+OK or an error).
The protocol described in this draft only uses the MIGRATE command, but this in turn will use RESTORE internally when connecting to another server, and DUMP is provided for symmetry.
Querying the cluster
====================
1) Reading the cluster config
-----------------------------
Clients of the cluster are required to have the cluster configuration loaded
into memory. The cluster configuration is the sum of the following info:
- Number of data nodes in the cluster, for instance, 10
- A map between hash slots and nodes, so for instnace:
hash slot 1 -> node 0
hash slot 2 -> node 5
hash slot 3 -> node 3
... and so forth ...
- Physical address of nodes, and their replicas.
node 0 addr -> 192.168.1.100
node 0 replicas -> 192.168.1.101, 192.168.1.105
- Configuration version: the SHA1 of the whole configuration
The configuration is stored in every single data node of the cluster.
A client without the configuration in memory is require, as a first step, to
read the config. In order to do so the client requires to have a list of IPs
that are with good probability data nodes of the cluster.
The client will try to get the config from all this nodes. If no node is found
responding, an error is reported to the user.
2) Caching and refreshing the configuration
-------------------------------------------
A node is allowed to cache the configuration in memory or in a different way
(for instance storing the configuration into a file), but every client is
required to check if the configuration changed at max every 10 seconds, asking
for the configuration version key with a single GET call, and checking if the
configuration version matches the one loaded in memory.
Also a client is required to refresh the configuration every time a node
replies with:
"-ERR wrong hash slot"
As this means that hash slots were reassigned in some way.
Checking the configuration every 10 seconds is not required in theory but is
a good protection against errors and failures that may happen in real world
environments. It is also very cheap to perform, as a GET operation from time
to time is going to have no impact in the overall performance.
3) Read query
-------------
To perform a read query the client hashes the key argument from the command
(in the intiial version of Redis Cluster only single-key commands are
allowed). Using the in memory configuration it maps the hash key to the
node ID.
If the client is configured to support read-after-write consistency, then
the "master" node for this hash slot is queried.
Otherwise the client picks a random node from the master and the replicas
available.
4) Write query
--------------
A write query is exactly like a read query, with the difference that the
write always targets the master node, instead of the replicas.
Creating a cluster
==================
In order to create a new cluster, the redis-cluster command line utility is
used. It gets a list of available nodes and replicas, in order to write the
initial configuration in all the nodes.
At this point the cluster is usable by clients.
Adding nodes to the cluster
===========================
The command line utility redis-cluster is used in order to add a node to the
cluster:
1) The cluster configuration is loaded.
2) A fair number of hash slots are assigned to the new data node.
3) Hash slots moved to the new node are marked as "REHASHING" in the old
nodes, using the HASHRING command:
HASHRING SETREHASHING 1 192.168.1.103 6380
The above command set the hash slot "1" in rehashing state, with the
"forwarding address" to 192.168.1.103:6380. As a result if this node receives
a query about a key hashing to hash slot 1, that *is not present* in the
current data set, it replies with:
"-MIGRATED 192.168.1.103:6380"
The client can then reissue the query against the new node.
Instead even if the hash slot is marked as rehashing but the requested key
is still there, the query is processed. This allows for non blocking
rehashing.
Note that no additional memory is used by Redis in order to provide such a
feature.
4) While the Hash slot is marked as "REHASHING", redis-cluster asks this node
the list of all the keys matching the specified hash slot. Then all the keys
are moved to the new node using the MIGRATE command.
5) Once all the keys are migrated, the hash slot is deleted from the old
node configuration with "HASHRING DEL 1". And the configuration is update.
Using this algorithm all the hash slots are migrated one after the other to the new node. In practical implementation before to start the migration the
redis-cluster utility should write a log into the configuration so that
in case of crash or any other problem the utility is able to recover from
were it left.
Fault tolerance
===============
Fault tolerance is reached replicating every data node M-1 times, so that we
have one master and M-1 replicas for a total of M nodes holding the same
hash slots. Up to M-1 nodes can go down without affecting the cluster.
The tricky part about fault tolerance is detecting when a node is failing and
signaling it to all the other clients.
When a master node is failing in a permanent way, promoting the first slave
is easy:
1) At some point a client will notice there are problems accessing a given node. It will try to refresh the config, but will notice that the config is already up to date.
2) In order to make sure the problem is not about the client connectivity itself, it will try to reach other nodes as well. If more than M-1 nodes appear to be down, it's either a client networking problem or alternatively the cluster can't be fixed as too many nodes are down anyway. So no action is taken, but an error is reported.
3) If instead only 1 or at max M-1 nodes appear to be down, the client promotes a slave as master and writes the new configuration to all the data nodes.
All the other clients will see the data node not working, and as a first step will try to refresh the configuration. They will successful refresh the configuration and the cluster will work again.
Every time a slave is promoted, the information is written in a log that is actually a Redis list, in all the data nodes, so that system administration tools can detect what happened in order to send notifications to the admin.
Intermittent problems
---------------------
In the above scenario a master was failing in a permanent way. Now instead
let's think to a case where a network cable is not working well so a node
appears to be a few seconds up and a few seconds down.
When this happens recovering can be much harder, as a client may notice the
problem and will promote a slave to master as a result, but then the host
will be up again and the other clients will not see the problem, writing to
the old master for at max 10 seconds (after 10 seconds all the clients are
required to perform a few GETs to check the configuration version of the
cluster and update if needed).
One way to fix this problem is to delegate the fail over mechanism to a
failover agent. When clients notice problems will not take any active action
but will just log the problem into a redis list in all the reachable nodes,
wait, check for configuration change, and retry.
The failover agent constantly monitor this logs: if some client is reporting
a failing node, it can take appropriate actions, checking if the failure is
permanent or not. If it's not he can send a SHUTDOWN command to the failing
master if possible. The failover agent can also consider better the problem
checking if the failing mode is advertised by all the clients or just a single
one, and can check itself if there is a real problem before to proceed with
the fail over.
Redis proxy
===========
In order to make the switch to the clustered version of Redis simpler, and
because the client-side protocol is non trivial to implement compared to the
usual Redis client lib protocol (where a minimal lib can be as small as
100 lines of code), a proxy will be provided to implement the cluster protocol
as a proxy.
Every client will talk to a redis-proxy node that is responsible of using
the new protocol and forwarding back the replies.
In the long run the aim is to switch all the major client libraries to the
new protocol in a native way.
Supported commands
==================
Because with this design we talk directly to data nodes and there is a single
"master" version of every value (that's the big gain dropping "P" from CAP!)
almost all the redis commands can be supported by the clustered version
including MULTI/EXEC and multi key commands as long as all the keys will hash
to the same hash slot. In order to guarantee this, key tags can be used,
where when a specific pattern is present in the key name, only that part is
hashed in order to obtain the hash index.
Random remarks
==============
- It's still not clear how to perform an atomic election of a slave to master.
- In normal conditions (all the nodes working) this new design is just
K clients talking to N nodes without intermediate layers, no routes:
this means it is horizontally scalable with O(1) lookups.
- The cluster should optionally be able to work with manual fail over
for environments where it's desirable to do so. For instance it's possible
to setup periodic checks on all the nodes, and switch IPs when needed
or other advanced configurations that can not be the default as they
are too environment dependent.
A few ideas about client-side slave election
============================================
Detecting failures in a collaborative way
-----------------------------------------
In order to take the node failure detection and slave election a distributed
effort, without any "control program" that is in some way a single point
of failure (the cluster will not stop when it stops, but errors are not
corrected without it running), it's possible to use a few consensus-alike
algorithms.
For instance all the nodes may take a list of errors detected by clients.
If Client-1 detects some failure accessing Node-3, for instance a connection
refused error or a timeout, it logs what happened with LPUSH commands against
all the other nodes. This "error messages" will have a timestamp and the Node
id. Something like:
LPUSH __cluster__:errors 3:1272545939
So if the error is reported many times in a small amount of time, at some
point a client can have enough hints about the need of performing a
slave election.
Atomic slave election
---------------------
In order to avoid races when electing a slave to master (that is in order to
avoid that some client can still contact the old master for that node in
the 10 seconds timeframe), the client performing the election may write
some hint in the configuration, change the configuration SHA1 accordingly and
wait for more than 10 seconds, in order to be sure all the clients will
refresh the configuration before a new access.
The config hint may be something like:
"we are switching to a new master, that is x.y.z.k:port, in a few seconds"
When a client updates the config and finds such a flag set, it starts to
continuously refresh the config until a change is noticed (this will take
at max 10-15 seconds).
The client performing the election will wait that famous 10 seconds time frame
and finally will update the config in a definitive way setting the new
slave as mater. All the clients at this point are guaranteed to have the new
config either because they refreshed or because in the next query their config
is already expired and they'll update the configuration.
EOF
redis.conf
+17 -1
View File
@@ -34,9 +34,10 @@ port 6379
# on a unix socket when not specified.
#
# unixsocket /tmp/redis.sock
# unixsocketperm 755
# Close the connection after a client is idle for N seconds (0 to disable)
timeout 300
timeout 0
# Set server verbosity to 'debug'
# it can be one of:
@@ -134,6 +135,21 @@ dir ./
#
slave-serve-stale-data yes
# Slaves send PINGs to server in a predefined interval. It's possible to change
# this interval with the repl_ping_slave_period option. The default value is 10
# seconds.
#
# repl-ping-slave-period 10
# The following option sets a timeout for both Bulk transfer I/O timeout and
# master data or ping response timeout. The default value is 60 seconds.
#
# It is important to make sure that this value is greater than the value
# specified for repl-ping-slave-period otherwise a timeout will be detected
# every time there is low traffic between the master and the slave.
#
# repl-timeout 60
################################## SECURITY ###################################
# Require clients to issue AUTH <PASSWORD> before processing any other
runtest
Executable
+9
View File
@@ -0,0 +1,9 @@
#!/bin/bash
TCL=tclsh8.5
which $TCL
if [ "$?" != "0" ]
then
echo "You need '$TCL' in order to run the Redis test"
exit 1
fi
$TCL tests/test_helper.tcl $*
src/Makefile
+7 -4
View File
@@ -58,7 +58,7 @@ PREFIX= /usr/local
INSTALL_BIN= $(PREFIX)/bin
INSTALL= cp -p
OBJ = adlist.o ae.o anet.o dict.o redis.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o vm.o pubsub.o multi.o debug.o sort.o intset.o syncio.o slowlog.o
OBJ = adlist.o ae.o anet.o dict.o redis.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o vm.o pubsub.o multi.o debug.o sort.o intset.o syncio.o slowlog.o bio.o
BENCHOBJ = ae.o anet.o redis-benchmark.o sds.o adlist.o zmalloc.o
CLIOBJ = anet.o sds.o adlist.o redis-cli.o zmalloc.o release.o
CHECKDUMPOBJ = redis-check-dump.o lzf_c.o lzf_d.o
@@ -84,6 +84,8 @@ ae_select.o: ae_select.c
anet.o: anet.c fmacros.h anet.h
aof.o: aof.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
bio.o: bio.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h bio.h
config.o: config.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
db.o: db.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
@@ -113,7 +115,8 @@ redis-check-dump.o: redis-check-dump.c lzf.h
redis-cli.o: redis-cli.c fmacros.h version.h ../deps/hiredis/hiredis.h \
sds.h zmalloc.h ../deps/linenoise/linenoise.h help.h
redis.o: redis.c redis.h fmacros.h config.h ae.h sds.h dict.h adlist.h \
zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h slowlog.h
zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h slowlog.h \
bio.h
release.o: release.c release.h
replication.o: replication.c redis.h fmacros.h config.h ae.h sds.h dict.h \
adlist.h zmalloc.h anet.h zipmap.h ziplist.h intset.h version.h util.h
@@ -195,8 +198,8 @@ clean:
dep:
$(CC) -MM *.c -I ../deps/hiredis -I ../deps/linenoise
test: redis-server
(cd ..; tclsh8.5 tests/test_helper.tcl --tags "${TAGS}")
test: redis-server redis-check-aof
@(cd ..; ./runtest)
bench:
./redis-benchmark
src/ae.c
+7
View File
@@ -118,6 +118,13 @@ void aeDeleteFileEvent(aeEventLoop *eventLoop, int fd, int mask)
aeApiDelEvent(eventLoop, fd, mask);
}
int aeGetFileEvents(aeEventLoop *eventLoop, int fd) {
if (fd >= AE_SETSIZE) return 0;
aeFileEvent *fe = &eventLoop->events[fd];
return fe->mask;
}
static void aeGetTime(long *seconds, long *milliseconds)
{
struct timeval tv;
src/ae.h
+1
View File
@@ -104,6 +104,7 @@ void aeStop(aeEventLoop *eventLoop);
int aeCreateFileEvent(aeEventLoop *eventLoop, int fd, int mask,
aeFileProc *proc, void *clientData);
void aeDeleteFileEvent(aeEventLoop *eventLoop, int fd, int mask);
int aeGetFileEvents(aeEventLoop *eventLoop, int fd);
long long aeCreateTimeEvent(aeEventLoop *eventLoop, long long milliseconds,
aeTimeProc *proc, void *clientData,
aeEventFinalizerProc *finalizerProc);
src/anet.c
+4 -1
View File
@@ -32,6 +32,7 @@
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <sys/un.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
@@ -291,7 +292,7 @@ int anetTcpServer(char *err, int port, char *bindaddr)
return s;
}
int anetUnixServer(char *err, char *path)
int anetUnixServer(char *err, char *path, mode_t perm)
{
int s;
struct sockaddr_un sa;
@@ -304,6 +305,8 @@ int anetUnixServer(char *err, char *path)
strncpy(sa.sun_path,path,sizeof(sa.sun_path)-1);
if (anetListen(err,s,(struct sockaddr*)&sa,sizeof(sa)) == ANET_ERR)
return ANET_ERR;
if (perm)
chmod(sa.sun_path, perm);
return s;
}
src/anet.h
+1 -1
View File
@@ -46,7 +46,7 @@ int anetUnixNonBlockConnect(char *err, char *path);
int anetRead(int fd, char *buf, int count);
int anetResolve(char *err, char *host, char *ipbuf);
int anetTcpServer(char *err, int port, char *bindaddr);
int anetUnixServer(char *err, char *path);
int anetUnixServer(char *err, char *path, mode_t perm);
int anetTcpAccept(char *err, int serversock, char *ip, int *port);
int anetUnixAccept(char *err, int serversock);
int anetWrite(int fd, char *buf, int count);
src/aof.c
+199 -61
View File
@@ -1,4 +1,5 @@
#include "redis.h"
#include "bio.h"
#include <signal.h>
#include <fcntl.h>
@@ -10,10 +11,14 @@
void aofUpdateCurrentSize(void);
void aof_background_fsync(int fd) {
bioCreateBackgroundJob(REDIS_BIO_AOF_FSYNC,(void*)(long)fd,NULL,NULL);
}
/* Called when the user switches from "appendonly yes" to "appendonly no"
* at runtime using the CONFIG command. */
void stopAppendOnly(void) {
flushAppendOnlyFile();
flushAppendOnlyFile(1);
aof_fsync(server.appendfd);
close(server.appendfd);
@@ -58,63 +63,121 @@ int startAppendOnly(void) {
* and the only way the client socket can get a write is entering when the
* the event loop, we accumulate all the AOF writes in a memory
* buffer and write it on disk using this function just before entering
* the event loop again. */
void flushAppendOnlyFile(void) {
time_t now;
* the event loop again.
*
* About the 'force' argument:
*
* When the fsync policy is set to 'everysec' we may delay the flush if there
* is still an fsync() going on in the background thread, since for instance
* on Linux write(2) will be blocked by the background fsync anyway.
* When this happens we remember that there is some aof buffer to be
* flushed ASAP, and will try to do that in the serverCron() function.
*
* However if force is set to 1 we'll write regardless of the background
* fsync. */
void flushAppendOnlyFile(int force) {
ssize_t nwritten;
int sync_in_progress = 0;
if (sdslen(server.aofbuf) == 0) return;
if (server.appendfsync == APPENDFSYNC_EVERYSEC)
sync_in_progress = bioPendingJobsOfType(REDIS_BIO_AOF_FSYNC) != 0;
if (server.appendfsync == APPENDFSYNC_EVERYSEC && !force) {
/* With this append fsync policy we do background fsyncing.
* If the fsync is still in progress we can try to delay
* the write for a couple of seconds. */
if (sync_in_progress) {
if (server.aof_flush_postponed_start == 0) {
/* No previous write postponinig, remember that we are
* postponing the flush and return. */
server.aof_flush_postponed_start = server.unixtime;
return;
} else if (server.unixtime - server.aof_flush_postponed_start < 2) {
/* We were already waiting for fsync to finish, but for less
* than two seconds this is still ok. Postpone again. */
return;
}
/* Otherwise fall trough, and go write since we can't wait
* over two seconds. */
redisLog(REDIS_NOTICE,"Asynchronous AOF fsync is taking too long (disk is busy?). Writing the AOF buffer without waiting for fsync to complete, this may slow down Redis.");
}
}
/* If you are following this code path, then we are going to write so
* set reset the postponed flush sentinel to zero. */
server.aof_flush_postponed_start = 0;
/* We want to perform a single write. This should be guaranteed atomic
* at least if the filesystem we are writing is a real physical one.
* While this will save us against the server being killed I don't think
* there is much to do about the whole server stopping for power problems
* or alike */
nwritten = write(server.appendfd,server.aofbuf,sdslen(server.aofbuf));
if (nwritten != (signed)sdslen(server.aofbuf)) {
nwritten = write(server.appendfd,server.aofbuf,sdslen(server.aofbuf));
if (nwritten != (signed)sdslen(server.aofbuf)) {
/* Ooops, we are in troubles. The best thing to do for now is
* aborting instead of giving the illusion that everything is
* working as expected. */
if (nwritten == -1) {
if (nwritten == -1) {
redisLog(REDIS_WARNING,"Exiting on error writing to the append-only file: %s",strerror(errno));
} else {
} else {
redisLog(REDIS_WARNING,"Exiting on short write while writing to the append-only file: %s",strerror(errno));
}
exit(1);
}
exit(1);
}
sdsfree(server.aofbuf);
server.aofbuf = sdsempty();
server.appendonly_current_size += nwritten;
/* Don't Fsync if no-appendfsync-on-rewrite is set to yes and we have
* childs performing heavy I/O on disk. */
/* Re-use AOF buffer when it is small enough. The maximum comes from the
* arena size of 4k minus some overhead (but is otherwise arbitrary). */
if ((sdslen(server.aofbuf)+sdsavail(server.aofbuf)) < 4000) {
sdsclear(server.aofbuf);
} else {
sdsfree(server.aofbuf);
server.aofbuf = sdsempty();
}
/* Don't fsync if no-appendfsync-on-rewrite is set to yes and there are
* children doing I/O in the background. */
if (server.no_appendfsync_on_rewrite &&
(server.bgrewritechildpid != -1 || server.bgsavechildpid != -1))
return;
/* Fsync if needed */
now = time(NULL);
if (server.appendfsync == APPENDFSYNC_ALWAYS ||
(server.appendfsync == APPENDFSYNC_EVERYSEC &&
now-server.lastfsync > 1))
{
/* Perform the fsync if needed. */
if (server.appendfsync == APPENDFSYNC_ALWAYS) {
/* aof_fsync is defined as fdatasync() for Linux in order to avoid
* flushing metadata. */
aof_fsync(server.appendfd); /* Let's try to get this data on the disk */
server.lastfsync = now;
server.lastfsync = server.unixtime;
} else if ((server.appendfsync == APPENDFSYNC_EVERYSEC &&
server.unixtime > server.lastfsync)) {
if (!sync_in_progress) aof_background_fsync(server.appendfd);
server.lastfsync = server.unixtime;
}
}
sds catAppendOnlyGenericCommand(sds buf, int argc, robj **argv) {
int j;
buf = sdscatprintf(buf,"*%d\r\n",argc);
sds catAppendOnlyGenericCommand(sds dst, int argc, robj **argv) {
char buf[32];
int len, j;
robj *o;
buf[0] = '*';
len = 1+ll2string(buf+1,sizeof(buf)-1,argc);
buf[len++] = '\r';
buf[len++] = '\n';
dst = sdscatlen(dst,buf,len);
for (j = 0; j < argc; j++) {
robj *o = getDecodedObject(argv[j]);
buf = sdscatprintf(buf,"$%lu\r\n",(unsigned long)sdslen(o->ptr));
buf = sdscatlen(buf,o->ptr,sdslen(o->ptr));
buf = sdscatlen(buf,"\r\n",2);
o = getDecodedObject(argv[j]);
buf[0] = '$';
len = 1+ll2string(buf+1,sizeof(buf)-1,sdslen(o->ptr));
buf[len++] = '\r';
buf[len++] = '\n';
dst = sdscatlen(dst,buf,len);
dst = sdscatlen(dst,o->ptr,sdslen(o->ptr));
dst = sdscatlen(dst,"\r\n",2);
decrRefCount(o);
}
return buf;
return dst;
}
sds catAppendOnlyExpireAtCommand(sds buf, robj *key, robj *seconds) {
@@ -264,6 +327,8 @@ int loadAppendOnlyFile(char *filename) {
}
if (buf[0] != '*') goto fmterr;
argc = atoi(buf+1);
if (argc < 1) goto fmterr;
argv = zmalloc(sizeof(robj*)*argc);
for (j = 0; j < argc; j++) {
if (fgets(buf,sizeof(buf),fp) == NULL) goto readerr;
@@ -626,6 +691,7 @@ int rewriteAppendOnlyFileBackground(void) {
}
redisLog(REDIS_NOTICE,
"Background append only file rewriting started by pid %d",childpid);
server.aofrewrite_scheduled = 0;
server.bgrewritechildpid = childpid;
updateDictResizePolicy();
/* We set appendseldb to -1 in order to force the next call to the
@@ -680,56 +746,128 @@ void backgroundRewriteDoneHandler(int statloc) {
int bysignal = WIFSIGNALED(statloc);
if (!bysignal && exitcode == 0) {
int fd;
int newfd, oldfd;
int nwritten;
char tmpfile[256];
long long now = ustime();
redisLog(REDIS_NOTICE,
"Background append only file rewriting terminated with success");
/* Now it's time to flush the differences accumulated by the parent */
snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof", (int) server.bgrewritechildpid);
fd = open(tmpfile,O_WRONLY|O_APPEND);
if (fd == -1) {
redisLog(REDIS_WARNING, "Not able to open the temp append only file produced by the child: %s", strerror(errno));
"Background AOF rewrite terminated with success");
/* Flush the differences accumulated by the parent to the
* rewritten AOF. */
snprintf(tmpfile,256,"temp-rewriteaof-bg-%d.aof",
(int)server.bgrewritechildpid);
newfd = open(tmpfile,O_WRONLY|O_APPEND);
if (newfd == -1) {
redisLog(REDIS_WARNING,
"Unable to open the temporary AOF produced by the child: %s", strerror(errno));
goto cleanup;
}
/* Flush our data... */
if (write(fd,server.bgrewritebuf,sdslen(server.bgrewritebuf)) !=
(signed) sdslen(server.bgrewritebuf)) {
redisLog(REDIS_WARNING, "Error or short write trying to flush the parent diff of the append log file in the child temp file: %s", strerror(errno));
close(fd);
nwritten = write(newfd,server.bgrewritebuf,sdslen(server.bgrewritebuf));
if (nwritten != (signed)sdslen(server.bgrewritebuf)) {
if (nwritten == -1) {
redisLog(REDIS_WARNING,
"Error trying to flush the parent diff to the rewritten AOF: %s", strerror(errno));
} else {
redisLog(REDIS_WARNING,
"Short write trying to flush the parent diff to the rewritten AOF: %s", strerror(errno));
}
close(newfd);
goto cleanup;
}
redisLog(REDIS_NOTICE,"Parent diff flushed into the new append log file with success (%lu bytes)",sdslen(server.bgrewritebuf));
/* Now our work is to rename the temp file into the stable file. And
* switch the file descriptor used by the server for append only. */
redisLog(REDIS_NOTICE,
"Parent diff successfully flushed to the rewritten AOF (%lu bytes)", nwritten);
/* The only remaining thing to do is to rename the temporary file to
* the configured file and switch the file descriptor used to do AOF
* writes. We don't want close(2) or rename(2) calls to block the
* server on old file deletion.
*
* There are two possible scenarios:
*
* 1) AOF is DISABLED and this was a one time rewrite. The temporary
* file will be renamed to the configured file. When this file already
* exists, it will be unlinked, which may block the server.
*
* 2) AOF is ENABLED and the rewritten AOF will immediately start
* receiving writes. After the temporary file is renamed to the
* configured file, the original AOF file descriptor will be closed.
* Since this will be the last reference to that file, closing it
* causes the underlying file to be unlinked, which may block the
* server.
*
* To mitigate the blocking effect of the unlink operation (either
* caused by rename(2) in scenario 1, or by close(2) in scenario 2), we
* use a background thread to take care of this. First, we
* make scenario 1 identical to scenario 2 by opening the target file
* when it exists. The unlink operation after the rename(2) will then
* be executed upon calling close(2) for its descriptor. Everything to
* guarantee atomicity for this switch has already happened by then, so
* we don't care what the outcome or duration of that close operation
* is, as long as the file descriptor is released again. */
if (server.appendfd == -1) {
/* AOF disabled */
/* Don't care if this fails: oldfd will be -1 and we handle that.
* One notable case of -1 return is if the old file does
* not exist. */
oldfd = open(server.appendfilename,O_RDONLY|O_NONBLOCK);
} else {
/* AOF enabled */
oldfd = -1; /* We'll set this to the current AOF filedes later. */
}
/* Rename the temporary file. This will not unlink the target file if
* it exists, because we reference it with "oldfd". */
if (rename(tmpfile,server.appendfilename) == -1) {
redisLog(REDIS_WARNING,"Can't rename the temp append only file into the stable one: %s", strerror(errno));
close(fd);
redisLog(REDIS_WARNING,
"Error trying to rename the temporary AOF: %s", strerror(errno));
close(newfd);
if (oldfd != -1) close(oldfd);
goto cleanup;
}
/* Mission completed... almost */
redisLog(REDIS_NOTICE,"Append only file successfully rewritten.");
if (server.appendfd != -1) {
/* If append only is actually enabled... */
close(server.appendfd);
server.appendfd = fd;
if (server.appendfsync != APPENDFSYNC_NO) aof_fsync(fd);
server.appendseldb = -1; /* Make sure it will issue SELECT */
redisLog(REDIS_NOTICE,"The new append only file was selected for future appends.");
if (server.appendfd == -1) {
/* AOF disabled, we don't need to set the AOF file descriptor
* to this new file, so we can close it. */
close(newfd);
} else {
/* AOF enabled, replace the old fd with the new one. */
oldfd = server.appendfd;
server.appendfd = newfd;
if (server.appendfsync == APPENDFSYNC_ALWAYS)
aof_fsync(newfd);
else if (server.appendfsync == APPENDFSYNC_EVERYSEC)
aof_background_fsync(newfd);
server.appendseldb = -1; /* Make sure SELECT is re-issued */
aofUpdateCurrentSize();
server.auto_aofrewrite_base_size = server.appendonly_current_size;
} else {
/* If append only is disabled we just generate a dump in this
* format. Why not? */
close(fd);
/* Clear regular AOF buffer since its contents was just written to
* the new AOF from the background rewrite buffer. */
sdsfree(server.aofbuf);
server.aofbuf = sdsempty();
}
redisLog(REDIS_NOTICE, "Background AOF rewrite successful");
/* Asynchronously close the overwritten AOF. */
if (oldfd != -1) bioCreateBackgroundJob(REDIS_BIO_CLOSE_FILE,(void*)(long)oldfd,NULL,NULL);
redisLog(REDIS_VERBOSE,
"Background AOF rewrite signal handler took %lldus", ustime()-now);
} else if (!bysignal && exitcode != 0) {
redisLog(REDIS_WARNING, "Background append only file rewriting error");
redisLog(REDIS_WARNING,
"Background AOF rewrite terminated with error");
} else {
redisLog(REDIS_WARNING,
"Background append only file rewriting terminated by signal %d",
"Background AOF rewrite terminated by signal %d",
WTERMSIG(statloc));
}
cleanup:
sdsfree(server.bgrewritebuf);
server.bgrewritebuf = sdsempty();
src/bio.c
+208
View File
@@ -0,0 +1,208 @@
/* Background I/O service for Redis.
*
* This file implements operations that we need to perform in the background.
* Currently there is only a single operation, that is a background close(2)
* system call. This is needed as when the process is the last owner of a
* reference to a file closing it means unlinking it, and the deletion of the
* file is slow, blocking the server.
*
* In the future we'll either continue implementing new things we need or
* we'll switch to libeio. However there are probably long term uses for this
* file as we may want to put here Redis specific background tasks (for instance
* it is not impossible that we'll need a non blocking FLUSHDB/FLUSHALL
* implementation).
*
* DESIGN
* ------
*
* The design is trivial, we have a structure representing a job to perform
* and a different thread and job queue for every job type.
* Every thread wait for new jobs in its queue, and process every job
* sequentially.
*
* Jobs of the same type are guaranteed to be processed from the least
* recently inserted to the most recently inserted (older jobs processed
* first).
*
* Currently there is no way for the creator of the job to be notified about
* the completion of the operation, this will only be added when/if needed.
*/
#include "redis.h"
#include "bio.h"
static pthread_mutex_t bio_mutex[REDIS_BIO_NUM_OPS];
static pthread_cond_t bio_condvar[REDIS_BIO_NUM_OPS];
static list *bio_jobs[REDIS_BIO_NUM_OPS];
/* The following array is used to hold the number of pending jobs for every
* OP type. This allows us to export the bioPendingJobsOfType() API that is
* useful when the main thread wants to perform some operation that may involve
* objects shared with the background thread. The main thread will just wait
* that there are no longer jobs of this type to be executed before performing
* the sensible operation. This data is also useful for reporting. */
static unsigned long long bio_pending[REDIS_BIO_NUM_OPS];
/* This structure represents a background Job. It is only used locally to this
* file as the API deos not expose the internals at all. */
struct bio_job {
time_t time; /* Time at which the job was created. */
/* Job specific arguments pointers. If we need to pass more than three
* arguments we can just pass a pointer to a structure or alike. */
void *arg1, *arg2, *arg3;
};
void *bioProcessBackgroundJobs(void *arg);
/* Make sure we have enough stack to perform all the things we do in the
* main thread. */
#define REDIS_THREAD_STACK_SIZE (1024*1024*4)
/* Initialize the background system, spawning the thread. */
void bioInit(void) {
pthread_attr_t attr;
pthread_t thread;
size_t stacksize;
int j;
/* Initialization of state vars and objects */
for (j = 0; j < REDIS_BIO_NUM_OPS; j++) {
pthread_mutex_init(&bio_mutex[j],NULL);
pthread_cond_init(&bio_condvar[j],NULL);
bio_jobs[j] = listCreate();
bio_pending[j] = 0;
}
/* Set the stack size as by default it may be small in some system */
pthread_attr_init(&attr);
pthread_attr_getstacksize(&attr,&stacksize);
if (!stacksize) stacksize = 1; /* The world is full of Solaris Fixes */
while (stacksize < REDIS_THREAD_STACK_SIZE) stacksize *= 2;
pthread_attr_setstacksize(&attr, stacksize);
/* Ready to spawn our threads. We use the single argument the thread
* function accepts in order to pass the job ID the thread is
* responsible of. */
for (j = 0; j < REDIS_BIO_NUM_OPS; j++) {
void *arg = (void*)(unsigned long) j;
if (pthread_create(&thread,&attr,bioProcessBackgroundJobs,arg) != 0) {
redisLog(REDIS_WARNING,"Fatal: Can't initialize Background Jobs.");
exit(1);
}
}
}
void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3) {
struct bio_job *job = zmalloc(sizeof(*job));
job->time = time(NULL);
job->arg1 = arg1;
job->arg2 = arg2;
job->arg3 = arg3;
pthread_mutex_lock(&bio_mutex[type]);
listAddNodeTail(bio_jobs[type],job);
bio_pending[type]++;
pthread_cond_signal(&bio_condvar[type]);
pthread_mutex_unlock(&bio_mutex[type]);
}
void *bioProcessBackgroundJobs(void *arg) {
struct bio_job *job;
unsigned long type = (unsigned long) arg;
pthread_detach(pthread_self());
pthread_mutex_lock(&bio_mutex[type]);
while(1) {
listNode *ln;
/* The loop always starts with the lock hold. */
if (listLength(bio_jobs[type]) == 0) {
pthread_cond_wait(&bio_condvar[type],&bio_mutex[type]);
continue;
}
/* Pop the job from the queue. */
ln = listFirst(bio_jobs[type]);
job = ln->value;
/* It is now possible to unlock the background system as we know have
* a stand alone job structure to process.*/
pthread_mutex_unlock(&bio_mutex[type]);
/* Process the job accordingly to its type. */
if (type == REDIS_BIO_CLOSE_FILE) {
close((long)job->arg1);
} else if (type == REDIS_BIO_AOF_FSYNC) {
aof_fsync((long)job->arg1);
} else {
redisPanic("Wrong job type in bioProcessBackgroundJobs().");
}
zfree(job);
/* Lock again before reiterating the loop, if there are no longer
* jobs to process we'll block again in pthread_cond_wait(). */
pthread_mutex_lock(&bio_mutex[type]);
listDelNode(bio_jobs[type],ln);
bio_pending[type]--;
}
}
/* Return the number of pending jobs of the specified type. */
unsigned long long bioPendingJobsOfType(int type) {
unsigned long long val;
pthread_mutex_lock(&bio_mutex[type]);
val = bio_pending[type];
pthread_mutex_unlock(&bio_mutex[type]);
return val;
}
#if 0 /* We don't use the following code for now, and bioWaitPendingJobsLE
probably needs a rewrite using conditional variables instead of the
current implementation. */
/* Wait until the number of pending jobs of the specified type are
* less or equal to the specified number.
*
* This function may block for long time, it should only be used to perform
* the following tasks:
*
* 1) To avoid that the main thread is pushing jobs of a given time so fast
* that the background thread can't process them at the same speed.
* So before creating a new job of a given type the main thread should
* call something like: bioWaitPendingJobsLE(job_type,10000);
* 2) In order to perform special operations that make it necessary to be sure
* no one is touching shared resourced in the background.
*/
void bioWaitPendingJobsLE(int type, unsigned long long num) {
unsigned long long iteration = 0;
/* We poll the jobs queue aggressively to start, and gradually relax
* the polling speed if it is going to take too much time. */
while(1) {
iteration++;
if (iteration > 1000 && iteration <= 10000) {
usleep(100);
} else if (iteration > 10000) {
usleep(1000);
}
if (bioPendingJobsOfType(type) <= num) break;
}
}
/* Return the older job of the specified type. */
time_t bioOlderJobOfType(int type) {
time_t time;
listNode *ln;
struct bio_job *job;
pthread_mutex_lock(&bio_mutex[type]);
ln = listFirst(bio_jobs[type]);
if (ln == NULL) {
pthread_mutex_unlock(&bio_mutex[type]);
return 0;
}
job = ln->value;
time = job->time;
pthread_mutex_unlock(&bio_mutex[type]);
return time;
}
#endif
src/bio.h
+11
View File
@@ -0,0 +1,11 @@
/* Exported API */
void bioInit(void);
void bioCreateBackgroundJob(int type, void *arg1, void *arg2, void *arg3);
unsigned long long bioPendingJobsOfType(int type);
void bioWaitPendingJobsLE(int type, unsigned long long num);
time_t bioOlderJobOfType(int type);
/* Background job opcodes */
#define REDIS_BIO_CLOSE_FILE 0 /* Deferred close(2) syscall. */
#define REDIS_BIO_AOF_FSYNC 1 /* Deferred AOF fsync. */
#define REDIS_BIO_NUM_OPS 2
src/config.c
+49 -6
View File
@@ -74,6 +74,12 @@ void loadServerConfig(char *filename) {
server.bindaddr = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"unixsocket") && argc == 2) {
server.unixsocket = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"unixsocketperm") && argc == 2) {
errno = 0;
server.unixsocketperm = (mode_t)strtol(argv[1], NULL, 8);
if (errno || server.unixsocketperm > 0777) {
err = "Invalid socket file permissions"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"save") && argc == 3) {
int seconds = atoi(argv[1]);
int changes = atoi(argv[2]);
@@ -189,6 +195,18 @@ void loadServerConfig(char *filename) {
server.masterhost = sdsnew(argv[1]);
server.masterport = atoi(argv[2]);
server.replstate = REDIS_REPL_CONNECT;
} else if (!strcasecmp(argv[0],"repl-ping-slave-period") && argc == 2) {
server.repl_ping_slave_period = atoi(argv[1]);
if (server.repl_ping_slave_period <= 0) {
err = "repl-ping-slave-period must be 1 or greater";
goto loaderr;
}
} else if (!strcasecmp(argv[0],"repl-timeout") && argc == 2) {
server.repl_timeout = atoi(argv[1]);
if (server.repl_timeout <= 0) {
err = "repl-timeout must be 1 or greater";
goto loaderr;
}
} else if (!strcasecmp(argv[0],"masterauth") && argc == 2) {
server.masterauth = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"slave-serve-stale-data") && argc == 2) {
@@ -360,7 +378,7 @@ void configSetCommand(redisClient *c) {
server.dbfilename = zstrdup(o->ptr);
} else if (!strcasecmp(c->argv[2]->ptr,"requirepass")) {
zfree(server.requirepass);
server.requirepass = zstrdup(o->ptr);
server.requirepass = ((char*)o->ptr)[0] ? zstrdup(o->ptr) : NULL;
} else if (!strcasecmp(c->argv[2]->ptr,"masterauth")) {
zfree(server.masterauth);
server.masterauth = zstrdup(o->ptr);
@@ -501,6 +519,18 @@ void configSetCommand(redisClient *c) {
} else if (!strcasecmp(c->argv[2]->ptr,"slowlog-max-len")) {
if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt;
server.slowlog_max_len = (unsigned)ll;
} else if (!strcasecmp(c->argv[2]->ptr,"loglevel")) {
if (!strcasecmp(o->ptr,"warning")) {
server.verbosity = REDIS_WARNING;
} else if (!strcasecmp(o->ptr,"notice")) {
server.verbosity = REDIS_NOTICE;
} else if (!strcasecmp(o->ptr,"verbose")) {
server.verbosity = REDIS_VERBOSE;
} else if (!strcasecmp(o->ptr,"debug")) {
server.verbosity = REDIS_DEBUG;
} else {
goto badfmt;
}
} else {
addReplyErrorFormat(c,"Unsupported CONFIG parameter: %s",
(char*)c->argv[2]->ptr);
@@ -526,12 +556,11 @@ void configGetCommand(redisClient *c) {
if (stringmatch(pattern,"dir",0)) {
char buf[1024];
addReplyBulkCString(c,"dir");
if (getcwd(buf,sizeof(buf)) == NULL) {
if (getcwd(buf,sizeof(buf)) == NULL)
buf[0] = '\0';
} else {
addReplyBulkCString(c,buf);
}
addReplyBulkCString(c,"dir");
addReplyBulkCString(c,buf);
matches++;
}
if (stringmatch(pattern,"dbfilename",0)) {
@@ -682,6 +711,20 @@ void configGetCommand(redisClient *c) {
addReplyBulkLongLong(c,server.slowlog_max_len);
matches++;
}
if (stringmatch(pattern,"loglevel",0)) {
char *s;
switch(server.verbosity) {
case REDIS_WARNING: s = "warning"; break;
case REDIS_VERBOSE: s = "verbose"; break;
case REDIS_NOTICE: s = "notice"; break;
case REDIS_DEBUG: s = "debug"; break;
default: s = "unknown"; break; /* too harmless to panic */
}
addReplyBulkCString(c,"loglevel");
addReplyBulkCString(c,s);
matches++;
}
setDeferredMultiBulkLength(c,replylen,matches*2);
}
src/db.c
+13 -4
View File
@@ -206,7 +206,13 @@ void flushallCommand(redisClient *c) {
kill(server.bgsavechildpid,SIGKILL);
rdbRemoveTempFile(server.bgsavechildpid);
}
rdbSave(server.dbfilename);
if (server.saveparamslen > 0) {
/* Normally rdbSave() will reset dirty, but we don't want this here
* as otherwise FLUSHALL will not be replicated nor put into the AOF. */
int saved_dirty = server.dirty;
rdbSave(server.dbfilename);
server.dirty = saved_dirty;
}
server.dirty++;
}
@@ -341,6 +347,7 @@ void shutdownCommand(redisClient *c) {
void renameGenericCommand(redisClient *c, int nx) {
robj *o;
time_t expire;
/* To use the same key as src and dst is probably an error */
if (sdscmp(c->argv[1]->ptr,c->argv[2]->ptr) == 0) {
@@ -352,16 +359,18 @@ void renameGenericCommand(redisClient *c, int nx) {
return;
incrRefCount(o);
expire = getExpire(c->db,c->argv[1]);
if (lookupKeyWrite(c->db,c->argv[2]) != NULL) {
if (nx) {
decrRefCount(o);
addReply(c,shared.czero);
return;
}
dbOverwrite(c->db,c->argv[2],o);
} else {
dbAdd(c->db,c->argv[2],o);
/* Overwrite: delete the old key before creating the new one with the same name. */
dbDelete(c->db,c->argv[2]);
}
dbAdd(c->db,c->argv[2],o);
if (expire != -1) setExpire(c->db,c->argv[2],expire);
dbDelete(c->db,c->argv[1]);
signalModifiedKey(c->db,c->argv[1]);
signalModifiedKey(c->db,c->argv[2]);
src/debug.c
+8 -2
View File
@@ -224,7 +224,7 @@ void debugCommand(redisClient *c) {
}
emptyDb();
if (rdbLoad(server.dbfilename) != REDIS_OK) {
addReply(c,shared.err);
addReplyError(c,"Error trying to load the RDB dump");
return;
}
redisLog(REDIS_WARNING,"DB reloaded by DEBUG RELOAD");
@@ -235,6 +235,7 @@ void debugCommand(redisClient *c) {
addReply(c,shared.err);
return;
}
server.dirty = 0; /* Prevent AOF / replication */
redisLog(REDIS_WARNING,"Append Only File loaded by DEBUG LOADAOF");
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"object") && c->argc == 3) {
@@ -337,15 +338,20 @@ void debugCommand(redisClient *c) {
}
void _redisAssert(char *estr, char *file, int line) {
bugReportStart();
redisLog(REDIS_WARNING,"=== ASSERTION FAILED ===");
redisLog(REDIS_WARNING,"==> %s:%d '%s' is not true",file,line,estr);
#ifdef HAVE_BACKTRACE
redisLog(REDIS_WARNING,"(forcing SIGSEGV in order to print the stack trace)");
server.assert_failed = estr;
server.assert_file = file;
server.assert_line = line;
redisLog(REDIS_WARNING,"(forcing SIGSEGV to print the bug report.)");
*((char*)-1) = 'x';
#endif
}
void _redisPanic(char *msg, char *file, int line) {
bugReportStart();
redisLog(REDIS_WARNING,"!!! Software Failure. Press left mouse button to continue");
redisLog(REDIS_WARNING,"Guru Meditation: %s #%s:%d",msg,file,line);
#ifdef HAVE_BACKTRACE
src/fmacros.h
+1 -1
View File
@@ -3,7 +3,7 @@
#define _BSD_SOURCE
#ifdef __linux__
#if defined(__linux__) || defined(__OpenBSD__)
#define _XOPEN_SOURCE 700
#else
#define _XOPEN_SOURCE
src/multi.c
+1 -1
View File
@@ -57,7 +57,7 @@ void discardCommand(redisClient *c) {
freeClientMultiState(c);
initClientMultiState(c);
c->flags &= (~REDIS_MULTI);
c->flags &= ~(REDIS_MULTI|REDIS_DIRTY_CAS);;
unwatchAllKeys(c);
addReply(c,shared.ok);
}
src/networking.c
+120 -43
View File
@@ -1,6 +1,8 @@
#include "redis.h"
#include <sys/uio.h>
static void setProtocolError(redisClient *c, int pos);
void *dupClientReplyValue(void *o) {
incrRefCount((robj*)o);
return o;
@@ -30,7 +32,7 @@ redisClient *createClient(int fd) {
c->reqtype = 0;
c->argc = 0;
c->argv = NULL;
c->cmd = NULL;
c->cmd = c->lastcmd = NULL;
c->multibulklen = 0;
c->bulklen = -1;
c->sentlen = 0;
@@ -388,6 +390,16 @@ void addReplyBulkLongLong(redisClient *c, long long ll) {
addReplyBulkCBuffer(c,buf,len);
}
/* Copy 'src' client output buffers into 'dst' client output buffers.
* The function takes care of freeing the old output buffers of the
* destination client. */
void copyClientOutputBuffer(redisClient *dst, redisClient *src) {
listRelease(dst->reply);
dst->reply = listDup(src->reply);
memcpy(dst->buf,src->buf,src->bufpos);
dst->bufpos = src->bufpos;
}
static void acceptCommonHandler(int fd) {
redisClient *c;
if ((c = createClient(fd)) == NULL) {
@@ -616,7 +628,7 @@ void sendReplyToClient(aeEventLoop *el, int fd, void *privdata, int mask) {
}
}
if (totwritten > 0) c->lastinteraction = time(NULL);
if (listLength(c->reply) == 0) {
if (c->bufpos == 0 && listLength(c->reply) == 0) {
c->sentlen = 0;
aeDeleteFileEvent(server.el,c->fd,AE_WRITABLE);
@@ -668,8 +680,13 @@ int processInlineBuffer(redisClient *c) {
size_t querylen;
/* Nothing to do without a \r\n */
if (newline == NULL)
if (newline == NULL) {
if (sdslen(c->querybuf) > REDIS_INLINE_MAX_SIZE) {
addReplyError(c,"Protocol error: too big inline request");
setProtocolError(c,0);
}
return REDIS_ERR;
}
/* Split the input buffer up to the \r\n */
querylen = newline-(c->querybuf);
@@ -698,6 +715,12 @@ int processInlineBuffer(redisClient *c) {
/* Helper function. Trims query buffer to make the function that processes
* multi bulk requests idempotent. */
static void setProtocolError(redisClient *c, int pos) {
if (server.verbosity >= REDIS_VERBOSE) {
sds client = getClientInfoString(c);
redisLog(REDIS_VERBOSE,
"Protocol error from client: %s", client);
sdsfree(client);
}
c->flags |= REDIS_CLOSE_AFTER_REPLY;
c->querybuf = sdsrange(c->querybuf,pos,-1);
}
@@ -713,8 +736,13 @@ int processMultibulkBuffer(redisClient *c) {
/* Multi bulk length cannot be read without a \r\n */
newline = strchr(c->querybuf,'\r');
if (newline == NULL)
if (newline == NULL) {
if (sdslen(c->querybuf) > REDIS_INLINE_MAX_SIZE) {
addReplyError(c,"Protocol error: too big mbulk count string");
setProtocolError(c,0);
}
return REDIS_ERR;
}
/* Buffer should also contain \n */
if (newline-(c->querybuf) > ((signed)sdslen(c->querybuf)-2))
@@ -748,8 +776,13 @@ int processMultibulkBuffer(redisClient *c) {
/* Read bulk length if unknown */
if (c->bulklen == -1) {
newline = strchr(c->querybuf+pos,'\r');
if (newline == NULL)
if (newline == NULL) {
if (sdslen(c->querybuf) > REDIS_INLINE_MAX_SIZE) {
addReplyError(c,"Protocol error: too big bulk count string");
setProtocolError(c,0);
}
break;
}
/* Buffer should also contain \n */
if (newline-(c->querybuf) > ((signed)sdslen(c->querybuf)-2))
@@ -790,9 +823,9 @@ int processMultibulkBuffer(redisClient *c) {
c->querybuf = sdsrange(c->querybuf,pos,-1);
/* We're done when c->multibulk == 0 */
if (c->multibulklen == 0) {
return REDIS_OK;
}
if (c->multibulklen == 0) return REDIS_OK;
/* Still not read to process the command */
return REDIS_ERR;
}
@@ -862,6 +895,16 @@ void readQueryFromClient(aeEventLoop *el, int fd, void *privdata, int mask) {
} else {
return;
}
if (sdslen(c->querybuf) > server.client_max_querybuf_len) {
sds ci = getClientInfoString(c), bytes = sdsempty();
bytes = sdscatrepr(bytes,c->querybuf,64);
redisLog(REDIS_WARNING,"Closing client that reached max query buffer length: %s (qbuf initial bytes: %s)", ci, bytes);
sdsfree(ci);
sdsfree(bytes);
freeClient(c);
return;
}
processInputBuffer(c);
}
@@ -883,47 +926,81 @@ void getClientsMaxBuffers(unsigned long *longest_output_list,
*biggest_input_buffer = bib;
}
/* Turn a Redis client into an sds string representing its state. */
sds getClientInfoString(redisClient *client) {
char ip[32], flags[16], events[3], *p;
int port;
time_t now = time(NULL);
int emask;
if (anetPeerToString(client->fd,ip,&port) == -1) {
ip[0] = '?';
ip[1] = '\0';
port = 0;
}
p = flags;
if (client->flags & REDIS_SLAVE) {
if (client->flags & REDIS_MONITOR)
*p++ = 'O';
else
*p++ = 'S';
}
if (client->flags & REDIS_MASTER) *p++ = 'M';
if (client->flags & REDIS_MULTI) *p++ = 'x';
if (client->flags & REDIS_BLOCKED) *p++ = 'b';
if (client->flags & REDIS_IO_WAIT) *p++ = 'i';
if (client->flags & REDIS_DIRTY_CAS) *p++ = 'd';
if (client->flags & REDIS_CLOSE_AFTER_REPLY) *p++ = 'c';
if (client->flags & REDIS_UNBLOCKED) *p++ = 'u';
if (p == flags) *p++ = 'N';
*p++ = '\0';
emask = client->fd == -1 ? 0 : aeGetFileEvents(server.el,client->fd);
p = events;
if (emask & AE_READABLE) *p++ = 'r';
if (emask & AE_WRITABLE) *p++ = 'w';
*p = '\0';
return sdscatprintf(sdsempty(),
"addr=%s:%d fd=%d idle=%ld flags=%s db=%d sub=%d psub=%d qbuf=%lu obl=%lu oll=%lu events=%s cmd=%s",
ip,port,client->fd,
(long)(now - client->lastinteraction),
flags,
client->db->id,
(int) dictSize(client->pubsub_channels),
(int) listLength(client->pubsub_patterns),
(unsigned long) sdslen(client->querybuf),
(unsigned long) client->bufpos,
(unsigned long) listLength(client->reply),
events,
client->lastcmd ? client->lastcmd->name : "NULL");
}
sds getAllClientsInfoString(void) {
listNode *ln;
listIter li;
redisClient *client;
sds o = sdsempty();
listRewind(server.clients,&li);
while ((ln = listNext(&li)) != NULL) {
sds cs;
client = listNodeValue(ln);
cs = getClientInfoString(client);
o = sdscatsds(o,cs);
sdsfree(cs);
o = sdscatlen(o,"\n",1);
}
return o;
}
void clientCommand(redisClient *c) {
listNode *ln;
listIter li;
redisClient *client;
if (!strcasecmp(c->argv[1]->ptr,"list") && c->argc == 2) {
sds o = sdsempty();
time_t now = time(NULL);
listRewind(server.clients,&li);
while ((ln = listNext(&li)) != NULL) {
char ip[32], flags[16], *p;
int port;
client = listNodeValue(ln);
if (anetPeerToString(client->fd,ip,&port) == -1) continue;
p = flags;
if (client->flags & REDIS_SLAVE) {
if (client->flags & REDIS_MONITOR)
*p++ = 'O';
else
*p++ = 'S';
}
if (client->flags & REDIS_MASTER) *p++ = 'M';
if (p == flags) *p++ = 'N';
if (client->flags & REDIS_MULTI) *p++ = 'x';
if (client->flags & REDIS_BLOCKED) *p++ = 'b';
if (client->flags & REDIS_IO_WAIT) *p++ = 'i';
if (client->flags & REDIS_DIRTY_CAS) *p++ = 'd';
if (client->flags & REDIS_CLOSE_AFTER_REPLY) *p++ = 'c';
if (client->flags & REDIS_UNBLOCKED) *p++ = 'u';
*p++ = '\0';
o = sdscatprintf(o,
"addr=%s:%d fd=%d idle=%ld flags=%s db=%d sub=%d psub=%d\n",
ip,port,client->fd,
(long)(now - client->lastinteraction),
flags,
client->db->id,
(int) dictSize(client->pubsub_channels),
(int) listLength(client->pubsub_patterns));
}
sds o = getAllClientsInfoString();
addReplyBulkCBuffer(c,o,sdslen(o));
sdsfree(o);
} else if (!strcasecmp(c->argv[1]->ptr,"kill") && c->argc == 3) {
src/rdb.c
+6 -1
View File
@@ -937,18 +937,23 @@ int rdbLoad(char *filename) {
long loops = 0;
fp = fopen(filename,"r");
if (!fp) return REDIS_ERR;
if (!fp) {
errno = ENOENT;
return REDIS_ERR;
}
if (fread(buf,9,1,fp) == 0) goto eoferr;
buf[9] = '\0';
if (memcmp(buf,"REDIS",5) != 0) {
fclose(fp);
redisLog(REDIS_WARNING,"Wrong signature trying to load DB from file");
errno = EINVAL;
return REDIS_ERR;
}
rdbver = atoi(buf+5);
if (rdbver < 1 || rdbver > 2) {
fclose(fp);
redisLog(REDIS_WARNING,"Can't handle RDB format version %d",rdbver);
errno = EINVAL;
return REDIS_ERR;
}
src/redis-benchmark.c
+19 -11
View File
@@ -53,9 +53,10 @@ static struct config {
int hostport;
const char *hostsocket;
int numclients;
int requests;
int liveclients;
int donerequests;
int requests;
int requests_issued;
int requests_finished;
int keysize;
int datasize;
int randomkeys;
@@ -148,7 +149,7 @@ static void randomizeClientKey(client c) {
}
static void clientDone(client c) {
if (config.donerequests == config.requests) {
if (config.requests_finished == config.requests) {
freeClient(c);
aeStop(config.el);
return;
@@ -189,8 +190,8 @@ static void readHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
exit(1);
}
if (config.donerequests < config.requests)
config.latency[config.donerequests++] = c->latency;
if (config.requests_finished < config.requests)
config.latency[config.requests_finished++] = c->latency;
clientDone(c);
}
}
@@ -202,8 +203,15 @@ static void writeHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
REDIS_NOTUSED(fd);
REDIS_NOTUSED(mask);
/* When nothing was written yet, randomize keys and set start time. */
/* Initialize request when nothing was written. */
if (c->written == 0) {
/* Enforce upper bound to number of requests. */
if (config.requests_issued++ >= config.requests) {
freeClient(c);
return;
}
/* Really initialize: randomize keys and set start time. */
if (config.randomkeys) randomizeClientKey(c);
c->start = ustime();
c->latency = -1;
@@ -286,10 +294,10 @@ static void showLatencyReport(void) {
int i, curlat = 0;
float perc, reqpersec;
reqpersec = (float)config.donerequests/((float)config.totlatency/1000);
reqpersec = (float)config.requests_finished/((float)config.totlatency/1000);
if (!config.quiet) {
printf("====== %s ======\n", config.title);
printf(" %d requests completed in %.2f seconds\n", config.donerequests,
printf(" %d requests completed in %.2f seconds\n", config.requests_finished,
(float)config.totlatency/1000);
printf(" %d parallel clients\n", config.numclients);
printf(" %d bytes payload\n", config.datasize);
@@ -314,7 +322,8 @@ static void benchmark(const char *title, const char *cmd, int len) {
client c;
config.title = title;
config.donerequests = 0;
config.requests_issued = 0;
config.requests_finished = 0;
c = createClient(cmd,len);
createMissingClients(c);
@@ -416,7 +425,7 @@ int showThroughput(struct aeEventLoop *eventLoop, long long id, void *clientData
REDIS_NOTUSED(clientData);
float dt = (float)(mstime()-config.start)/1000.0;
float rps = (float)config.donerequests/dt;
float rps = (float)config.requests_finished/dt;
printf("%s: %.2f\r", config.title, rps);
fflush(stdout);
return 250; /* every 250ms */
@@ -438,7 +447,6 @@ int main(int argc, const char **argv) {
config.el = aeCreateEventLoop();
aeCreateTimeEvent(config.el,1,showThroughput,NULL,NULL);
config.keepalive = 1;
config.donerequests = 0;
config.datasize = 3;
config.randomkeys = 0;
config.randomkeys_keyspacelen = 0;
src/redis-check-dump.c
+13 -3
View File
@@ -16,6 +16,10 @@
#define REDIS_SET 2
#define REDIS_ZSET 3
#define REDIS_HASH 4
#define REDIS_HASH_ZIPMAP 9
#define REDIS_LIST_ZIPLIST 10
#define REDIS_SET_INTSET 11
#define REDIS_ZSET_ZIPLIST 12
/* Objects encoding. Some kind of objects like Strings and Hashes can be
* internally represented in multiple ways. The 'encoding' field of the object
@@ -132,7 +136,7 @@ int processHeader() {
}
dump_version = (int)strtol(buf + 5, NULL, 10);
if (dump_version != 1) {
if (dump_version < 1 || dump_version > 2) {
ERROR("Unknown RDB format version: %d\n", dump_version);
}
return 1;
@@ -144,7 +148,7 @@ int loadType(entry *e) {
/* this byte needs to qualify as type */
unsigned char t;
if (readBytes(&t, 1)) {
if (t <= 4 || t >= 253) {
if (t <= 4 || (t >=9 && t <= 12) || t >= 253) {
e->type = t;
return 1;
} else {
@@ -160,7 +164,8 @@ int loadType(entry *e) {
int peekType() {
unsigned char t;
if (readBytes(&t, -1) && (t <= 4 || t >= 253)) return t;
if (readBytes(&t, -1) && (t <= 4 || (t >=9 && t <= 12) || t >= 253))
return t;
return -1;
}
@@ -375,6 +380,10 @@ int loadPair(entry *e) {
switch(e->type) {
case REDIS_STRING:
case REDIS_HASH_ZIPMAP:
case REDIS_LIST_ZIPLIST:
case REDIS_SET_INTSET:
case REDIS_ZSET_ZIPLIST:
if (!processStringObject(NULL)) {
SHIFT_ERROR(offset, "Error reading entry value");
return 0;
@@ -591,6 +600,7 @@ void process() {
/* advance position */
positions[0] = positions[1];
}
free(entry.key);
}
/* because there is another potential error,
src/redis-cli.c
+45 -1
View File
@@ -61,6 +61,7 @@ static struct config {
int shutdown;
int monitor_mode;
int pubsub_mode;
int latency_mode;
int stdinarg; /* get last arg from stdin. (-x option) */
char *auth;
int raw_output; /* output mode per command */
@@ -564,6 +565,8 @@ static int parseOptions(int argc, char **argv) {
i++;
} else if (!strcmp(argv[i],"--raw")) {
config.raw_output = 1;
} else if (!strcmp(argv[i],"--latency")) {
config.latency_mode = 1;
} else if (!strcmp(argv[i],"-d") && !lastarg) {
sdsfree(config.mb_delim);
config.mb_delim = sdsnew(argv[i+1]);
@@ -614,6 +617,7 @@ static void usage() {
" -x Read last argument from STDIN\n"
" -d <delimiter> Multi-bulk delimiter in for raw formatting (default: \\n)\n"
" --raw Use raw formatting for replies (default when STDOUT is not a tty)\n"
" --latency Enter a special mode continuously sampling latency.\n"
" --help Output this help and exit\n"
" --version Output version and exit\n"
"\n"
@@ -672,6 +676,7 @@ static void repl() {
if (argv == NULL) {
printf("Invalid argument(s)\n");
free(line);
continue;
} else if (argc > 0) {
if (strcasecmp(argv[0],"quit") == 0 ||
@@ -690,7 +695,7 @@ static void repl() {
int repeat, skipargs = 0;
repeat = atoi(argv[0]);
if (repeat) {
if (argc > 1 && repeat) {
skipargs = 1;
} else {
repeat = 1;
@@ -736,6 +741,38 @@ static int noninteractive(int argc, char **argv) {
return retval;
}
static void latencyMode(void) {
redisReply *reply;
long long start, latency, min, max, tot, count = 0;
double avg;
if (!context) exit(1);
while(1) {
start = mstime();
reply = redisCommand(context,"PING");
if (reply == NULL) {
fprintf(stderr,"\nI/O error\n");
exit(1);
}
latency = mstime()-start;
freeReplyObject(reply);
count++;
if (count == 1) {
min = max = tot = latency;
avg = (double) latency;
} else {
if (latency < min) min = latency;
if (latency > max) max = latency;
tot += latency;
avg = (double) tot/count;
}
printf("\x1b[0G\x1b[2Kmin: %lld, max: %lld, avg: %.2f (%lld samples)",
min, max, avg, count);
fflush(stdout);
usleep(10000);
}
}
int main(int argc, char **argv) {
int firstarg;
@@ -749,6 +786,7 @@ int main(int argc, char **argv) {
config.shutdown = 0;
config.monitor_mode = 0;
config.pubsub_mode = 0;
config.latency_mode = 0;
config.stdinarg = 0;
config.auth = NULL;
config.raw_output = !isatty(fileno(stdout)) && (getenv("FAKETTY") == NULL);
@@ -759,6 +797,12 @@ int main(int argc, char **argv) {
argc -= firstarg;
argv += firstarg;
/* Start in latency mode if appropriate */
if (config.latency_mode) {
cliConnect(0);
latencyMode();
}
/* Start interactive mode when no command is provided */
if (argc == 0) {
/* Note that in repl mode we don't abort on connection error.
src/redis.c
+150 -36
View File
@@ -29,6 +29,7 @@
#include "redis.h"
#include "slowlog.h"
#include "bio.h"
#ifdef HAVE_BACKTRACE
#include <execinfo.h>
@@ -58,7 +59,7 @@
struct sharedObjectsStruct shared;
/* Global vars that are actally used as constants. The following double
/* Global vars that are actually used as constants. The following double
* values are used for double on-disk serialization, and are initialized
* at runtime to avoid strange compiler optimizations. */
@@ -530,6 +531,7 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
* in objects at every object access, and accuracy is not needed.
* To access a global var is faster than calling time(NULL) */
server.unixtime = time(NULL);
/* We have just 22 bits per object for LRU information.
* So we use an (eventually wrapping) LRU clock with 10 seconds resolution.
* 2^22 bits with 10 seconds resoluton is more or less 1.5 years.
@@ -635,7 +637,7 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
server.auto_aofrewrite_perc &&
server.appendonly_current_size > server.auto_aofrewrite_min_size)
{
int base = server.auto_aofrewrite_base_size ?
long long base = server.auto_aofrewrite_base_size ?
server.auto_aofrewrite_base_size : 1;
long long growth = (server.appendonly_current_size*100/base) - 100;
if (growth >= server.auto_aofrewrite_perc) {
@@ -645,6 +647,11 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
}
}
/* If we postponed an AOF buffer flush, let's try to do it every time the
* cron function is called. */
if (server.aof_flush_postponed_start) flushAppendOnlyFile(0);
/* Expire a few keys per cycle, only if this is a master.
* On slaves we wait for DEL operations synthesized by the master
* in order to guarantee a strict consistency. */
@@ -728,7 +735,7 @@ void beforeSleep(struct aeEventLoop *eventLoop) {
}
/* Write the AOF buffer on disk */
flushAppendOnlyFile();
flushAppendOnlyFile(0);
}
/* =========================== Server initialization ======================== */
@@ -791,11 +798,13 @@ void initServerConfig() {
server.port = REDIS_SERVERPORT;
server.bindaddr = NULL;
server.unixsocket = NULL;
server.unixsocketperm = 0;
server.ipfd = -1;
server.sofd = -1;
server.dbnum = REDIS_DEFAULT_DBNUM;
server.verbosity = REDIS_VERBOSE;
server.maxidletime = REDIS_MAXIDLETIME;
server.client_max_querybuf_len = REDIS_MAX_QUERYBUF_LEN;
server.saveparams = NULL;
server.loading = 0;
server.logfile = NULL; /* NULL = log on standard output */
@@ -813,6 +822,7 @@ void initServerConfig() {
server.lastfsync = time(NULL);
server.appendfd = -1;
server.appendseldb = -1; /* Make sure the first time will not match */
server.aof_flush_postponed_start = 0;
server.pidfile = zstrdup("/var/run/redis.pid");
server.dbfilename = zstrdup("dump.rdb");
server.appendfilename = zstrdup("appendonly.aof");
@@ -839,6 +849,8 @@ void initServerConfig() {
server.zset_max_ziplist_entries = REDIS_ZSET_MAX_ZIPLIST_ENTRIES;
server.zset_max_ziplist_value = REDIS_ZSET_MAX_ZIPLIST_VALUE;
server.shutdown_asap = 0;
server.repl_ping_slave_period = REDIS_REPL_PING_SLAVE_PERIOD;
server.repl_timeout = REDIS_REPL_TIMEOUT;
updateLRUClock();
resetServerSaveParams();
@@ -874,6 +886,12 @@ void initServerConfig() {
/* Slow log */
server.slowlog_log_slower_than = REDIS_SLOWLOG_LOG_SLOWER_THAN;
server.slowlog_max_len = REDIS_SLOWLOG_MAX_LEN;
/* Assert */
server.assert_failed = "<no assertion failed>";
server.assert_file = "<no file>";
server.assert_line = 0;
server.bug_report_start = 0;
}
void initServer() {
@@ -900,13 +918,14 @@ void initServer() {
if (server.port != 0) {
server.ipfd = anetTcpServer(server.neterr,server.port,server.bindaddr);
if (server.ipfd == ANET_ERR) {
redisLog(REDIS_WARNING, "Opening port: %s", server.neterr);
redisLog(REDIS_WARNING, "Opening port %d: %s",
server.port, server.neterr);
exit(1);
}
}
if (server.unixsocket != NULL) {
unlink(server.unixsocket); /* don't care if this fails */
server.sofd = anetUnixServer(server.neterr,server.unixsocket);
server.sofd = anetUnixServer(server.neterr,server.unixsocket,server.unixsocketperm);
if (server.sofd == ANET_ERR) {
redisLog(REDIS_WARNING, "Opening socket: %s", server.neterr);
exit(1);
@@ -963,6 +982,7 @@ void initServer() {
if (server.vm_enabled) vmInit();
slowlogInit();
bioInit();
srand(time(NULL)^getpid());
}
@@ -1006,9 +1026,9 @@ void call(redisClient *c) {
duration = ustime()-start;
slowlogPushEntryIfNeeded(c->argv,c->argc,duration);
if (server.appendonly && dirty)
if (server.appendonly && dirty > 0)
feedAppendOnlyFile(c->cmd,c->db->id,c->argv,c->argc);
if ((dirty || c->cmd->flags & REDIS_CMD_FORCE_REPLICATION) &&
if ((dirty > 0 || c->cmd->flags & REDIS_CMD_FORCE_REPLICATION) &&
listLength(server.slaves))
replicationFeedSlaves(server.slaves,c->db->id,c->argv,c->argc);
if (listLength(server.monitors))
@@ -1037,7 +1057,7 @@ int processCommand(redisClient *c) {
/* Now lookup the command and check ASAP about trivial error conditions
* such as wrong arity, bad command name and so forth. */
c->cmd = lookupCommand(c->argv[0]->ptr);
c->cmd = c->lastcmd = lookupCommand(c->argv[0]->ptr);
if (!c->cmd) {
addReplyErrorFormat(c,"unknown command '%s'",
(char*)c->argv[0]->ptr);
@@ -1115,20 +1135,29 @@ int processCommand(redisClient *c) {
/*================================== Shutdown =============================== */
int prepareForShutdown() {
redisLog(REDIS_WARNING,"User requested shutdown, saving DB...");
redisLog(REDIS_WARNING,"User requested shutdown...");
/* Kill the saving child if there is a background saving in progress.
We want to avoid race conditions, for instance our saving child may
overwrite the synchronous saving did by SHUTDOWN. */
if (server.bgsavechildpid != -1) {
redisLog(REDIS_WARNING,"There is a live saving child. Killing it!");
redisLog(REDIS_WARNING,"There is a child saving an .rdb. Killing it!");
kill(server.bgsavechildpid,SIGKILL);
rdbRemoveTempFile(server.bgsavechildpid);
}
if (server.appendonly) {
/* Kill the AOF saving child as the AOF we already have may be longer
* but contains the full dataset anyway. */
if (server.bgrewritechildpid != -1) {
redisLog(REDIS_WARNING,
"There is a child rewriting the AOF. Killing it!");
kill(server.bgrewritechildpid,SIGKILL);
}
/* Append only file: fsync() the AOF and exit */
redisLog(REDIS_NOTICE,"Calling fsync() on the AOF file.");
aof_fsync(server.appendfd);
if (server.vm_enabled) unlink(server.vm_swap_file);
} else if (server.saveparamslen > 0) {
}
if (server.saveparamslen > 0) {
redisLog(REDIS_NOTICE,"Saving the final RDB snapshot before exiting.");
/* Snapshotting. Perform a SYNC SAVE and exit */
if (rdbSave(server.dbfilename) != REDIS_OK) {
/* Ooops.. error saving! The best we can do is to continue
@@ -1136,21 +1165,36 @@ int prepareForShutdown() {
* in the next cron() Redis will be notified that the background
* saving aborted, handling special stuff like slaves pending for
* synchronization... */
redisLog(REDIS_WARNING,"Error trying to save the DB, can't exit");
redisLog(REDIS_WARNING,"Error trying to save the DB, can't exit.");
return REDIS_ERR;
}
} else {
redisLog(REDIS_WARNING,"Not saving DB.");
}
if (server.daemonize) unlink(server.pidfile);
redisLog(REDIS_WARNING,"Server exit now, bye bye...");
if (server.vm_enabled) {
redisLog(REDIS_NOTICE,"Removing the swap file.");
unlink(server.vm_swap_file);
}
if (server.daemonize) {
redisLog(REDIS_NOTICE,"Removing the pid file.");
unlink(server.pidfile);
}
/* Close the listening sockets. Apparently this allows faster restarts. */
if (server.ipfd != -1) close(server.ipfd);
if (server.sofd != -1) close(server.sofd);
if (server.unixsocket) {
redisLog(REDIS_NOTICE,"Removing the unix socket file.");
unlink(server.unixsocket); /* don't care if this fails */
}
redisLog(REDIS_WARNING,"Redis is now ready to exit, bye bye...");
return REDIS_OK;
}
/*================================== Commands =============================== */
void authCommand(redisClient *c) {
if (!server.requirepass || !strcmp(c->argv[1]->ptr, server.requirepass)) {
if (!server.requirepass) {
addReplyError(c,"Client sent AUTH, but no password is set");
} else if (!strcmp(c->argv[1]->ptr, server.requirepass)) {
c->authenticated = 1;
addReply(c,shared.ok);
} else {
@@ -1211,6 +1255,7 @@ sds genRedisInfoString(void) {
"redis_git_dirty:%d\r\n"
"arch_bits:%s\r\n"
"multiplexing_api:%s\r\n"
"gcc_version:%d.%d.%d\r\n"
"process_id:%ld\r\n"
"uptime_in_seconds:%ld\r\n"
"uptime_in_days:%ld\r\n"
@@ -1243,8 +1288,6 @@ sds genRedisInfoString(void) {
"evicted_keys:%lld\r\n"
"keyspace_hits:%lld\r\n"
"keyspace_misses:%lld\r\n"
"hash_max_zipmap_entries:%zu\r\n"
"hash_max_zipmap_value:%zu\r\n"
"pubsub_channels:%ld\r\n"
"pubsub_patterns:%u\r\n"
"latest_fork_usec:%lld\r\n"
@@ -1255,14 +1298,19 @@ sds genRedisInfoString(void) {
strtol(redisGitDirty(),NULL,10) > 0,
(sizeof(long) == 8) ? "64" : "32",
aeGetApiName(),
#ifdef __GNUC__
__GNUC__,__GNUC_MINOR__,__GNUC_PATCHLEVEL__,
#else
0,0,0,
#endif
(long) getpid(),
uptime,
uptime/(3600*24),
(unsigned long) server.lruclock,
(float)self_ru.ru_utime.tv_sec+(float)self_ru.ru_utime.tv_usec/1000000,
(float)self_ru.ru_stime.tv_sec+(float)self_ru.ru_stime.tv_usec/1000000,
(float)c_ru.ru_utime.tv_sec+(float)c_ru.ru_utime.tv_usec/1000000,
(float)self_ru.ru_utime.tv_sec+(float)self_ru.ru_utime.tv_usec/1000000,
(float)c_ru.ru_stime.tv_sec+(float)c_ru.ru_stime.tv_usec/1000000,
(float)c_ru.ru_utime.tv_sec+(float)c_ru.ru_utime.tv_usec/1000000,
listLength(server.clients)-listLength(server.slaves),
listLength(server.slaves),
lol, bib,
@@ -1286,8 +1334,6 @@ sds genRedisInfoString(void) {
server.stat_evictedkeys,
server.stat_keyspace_hits,
server.stat_keyspace_misses,
server.hash_max_zipmap_entries,
server.hash_max_zipmap_value,
dictSize(server.pubsub_channels),
listLength(server.pubsub_patterns),
server.stat_fork_time,
@@ -1299,10 +1345,47 @@ sds genRedisInfoString(void) {
info = sdscatprintf(info,
"aof_current_size:%lld\r\n"
"aof_base_size:%lld\r\n"
"aof_pending_rewrite:%d\r\n",
"aof_pending_rewrite:%d\r\n"
"aof_buffer_length:%zu\r\n"
"aof_pending_bio_fsync:%llu\r\n",
(long long) server.appendonly_current_size,
(long long) server.auto_aofrewrite_base_size,
server.aofrewrite_scheduled);
server.aofrewrite_scheduled,
sdslen(server.aofbuf),
bioPendingJobsOfType(REDIS_BIO_AOF_FSYNC));
}
/* List connected slaves */
if (listLength(server.slaves)) {
int slaveid = 0;
listNode *ln;
listIter li;
listRewind(server.slaves,&li);
while((ln = listNext(&li))) {
redisClient *slave = listNodeValue(ln);
char *state = NULL;
char ip[32];
int port;
if (anetPeerToString(slave->fd,ip,&port) == -1) continue;
switch(slave->replstate) {
case REDIS_REPL_WAIT_BGSAVE_START:
case REDIS_REPL_WAIT_BGSAVE_END:
state = "wait_bgsave";
break;
case REDIS_REPL_SEND_BULK:
state = "send_bulk";
break;
case REDIS_REPL_ONLINE:
state = "online";
break;
}
if (state == NULL) continue;
info = sdscatprintf(info,"slave%d:%s,%d,%s\r\n",
slaveid,ip,port,state);
slaveid++;
}
}
if (server.masterhost) {
@@ -1620,8 +1703,13 @@ int main(int argc, char **argv) {
if (loadAppendOnlyFile(server.appendfilename) == REDIS_OK)
redisLog(REDIS_NOTICE,"DB loaded from append only file: %ld seconds",time(NULL)-start);
} else {
if (rdbLoad(server.dbfilename) == REDIS_OK)
redisLog(REDIS_NOTICE,"DB loaded from disk: %ld seconds",time(NULL)-start);
if (rdbLoad(server.dbfilename) == REDIS_OK) {
redisLog(REDIS_NOTICE,"DB loaded from disk: %ld seconds",
time(NULL)-start);
} else if (errno != ENOENT) {
redisLog(REDIS_WARNING,"Fatal error loading the DB. Exiting.");
exit(1);
}
}
if (server.ipfd > 0)
redisLog(REDIS_NOTICE,"The server is now ready to accept connections on port %d", server.port);
@@ -1642,8 +1730,10 @@ static void *getMcontextEip(ucontext_t *uc) {
#elif defined(__APPLE__) && !defined(MAC_OS_X_VERSION_10_6)
#if __x86_64__
return (void*) uc->uc_mcontext->__ss.__rip;
#else
#elif __i386__
return (void*) uc->uc_mcontext->__ss.__eip;
#else
return (void*) uc->uc_mcontext->__ss.__srr0;
#endif
#elif defined(__APPLE__) && defined(MAC_OS_X_VERSION_10_6)
#if defined(_STRUCT_X86_THREAD_STATE64) && !defined(__i386__)
@@ -1662,32 +1752,56 @@ static void *getMcontextEip(ucontext_t *uc) {
#endif
}
void bugReportStart(void) {
if (server.bug_report_start == 0) {
redisLog(REDIS_WARNING,
"=== REDIS BUG REPORT START: Cut & paste starting from here ===");
server.bug_report_start = 1;
}
}
static void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
void *trace[100];
char **messages = NULL;
int i, trace_size = 0;
ucontext_t *uc = (ucontext_t*) secret;
sds infostring;
sds infostring, clients;
struct sigaction act;
REDIS_NOTUSED(info);
bugReportStart();
redisLog(REDIS_WARNING,
"======= Ooops! Redis %s got signal: -%d- =======", REDIS_VERSION, sig);
infostring = genRedisInfoString();
redisLog(REDIS_WARNING, "%s",infostring);
/* It's not safe to sdsfree() the returned string under memory
* corruption conditions. Let it leak as we are going to abort */
" Redis %s crashed by signal: %d", REDIS_VERSION, sig);
redisLog(REDIS_WARNING,
" Failed assertion: %s (%s:%d)", server.assert_failed,
server.assert_file, server.assert_line);
/* Generate the stack trace */
trace_size = backtrace(trace, 100);
/* overwrite sigaction with caller's address */
if (getMcontextEip(uc) != NULL) {
trace[1] = getMcontextEip(uc);
}
messages = backtrace_symbols(trace, trace_size);
redisLog(REDIS_WARNING, "--- STACK TRACE");
for (i=1; i<trace_size; ++i)
redisLog(REDIS_WARNING,"%s", messages[i]);
/* Log INFO and CLIENT LIST */
redisLog(REDIS_WARNING, "--- INFO OUTPUT");
infostring = genRedisInfoString();
redisLog(REDIS_WARNING, infostring);
redisLog(REDIS_WARNING, "--- CLIENT LIST OUTPUT");
clients = getAllClientsInfoString();
redisLog(REDIS_WARNING, clients);
/* Don't sdsfree() strings to avoid a crash. Memory may be corrupted. */
redisLog(REDIS_WARNING,
"=== REDIS BUG REPORT END. Make sure to include from START to END. ===\n\n"
" Please report the crash opening an issue on github:\n\n"
" http://github.com/antirez/redis/issues\n\n"
);
/* free(messages); Don't call free() with possibly corrupted memory. */
if (server.daemonize) unlink(server.pidfile);
src/redis.h
+24 -5
View File
@@ -37,8 +37,9 @@
/* Static server configuration */
#define REDIS_SERVERPORT 6379 /* TCP port */
#define REDIS_MAXIDLETIME (60*5) /* default client timeout */
#define REDIS_IOBUF_LEN 1024
#define REDIS_MAXIDLETIME 0 /* default client timeout: infinite */
#define REDIS_MAX_QUERYBUF_LEN (1024*1024*1024) /* 1GB max query buffer. */
#define REDIS_IOBUF_LEN (1024*16)
#define REDIS_LOADBUF_LEN 1024
#define REDIS_DEFAULT_DBNUM 16
#define REDIS_CONFIGLINE_MAX 1024
@@ -48,12 +49,16 @@
#define REDIS_REQUEST_MAX_SIZE (1024*1024*256) /* max bytes in inline command */
#define REDIS_SHARED_INTEGERS 10000
#define REDIS_REPLY_CHUNK_BYTES (5*1500) /* 5 TCP packets with default MTU */
#define REDIS_MAX_LOGMSG_LEN 1024 /* Default maximum length of syslog messages */
#define REDIS_INLINE_MAX_SIZE (1024*64) /* Max size of inline reads */
#define REDIS_MAX_LOGMSG_LEN 4096 /* Default maximum length of syslog messages */
#define REDIS_AUTO_AOFREWRITE_PERC 100
#define REDIS_AUTO_AOFREWRITE_MIN_SIZE (1024*1024)
#define REDIS_SLOWLOG_LOG_SLOWER_THAN 10000
#define REDIS_SLOWLOG_MAX_LEN 64
#define REDIS_REPL_TIMEOUT 60
#define REDIS_REPL_PING_SLAVE_PERIOD 10
/* Hash table parameters */
#define REDIS_HT_MINFILL 10 /* Minimal hash table fill 10% */
@@ -232,6 +237,7 @@
#define redisPanic(_e) _redisPanic(#_e,__FILE__,__LINE__),_exit(1)
void _redisAssert(char *estr, char *file, int line);
void _redisPanic(char *msg, char *file, int line);
void bugReportStart(void);
/*-----------------------------------------------------------------------------
* Data types
@@ -328,7 +334,7 @@ typedef struct redisClient {
sds querybuf;
int argc;
robj **argv;
struct redisCommand *cmd;
struct redisCommand *cmd, *lastcmd;
int reqtype;
int multibulklen; /* number of multi bulk arguments left to read */
long bulklen; /* length of bulk argument in multi bulk request */
@@ -378,6 +384,7 @@ struct redisServer {
int port;
char *bindaddr;
char *unixsocket;
mode_t unixsocketperm;
int ipfd;
int sofd;
redisDb *db;
@@ -414,6 +421,7 @@ struct redisServer {
/* Configuration */
int verbosity;
int maxidletime;
size_t client_max_querybuf_len;
int dbnum;
int daemonize;
int appendonly;
@@ -428,6 +436,7 @@ struct redisServer {
time_t lastfsync;
int appendfd;
int appendseldb;
time_t aof_flush_postponed_start;
char *pidfile;
pid_t bgsavechildpid;
pid_t bgrewritechildpid;
@@ -450,6 +459,8 @@ struct redisServer {
char *masterauth;
char *masterhost;
int masterport;
int repl_ping_slave_period;
int repl_timeout;
redisClient *master; /* client that is master for this slave */
int repl_syncio_timeout; /* timeout for synchronous I/O calls */
int replstate; /* replication status if the instance is a slave */
@@ -525,6 +536,11 @@ struct redisServer {
/* Misc */
unsigned lruclock:22; /* clock incrementing every minute, for LRU */
unsigned lruclock_padding:10;
/* Assert & bug reportign */
char *assert_failed;
char *assert_file;
int assert_line;
int bug_report_start; /* True if bug report header already logged. */
};
typedef struct pubsubPattern {
@@ -690,9 +706,12 @@ void addReplyStatus(redisClient *c, char *status);
void addReplyDouble(redisClient *c, double d);
void addReplyLongLong(redisClient *c, long long ll);
void addReplyMultiBulkLen(redisClient *c, long length);
void copyClientOutputBuffer(redisClient *dst, redisClient *src);
void *dupClientReplyValue(void *o);
void getClientsMaxBuffers(unsigned long *longest_output_list,
unsigned long *biggest_input_buffer);
sds getClientInfoString(redisClient *client);
sds getAllClientsInfoString(void);
void rewriteClientCommandVector(redisClient *c, int argc, ...);
#ifdef __GNUC__
@@ -796,7 +815,7 @@ void backgroundSaveDoneHandler(int statloc);
int getObjectSaveType(robj *o);
/* AOF persistence */
void flushAppendOnlyFile(void);
void flushAppendOnlyFile(int force);
void feedAppendOnlyFile(struct redisCommand *cmd, int dictid, robj **argv, int argc);
void aofRemoveTempFile(pid_t childpid);
int rewriteAppendOnlyFileBackground(void);
src/replication.c
+34 -8
View File
@@ -122,8 +122,7 @@ void syncCommand(redisClient *c) {
if (ln) {
/* Perfect, the server is already registering differences for
* another slave. Set the right state, and copy the buffer. */
listRelease(c->reply);
c->reply = listDup(slave->reply);
copyClientOutputBuffer(c,slave);
c->replstate = REDIS_REPL_WAIT_BGSAVE_END;
redisLog(REDIS_NOTICE,"Waiting for end of BGSAVE for SYNC");
} else {
@@ -382,6 +381,13 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
* function is called, so we can delete it. */
aeDeleteFileEvent(server.el,fd,AE_READABLE|AE_WRITABLE);
/* If this event fired after the user turned the instance into a master
* with SLAVEOF NO ONE we must just return ASAP. */
if (server.replstate == REDIS_REPL_NONE) {
close(fd);
return;
}
/* AUTH with the master if required. */
if(server.masterauth) {
char authcmd[1024];
@@ -464,11 +470,24 @@ int connectWithMaster(void) {
return REDIS_ERR;
}
server.repl_transfer_lastio = time(NULL);
server.repl_transfer_s = fd;
server.replstate = REDIS_REPL_CONNECTING;
return REDIS_OK;
}
/* This function can be called when a non blocking connection is currently
* in progress to undo it. */
void undoConnectWithMaster(void) {
int fd = server.repl_transfer_s;
redisAssert(server.replstate == REDIS_REPL_CONNECTING);
aeDeleteFileEvent(server.el,fd,AE_READABLE|AE_WRITABLE);
close(fd);
server.repl_transfer_s = -1;
server.replstate = REDIS_REPL_CONNECT;
}
void slaveofCommand(redisClient *c) {
if (!strcasecmp(c->argv[1]->ptr,"no") &&
!strcasecmp(c->argv[2]->ptr,"one")) {
@@ -478,6 +497,8 @@ void slaveofCommand(redisClient *c) {
if (server.master) freeClient(server.master);
if (server.replstate == REDIS_REPL_TRANSFER)
replicationAbortSyncTransfer();
else if (server.replstate == REDIS_REPL_CONNECTING)
undoConnectWithMaster();
server.replstate = REDIS_REPL_NONE;
redisLog(REDIS_NOTICE,"MASTER MODE enabled (user request)");
}
@@ -497,13 +518,18 @@ void slaveofCommand(redisClient *c) {
/* --------------------------- REPLICATION CRON ---------------------------- */
#define REDIS_REPL_TIMEOUT 60
#define REDIS_REPL_PING_SLAVE_PERIOD 10
void replicationCron(void) {
/* Non blocking connection timeout? */
if (server.masterhost && server.replstate == REDIS_REPL_CONNECTING &&
(time(NULL)-server.repl_transfer_lastio) > server.repl_timeout)
{
redisLog(REDIS_WARNING,"Timeout connecting to the MASTER...");
undoConnectWithMaster();
}
/* Bulk transfer I/O timeout? */
if (server.masterhost && server.replstate == REDIS_REPL_TRANSFER &&
(time(NULL)-server.repl_transfer_lastio) > REDIS_REPL_TIMEOUT)
(time(NULL)-server.repl_transfer_lastio) > server.repl_timeout)
{
redisLog(REDIS_WARNING,"Timeout receiving bulk data from MASTER...");
replicationAbortSyncTransfer();
@@ -511,7 +537,7 @@ void replicationCron(void) {
/* Timed out master when we are an already connected slave? */
if (server.masterhost && server.replstate == REDIS_REPL_CONNECTED &&
(time(NULL)-server.master->lastinteraction) > REDIS_REPL_TIMEOUT)
(time(NULL)-server.master->lastinteraction) > server.repl_timeout)
{
redisLog(REDIS_WARNING,"MASTER time out: no data nor PING received...");
freeClient(server.master);
@@ -529,7 +555,7 @@ void replicationCron(void) {
* So slaves can implement an explicit timeout to masters, and will
* be able to detect a link disconnection even if the TCP connection
* will not actually go down. */
if (!(server.cronloops % (REDIS_REPL_PING_SLAVE_PERIOD*10))) {
if (!(server.cronloops % (server.repl_ping_slave_period*10))) {
listIter li;
listNode *ln;
src/sds.c
+33 -2
View File
@@ -92,6 +92,13 @@ void sdsupdatelen(sds s) {
sh->len = reallen;
}
void sdsclear(sds s) {
struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));
sh->free += sh->len;
sh->len = 0;
sh->buf[0] = '\0';
}
static sds sdsMakeRoomFor(sds s, size_t addlen) {
struct sdshdr *sh, *newsh;
size_t free = sdsavail(s);
@@ -149,6 +156,10 @@ sds sdscat(sds s, char *t) {
return sdscatlen(s, t, strlen(t));
}
sds sdscatsds(sds s, sds t) {
return sdscatlen(s, t, sdslen(t));
}
sds sdscpylen(sds s, char *t, size_t len) {
struct sdshdr *sh = (void*) (s-(sizeof(struct sdshdr)));
size_t totlen = sh->free+sh->len;
@@ -467,7 +478,8 @@ sds *sdssplitargs(char *line, int *argc) {
while(*p && isspace(*p)) p++;
if (*p) {
/* get a token */
int inq=0; /* set to 1 if we are in "quotes" */
int inq=0; /* set to 1 if we are in "quotes" */
int insq=0; /* set to 1 if we are in 'single quotes' */
int done=0;
if (current == NULL) current = sdsempty();
@@ -497,7 +509,23 @@ sds *sdssplitargs(char *line, int *argc) {
}
current = sdscatlen(current,&c,1);
} else if (*p == '"') {
/* closing quote must be followed by a space */
/* closing quote must be followed by a space or
* nothing at all. */
if (*(p+1) && !isspace(*(p+1))) goto err;
done=1;
} else if (!*p) {
/* unterminated quotes */
goto err;
} else {
current = sdscatlen(current,p,1);
}
} else if (insq) {
if (*p == '\\' && *(p+1) == '\'') {
p++;
current = sdscatlen(current,"'",1);
} else if (*p == '\'') {
/* closing quote must be followed by a space or
* nothing at all. */
if (*(p+1) && !isspace(*(p+1))) goto err;
done=1;
} else if (!*p) {
@@ -518,6 +546,9 @@ sds *sdssplitargs(char *line, int *argc) {
case '"':
inq=1;
break;
case '\'':
insq=1;
break;
default:
current = sdscatlen(current,p,1);
break;
src/sds.h
+2
View File
@@ -62,6 +62,7 @@ size_t sdsavail(sds s);
sds sdsgrowzero(sds s, size_t len);
sds sdscatlen(sds s, void *t, size_t len);
sds sdscat(sds s, char *t);
sds sdscatsds(sds s, sds t);
sds sdscpylen(sds s, char *t, size_t len);
sds sdscpy(sds s, char *t);
@@ -76,6 +77,7 @@ sds sdscatprintf(sds s, const char *fmt, ...);
sds sdstrim(sds s, const char *cset);
sds sdsrange(sds s, int start, int end);
void sdsupdatelen(sds s);
void sdsclear(sds s);
int sdscmp(sds s1, sds s2);
sds *sdssplitlen(char *s, int len, char *sep, int seplen, int *count);
void sdsfreesplitres(sds *tokens, int count);
src/sort.c
+9 -12
View File
@@ -141,11 +141,7 @@ void sortCommand(redisClient *c) {
/* Lookup the key to sort. It must be of the right types */
sortval = lookupKeyRead(c->db,c->argv[1]);
if (sortval == NULL) {
addReply(c,shared.emptymultibulk);
return;
}
if (sortval->type != REDIS_SET && sortval->type != REDIS_LIST &&
if (sortval && sortval->type != REDIS_SET && sortval->type != REDIS_LIST &&
sortval->type != REDIS_ZSET)
{
addReply(c,shared.wrongtypeerr);
@@ -161,7 +157,10 @@ void sortCommand(redisClient *c) {
/* Now we need to protect sortval incrementing its count, in the future
* SORT may have options able to overwrite/delete keys during the sorting
* and the sorted key itself may get destroied */
incrRefCount(sortval);
if (sortval)
incrRefCount(sortval);
else
sortval = createListObject();
/* The SORT command has an SQL-alike syntax, parse it */
while(j < c->argc) {
@@ -200,7 +199,8 @@ void sortCommand(redisClient *c) {
}
/* Destructively convert encoded sorted sets for SORT. */
if (sortval->type == REDIS_ZSET) zsetConvert(sortval, REDIS_ENCODING_SKIPLIST);
if (sortval->type == REDIS_ZSET)
zsetConvert(sortval, REDIS_ENCODING_SKIPLIST);
/* Load the sorting vector with all the objects to sort */
switch(sortval->type) {
@@ -366,12 +366,9 @@ void sortCommand(redisClient *c) {
}
}
}
setKey(c->db,storekey,sobj);
if (outputlen) setKey(c->db,storekey,sobj);
decrRefCount(sobj);
/* Note: we add 1 because the DB is dirty anyway since even if the
* SORT result is empty a new key is set and maybe the old content
* replaced. */
server.dirty += 1+outputlen;
server.dirty += outputlen;
addReplyLongLong(c,outputlen);
}
src/t_hash.c
+4 -1
View File
@@ -403,8 +403,11 @@ void hdelCommand(redisClient *c) {
for (j = 2; j < c->argc; j++) {
if (hashTypeDelete(o,c->argv[j])) {
if (hashTypeLength(o) == 0) dbDelete(c->db,c->argv[1]);
deleted++;
if (hashTypeLength(o) == 0) {
dbDelete(c->db,c->argv[1]);
break;
}
}
}
if (deleted) {
src/t_list.c
+12 -5
View File
@@ -519,7 +519,12 @@ void lrangeCommand(redisClient *c) {
p = ziplistNext(o->ptr,p);
}
} else if (o->encoding == REDIS_ENCODING_LINKEDLIST) {
listNode *ln = listIndex(o->ptr,start);
listNode *ln;
/* If we are nearest to the end of the list, reach the element
* starting from tail and going backward, as it is faster. */
if (start > llen/2) start -= llen;
ln = listIndex(o->ptr,start);
while(rangelen--) {
addReplyBulk(c,ln->value);
@@ -643,7 +648,7 @@ void lremCommand(redisClient *c) {
void rpoplpushHandlePush(redisClient *origclient, redisClient *c, robj *dstkey, robj *dstobj, robj *value) {
robj *aux;
if (!handleClientsWaitingListPush(c,dstkey,value)) {
if (!handleClientsWaitingListPush(origclient,dstkey,value)) {
/* Create the list if the key does not exist */
if (!dstobj) {
dstobj = createZiplistObject();
@@ -653,10 +658,12 @@ void rpoplpushHandlePush(redisClient *origclient, redisClient *c, robj *dstkey,
}
listTypePush(dstobj,value,REDIS_HEAD);
/* If we are pushing as a result of LPUSH against a key
* watched by BLPOPLPUSH, we need to rewrite the command vector.
* But if this is called directly by RPOPLPUSH (either directly
* watched by BRPOPLPUSH, we need to rewrite the command vector
* as an LPUSH.
*
* If this is called directly by RPOPLPUSH (either directly
* or via a BRPOPLPUSH where the popped list exists)
* we should replicate the BRPOPLPUSH command itself. */
* we should replicate the RPOPLPUSH command itself. */
if (c != origclient) {
aux = createStringObject("LPUSH",5);
rewriteClientCommandVector(origclient,3,aux,dstkey,value);
src/t_zset.c
+5 -1
View File
@@ -17,7 +17,7 @@
/* This skiplist implementation is almost a C translation of the original
* algorithm described by William Pugh in "Skip Lists: A Probabilistic
* Alternative to Balanced Trees", modified in three ways:
* a) this implementation allows for repeated values.
* a) this implementation allows for repeated scores.
* b) the comparison is not just by key (our 'score') but by satellite data.
* c) there is a back pointer, so it's a doubly linked list with the back
* pointers being only at "level 1". This allows to traverse the list
@@ -76,6 +76,7 @@ zskiplistNode *zslInsert(zskiplist *zsl, double score, robj *obj) {
unsigned int rank[ZSKIPLIST_MAXLEVEL];
int i, level;
redisAssert(!isnan(score));
x = zsl->header;
for (i = zsl->level-1; i >= 0; i--) {
/* store rank that is crossed to reach the insert position */
@@ -1547,6 +1548,8 @@ void zunionInterGenericCommand(redisClient *c, robj *dstkey, int op) {
double score, value;
score = src[0].weight * zval.score;
if (isnan(score)) score = 0;
for (j = 1; j < setnum; j++) {
/* It is not safe to access the zset we are
* iterating, so explicitly check for equal object. */
@@ -1589,6 +1592,7 @@ void zunionInterGenericCommand(redisClient *c, robj *dstkey, int op) {
/* Initialize score */
score = src[i].weight * zval.score;
if (isnan(score)) score = 0;
/* Because the inputs are sorted by size, it's only possible
* for sets at larger indices to hold this element. */
src/version.h
+1 -1
View File
@@ -1 +1 @@
#define REDIS_VERSION "2.3.6"
#define REDIS_VERSION "2.4.6"
src/zmalloc.c
+1 -1
View File
@@ -38,7 +38,7 @@
#ifdef HAVE_MALLOC_SIZE
#define PREFIX_SIZE (0)
#else
#if defined(__sun)
#if defined(__sun) || defined(__sparc) || defined(__sparc__)
#define PREFIX_SIZE (sizeof(long long))
#else
#define PREFIX_SIZE (sizeof(size_t))
tests/helpers/gen_write_load.tcl
+15
View File
@@ -0,0 +1,15 @@
source tests/support/redis.tcl
proc gen_write_load {host port seconds} {
set start_time [clock seconds]
set r [redis $host $port 1]
$r select 9
while 1 {
$r set [expr rand()] [expr rand()]
if {[clock seconds]-$start_time > $seconds} {
exit 0
}
}
}
gen_write_load [lindex $argv 0] [lindex $argv 1] [lindex $argv 2]
tests/integration/aof-race.tcl
+35
View File
@@ -0,0 +1,35 @@
set defaults { appendonly {yes} appendfilename {appendonly.aof} }
set server_path [tmpdir server.aof]
set aof_path "$server_path/appendonly.aof"
proc start_server_aof {overrides code} {
upvar defaults defaults srv srv server_path server_path
set config [concat $defaults $overrides]
start_server [list overrides $config] $code
}
tags {"aof"} {
# Specific test for a regression where internal buffers were not properly
# cleaned after a child responsible for an AOF rewrite exited. This buffer
# was subsequently appended to the new AOF, resulting in duplicate commands.
start_server_aof [list dir $server_path] {
set client [redis [srv host] [srv port]]
set bench [open "|src/redis-benchmark -q -p [srv port] -c 20 -n 20000 incr foo" "r+"]
after 100
# Benchmark should be running by now: start background rewrite
$client bgrewriteaof
# Read until benchmark pipe reaches EOF
while {[string length [read $bench]] > 0} {}
# Check contents of foo
assert_equal 20000 [$client get foo]
}
# Restart server to replay AOF
start_server_aof [list dir $server_path] {
set client [redis [srv host] [srv port]]
assert_equal 20000 [$client get foo]
}
}
tests/integration/aof.tcl
+26 -14
View File
@@ -31,14 +31,20 @@ tags {"aof"} {
}
start_server_aof [list dir $server_path] {
test "Unfinished MULTI: Server should not have been started" {
if {$::valgrind} {after 2000}
assert_equal 0 [is_alive $srv]
}
test "Unfinished MULTI: Server should have logged an error" {
set result [exec cat [dict get $srv stdout] | tail -n1]
assert_match "*Unexpected end of file reading the append only file*" $result
set pattern "*Unexpected end of file reading the append only file*"
set retry 10
while {$retry} {
set result [exec cat [dict get $srv stdout] | tail -n1]
if {[string match $pattern $result]} {
break
}
incr retry -1
after 1000
}
if {$retry == 0} {
error "assertion:expected error not found on config file"
}
}
}
@@ -49,14 +55,20 @@ tags {"aof"} {
}
start_server_aof [list dir $server_path] {
test "Short read: Server should not have been started" {
if {$::valgrind} {after 2000}
assert_equal 0 [is_alive $srv]
}
test "Short read: Server should have logged an error" {
set result [exec cat [dict get $srv stdout] | tail -n1]
assert_match "*Bad file format reading the append only file*" $result
set pattern "*Bad file format reading the append only file*"
set retry 10
while {$retry} {
set result [exec cat [dict get $srv stdout] | tail -n1]
if {[string match $pattern $result]} {
break
}
incr retry -1
after 1000
}
if {$retry == 0} {
error "assertion:expected error not found on config file"
}
}
}
tests/integration/replication.tcl
+74
View File
@@ -57,5 +57,79 @@ start_server {tags {"repl"}} {
if {$::valgrind} {after 2000}
r 0 get mykey
} {bar}
test {FLUSHALL should replicate} {
r -1 flushall
if {$::valgrind} {after 2000}
list [r -1 dbsize] [r 0 dbsize]
} {0 0}
}
}
proc start_write_load {host port seconds} {
exec tclsh8.5 tests/helpers/gen_write_load.tcl $host $port $seconds &
}
proc stop_write_load {handle} {
catch {exec /bin/kill -9 $handle}
}
start_server {tags {"repl"}} {
set master [srv 0 client]
set master_host [srv 0 host]
set master_port [srv 0 port]
set slaves {}
set load_handle0 [start_write_load $master_host $master_port 20]
set load_handle1 [start_write_load $master_host $master_port 20]
set load_handle2 [start_write_load $master_host $master_port 20]
set load_handle3 [start_write_load $master_host $master_port 20]
set load_handle4 [start_write_load $master_host $master_port 20]
after 2000
start_server {} {
lappend slaves [srv 0 client]
start_server {} {
lappend slaves [srv 0 client]
start_server {} {
lappend slaves [srv 0 client]
test "Connect multiple slaves at the same time (issue #141)" {
[lindex $slaves 0] slaveof $master_host $master_port
[lindex $slaves 1] slaveof $master_host $master_port
[lindex $slaves 2] slaveof $master_host $master_port
# Wait for all the three slaves to reach the "online" state
set retry 100
while {$retry} {
set info [r -3 info]
if {[string match {*slave0:*,online*slave1:*,online*slave2:*,online*} $info]} {
break
} else {
incr retry -1
after 100
}
}
if {$retry == 0} {
error "assertion:Slaves not correctly synchronized"
}
stop_write_load $load_handle0
stop_write_load $load_handle1
stop_write_load $load_handle2
stop_write_load $load_handle3
stop_write_load $load_handle4
after 1000
set digest [$master debug digest]
set digest0 [[lindex $slaves 0] debug digest]
set digest1 [[lindex $slaves 1] debug digest]
set digest2 [[lindex $slaves 2] debug digest]
assert {$digest ne 0000000000000000000000000000000000000000}
assert {$digest eq $digest0}
assert {$digest eq $digest1}
assert {$digest eq $digest2}
#puts [$master dbsize]
#puts [[lindex $slaves 0] dbsize]
#puts [[lindex $slaves 1] dbsize]
#puts [[lindex $slaves 2] dbsize]
}
}
}
}
}
tests/support/server.tcl
+2 -1
View File
@@ -154,7 +154,8 @@ proc start_server {options {code undefined}} {
dict set config dir [tmpdir server]
# start every server on a different port
dict set config port [incr ::port]
set ::port [find_available_port [expr {$::port+1}]]
dict set config port $::port
# apply overrides from global space and arguments
foreach {directive arguments} [concat $::global_overrides $overrides] {
tests/support/test.tcl
+2 -2
View File
@@ -4,8 +4,8 @@ set ::num_failed 0
set ::tests_failed {}
proc assert {condition} {
if {![uplevel 1 expr $condition]} {
error "assertion:Expected '$value' to be true"
if {![uplevel 1 [list expr $condition]]} {
error "assertion:Expected condition '$condition' to be true ([uplevel 1 [list subst -nocommands $condition]])"
}
}
tests/test_helper.tcl
+48 -28
View File
@@ -31,15 +31,18 @@ set ::all_tests {
integration/aof
unit/pubsub
unit/slowlog
unit/maxmemory
unit/introspection
}
# Index to the next test to run in the ::all_tests list.
set ::next_test 0
set ::host 127.0.0.1
set ::port 16379
set ::port 21111
set ::traceleaks 0
set ::valgrind 0
set ::verbose 0
set ::quiet 0
set ::denytags {}
set ::allowtags {}
set ::external 0; # If "1" this means, we are running against external instance
@@ -110,7 +113,7 @@ proc reconnect {args} {
}
# re-set $srv in the servers list
set ::servers [lreplace $::servers end+$level 1 $srv]
lset ::servers end+$level $srv
}
proc redis_deferring_client {args} {
@@ -140,43 +143,47 @@ proc s {args} {
}
proc cleanup {} {
puts -nonewline "Cleanup: may take some time... "
if {!$::quiet} {puts -nonewline "Cleanup: may take some time... "}
flush stdout
catch {exec rm -rf {*}[glob tests/tmp/redis.conf.*]}
catch {exec rm -rf {*}[glob tests/tmp/server.*]}
puts "OK"
if {!$::quiet} {puts "OK"}
}
proc find_available_port start {
for {set j $start} {$j < $start+1024} {incr j} {
if {[catch {
set fd [socket 127.0.0.1 $j]
}]} {
return $j
} else {
close $fd
}
}
if {$j == $start+1024} {
error "Can't find a non busy port in the $start-[expr {$start+1023}] range."
}
}
proc test_server_main {} {
cleanup
# Open a listening socket, trying different ports in order to find a
# non busy one.
set port 11111
while 1 {
set port [find_available_port 11111]
if {!$::quiet} {
puts "Starting test server at port $port"
if {[catch {socket -server accept_test_clients $port} e]} {
if {[string match {*address already in use*} $e]} {
if {$port == 20000} {
puts "Can't find an available TCP port for test server."
exit 1
} else {
incr port
}
} else {
puts "Fatal error starting test server: $e"
exit 1
}
} else {
break
}
}
socket -server accept_test_clients $port
# Start the client instances
set ::clients_pids {}
set start_port [expr {$::port+100}]
for {set j 0} {$j < $::numclients} {incr j} {
set start_port [find_available_port $start_port]
set p [exec tclsh8.5 [info script] {*}$::argv \
--client $port --port [expr {$::port+($j*10)}] &]
--client $port --port $start_port &]
lappend ::clients_pids $p
incr start_port 10
}
# Setup global state for the test server
@@ -219,16 +226,22 @@ proc read_from_test_client fd {
set payload [read $fd $bytes]
foreach {status data} $payload break
if {$status eq {ready}} {
puts "\[$status\]: $data"
if {!$::quiet} {
puts "\[$status\]: $data"
}
signal_idle_client $fd
} elseif {$status eq {done}} {
set elapsed [expr {[clock seconds]-$::clients_start_time($fd)}]
puts "\[[colorstr yellow $status]\]: $data ($elapsed seconds)"
puts "+++ [expr {[llength $::active_clients]-1}] units still in execution."
set all_tests_count [llength $::all_tests]
set running_tests_count [expr {[llength $::active_clients]-1}]
set completed_tests_count [expr {$::next_test-$running_tests_count}]
puts "\[$completed_tests_count/$all_tests_count [colorstr yellow $status]\]: $data ($elapsed seconds)"
lappend ::clients_time_history $elapsed $data
signal_idle_client $fd
} elseif {$status eq {ok}} {
puts "\[[colorstr green $status]\]: $data"
if {!$::quiet} {
puts "\[[colorstr green $status]\]: $data"
}
} elseif {$status eq {err}} {
set err "\[[colorstr red $status]\]: $data"
puts $err
@@ -242,7 +255,9 @@ proc read_from_test_client fd {
} elseif {$status eq {testing}} {
# No op
} else {
puts "\[$status\]: $data"
if {!$::quiet} {
puts "\[$status\]: $data"
}
}
}
@@ -254,7 +269,9 @@ proc signal_idle_client fd {
[lsearch -all -inline -not -exact $::active_clients $fd]
# New unit to process?
if {$::next_test != [llength $::all_tests]} {
puts [colorstr bold-white "Testing [lindex $::all_tests $::next_test]"]
if {!$::quiet} {
puts [colorstr bold-white "Testing [lindex $::all_tests $::next_test]"]
}
set ::clients_start_time($fd) [clock seconds]
send_data_packet $fd run [lindex $::all_tests $::next_test]
lappend ::active_clients $fd
@@ -318,6 +335,7 @@ proc print_help_screen {} {
puts [join {
"--valgrind Run the test over valgrind."
"--accurate Run slow randomized tests for more iterations."
"--quiet Don't show individual tests."
"--single <unit> Just execute the specified unit (see next option)."
"--list-tests List all the available test units."
"--force-failure Force the execution of a test that always fails."
@@ -340,6 +358,8 @@ for {set j 0} {$j < [llength $argv]} {incr j} {
incr j
} elseif {$opt eq {--valgrind}} {
set ::valgrind 1
} elseif {$opt eq {--quiet}} {
set ::quiet 1
} elseif {$opt eq {--host}} {
set ::external 1
set ::host $arg
tests/unit/auth.tcl
+14 -2
View File
@@ -1,15 +1,27 @@
start_server {tags {"auth"}} {
test {AUTH fails if there is no password configured server side} {
catch {r auth foo} err
set _ $err
} {ERR*no password*}
}
start_server {tags {"auth"} overrides {requirepass foobar}} {
test {AUTH fails when a wrong password is given} {
catch {r auth wrong!} err
format $err
set _ $err
} {ERR*invalid password}
test {Arbitrary command gives an error when AUTH is required} {
catch {r set foo bar} err
format $err
set _ $err
} {ERR*operation not permitted}
test {AUTH succeeds when the right password is given} {
r auth foobar
} {OK}
test {Once AUTH succeeded we can actually send commands to the server} {
r set foo 100
r incr foo
} {101}
}
tests/unit/basic.tcl
+19
View File
@@ -262,6 +262,25 @@ start_server {tags {"basic"}} {
format $err
} {ERR*}
test {RENAME with volatile key, should move the TTL as well} {
r del mykey mykey2
r set mykey foo
r expire mykey 100
assert {[r ttl mykey] > 95 && [r ttl mykey] <= 100}
r rename mykey mykey2
assert {[r ttl mykey2] > 95 && [r ttl mykey2] <= 100}
}
test {RENAME with volatile key, should not inherit TTL of target key} {
r del mykey mykey2
r set mykey foo
r set mykey2 bar
r expire mykey2 100
assert {[r ttl mykey] == -1 && [r ttl mykey2] > 0}
r rename mykey mykey2
r ttl mykey2
} {-1}
test {DEL all keys again (DB 0)} {
foreach key [r keys *] {
r del $key
tests/unit/cas.tcl
+23 -2
View File
@@ -125,11 +125,32 @@ start_server {tags {"cas"}} {
test {WATCH will not consider touched expired keys} {
r del x
r set x foo
r expire x 2
r expire x 1
r watch x
after 3000
after 1100
r multi
r ping
r exec
} {PONG}
test {DISCARD should clear the WATCH dirty flag on the client} {
r watch x
r set x 10
r multi
r discard
r multi
r incr x
r exec
} {11}
test {DISCARD should UNWATCH all the keys} {
r watch x
r set x 10
r multi
r discard
r set x 10
r multi
r incr x
r exec
} {11}
}
tests/unit/introspection.tcl
+5
View File
@@ -0,0 +1,5 @@
start_server {tags {"introspection"}} {
test {CLIENT LIST} {
r client list
} {*addr=*:* fd=* idle=* flags=N db=9 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=client*}
}
tests/unit/maxmemory.tcl
+120
View File
@@ -0,0 +1,120 @@
start_server {tags {"maxmemory"}} {
foreach policy {
allkeys-random allkeys-lru volatile-lru volatile-random volatile-ttl
} {
test "maxmemory - is the memory limit honoured? (policy $policy)" {
# make sure to start with a blank instance
r flushall
# Get the current memory limit and calculate a new limit.
# We just add 100k to the current memory size so that it is
# fast for us to reach that limit.
set used [s used_memory]
set limit [expr {$used+100*1024}]
r config set maxmemory $limit
r config set maxmemory-policy $policy
# Now add keys until the limit is almost reached.
set numkeys 0
while 1 {
r setex [randomKey] 10000 x
incr numkeys
if {[s used_memory]+4096 > $limit} {
assert {$numkeys > 10}
break
}
}
# If we add the same number of keys already added again, we
# should still be under the limit.
for {set j 0} {$j < $numkeys} {incr j} {
r setex [randomKey] 10000 x
}
assert {[s used_memory] < ($limit+4096)}
}
}
foreach policy {
allkeys-random allkeys-lru volatile-lru volatile-random volatile-ttl
} {
test "maxmemory - only allkeys-* should remove non-volatile keys ($policy)" {
# make sure to start with a blank instance
r flushall
# Get the current memory limit and calculate a new limit.
# We just add 100k to the current memory size so that it is
# fast for us to reach that limit.
set used [s used_memory]
set limit [expr {$used+100*1024}]
r config set maxmemory $limit
r config set maxmemory-policy $policy
# Now add keys until the limit is almost reached.
set numkeys 0
while 1 {
r set [randomKey] x
incr numkeys
if {[s used_memory]+4096 > $limit} {
assert {$numkeys > 10}
break
}
}
# If we add the same number of keys already added again and
# the policy is allkeys-* we should still be under the limit.
# Otherwise we should see an error reported by Redis.
set err 0
for {set j 0} {$j < $numkeys} {incr j} {
if {[catch {r set [randomKey] x} e]} {
if {[string match {*used memory*} $e]} {
set err 1
}
}
}
if {[string match allkeys-* $policy]} {
assert {[s used_memory] < ($limit+4096)}
} else {
assert {$err == 1}
}
}
}
foreach policy {
volatile-lru volatile-random volatile-ttl
} {
test "maxmemory - policy $policy should only remove volatile keys." {
# make sure to start with a blank instance
r flushall
# Get the current memory limit and calculate a new limit.
# We just add 100k to the current memory size so that it is
# fast for us to reach that limit.
set used [s used_memory]
set limit [expr {$used+100*1024}]
r config set maxmemory $limit
r config set maxmemory-policy $policy
# Now add keys until the limit is almost reached.
set numkeys 0
while 1 {
# Odd keys are volatile
# Even keys are non volatile
if {$numkeys % 2} {
r setex "key:$numkeys" 10000 x
} else {
r set "key:$numkeys" x
}
if {[s used_memory]+4096 > $limit} {
assert {$numkeys > 10}
break
}
incr numkeys
}
# Now we add the same number of volatile keys already added.
# We expect Redis to evict only volatile keys in order to make
# space.
set err 0
for {set j 0} {$j < $numkeys} {incr j} {
catch {r setex "foo:$j" 10000 x}
}
# We should still be under the limit.
assert {[s used_memory] < ($limit+4096)}
# However all our non volatile keys should be here.
for {set j 0} {$j < $numkeys} {incr j 2} {
assert {[r exists "key:$j"]}
}
}
}
}
tests/unit/protocol.tcl
+31
View File
@@ -59,4 +59,35 @@ start_server {tags {"protocol"}} {
reconnect
assert_error "*wrong*arguments*ping*" {r ping x y z}
}
set c 0
foreach seq [list "\x00" "*\x00" "$\x00"] {
incr c
test "Protocol desync regression test #$c" {
set s [socket [srv 0 host] [srv 0 port]]
puts -nonewline $s $seq
set payload [string repeat A 1024]"\n"
set test_start [clock seconds]
set test_time_limit 5
while 1 {
if {[catch {
puts -nonewline $s payload
flush $s
incr payload_size [string length $payload]
}]} {
set retval [gets $s]
close $s
break
} else {
set elapsed [expr {[clock seconds]-$test_start}]
if {$elapsed > $test_time_limit} {
close $s
error "assertion:Redis did not closed connection after protocol desync"
}
}
}
set retval
} {*Protocol error*}
}
unset c
}
tests/unit/sort.tcl
+12
View File
@@ -134,6 +134,18 @@ start_server {
assert_equal [lsort -real $floats] [r sort mylist]
}
test "SORT with STORE returns zero if result is empty (github isse 224)" {
r flushdb
r sort foo store bar
} {0}
test "SORT with STORE does not create empty lists (github issue 224)" {
r flushdb
r lpush foo bar
r sort foo limit 10 10 store zap
r exists zap
} {0}
tags {"slow"} {
set num 100
set res [create_random_dataset $num lpush]
tests/unit/type/hash.tcl
+7
View File
@@ -235,6 +235,13 @@ start_server {tags {"hash"}} {
r hgetall myhash
} {b 2}
test {HDEL - hash becomes empty before deleting all specified fields} {
r del myhash
r hmset myhash a 1 b 2 c 3
assert_equal 3 [r hdel myhash a b c d e]
assert_equal 0 [r exists myhash]
}
test {HEXISTS} {
set rv {}
set k [lindex [array names smallhash *] 0]
tests/unit/type/list.tcl
+14
View File
@@ -728,4 +728,18 @@ start_server {
assert_equal 3 [r llen myotherlist]
}
}
test "Regression for bug 593 - chaining BRPOPLPUSH with other blocking cmds" {
set rd1 [redis_deferring_client]
set rd2 [redis_deferring_client]
$rd1 brpoplpush a b 0
$rd1 brpoplpush a b 0
$rd2 brpoplpush b c 0
after 1000
r lpush a data
$rd1 close
$rd2 close
r ping
} {PONG}
}
tests/unit/type/zset.tcl
+6
View File
@@ -518,6 +518,12 @@ start_server {tags {"zset"}} {
r zinterstore set3 2 set1 set2
} {0}
test {ZUNIONSTORE regression, should not create NaN in scores} {
r zadd z -inf neginf
r zunionstore out 1 z weights 0
r zrange out 0 -1 withscores
} {neginf 0}
proc stressers {encoding} {
if {$encoding == "ziplist"} {
# Little extra to allow proper fuzzing in the sorting stresser
utils/install_server.sh
Executable
+156
View File
@@ -0,0 +1,156 @@
#! /bin/sh
# Copyright 2011 Dvir Volk <dvirsk at gmail dot com>. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification, are
# permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice, this list of
# conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice, this list
# of conditions and the following disclaimer in the documentation and/or other materials
# provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
# FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL Dvir Volk OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
#
# # # # # # # # # # # # # # # # # # # # # # # # # #
#
# Interactive service installer for redis server
# this generates a redis config file and an /etc/init.d script, and installs them
# this scripts should be run as root
#
die () {
echo "ERROR: $1. Aborting!"
exit 1
}
#Initial defaults
_REDIS_PORT=6379
echo "Welcome to the redis service installer"
echo "This script will help you easily set up a running redis server
"
#check for root user TODO: replace this with a call to "id"
if [ `whoami` != "root" ] ; then
echo "You must run this script as root. Sorry!"
exit 1
fi
#Read the redis port
read -p "Please select the redis port for this instance: [$_REDIS_PORT] " REDIS_PORT
if [ ! `echo $REDIS_PORT | egrep "^[0-9]+\$"` ] ; then
echo "Selecting default: $_REDIS_PORT"
REDIS_PORT=$_REDIS_PORT
fi
#read the redis config file
_REDIS_CONFIG_FILE="/etc/redis/$REDIS_PORT.conf"
read -p "Please select the redis config file name [$_REDIS_CONFIG_FILE] " REDIS_CONFIG_FILE
if [ !"$REDIS_CONFIG_FILE" ] ; then
REDIS_CONFIG_FILE=$_REDIS_CONFIG_FILE
echo "Selected default - $REDIS_CONFIG_FILE"
fi
#try and create it
mkdir -p `dirname "$REDIS_CONFIG_FILE"` || die "Could not create redis config directory"
#read the redis log file path
_REDIS_LOG_FILE="/var/log/redis_$REDIS_PORT.log"
read -p "Please select the redis log file name [$_REDIS_LOG_FILE] " REDIS_LOG_FILE
if [ !"$REDIS_LOG_FILE" ] ; then
REDIS_LOG_FILE=$_REDIS_LOG_FILE
echo "Selected default - $REDIS_LOG_FILE"
fi
#get the redis data directory
_REDIS_DATA_DIR="/var/lib/redis/$REDIS_PORT"
read -p "Please select the data directory for this instance [$_REDIS_DATA_DIR] " REDIS_DATA_DIR
if [ !"$REDIS_DATA_DIR" ] ; then
REDIS_DATA_DIR=$_REDIS_DATA_DIR
echo "Selected default - $REDIS_DATA_DIR"
fi
mkdir -p $REDIS_DATA_DIR || die "Could not create redis data directory"
#get the redis executable path
_REDIS_EXECUTABLE=`which redis-server`
read -p "Please select the redis executable path [$_REDIS_EXECUTABLE] " REDIS_EXECUTABLE
if [ ! -f "$REDIS_EXECUTABLE" ] ; then
REDIS_EXECUTABLE=$_REDIS_EXECUTABLE
if [ ! -f "$REDIS_EXECUTABLE" ] ; then
echo "Mmmmm... it seems like you don't have a redis executable. Did you run make install yet?"
exit 1
fi
fi
#render the tmplates
TMP_FILE="/tmp/$REDIS_PORT.conf"
TPL_FILE="./redis.conf.tpl"
INIT_TPL_FILE="./redis_init_script.tpl"
INIT_SCRIPT_DEST="/etc/init.d/redis_$REDIS_PORT"
PIDFILE="/var/run/redis_$REDIS_PORT.pid"
#check the default for redis cli
CLI_EXEC=`which redis-cli`
if [ ! "$CLI_EXEC" ] ; then
CLI_EXEC=`dirname $REDIS_EXECUTABLE`"/redis-cli"
fi
#Generate config file from template
echo "## Generated by install_server.sh ##" > $TMP_FILE
cat $TPL_FILE | while read line; do eval "echo \"$line\"" >> $TMP_FILE; done
cp -f $TMP_FILE $REDIS_CONFIG_FILE || exit 1
#Generate sample script from template file
rm -f $TMP_FILE
#we hard code the configs here to avoid issues with templates containing env vars
#kinda lame but works!
REDIS_INIT_HEADER=\
"#/bin/sh\n
#Configurations injected by install_server below....\n\n
EXEC=$REDIS_EXECUTABLE\n
CLIEXEC=$CLI_EXEC\n
PIDFILE=$PIDFILE\n
CONF=\"$REDIS_CONFIG_FILE\"\n\n
REDISPORT=\"$REDIS_PORT\"\n\n
###############\n\n"
#combine the header and the template (which is actually a static footer)
echo $REDIS_INIT_HEADER > $TMP_FILE && cat $INIT_TPL_FILE >> $TMP_FILE || die "Could not write init script to $TMP_FILE"
#copy to /etc/init.d
cp -f $TMP_FILE $INIT_SCRIPT_DEST && chmod +x $INIT_SCRIPT_DEST || die "Could not copy redis init script to $INIT_SCRIPT_DEST"
echo "Copied $TMP_FILE => $INIT_SCRIPT_DEST"
#Install the service
echo "Installing service..."
update-rc.d redis_$REDIS_PORT defaults && echo "Success!"
/etc/init.d/redis_$REDIS_PORT start || die "Failed starting service..."
#tada
echo "Installation successful!"
exit 0
utils/redis.conf.tpl
+402
View File
@@ -0,0 +1,402 @@
# Redis configuration file example
# Note on units: when memory size is needed, it is possible to specifiy
# it in the usual form of 1k 5GB 4M and so forth:
#
# 1k => 1000 bytes
# 1kb => 1024 bytes
# 1m => 1000000 bytes
# 1mb => 1024*1024 bytes
# 1g => 1000000000 bytes
# 1gb => 1024*1024*1024 bytes
#
# units are case insensitive so 1GB 1Gb 1gB are all the same.
# By default Redis does not run as a daemon. Use 'yes' if you need it.
# Note that Redis will write a pid file in /var/run/redis.pid when daemonized.
daemonize yes
# When running daemonized, Redis writes a pid file in /var/run/redis.pid by
# default. You can specify a custom pid file location here.
pidfile $PIDFILE
# Accept connections on the specified port, default is 6379.
# If port 0 is specified Redis will not listen on a TCP socket.
port $REDIS_PORT
# If you want you can bind a single interface, if the bind option is not
# specified all the interfaces will listen for incoming connections.
#
# bind 127.0.0.1
# Specify the path for the unix socket that will be used to listen for
# incoming connections. There is no default, so Redis will not listen
# on a unix socket when not specified.
#
# unixsocket /tmp/redis.sock
# Close the connection after a client is idle for N seconds (0 to disable)
timeout 300
# Set server verbosity to 'debug'
# it can be one of:
# debug (a lot of information, useful for development/testing)
# verbose (many rarely useful info, but not a mess like the debug level)
# notice (moderately verbose, what you want in production probably)
# warning (only very important / critical messages are logged)
loglevel verbose
# Specify the log file name. Also 'stdout' can be used to force
# Redis to log on the standard output. Note that if you use standard
# output for logging but daemonize, logs will be sent to /dev/null
logfile $REDIS_LOG_FILE
# To enable logging to the system logger, just set 'syslog-enabled' to yes,
# and optionally update the other syslog parameters to suit your needs.
# syslog-enabled no
# Specify the syslog identity.
# syslog-ident redis
# Specify the syslog facility. Must be USER or between LOCAL0-LOCAL7.
# syslog-facility local0
# Set the number of databases. The default database is DB 0, you can select
# a different one on a per-connection basis using SELECT <dbid> where
# dbid is a number between 0 and 'databases'-1
databases 16
################################ SNAPSHOTTING #################################
#
# Save the DB on disk:
#
# save <seconds> <changes>
#
# Will save the DB if both the given number of seconds and the given
# number of write operations against the DB occurred.
#
# In the example below the behaviour will be to save:
# after 900 sec (15 min) if at least 1 key changed
# after 300 sec (5 min) if at least 10 keys changed
# after 60 sec if at least 10000 keys changed
#
# Note: you can disable saving at all commenting all the "save" lines.
save 900 1
save 300 10
save 60 10000
# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes
# The filename where to dump the DB
dbfilename dump.rdb
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# Also the Append Only File will be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir $REDIS_DATA_DIR
################################# REPLICATION #################################
# Master-Slave replication. Use slaveof to make a Redis instance a copy of
# another Redis server. Note that the configuration is local to the slave
# so for example it is possible to configure the slave to save the DB with a
# different interval, or to listen to another port, and so on.
#
# slaveof <masterip> <masterport>
# If the master is password protected (using the "requirepass" configuration
# directive below) it is possible to tell the slave to authenticate before
# starting the replication synchronization process, otherwise the master will
# refuse the slave request.
#
# masterauth <master-password>
# When a slave lost the connection with the master, or when the replication
# is still in progress, the slave can act in two different ways:
#
# 1) if slave-serve-stale-data is set to 'yes' (the default) the slave will
# still reply to client requests, possibly with out of data data, or the
# data set may just be empty if this is the first synchronization.
#
# 2) if slave-serve-stale data is set to 'no' the slave will reply with
# an error "SYNC with master in progress" to all the kind of commands
# but to INFO and SLAVEOF.
#
slave-serve-stale-data yes
################################## SECURITY ###################################
# Require clients to issue AUTH <PASSWORD> before processing any other
# commands. This might be useful in environments in which you do not trust
# others with access to the host running redis-server.
#
# This should stay commented out for backward compatibility and because most
# people do not need auth (e.g. they run their own servers).
#
# Warning: since Redis is pretty fast an outside user can try up to
# 150k passwords per second against a good box. This means that you should
# use a very strong password otherwise it will be very easy to break.
#
# requirepass foobared
# Command renaming.
#
# It is possilbe to change the name of dangerous commands in a shared
# environment. For instance the CONFIG command may be renamed into something
# of hard to guess so that it will be still available for internal-use
# tools but not available for general clients.
#
# Example:
#
# rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52
#
# It is also possilbe to completely kill a command renaming it into
# an empty string:
#
# rename-command CONFIG ""
################################### LIMITS ####################################
# Set the max number of connected clients at the same time. By default there
# is no limit, and it's up to the number of file descriptors the Redis process
# is able to open. The special value '0' means no limits.
# Once the limit is reached Redis will close all the new connections sending
# an error 'max number of clients reached'.
#
# maxclients 128
# Don't use more memory than the specified amount of bytes.
# When the memory limit is reached Redis will try to remove keys with an
# EXPIRE set. It will try to start freeing keys that are going to expire
# in little time and preserve keys with a longer time to live.
# Redis will also try to remove objects from free lists if possible.
#
# If all this fails, Redis will start to reply with errors to commands
# that will use more memory, like SET, LPUSH, and so on, and will continue
# to reply to most read-only commands like GET.
#
# WARNING: maxmemory can be a good idea mainly if you want to use Redis as a
# 'state' server or cache, not as a real DB. When Redis is used as a real
# database the memory usage will grow over the weeks, it will be obvious if
# it is going to use too much memory in the long run, and you'll have the time
# to upgrade. With maxmemory after the limit is reached you'll start to get
# errors for write operations, and this may even lead to DB inconsistency.
#
# maxmemory <bytes>
# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
# is reached? You can select among five behavior:
#
# volatile-lru -> remove the key with an expire set using an LRU algorithm
# allkeys-lru -> remove any key accordingly to the LRU algorithm
# volatile-random -> remove a random key with an expire set
# allkeys->random -> remove a random key, any key
# volatile-ttl -> remove the key with the nearest expire time (minor TTL)
# noeviction -> don't expire at all, just return an error on write operations
#
# Note: with all the kind of policies, Redis will return an error on write
# operations, when there are not suitable keys for eviction.
#
# At the date of writing this commands are: set setnx setex append
# incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd
# sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby
# zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby
# getset mset msetnx exec sort
#
# The default is:
#
# maxmemory-policy volatile-lru
# LRU and minimal TTL algorithms are not precise algorithms but approximated
# algorithms (in order to save memory), so you can select as well the sample
# size to check. For instance for default Redis will check three keys and
# pick the one that was used less recently, you can change the sample size
# using the following configuration directive.
#
# maxmemory-samples 3
############################## APPEND ONLY MODE ###############################
# By default Redis asynchronously dumps the dataset on disk. If you can live
# with the idea that the latest records will be lost if something like a crash
# happens this is the preferred way to run Redis. If instead you care a lot
# about your data and don't want to that a single record can get lost you should
# enable the append only mode: when this mode is enabled Redis will append
# every write operation received in the file appendonly.aof. This file will
# be read on startup in order to rebuild the full dataset in memory.
#
# Note that you can have both the async dumps and the append only file if you
# like (you have to comment the "save" statements above to disable the dumps).
# Still if append only mode is enabled Redis will load the data from the
# log file at startup ignoring the dump.rdb file.
#
# IMPORTANT: Check the BGREWRITEAOF to check how to rewrite the append
# log file in background when it gets too big.
appendonly no
# The name of the append only file (default: "appendonly.aof")
# appendfilename appendonly.aof
# The fsync() call tells the Operating System to actually write data on disk
# instead to wait for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log . Slow, Safest.
# everysec: fsync only if one second passed since the last fsync. Compromise.
#
# The default is "everysec" that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# If unsure, use "everysec".
# appendfsync always
appendfsync everysec
# appendfsync no
# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving the durability of Redis is
# the same as "appendfsync none", that in pratical terms means that it is
# possible to lost up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.
no-appendfsync-on-rewrite no
# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size will growth by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (or if no rewrite happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a precentage of zero in order to disable the automatic AOF
# rewrite feature.
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
################################ LUA SCRIPTING ###############################
# Max execution time of a Lua script in milliseconds.
# This prevents that a programming error generating an infinite loop will block
# your server forever. Set it to 0 or a negative value for unlimited execution.
#lua-time-limit 60000
################################## SLOW LOG ###################################
# The Redis Slow Log is a system to log queries that exceeded a specified
# execution time. The execution time does not include the I/O operations
# like talking with the client, sending the reply and so forth,
# but just the time needed to actually execute the command (this is the only
# stage of command execution where the thread is blocked and can not serve
# other requests in the meantime).
#
# You can configure the slow log with two parameters: one tells Redis
# what is the execution time, in microseconds, to exceed in order for the
# command to get logged, and the other parameter is the length of the
# slow log. When a new command is logged the oldest one is removed from the
# queue of logged commands.
# The following time is expressed in microseconds, so 1000000 is equivalent
# to one second. Note that a negative number disables the slow log, while
# a value of zero forces the logging of every command.
slowlog-log-slower-than 10000
# There is no limit to this length. Just be aware that it will consume memory.
# You can reclaim memory used by the slow log with SLOWLOG RESET.
slowlog-max-len 1024
############################### ADVANCED CONFIG ###############################
# Hashes are encoded in a special way (much more memory efficient) when they
# have at max a given numer of elements, and the biggest element does not
# exceed a given threshold. You can configure this limits with the following
# configuration directives.
hash-max-zipmap-entries 512
hash-max-zipmap-value 64
# Similarly to hashes, small lists are also encoded in a special way in order
# to save a lot of space. The special representation is only used when
# you are under the following limits:
list-max-ziplist-entries 512
list-max-ziplist-value 64
# Sets have a special encoding in just one case: when a set is composed
# of just strings that happens to be integers in radix 10 in the range
# of 64 bit signed integers.
# The following configuration setting sets the limit in the size of the
# set in order to use this special memory saving encoding.
set-max-intset-entries 512
# Similarly to hashes and lists, sorted sets are also specially encoded in
# order to save a lot of space. This encoding is only used when the length and
# elements of a sorted set are below the following limits:
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
# Active rehashing uses 1 millisecond every 100 milliseconds of CPU time in
# order to help rehashing the main Redis hash table (the one mapping top-level
# keys to values). The hash table implementation redis uses (see dict.c)
# performs a lazy rehashing: the more operation you run into an hash table
# that is rhashing, the more rehashing "steps" are performed, so if the
# server is idle the rehashing is never complete and some more memory is used
# by the hash table.
#
# The default is to use this millisecond 10 times every second in order to
# active rehashing the main dictionaries, freeing memory when possible.
#
# If unsure:
# use "activerehashing no" if you have hard latency requirements and it is
# not a good thing in your environment that Redis can reply form time to time
# to queries with 2 milliseconds delay.
#
# use "activerehashing yes" if you don't have such hard requirements but
# want to free memory asap when possible.
activerehashing yes
################################## INCLUDES ###################################
# Include one or more other config files here. This is useful if you
# have a standard template that goes to all redis server but also need
# to customize a few per-server settings. Include files can include
# other files, so use this wisely.
#
# include /path/to/local.conf
# include /path/to/other.conf
utils/redis_init_script.tpl
Executable
+31
View File
@@ -0,0 +1,31 @@
case "$1" in
start)
if [ -f $$PIDFILE ]
then
echo "$PIDFILE exists, process is already running or crashed"
else
echo "Starting Redis server..."
$EXEC $CONF
fi
;;
stop)
if [ ! -f $PIDFILE ]
then
echo "$PIDFILE does not exist, process is not running"
else
PID=$(cat $PIDFILE)
echo "Stopping ..."
$CLIEXEC -p $REDISPORT shutdown
while [ -x /proc/${PID} ]
do
echo "Waiting for Redis to shutdown ..."
sleep 1
done
echo "Redis stopped"
fi
;;
*)
echo "Please use start or stop as first argument"
;;
esac