Redis/redis
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Compare commits

base: Redis/redis:3.2.0
Branches Tags
Redis/redis:6.2 Redis/redis:unstable Redis/redis:7.2 Redis/redis:7.4 Redis/redis:8.0
Redis/redis:7.2.10 Redis/redis:6.2.19 Redis/redis:7.4.5 Redis/redis:8.0.3 Redis/redis:8.2-rc1 Redis/redis:8.2-rc1-int Redis/redis:8.2-m01 Redis/redis:8.2-m01-int2 Redis/redis:8.2-m01-int Redis/redis:8.0.2 Redis/redis:7.2.9 Redis/redis:7.4.4 Redis/redis:8.0.1 Redis/redis:8.0.1-int Redis/redis:8.0.0 Redis/redis:8.0-rc2-int Redis/redis:7.4.3 Redis/redis:7.2.8 Redis/redis:6.2.18 Redis/redis:8.0-rc1 Redis/redis:8.0-rc1-int2 Redis/redis:8.0-rc1-int Redis/redis:8.0-m04 Redis/redis:8.0-m04-int Redis/redis:8.0-m03 Redis/redis:7.2.7 Redis/redis:7.4.2 Redis/redis:6.2.17 Redis/redis:8.0-m02 Redis/redis:6.2.16 Redis/redis:7.2.6 Redis/redis:7.4.1 Redis/redis:8.0-m01 Redis/redis:7.4.0 Redis/redis:7.4-rc2 Redis/redis:7.4-rc1 Redis/redis:7.2.5 Redis/redis:7.2.4 Redis/redis:7.0.15 Redis/redis:7.2.3 Redis/redis:7.2.2 Redis/redis:7.0.14 Redis/redis:6.2.15 Redis/redis:6.2.14 Redis/redis:7.2.1 Redis/redis:7.0.13 Redis/redis:7.2.0 Redis/redis:7.2-rc3 Redis/redis:7.0.12 Redis/redis:6.2.13 Redis/redis:6.0.20 Redis/redis:7.2-rc2 Redis/redis:6.0.19 Redis/redis:6.2.12 Redis/redis:7.0.11 Redis/redis:7.2-rc1 Redis/redis:7.0.10 Redis/redis:7.0.9 Redis/redis:6.2.11 Redis/redis:6.0.18 Redis/redis:6.2.10 Redis/redis:6.0.17 Redis/redis:6.2.9 Redis/redis:7.0.8 Redis/redis:7.0.7 Redis/redis:7.0.6 Redis/redis:6.2.8 Redis/redis:7.0.5 Redis/redis:7.0.4 Redis/redis:7.0.3 Redis/redis:7.0.2 Redis/redis:7.0.1 Redis/redis:7.0.0 Redis/redis:6.2.7 Redis/redis:7.0-rc3 Redis/redis:7.0-rc2 Redis/redis:7.0-rc1 Redis/redis:6.2.6 Redis/redis:6.0.16 Redis/redis:5.0.14 Redis/redis:6.2.5 Redis/redis:5.0.13 Redis/redis:6.0.15 Redis/redis:6.0.14 Redis/redis:6.2.4 Redis/redis:6.2.3 Redis/redis:6.0.13 Redis/redis:6.2.2 Redis/redis:6.2.1 Redis/redis:6.0.12 Redis/redis:5.0.12 Redis/redis:6.0.11 Redis/redis:6.2.0 Redis/redis:5.0.11 Redis/redis:6.2-rc3 Redis/redis:6.0.10 Redis/redis:6.2-rc2 Redis/redis:6.2-rc1 Redis/redis:6.0.9 Redis/redis:5.0.10 Redis/redis:6.0.8 Redis/redis:6.0.7 Redis/redis:6.0.6 Redis/redis:6.0.5 Redis/redis:6.0.4 Redis/redis:6.0.3 Redis/redis:6.0.2 Redis/redis:6.0.1 Redis/redis:6.0.0 Redis/redis:5.0.9 Redis/redis:6.0-rc4 Redis/redis:6.0-rc3 Redis/redis:5.0.8 Redis/redis:6.0-rc2 Redis/redis:6.0-rc1 Redis/redis:5.0.7 Redis/redis:5.0.6 Redis/redis:5.0.5 Redis/redis:3.2.13 Redis/redis:4.0.14 Redis/redis:5.0.4 Redis/redis:4.0.13 Redis/redis:5.0.3 Redis/redis:4.0.12 Redis/redis:5.0.2 Redis/redis:5.0.1 Redis/redis:5.0.0 Redis/redis:5.0-rc6 Redis/redis:5.0-rc5 Redis/redis:4.0.11 Redis/redis:5.0-rc4 Redis/redis:5.0-rc3 Redis/redis:5.0-rc2 Redis/redis:4.0.10 Redis/redis:3.2.12 Redis/redis:5.0-rc1 Redis/redis:4.0.9 Redis/redis:4.0.8 Redis/redis:4.0.7 Redis/redis:4.0.6 Redis/redis:4.0.5 Redis/redis:4.0.4 Redis/redis:4.0.3 Redis/redis:3.2.11 Redis/redis:4.0.2 Redis/redis:3.2.10 Redis/redis:4.0.1 Redis/redis:4.0.0 Redis/redis:3.2.9 Redis/redis:4.0-rc3 Redis/redis:3.2.8 Redis/redis:3.2.7 Redis/redis:3.2.6 Redis/redis:4.0-rc2 Redis/redis:4.0-rc1 Redis/redis:3.2.5 Redis/redis:3.2.4 Redis/redis:3.2.3 Redis/redis:3.2.2 Redis/redis:3.2.1 Redis/redis:3.2.0 Redis/redis:3.2.0-rc3 Redis/redis:3.0.7 Redis/redis:3.2.0-rc2 Redis/redis:3.2-rc1 Redis/redis:3.0.6 Redis/redis:2.8.24 Redis/redis:3.0.5 Redis/redis:2.8.23 Redis/redis:2.8.22 Redis/redis:3.0.4 Redis/redis:3.0.3 Redis/redis:3.0.2 Redis/redis:2.8.21 Redis/redis:2.8.20 Redis/redis:3.0.1 Redis/redis:3.0.0 Redis/redis:3.0.0-rc6 Redis/redis:3.0.0-rc5 Redis/redis:3.0.0-rc4 Redis/redis:3.0.0-rc3 Redis/redis:3.0.0-rc2 Redis/redis:2.8.19 Redis/redis:2.8.18 Redis/redis:3.0.0-rc1 Redis/redis:2.8.17 Redis/redis:2.8.16 Redis/redis:2.8.15 Redis/redis:2.8.14 Redis/redis:3.0.0-beta8 Redis/redis:2.8.13 Redis/redis:3.0.0-beta7 Redis/redis:2.8.12 Redis/redis:2.8.11 Redis/redis:3.0.0-beta6 Redis/redis:2.8.10 Redis/redis:3.0.0-beta5 Redis/redis:3.0.0-beta4 Redis/redis:3.0.0-beta3 Redis/redis:2.8.9 Redis/redis:2.8.8 Redis/redis:3.0.0-beta2 Redis/redis:2.8.7 Redis/redis:2.8.6 Redis/redis:3.0.0-beta1 Redis/redis:2.8.5 Redis/redis:2.8.4 Redis/redis:2.6.17 Redis/redis:2.8.3 Redis/redis:2.8.2 Redis/redis:2.8.1 Redis/redis:2.8.0 Redis/redis:2.8.0-rc6 Redis/redis:2.8.0-rc5 Redis/redis:2.8.0-rc4 Redis/redis:2.6.16 Redis/redis:2.6.15 Redis/redis:2.8.0-rc3 Redis/redis:2.8.0-rc2 Redis/redis:2.8.0-rc1 Redis/redis:2.6.14-2 Redis/redis:2.6.14-1 Redis/redis:2.6.14 Redis/redis:2.6.13 Redis/redis:2.6.12 Redis/redis:2.6.10-3 Redis/redis:2.6.10-2 Redis/redis:2.6.11 Redis/redis:2.6.10-1 Redis/redis:2.6.10 Redis/redis:2.6.9-1 Redis/redis:2.6.9 Redis/redis:2.6.8-1 Redis/redis:2.6.8 Redis/redis:2.6.7-1 Redis/redis:2.6.7 Redis/redis:2.6.6 Redis/redis:2.4.18 Redis/redis:2.6.5 Redis/redis:2.6.4 Redis/redis:2.6.3 Redis/redis:2.6.2 Redis/redis:2.6.1 Redis/redis:2.6.0 Redis/redis:2.6.0-rc8 Redis/redis:2.6.0-rc7 Redis/redis:2.4.17 Redis/redis:2.6.0-rc6 Redis/redis:2.4.16 Redis/redis:2.4.15 Redis/redis:2.6.0-rc5 Redis/redis:2.6.0-rc4 Redis/redis:2.4.14 Redis/redis:2.6.0-rc3 Redis/redis:2.6.0-rc2 Redis/redis:2.4.13 Redis/redis:2.4.12 Redis/redis:2.6.0-rc1 Redis/redis:2.4.11 Redis/redis:2.4.10 Redis/redis:2.4.9 Redis/redis:2.4.8 Redis/redis:2.4.7 Redis/redis:2.4.6 Redis/redis:2.4.5 Redis/redis:2.4.4 Redis/redis:2.4.3 Redis/redis:2.4.2 Redis/redis:2.2.15 Redis/redis:2.4.1 Redis/redis:2.4.0 Redis/redis:2.2.14 Redis/redis:2.4.0-rc8 Redis/redis:2.2.13 Redis/redis:2.4.0-rc7 Redis/redis:2.4.0-rc6 Redis/redis:2.4.0-rc5 Redis/redis:2.4.0-rc4 Redis/redis:2.2.12 Redis/redis:2.4.0-rc3 Redis/redis:with-deprecated-diskstore Redis/redis:2.2.111-scripting Redis/redis:2.2.11 Redis/redis:2.2.110-scripting Redis/redis:2.2.10 Redis/redis:2.2.9 Redis/redis:2.4.0-rc2 Redis/redis:2.4.0-rc1 Redis/redis:2.2.8 Redis/redis:2.2.107-scripting Redis/redis:2.2.7 Redis/redis:2.2.106-scripting Redis/redis:2.2.6 Redis/redis:2.2.105-scripting Redis/redis:2.2.5 Redis/redis:2.2.4 Redis/redis:2.2.3 Redis/redis:3.0-alpha0 Redis/redis:2.0.5 Redis/redis:2.2.2 Redis/redis:2.2.1 Redis/redis:2.2.0 Redis/redis:2.2.0-rc4 Redis/redis:2.3-alpha0 Redis/redis:2.2.0-rc3 Redis/redis:2.2.0-rc2 Redis/redis:2.2.0-rc1 Redis/redis:2.2-alpha6 Redis/redis:2.2-alpha5 Redis/redis:v2.0.4-stable Redis/redis:2.0.4 Redis/redis:2.2-alpha4 Redis/redis:2.2-alpha3 Redis/redis:v2.0.3-stable Redis/redis:2.0.3 Redis/redis:v2.0.2-stable Redis/redis:2.0.2 Redis/redis:2.2-alpha2 Redis/redis:v2.0.1-stable Redis/redis:2.0.1 Redis/redis:v2.0.0-stable Redis/redis:2.0.0 Redis/redis:2.2-alpha1 Redis/redis:twitter-20100825 Redis/redis:twitter-20100804 Redis/redis:2.2-alpha0 Redis/redis:v2.0.0-rc4 Redis/redis:v2.0.0-rc3 Redis/redis:v2.0.0-rc2 Redis/redis:v2.0.0-rc1b Redis/redis:v2.1.1-watch Redis/redis:v2.0.0-rc1 Redis/redis:v1.3.12 Redis/redis:v1.3.11 Redis/redis:v1.3.10 Redis/redis:v1.3.9 Redis/redis:v1.3.8 Redis/redis:v1.3.7 Redis/redis:1.3.6 Redis/redis:vm-playpen
...
compare: Redis/redis:3.2.13
Branches Tags
Redis/redis:unstable Redis/redis:7.2 Redis/redis:6.2 Redis/redis:7.4 Redis/redis:8.0
Redis/redis:7.2.10 Redis/redis:6.2.19 Redis/redis:7.4.5 Redis/redis:8.0.3 Redis/redis:8.2-rc1 Redis/redis:8.2-rc1-int Redis/redis:8.2-m01 Redis/redis:8.2-m01-int2 Redis/redis:8.2-m01-int Redis/redis:8.0.2 Redis/redis:7.2.9 Redis/redis:7.4.4 Redis/redis:8.0.1 Redis/redis:8.0.1-int Redis/redis:8.0.0 Redis/redis:8.0-rc2-int Redis/redis:7.4.3 Redis/redis:7.2.8 Redis/redis:6.2.18 Redis/redis:8.0-rc1 Redis/redis:8.0-rc1-int2 Redis/redis:8.0-rc1-int Redis/redis:8.0-m04 Redis/redis:8.0-m04-int Redis/redis:8.0-m03 Redis/redis:7.2.7 Redis/redis:7.4.2 Redis/redis:6.2.17 Redis/redis:8.0-m02 Redis/redis:6.2.16 Redis/redis:7.2.6 Redis/redis:7.4.1 Redis/redis:8.0-m01 Redis/redis:7.4.0 Redis/redis:7.4-rc2 Redis/redis:7.4-rc1 Redis/redis:7.2.5 Redis/redis:7.2.4 Redis/redis:7.0.15 Redis/redis:7.2.3 Redis/redis:7.2.2 Redis/redis:7.0.14 Redis/redis:6.2.15 Redis/redis:6.2.14 Redis/redis:7.2.1 Redis/redis:7.0.13 Redis/redis:7.2.0 Redis/redis:7.2-rc3 Redis/redis:7.0.12 Redis/redis:6.2.13 Redis/redis:6.0.20 Redis/redis:7.2-rc2 Redis/redis:6.0.19 Redis/redis:6.2.12 Redis/redis:7.0.11 Redis/redis:7.2-rc1 Redis/redis:7.0.10 Redis/redis:7.0.9 Redis/redis:6.2.11 Redis/redis:6.0.18 Redis/redis:6.2.10 Redis/redis:6.0.17 Redis/redis:6.2.9 Redis/redis:7.0.8 Redis/redis:7.0.7 Redis/redis:7.0.6 Redis/redis:6.2.8 Redis/redis:7.0.5 Redis/redis:7.0.4 Redis/redis:7.0.3 Redis/redis:7.0.2 Redis/redis:7.0.1 Redis/redis:7.0.0 Redis/redis:6.2.7 Redis/redis:7.0-rc3 Redis/redis:7.0-rc2 Redis/redis:7.0-rc1 Redis/redis:6.2.6 Redis/redis:6.0.16 Redis/redis:5.0.14 Redis/redis:6.2.5 Redis/redis:5.0.13 Redis/redis:6.0.15 Redis/redis:6.0.14 Redis/redis:6.2.4 Redis/redis:6.2.3 Redis/redis:6.0.13 Redis/redis:6.2.2 Redis/redis:6.2.1 Redis/redis:6.0.12 Redis/redis:5.0.12 Redis/redis:6.0.11 Redis/redis:6.2.0 Redis/redis:5.0.11 Redis/redis:6.2-rc3 Redis/redis:6.0.10 Redis/redis:6.2-rc2 Redis/redis:6.2-rc1 Redis/redis:6.0.9 Redis/redis:5.0.10 Redis/redis:6.0.8 Redis/redis:6.0.7 Redis/redis:6.0.6 Redis/redis:6.0.5 Redis/redis:6.0.4 Redis/redis:6.0.3 Redis/redis:6.0.2 Redis/redis:6.0.1 Redis/redis:6.0.0 Redis/redis:5.0.9 Redis/redis:6.0-rc4 Redis/redis:6.0-rc3 Redis/redis:5.0.8 Redis/redis:6.0-rc2 Redis/redis:6.0-rc1 Redis/redis:5.0.7 Redis/redis:5.0.6 Redis/redis:5.0.5 Redis/redis:3.2.13 Redis/redis:4.0.14 Redis/redis:5.0.4 Redis/redis:4.0.13 Redis/redis:5.0.3 Redis/redis:4.0.12 Redis/redis:5.0.2 Redis/redis:5.0.1 Redis/redis:5.0.0 Redis/redis:5.0-rc6 Redis/redis:5.0-rc5 Redis/redis:4.0.11 Redis/redis:5.0-rc4 Redis/redis:5.0-rc3 Redis/redis:5.0-rc2 Redis/redis:4.0.10 Redis/redis:3.2.12 Redis/redis:5.0-rc1 Redis/redis:4.0.9 Redis/redis:4.0.8 Redis/redis:4.0.7 Redis/redis:4.0.6 Redis/redis:4.0.5 Redis/redis:4.0.4 Redis/redis:4.0.3 Redis/redis:3.2.11 Redis/redis:4.0.2 Redis/redis:3.2.10 Redis/redis:4.0.1 Redis/redis:4.0.0 Redis/redis:3.2.9 Redis/redis:4.0-rc3 Redis/redis:3.2.8 Redis/redis:3.2.7 Redis/redis:3.2.6 Redis/redis:4.0-rc2 Redis/redis:4.0-rc1 Redis/redis:3.2.5 Redis/redis:3.2.4 Redis/redis:3.2.3 Redis/redis:3.2.2 Redis/redis:3.2.1 Redis/redis:3.2.0 Redis/redis:3.2.0-rc3 Redis/redis:3.0.7 Redis/redis:3.2.0-rc2 Redis/redis:3.2-rc1 Redis/redis:3.0.6 Redis/redis:2.8.24 Redis/redis:3.0.5 Redis/redis:2.8.23 Redis/redis:2.8.22 Redis/redis:3.0.4 Redis/redis:3.0.3 Redis/redis:3.0.2 Redis/redis:2.8.21 Redis/redis:2.8.20 Redis/redis:3.0.1 Redis/redis:3.0.0 Redis/redis:3.0.0-rc6 Redis/redis:3.0.0-rc5 Redis/redis:3.0.0-rc4 Redis/redis:3.0.0-rc3 Redis/redis:3.0.0-rc2 Redis/redis:2.8.19 Redis/redis:2.8.18 Redis/redis:3.0.0-rc1 Redis/redis:2.8.17 Redis/redis:2.8.16 Redis/redis:2.8.15 Redis/redis:2.8.14 Redis/redis:3.0.0-beta8 Redis/redis:2.8.13 Redis/redis:3.0.0-beta7 Redis/redis:2.8.12 Redis/redis:2.8.11 Redis/redis:3.0.0-beta6 Redis/redis:2.8.10 Redis/redis:3.0.0-beta5 Redis/redis:3.0.0-beta4 Redis/redis:3.0.0-beta3 Redis/redis:2.8.9 Redis/redis:2.8.8 Redis/redis:3.0.0-beta2 Redis/redis:2.8.7 Redis/redis:2.8.6 Redis/redis:3.0.0-beta1 Redis/redis:2.8.5 Redis/redis:2.8.4 Redis/redis:2.6.17 Redis/redis:2.8.3 Redis/redis:2.8.2 Redis/redis:2.8.1 Redis/redis:2.8.0 Redis/redis:2.8.0-rc6 Redis/redis:2.8.0-rc5 Redis/redis:2.8.0-rc4 Redis/redis:2.6.16 Redis/redis:2.6.15 Redis/redis:2.8.0-rc3 Redis/redis:2.8.0-rc2 Redis/redis:2.8.0-rc1 Redis/redis:2.6.14-2 Redis/redis:2.6.14-1 Redis/redis:2.6.14 Redis/redis:2.6.13 Redis/redis:2.6.12 Redis/redis:2.6.10-3 Redis/redis:2.6.10-2 Redis/redis:2.6.11 Redis/redis:2.6.10-1 Redis/redis:2.6.10 Redis/redis:2.6.9-1 Redis/redis:2.6.9 Redis/redis:2.6.8-1 Redis/redis:2.6.8 Redis/redis:2.6.7-1 Redis/redis:2.6.7 Redis/redis:2.6.6 Redis/redis:2.4.18 Redis/redis:2.6.5 Redis/redis:2.6.4 Redis/redis:2.6.3 Redis/redis:2.6.2 Redis/redis:2.6.1 Redis/redis:2.6.0 Redis/redis:2.6.0-rc8 Redis/redis:2.6.0-rc7 Redis/redis:2.4.17 Redis/redis:2.6.0-rc6 Redis/redis:2.4.16 Redis/redis:2.4.15 Redis/redis:2.6.0-rc5 Redis/redis:2.6.0-rc4 Redis/redis:2.4.14 Redis/redis:2.6.0-rc3 Redis/redis:2.6.0-rc2 Redis/redis:2.4.13 Redis/redis:2.4.12 Redis/redis:2.6.0-rc1 Redis/redis:2.4.11 Redis/redis:2.4.10 Redis/redis:2.4.9 Redis/redis:2.4.8 Redis/redis:2.4.7 Redis/redis:2.4.6 Redis/redis:2.4.5 Redis/redis:2.4.4 Redis/redis:2.4.3 Redis/redis:2.4.2 Redis/redis:2.2.15 Redis/redis:2.4.1 Redis/redis:2.4.0 Redis/redis:2.2.14 Redis/redis:2.4.0-rc8 Redis/redis:2.2.13 Redis/redis:2.4.0-rc7 Redis/redis:2.4.0-rc6 Redis/redis:2.4.0-rc5 Redis/redis:2.4.0-rc4 Redis/redis:2.2.12 Redis/redis:2.4.0-rc3 Redis/redis:with-deprecated-diskstore Redis/redis:2.2.111-scripting Redis/redis:2.2.11 Redis/redis:2.2.110-scripting Redis/redis:2.2.10 Redis/redis:2.2.9 Redis/redis:2.4.0-rc2 Redis/redis:2.4.0-rc1 Redis/redis:2.2.8 Redis/redis:2.2.107-scripting Redis/redis:2.2.7 Redis/redis:2.2.106-scripting Redis/redis:2.2.6 Redis/redis:2.2.105-scripting Redis/redis:2.2.5 Redis/redis:2.2.4 Redis/redis:2.2.3 Redis/redis:3.0-alpha0 Redis/redis:2.0.5 Redis/redis:2.2.2 Redis/redis:2.2.1 Redis/redis:2.2.0 Redis/redis:2.2.0-rc4 Redis/redis:2.3-alpha0 Redis/redis:2.2.0-rc3 Redis/redis:2.2.0-rc2 Redis/redis:2.2.0-rc1 Redis/redis:2.2-alpha6 Redis/redis:2.2-alpha5 Redis/redis:v2.0.4-stable Redis/redis:2.0.4 Redis/redis:2.2-alpha4 Redis/redis:2.2-alpha3 Redis/redis:v2.0.3-stable Redis/redis:2.0.3 Redis/redis:v2.0.2-stable Redis/redis:2.0.2 Redis/redis:2.2-alpha2 Redis/redis:v2.0.1-stable Redis/redis:2.0.1 Redis/redis:v2.0.0-stable Redis/redis:2.0.0 Redis/redis:2.2-alpha1 Redis/redis:twitter-20100825 Redis/redis:twitter-20100804 Redis/redis:2.2-alpha0 Redis/redis:v2.0.0-rc4 Redis/redis:v2.0.0-rc3 Redis/redis:v2.0.0-rc2 Redis/redis:v2.0.0-rc1b Redis/redis:v2.1.1-watch Redis/redis:v2.0.0-rc1 Redis/redis:v1.3.12 Redis/redis:v1.3.11 Redis/redis:v1.3.10 Redis/redis:v1.3.9 Redis/redis:v1.3.8 Redis/redis:v1.3.7 Redis/redis:1.3.6 Redis/redis:vm-playpen
171 Commits
3.2.0 ... 3.2.13
Author SHA1 Message Date
antirez 990cd2c835 Redis 3.2.13. 2019-03-18 17:25:02 +01:00
antirez 635119ded1 HyperLogLog: handle wrong offset in the base case. 2019-03-18 11:31:38 +01:00
John Sully f11666847a Fix hyperloglog corruption 2019-03-18 11:31:33 +01:00
John Sully 3983dac12b Replicas aren't allowed to run the replicaof command 2019-03-18 11:28:42 +01:00
antirez befcbfbe7f Sentinel: add an option to deny online script reconfiguration. 
The ability of "SENTINEL SET" to change the reconfiguration script at
runtime is a problem even in the security model of Redis: any client
inside the network may set any executable to be ran once a failover is
triggered.

This option adds protection for this problem: by default the two
SENTINEL SET subcommands modifying scripts paths are denied. However the
user is still able to rever that using the Sentinel configuration file
in order to allow such a feature.
 2018-06-29 13:27:58 +02:00
antirez 590f537420 Redis 3.2.12. 2018-06-13 12:47:39 +02:00
antirez 37578f2ecf Security: fix redis-cli buffer overflow. 
Thanks to Fakhri Zulkifli for reporting it.

The fix switched to dynamic allocation, copying the final prompt in the
static buffer only at the end.
 2018-06-13 12:40:50 +02:00
antirez 299d5a4b2c Security: fix Lua struct package offset handling. 
After the first fix to the struct package I found another similar
problem, which is fixed by this patch. It could be reproduced easily by
running the following script:

    return struct.unpack('f', "xxxxxxxxxxxxx",-3)

The above will access bytes before the 'data' pointer.
 2018-06-13 12:40:50 +02:00
antirez 3dcf42697b Security: more cmsgpack fixes by @soloestoy. 
@soloestoy sent me this additional fixes, after searching for similar
problems to the one reported in mp_pack(). I'm committing the changes
because it was not possible during to make a public PR to protect Redis
users and give Redis providers some time to patch their systems.
 2018-06-13 12:40:50 +02:00
antirez cd13249b55 Security: update Lua struct package for security. 
During an auditing Apple found that the "struct" Lua package
we ship with Redis (http://www.inf.puc-rio.br/~roberto/struct/) contains
a security problem. A bound-checking statement fails because of integer
overflow. The bug exists since we initially integrated this package with
Lua, when scripting was introduced, so every version of Redis with
EVAL/EVALSHA capabilities exposed is affected.

Instead of just fixing the bug, the library was updated to the latest
version shipped by the author.
 2018-06-13 12:40:50 +02:00
antirez 75d66a7afd Security: fix Lua cmsgpack library stack overflow. 
During an auditing effort, the Apple Vulnerability Research team discovered
a critical Redis security issue affecting the Lua scripting part of Redis.

-- Description of the problem

Several years ago I merged a pull request including many small changes at
the Lua MsgPack library (that originally I authored myself). The Pull
Request entered Redis in commit 90b6337c1, in 2014.
Unfortunately one of the changes included a variadic Lua function that
lacked the check for the available Lua C stack. As a result, calling the
"pack" MsgPack library function with a large number of arguments, results
into pushing into the Lua C stack a number of new values proportional to
the number of arguments the function was called with. The pushed values,
moreover, are controlled by untrusted user input.

This in turn causes stack smashing which we believe to be exploitable,
while not very deterministic, but it is likely that an exploit could be
created targeting specific versions of Redis executables. However at its
minimum the issue results in a DoS, crashing the Redis server.

-- Versions affected

Versions greater or equal to Redis 2.8.18 are affected.

-- Reproducing

Reproduce with this (based on the original reproduction script by
Apple security team):

https://gist.github.com/antirez/82445fcbea6d9b19f97014cc6cc79f8a

-- Verification of the fix

The fix was tested in the following way:

1) I checked that the problem is no longer observable running the trigger.
2) The Lua code was analyzed to understand the stack semantics, and that
actually enough stack is allocated in all the cases of mp_pack() calls.
3) The mp_pack() function was modified in order to show exactly what items
in the stack were being set, to make sure that there is no silent overflow
even after the fix.

-- Credits

Thank you to the Apple team and to the other persons that helped me
checking the patch and coordinating this communication.
 2018-06-13 12:40:50 +02:00
赵磊 1565106673 Fix dictScan(): It can't scan all buckets when dict is shrinking. 2018-06-08 11:02:54 +02:00
antirez 2717be6392 Actually use ae_flags to add AE_BARRIER if needed. 
Many thanks to @Plasma that spotted this problem reviewing the code.
 2018-02-28 18:05:53 +01:00
antirez 9d797fe128 ae.c: insetad of not firing, on AE_BARRIER invert the sequence. 
AE_BARRIER was implemented like:

    - Fire the readable event.
    - Do not fire the writabel event if the readable fired.

However this may lead to the writable event to never be called if the
readable event is always fired. There is an alterantive, we can just
invert the sequence of the calls in case AE_BARRIER is set. This commit
does that.
 2018-02-27 16:19:58 +01:00
antirez 50571f5738 AOF: fix a bug that may prevent proper fsyncing when fsync=always. 
In case the write handler is already installed, it could happen that we
serve the reply of a query in the same event loop cycle we received it,
preventing beforeSleep() from guaranteeing that we do the AOF fsync
before sending the reply to the client.

The AE_BARRIER mechanism, introduced in a previous commit, prevents this
problem. This commit makes actual use of this new feature to fix the
bug.
 2018-02-27 16:19:56 +01:00
antirez 9176f4b945 Cluster: improve crash-recovery safety after failover auth vote. 
Add AE_BARRIER to the writable event loop so that slaves requesting
votes can't be served before we re-enter the event loop in the next
iteration, so clusterBeforeSleep() will fsync to disk in time.
Also add the call to explicitly fsync, given that we modified the last
vote epoch variable.
 2018-02-27 16:19:53 +01:00
antirez e6043981f5 ae.c: introduce the concept of read->write barrier. 
AOF fsync=always, and certain Redis Cluster bus operations, require to
fsync data on disk before replying with an acknowledge.
In such case, in order to implement Group Commits, we want to be sure
that queries that are read in a given cycle of the event loop, are never
served to clients in the same event loop iteration. This way, by using
the event loop "before sleep" callback, we can fsync the information
just one time before returning into the event loop for the next cycle.
This is much more efficient compared to calling fsync() multiple times.

Unfortunately because of a bug, this was not always guaranteed: the
actual way the events are installed was the sole thing that could
control. Normally this problem is hard to trigger when AOF is enabled
with fsync=always, because we try to flush the output buffers to the
socekt directly in the beforeSleep() function of Redis. However if the
output buffers are full, we actually install a write event, and in such
a case, this bug could happen.

This change to ae.c modifies the event loop implementation to make this
concept explicit. Write events that are registered with:

    AE_WRITABLE|AE_BARRIER

Are guaranteed to never fire after the readable event was fired for the
same file descriptor. In this way we are sure that data is persisted to
disk before the client performing the operation receives an
acknowledged.

However note that this semantics does not provide all the guarantees
that one may believe are automatically provided. Take the example of the
blocking list operations in Redis.

With AOF and fsync=always we could have:

    Client A doing: BLPOP myqueue 0
    Client B doing: RPUSH myqueue a b c

In this scenario, Client A will get the "a" elements immediately after
the Client B RPUSH will be executed, even before the operation is persisted.
However when Client B will get the acknowledge, it can be sure that
"b,c" are already safe on disk inside the list.

What to note here is that it cannot be assumed that Client A receiving
the element is a guaranteed that the operation succeeded from the point
of view of Client B.

This is due to the fact that the barrier exists within the same socket,
and not between different sockets. However in the case above, the
element "a" was not going to be persisted regardless, so it is a pretty
synthetic argument.
 2018-02-27 16:19:50 +01:00
zhaozhao.zz fd5f229ff0 rehash: handle one db until finished 2017-11-21 09:50:18 +01:00
kmiku7 bab437bef3 fix boundary case for _dictNextPower 2017-11-08 17:09:05 +01:00
antirez 0aa0cdf7cc Fix saving of zero-length lists. 
Normally in modern Redis you can't create zero-len lists, however it's
possible to load them from old RDB files generated, for instance, using
Redis 2.8 (see issue #4409). The "Right Thing" would be not loading such
lists at all, but this requires to hook in rdb.c random places in a not
great way, for a problem that is at this point, at best, minor.

Here in this commit instead I just fix the fact that zero length lists,
materialized as quicklists with the first node set to NULL, were
iterated in the wrong way while they are saved, leading to a crash.

The other parts of the list implementation are apparently able to deal
with empty lists correctly, even if they are no longer a thing.
 2017-11-06 12:37:30 +01:00
antirez 5727d7ec2f Fix buffer overflows occurring reading redis.conf. 
There was not enough sanity checking in the code loading the slots of
Redis Cluster from the nodes.conf file, this resulted into the
attacker's ability to write data at random addresses in the process
memory, by manipulating the index of the array. The bug seems
exploitable using the following techique: the config file may be altered so
that one of the nodes gets, as node ID (which is the first field inside the
structure) some data that is actually executable: then by writing this
address in selected places, this node ID part can be executed after a
jump. So it is mostly just a matter of effort in order to exploit the
bug. In practice however the issue is not very critical because the
bug requires an unprivileged user to be able to modify the Redis cluster
nodes configuration, and at the same time this should result in some
gain. However Redis normally is unprivileged as well. Yet much better to
have this fixed indeed.

Fix #4278.
 2017-10-31 09:52:59 +01:00
antirez 1cb6effdc8 Redis 3.2.11. 2017-09-21 16:20:18 +02:00
Oran Agra 13e8e53806 Flush append only buffers before existing. 
when SHUTDOWN command is recived it is possible that some of the recent
command were not yet flushed from the AOF buffer, and the server
experiences data loss at shutdown.
 2017-09-18 12:04:33 +02:00
antirez c9ab43ef7c Redis 3.2.10. 2017-07-28 17:59:00 +02:00
Jan-Erik Rediger bf50875353 Check that the whole first argument is a number 
Fixes #2258
 2017-07-24 15:21:44 +02:00
WuYunlong 8774228cc5 fix rewrite config: auto-aof-rewrite-min-size 2017-07-24 15:21:14 +02:00
Byron Grobe ab900303b8 Fixed issue #1996 (Missing '-' in help message for redis-benchmark) 2017-07-24 15:21:14 +02:00
Jan-Erik Rediger 7fcca9baef Don't use extended Regexp Syntax 
It's not POSIX (BSD systems have -E instead) and we don't actually need it.

Closes #1922
 2017-07-24 15:21:14 +02:00
Leon Chen 41963fe676 fix return wrong value of clusterDelNodeSlots 2017-07-24 14:18:56 +02:00
Leon Chen 97d50344c4 fix mismatch argument 2017-07-24 14:18:56 +02:00
liangsijian e39849808c Fix lua ldb command log 2017-07-24 14:11:36 +02:00
antirez 84a4f20278 Make representClusterNodeFlags() more robust. 
This function failed when an internal-only flag was set as an only flag
in a node: the string was trimmed expecting a final comma before
exiting the function, causing a crash. See issue #4142.
Moreover generation of flags representation only needed at DEBUG log
level was always performed: a waste of CPU time. This is fixed as well
by this commit.
 2017-07-24 14:09:50 +02:00
antirez 5aa2525006 Fix isHLLObjectOrReply() to handle integer encoded strings. 
Close #3766.
 2017-07-11 12:46:30 +02:00
antirez 7018d27dfe Fix abort typo in Lua debugger help screen. 2017-06-30 12:12:17 +02:00
antirez d557144ebb Added GEORADIUS(BYMEMBER)_RO variants for read-only operations. 
Issue #4084 shows how for a design error, GEORADIUS is a write command
because of the STORE option. Because of this it does not work
on readonly slaves, gets redirected to masters in Redis Cluster even
when the connection is in READONLY mode and so forth.

To break backward compatibility at this stage, with Redis 4.0 to be in
advanced RC state, is problematic for the user base. The API can be
fixed into the unstable branch soon if we'll decide to do so in order to
be more consistent, and reease Redis 5.0 with this incompatibility in
the future. This is still unclear.

However, the ability to scale GEO queries in slaves easily is too
important so this commit adds two read-only variants to the GEORADIUS
and GEORADIUSBYMEMBER command: GEORADIUS_RO and GEORADIUSBYMEMBER_RO.
The commands are exactly as the original commands, but they do not
accept the STORE and STOREDIST options.
 2017-06-30 11:53:45 +02:00
Suraj Narkhede a309388dff Fix following issues in blocking commands: 
1. brpop last key index, thus checking all keys for slots.
2. Memory leak in clusterRedirectBlockedClientIfNeeded.
3. Remove while loop in clusterRedirectBlockedClientIfNeeded.
 2017-06-27 18:06:22 +02:00
Suraj Narkhede 55442262e1 Fix brpop command table entry and redirect blocked clients. 2017-06-27 18:06:02 +02:00
antirez e3641c8dee Fix PERSIST expired key resuscitation issue #4048. 2017-06-27 18:05:37 +02:00
Antonio Mallia 9ce105add5 Removed duplicate 'sys/socket.h' include 2017-06-27 18:03:22 +02:00
Zachary Marquez e084a394a6 Prevent expirations and evictions while paused 
Proposed fix to https://github.com/antirez/redis/issues/4027
 2017-06-27 18:03:00 +02:00
antirez 61c78a5215 Collect fork() timing info only if fork succeeded. 2017-06-27 17:56:07 +02:00
antirez d067e3342f Aesthetic changes to #4068 PR to conform to Redis coding standard. 
1. Inline if ... statement if short.
2. No lines over 80 columns.
 2017-06-22 11:01:51 +02:00
xuzhou 2b0f03e4f2 Optimize set command with ex/px when updating aof. 2017-06-22 11:01:51 +02:00
antirez ad949f2329 redis-benchmark: add -t hset target. 2017-06-22 11:01:47 +02:00
xuzhou 351663bd7b Fix set with ex/px option when propagated to aof 2017-06-22 11:01:39 +02:00
minghang.zmh d70ac1d105 fix server.stat_net_output_bytes calc bug 2017-06-20 17:03:35 +02:00
xuchengxuan 8da9a167e0 Fixed comments of slowlog duration 2017-06-20 16:56:49 +02:00
cbgbt 13546adcb9 cli: Only print elapsed time on OUTPUT_STANDARD 2017-06-20 16:54:47 +02:00
Aric Huang 373facf81c (fix) Update create-cluster README 
Fix a few typos/adjust wording in `create-cluster` README
 2017-06-20 16:54:47 +02:00
Salvatore Sanfilippo 29e3ff9bec Merge pull request #3926 from QuChen88/3.2 
Implement getKeys procedure for georadius and georadiusbymember commands
 2017-06-14 09:12:17 -07:00
antirez 5116130d42 Redis 3.2.9. 2017-05-17 17:39:00 +02:00
antirez 3b46cf97de redis-cli --bigkeys: show error when TYPE fails. 
Close #3993.
 2017-05-15 11:23:57 +02:00
antirez f59b4b93d9 Fix preprocessor if/else chain broken in order to fix #3927. 2017-04-21 11:28:02 +02:00
antirez dd80fedf6f Fix zmalloc_get_memory_size() ifdefs to actually use the else branch. 
Close #3927.
 2017-04-21 11:27:28 +02:00
antirez 697d3abeff Set lua-time-limit default value at safe place. 
Otherwise, as it was, it will overwrite whatever the user set.

Close #3703.
 2017-04-21 11:26:27 +02:00
antirez c9c04b11b9 Fix #3848 by closing the descriptor on error. 2017-04-21 11:25:08 +02:00
张文康 d3b4992450 update block->free after some diff data are written to the child process 2017-04-20 07:59:46 +02:00
antirez 6a33952b8a Test: fix, hopefully, false PSYNC failure like in issue #2715. 
And many other related Github issues... all reporting the same problem.
There was probably just not enough backlog in certain unlucky runs.
I'll ask people that can reporduce if they see now this as fixed as
well.
 2017-04-14 17:54:06 +02:00
Qu Chen bacb04f89f Implement getKeys procedure for georadius and georadiusbymember 
commands.
 2017-04-08 18:37:02 +00:00
John.Koepi b83f9fea81 fix #2883, #2857 pipe fds leak when fork() failed on bg aof rw 2017-02-20 10:28:53 +01:00
antirez 10dbb5cd33 Don't leak file descriptor on syncWithMaster(). 
Close #3804.
 2017-02-20 10:28:52 +01:00
antirez db8a945cbb Redis 3.2.8. 2017-02-12 16:14:57 +01:00
antirez 7178cac031 Revert "Jemalloc updated to 4.4.0." 
This reverts commit 153f2f00ea.

Jemalloc 4.4.0 is apparently causing deadlocks in certain
systems. See for example https://github.com/antirez/redis/issues/3799.
As a cautionary step we are reverting the commit back and
releasing a new stable Redis version.
 2017-02-12 16:07:13 +01:00
antirez 33fad43c0f Fix MIGRATE closing of cached socket on error. 
After investigating issue #3796, it was discovered that MIGRATE
could call migrateCloseSocket() after the original MIGRATE c->argv
was already rewritten as a DEL operation. As a result the host/port
passed to migrateCloseSocket() could be anything, often a NULL pointer
that gets deferenced crashing the server.

Now the socket is closed at an earlier time when there is a socket
error in a later stage where no retry will be performed, before we
rewrite the argument vector. Moreover a check was added so that later,
in the socket_err label, there is no further attempt at closing the
socket if the argument was rewritten.

This fix should resolve the bug reported in #3796.
 2017-02-09 10:15:52 +01:00
antirez af12f8ec3c Redis 3.2.7. 2017-01-31 16:32:23 +01:00
antirez 3876d985a8 Ziplist: insertion bug under particular conditions fixed. 
Ziplists had a bug that was discovered while investigating a different
issue, resulting in a corrupted ziplist representation, and a likely
segmentation foult and/or data corruption of the last element of the
ziplist, once the ziplist is accessed again.

The bug happens when a specific set of insertions / deletions is
performed so that an entry is encoded to have a "prevlen" field (the
length of the previous entry) of 5 bytes but with a count that could be
encoded in a "prevlen" field of a since byte. This could happen when the
"cascading update" process called by ziplistInsert()/ziplistDelete() in
certain contitious forces the prevlen to be bigger than necessary in
order to avoid too much data moving around.

Once such an entry is generated, inserting a very small entry
immediately before it will result in a resizing of the ziplist for a
count smaller than the current ziplist length (which is a violation,
inserting code expects the ziplist to get bigger actually). So an FF
byte is inserted in a misplaced position. Moreover a realloc() is
performed with a count smaller than the ziplist current length so the
final bytes could be trashed as well.

SECURITY IMPLICATIONS:

Currently it looks like an attacker can only crash a Redis server by
providing specifically choosen commands. However a FF byte is written
and there are other memory operations that depend on a wrong count, so
even if it is not immediately apparent how to mount an attack in order
to execute code remotely, it is not impossible at all that this could be
done. Attacks always get better... and we did not spent enough time in
order to think how to exploit this issue, but security researchers
or malicious attackers could.

REPRODUCING:

The bug can be reproduced with the following commands.

    redis-cli del list
    redis-cli rpush list one
    redis-cli rpush list two
    redis-cli rpush list
    AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
    redis-cli rpush list
    AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
    redis-cli rpush list three
    redis-cli rpush list a
    redis-cli lrem list 1
    AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
    redis-cli linsert list after
    AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
    10
    redis-cli lrange list 0 -1

Instead of "rpush list a", use "rpush list 10" in order to trigger a
data corruption instead of a crash.
 2017-01-30 16:34:13 +01:00
antirez 153f2f00ea Jemalloc updated to 4.4.0. 
The original jemalloc source tree was modified to:

1. Remove the configure error that prevents nested builds.
2. Insert the Redis private Jemalloc API in order to allow the
Redis fragmentation function to work.
 2017-01-30 10:11:36 +01:00
miter ca532c94ef Change switch statment to if statment 2017-01-30 10:10:30 +01:00
oranagra a73503530e fix rare assertion in DEBUG DIGEST 
getExpire calls dictFind which can do rehashing.
found by calling computeDatasetDigest from serverCron and running the test suite.
 2017-01-30 10:10:21 +01:00
Itamar Haber b917e3fdb3 Verify pairs are provided after subcommands 
Fixes https://github.com/antirez/redis/issues/3639
 2017-01-30 10:10:14 +01:00
antirez 1177cf6395 Avoid geo.c warning in initialization. 
This problem was properly solved in 4.0 / unstable. Here is just a quick
fix for the warning.
 2017-01-30 09:16:05 +01:00
antirez 874804da0c Security: Cross Protocol Scripting protection. 
This is an attempt at mitigating problems due to cross protocol
scripting, an attack targeting services using line oriented protocols
like Redis that can accept HTTP requests as valid protocol, by
discarding the invalid parts and accepting the payloads sent, for
example, via a POST request.

For this to be effective, when we detect POST and Host: and terminate
the connection asynchronously, the networking code was modified in order
to never process further input. It was later verified that in a
pipelined request containing a POST command, the successive commands are
not executed.
 2017-01-30 09:08:58 +01:00
antirez 273cd7ff51 Ziplist: remove static from functions, they prevent good crash reports. 2017-01-27 10:49:53 +01:00
Jan-Erik Rediger 389b9f5945 Initialize help only in repl mode 2017-01-27 10:49:47 +01:00
Yossi Gottlieb 1370a88887 Fix redis-cli rare crash. 
This happens if the server (mysteriously) returns an unexpected response
to the COMMAND command.
 2016-12-21 11:03:44 +01:00
antirez 68aab8e811 MIGRATE: Remove upfront ttl initialization. 
After the fix for #3673 the ttl var is always initialized inside the
loop itself, so the early initialization is not needed.

Variables declaration also moved to a more local scope.
 2016-12-14 12:44:32 +01:00
Jan-Erik Rediger 788e892534 Reset the ttl for additional keys 
Before, if a previous key had a TTL set but the current one didn't, the
TTL was reused and thus resulted in wrong expirations set.

This behaviour was experienced, when `MigrateDefaultPipeline` in
redis-trib was set to >1

Fixes #3655
 2016-12-14 12:41:21 +01:00
antirez 77de5c2b71 Redis 3.2.6. 2016-12-06 09:38:33 +01:00
antirez f20f3ead1b Geo: improve fuzz test. 
The test now uses more diverse radius sizes, especially sizes near or
greater the whole earth surface are used, that are known to trigger edge
cases. Moreover the PRNG seeding was probably resulting into the same
sequence tested over and over again, now seeding unsing the current unix
time in milliseconds.

Related to #3631.
 2016-12-05 14:19:03 +01:00
antirez 8c22086c34 Geo: fix computation of bounding box. 
A bug was reported in the context in issue #3631. The root cause of the
bug was that certain neighbor boxes were zeroed after the "inside the
bounding box or not" check, simply because the bounding box computation
function was wrong.

A few debugging infos where enhanced and moved in other parts of the
code. A check to avoid steps=0 was added, but is unrelated to this
issue and I did not verified it was an actual bug in practice.
 2016-12-05 14:18:59 +01:00
antirez 92958df3b1 Cluster: handle zero bytes at the end of nodes.conf. 2016-11-16 14:13:13 +01:00
antirez 0ee28ae40c Test: regression test for #3564 added. 2016-10-31 15:47:09 +01:00
sunhe 289fadb2d0 bitops.c/bitfieldCommand: update higest_write_offset with check 2016-10-31 15:41:40 +01:00
jybaek f4f7d2c37f Add missing fclose() 2016-10-31 15:36:54 +01:00
antirez f61d6f0c4a Redis 3.2.5. 2016-10-26 09:17:45 +02:00
zach shipko 473640790f BSDs don't have -ldl 2016-10-26 09:11:04 +02:00
antirez 9ada818b86 Fix modules compilation when libc malloc is used. 
Compiling Redis worked as a side effect of jemalloc target specifying
-ldl as needed linker options, otherwise it is not provided during
linking and dlopen() API will remain unresolved symbols.
 2016-10-26 09:11:01 +02:00
antirez 381651fac0 3.2.4 release notes clarifications. 2016-09-26 09:18:59 +02:00
antirez 070d047179 Redis 3.2.4. 2016-09-26 09:10:17 +02:00
antirez 0539634777 Security: CONFIG SET client-output-buffer-limit overflow fixed. 
This commit fixes a vunlerability reported by Cory Duplantis
of Cisco Talos, see TALOS-2016-0206 for reference.

CONFIG SET client-output-buffer-limit accepts as client class "master"
which is actually only used to implement CLIENT KILL. The "master" class
has ID 3. What happens is that the global structure:

    server.client_obuf_limits[class]

Is accessed with class = 3. However it is a 3 elements array, so writing
the 4th element means to write up to 24 bytes of memory *after* the end
of the array, since the structure is defined as:

    typedef struct clientBufferLimitsConfig {
        unsigned long long hard_limit_bytes;
        unsigned long long soft_limit_bytes;
        time_t soft_limit_seconds;
    } clientBufferLimitsConfig;

EVALUATION OF IMPACT:

Checking what's past the boundaries of the array in the global
'server' structure, we find AOF state fields:

    clientBufferLimitsConfig client_obuf_limits[CLIENT_TYPE_OBUF_COUNT];
    /* AOF persistence */
    int aof_state;                  /* AOF_(ON|OFF|WAIT_REWRITE) */
    int aof_fsync;                  /* Kind of fsync() policy */
    char *aof_filename;             /* Name of the AOF file */
    int aof_no_fsync_on_rewrite;    /* Don't fsync if a rewrite is in prog. */
    int aof_rewrite_perc;           /* Rewrite AOF if % growth is > M and... */
    off_t aof_rewrite_min_size;     /* the AOF file is at least N bytes. */
    off_t aof_rewrite_base_size;    /* AOF size on latest startup or rewrite. */
    off_t aof_current_size;         /* AOF current size. */

Writing to most of these fields should be harmless and only cause problems in
Redis persistence that should not escalate to security problems.
However unfortunately writing to "aof_filename" could be potentially a
security issue depending on the access pattern.

Searching for "aof.filename" accesses in the source code returns many different
usages of the field, including using it as input for open(), logging to the
Redis log file or syslog, and calling the rename() syscall.

It looks possible that attacks could lead at least to informations
disclosure of the state and data inside Redis. However note that the
attacker must already have access to the server. But, worse than that,
it looks possible that being able to change the AOF filename can be used
to mount more powerful attacks: like overwriting random files with AOF
data (easily a potential security issue as demostrated here:
http://antirez.com/news/96), or even more subtle attacks where the
AOF filename is changed to a path were a malicious AOF file is loaded
in order to exploit other potential issues when the AOF parser is fed
with untrusted input (no known issue known currently).

The fix checks the places where the 'master' class is specifiedf in
order to access configuration data structures, and return an error in
this cases.

WHO IS AT RISK?

The "master" client class was introduced in Redis in Jul 28 2015.
Every Redis instance released past this date is not vulnerable
while all the releases after this date are. Notably:

    Redis 3.0.x is NOT vunlerable.
    Redis 3.2.x IS vulnerable.
    Redis unstable is vulnerable.

In order for the instance to be at risk, at least one of the following
conditions must be true:

    1. The attacker can access Redis remotely and is able to send
       the CONFIG SET command (often banned in managed Redis instances).

    2. The attacker is able to control the "redis.conf" file and
       can wait or trigger a server restart.

The problem was fixed 26th September 2016 in all the releases affected.
 2016-09-26 08:56:37 +02:00
antirez c01abcdebf fix the fix for the TCP binding. 
This commit attempts to fix a problem with PR #3467.
 2016-09-12 10:56:25 +02:00
oranagra a6d0698b22 fix tcp binding when IPv6 is unsupported 2016-09-12 10:56:21 +02:00
antirez 22b6c28d25 debug.c: no need to define _GNU_SOURCE, is defined in fmacros.h. 2016-09-09 11:13:29 +02:00
antirez 9e9d398beb crash log - improve code dump with more info and called symbols. 2016-09-09 11:13:24 +02:00
oranagra 3745c5dc81 crash log - add hex dump of function code 2016-09-09 11:13:21 +02:00
antirez c1cc07b25d Sentinel example config: warn about protected mode. 2016-08-11 19:53:15 +02:00
rojingeorge 011dc9f9a5 Display the nodes summary once the cluster is established using redis-trib.rb 
Display the nodes summary once the cluster is established using
redis-trib.rb
After the cluster meet and join was done, when the summary was shown, it
was giving info regarding the nodes. This fix ensures that confusion
where the slaves were shown as masters.
Fix would be to reset the nodes and reload the cluster information
before checking the cluster status after creating it.
 2016-08-05 15:11:21 +02:00
Guo Xiao f4e3a940c4 Use the standard predefined identifier __func__ (since C99) 
Fix warning: ISO C does not support '__FUNCTION__' predefined identifier
[-Wpedantic]
 2016-08-04 10:26:23 +02:00
antirez 44bf2533cd Redis 3.2.3. 2016-08-02 11:00:29 +02:00
Salvatore Sanfilippo 2444c82a3f Merge pull request #3426 from QuChen88/3.2 
Fix a bug to delay bgsave while AOF rewrite in progress for replication
 2016-08-02 10:43:05 +02:00
Qu Chen e67ad1d1eb Fix a bug to delay bgsave while AOF rewrite in progress for replication 2016-08-01 12:16:17 -07:00
antirez 71536684a7 Update linenoise to fix insecure redis-cli history file creation. 
The problem was fixed in antirez/linenoise repository applying a patch
contributed by @lamby. Here the new version is updated in the Redis
source tree.

Close #1418
Close #3322
 2016-07-29 11:29:47 +02:00
antirez 7c6e288d76 Redis 3.2.2. 2016-07-28 14:53:24 +02:00
antirez 0a45fbc302 Ability of slave to announce arbitrary ip/port to master. 
This feature is useful, especially in deployments using Sentinel in
order to setup Redis HA, where the slave is executed with NAT or port
forwarding, so that the auto-detected port/ip addresses, as listed in
the "INFO replication" output of the master, or as provided by the
"ROLE" command, don't match the real addresses at which the slave is
reachable for connections.
 2016-07-28 13:05:19 +02:00
antirez c3982c0905 redis-benchmark: new option to show server errors on stdout. 
Disabled by default, can be activated with -e. Maybe the reverse was
more safe but departs from the past behavior.
 2016-07-28 13:04:42 +02:00
antirez fdafe23315 Multiple GEORADIUS bugs fixed. 
By grepping the continuous integration errors log a number of GEORADIUS
tests failures were detected.

Fortunately when a GEORADIUS failure happens, the test suite logs enough
information in order to reproduce the problem: the PRNG seed,
coordinates and radius of the query.

By reproducing the issues, three different bugs were discovered and
fixed in this commit. This commit also improves the already good
reporting of the fuzzer and adds the failure vectors as regression
tests.

The issues found:

1. We need larger squares around the poles in order to cover the area
requested by the user. There were already checks in order to use a
smaller step (larger squares) but the limit set (+/- 67 degrees) is not
enough in certain edge cases, so 66 is used now.

2. Even near the equator, when the search area center is very near the
edge of the square, the north, south, west or ovest square may not be
able to fully cover the specified radius. Now a test is performed at the
edge of the initial guessed search area, and larger squares are used in
case the test fails.

3. Because of rounding errors between Redis and Tcl, sometimes the test
signaled false positives. This is now addressed.

Whenever possible the original code was improved a bit in other ways. A
debugging example stanza was added in order to make the next debugging
session simpler when the next bug is found.
 2016-07-27 12:11:31 +02:00
antirez a1bfe22a80 Replication: when possible start RDB saving ASAP. 
In a previous commit the replication code was changed in order to
centralize the BGSAVE for replication trigger in replicationCron(),
however after further testings, the 1 second delay imposed by this
change is not acceptable.

So now the BGSAVE is only delayed if the AOF rewriting process is
active. However past comments made sure that replicationCron() is always
able to trigger the BGSAVE when needed, making the code generally more
robust.

The new code is more similar to the initial @oranagra patch where the
BGSAVE was delayed only if an AOF rewrite was in progress.

Trivia: delaying the BGSAVE uncovered a minor Sentinel issue that is now
fixed.
 2016-07-27 12:08:32 +02:00
antirez 7ca69aff26 Sentinel: new test unit 07 that tests master down conditions. 2016-07-27 12:08:25 +02:00
antirez 5b5e65203f Sentinel: check Slave INFO state more often when disconnected. 
During the initial handshake with the master a slave will report to have
a very high disconnection time from its master (since technically it was
disconnected since forever, so the current UNIX time in seconds is
reported).

However when the slave is connected again the Sentinel may re-scan the
INFO output again only after 10 seconds, which is a long time. During
this time Sentinels will consider this instance unable to failover, so
a useless delay is introduced.

Actaully this hardly happened in the practice because when a slave's
master is down, the INFO period for slaves changes to 1 second. However
when a manual failover is attempted immediately after adding slaves
(like in the case of the Sentinel unit test), this problem may happen.

This commit changes the INFO period to 1 second even in the case the
slave's master is not down, but the slave reported to be disconnected
from the master (by publishing, last time we checked, a master
disconnection time field in INFO).

This change is required as a result of an unrelated change in the
replication code that adds a small delay in the master-slave first
synchronization.
 2016-07-27 12:08:17 +02:00
antirez 21cffc2681 Avoid simultaneous RDB and AOF child process. 
This patch, written in collaboration with Oran Agra (@oranagra) is a companion
to 780a8b1. Together the two patches should avoid that the AOF and RDB saving
processes can be spawned at the same time. Previously conditions that
could lead to two saving processes at the same time were:

1. When AOF is enabled via CONFIG SET and an RDB saving process is
   already active.

2. When the SYNC command decides to start an RDB saving process ASAP in
   order to serve a new slave that cannot partially resynchronize (but
   only if we have a disk target for replication, for diskless
   replication there is not such a problem).

Condition "1" is not very severe but "2" can happen often and is
definitely good at degrading Redis performances in an unexpected way.

The two commits have the effect of always spawning RDB savings for
replication in replicationCron() instead of attempting to start an RDB
save synchronously. Moreover when a BGSAVE or AOF rewrite must be
performed, they are instead just postponed using flags that will try to
perform such operations ASAP.

Finally the BGSAVE command was modified in order to accept a SCHEDULE
option so that if an AOF rewrite is in progress, when this option is
given, the command no longer returns an error, but instead schedules an
RDB rewrite operation for when it will be possible to start it.
 2016-07-27 12:08:12 +02:00
antirez 017378eca9 Replication: start BGSAVE for replication always in replicationCron(). 
This makes the replication code conceptually simpler by removing the
synchronous BGSAVE trigger in syncCommand(). This also means that
socket and disk BGSAVE targets are handled by the same code.
 2016-07-27 12:08:08 +02:00
antirez 940be9ab54 Regression test for issue #3333. 2016-07-06 11:50:13 +02:00
antirez 21736b41a2 getLongLongFromObject: use string2ll() instead of strict_strtoll(). 
strict_strtoll() has a bug that reports the empty string as ok and
parses it as zero.

Apparently nobody ever replaced this old call with the faster/saner
string2ll() which is used otherwise in the rest of the Redis core.

This commit close #3333.
 2016-07-06 11:47:07 +02:00
antirez 0b748e9139 redis-cli: check SELECT reply type just in state updated. 
In issues #3361 / #3365 a problem was reported / fixed with redis-cli
not updating correctly the current DB on error after SELECT.

In theory this bug was fixed in 0042fb0e, but actually the commit only
fixed the prompt updating, not the fact the state was set in a wrong
way.

This commit removes the check in the prompt update, now that hopefully
it is the state that is correct, there is no longer need for this check.
 2016-07-05 17:40:32 +02:00
sskorgal 1158386bb8 Fix for redis_cli printing default DB when select command fails. 2016-07-05 17:40:32 +02:00
antirez 026f9fc7b0 Sentinel: fix cross-master Sentinel address update. 
This commit both fixes the crash reported with issue #3364 and
also properly closes the old links after the Sentinel address for the
other masters gets updated.

The two problems where:

1. The Sentinel that switched address may not monitor all the masters,
   it is possible that there is no match, and the 'match' variable is
   NULL. Now we check for no match and 'continue' to the next master.

2. By ispecting the code because of issue "1" I noticed that there was a
   problem in the code that disconnects the link of the Sentinel that
   needs the address update. Basically link->disconnected is non-zero
   even if just *a single link* (cc -- command link or pc -- pubsub
   link) are disconnected, so to check with if (link->disconnected)
   in order to close the links risks to leave one link connected.

I was able to manually reproduce the crash at "1" and verify that the
commit resolves the issue.

Close #3364.
 2016-07-04 18:50:40 +02:00
antirez 11523b3e0e CONFIG GET is now no longer case sensitive. 
Like CONFIG SET always was. Close #3369.
 2016-07-04 16:09:07 +02:00
antirez f5a7f4f2d9 Fix test for new RDB checksum failure message. 2016-07-04 12:41:25 +02:00
antirez 4c6ff74c07 Make tcp-keepalive default to 300 in internal conf. 
We already changed the default in the redis.conf template, but I forgot
to change the internal config as well.
 2016-07-04 12:33:29 +02:00
antirez 27dbec2a36 In Redis RDB check: more details in error reportings. 2016-07-04 12:33:28 +02:00
antirez 41f300473a In Redis RDB check: log decompression errors. 2016-07-04 12:24:15 +02:00
antirez 278fe3e965 In Redis RDB check: log object type on error. 2016-07-04 12:24:08 +02:00
antirez a117dfa807 Added a trivial program to randomly corrupt RDB files in /utils. 2016-07-04 12:24:05 +02:00
antirez f5110c3c7c In Redis RDB check: minor output message changes. 2016-07-04 12:24:02 +02:00
antirez 35b18bfba3 In Redis RDB check: better error reporting. 2016-07-04 12:23:59 +02:00
antirez f578f08544 In Redis RDB check: initial POC. 
So far we used an external program (later executed within Redis) and
parser in order to check RDB files for correctness. This forces, at each
RDB format update, to have two copies of the same format implementation
that are hard to keep in sync. Morover the former RDB checker only
checked the very high-level format of the file, without actually trying
to load things in memory. Certain corruptions can only be handled by
really loading key-value pairs.

This first commit attempts to unify the Redis RDB loadig code with the
task of checking the RDB file for correctness. More work is needed but
it looks like a sounding direction so far.
 2016-07-04 12:23:47 +02:00
tielei 7f1e1caee7 A string with 21 chars is not representable as a 64-bit integer. 2016-07-04 12:10:22 +02:00
antirez 7a3a595fb4 Test: new randomized stress tester for #3343 alike bugs. 2016-06-30 16:50:12 +02:00
antirez c75ca104f4 Stress tester WIP. 2016-06-30 16:50:10 +02:00
antirez 2c3fcf87cc Regression test for issue #3343 exact min crash sequence. 
Note: it was verified that it can crash the test suite without the patch
applied.
 2016-06-30 16:50:06 +02:00
antirez 704196790e Fix quicklistReplaceAtIndex() by updating the quicklist ziplist size. 
The quicklist takes a cached version of the ziplist representation size
in bytes. The implementation must update this length every time the
underlying ziplist changes. However quicklistReplaceAtIndex() failed to
fix the length.

During LSET calls, the size of the ziplist blob and the cached size
inside the quicklist diverged. Later, when this size is used in an
authoritative way, for example during nodes splitting in order to copy
the nodes, we end with a duplicated node that may contain random
garbage.

This commit should fix issue #3343, however several problems were found
reviewing the quicklist.c code in search of this bug that should be
addressed soon or later.

For example:

1. To take a cached ziplist length is fragile since failing to update it
leads to this kind of issues.

2. The node splitting code needs auditing. For example it works just for
a side effect of ziplistDeleteRange() to be able to cope with a wrong
count of elements to remove. The code inside quicklist.c assumes that
-1 means "delete till the end" while actually it's just a count of how
many elements to delete, and is an unsigned count. So -1 gets converted
into the maximum integer, and just by chance the ziplist code stops
deleting elements after there are no more to delete.

3. Node splitting is extremely inefficient, it copies the node and
removes elements from both nodes even when actually there is to move a
single entry from one node to the other, or when the new resulting node
is empty at all so there is nothing to copy but just to create a new
node.

However at least for Redis 3.2 to introduce fresh code inside
quicklist.c may be even more risky, so instead I'm writing a better
fuzzy tester to stress the internals a bit more in order to anticipate
other possible bugs.

This bug was found using a fuzzy tester written after having some clue
about where the bug could be. The tester eventually created a ~2000
commands sequence able to always crash Redis. I wrote a better version
of the tester that searched for the smallest sequence that could crash
Redis automatically. Later this smaller sequence was minimized by
removing random commands till it still crashed the server. This resulted
into a sequence of 7 commands. With this small sequence it was just a
matter of filling the code with enough printf() to understand enough
state to fix the bug.
 2016-06-27 18:12:12 +02:00
antirez 04c7261f03 Redis 3.2.1. 2016-06-17 15:15:21 +02:00
oranagra 8207e82804 config set list-max-ziplist-size didn't support negative values, unlike config file 2016-06-17 14:49:37 +02:00
antirez 6ad0371c9b Fix Sentinel pending commands counting. 
This bug most experienced effect was an inability of Redis to
reconfigure back old masters to slaves after they are reachable again
after a failover. This was due to failing to reset the count of the
pending commands properly, so the master appeared fovever down.

Was introduced in Redis 3.2 new Sentinel connection sharing feature
which is a lot more complex than the 3.0 code, but more scalable.

Many thanks to people reporting the issue, and especially to
@sskorgal for investigating the issue in depth.

Hopefully closes #3285.
 2016-06-16 19:24:34 +02:00
antirez 58f1d446c3 redis-cli: really connect to the right server. 
I recently introduced populating the autocomplete help array with the
COMMAND command if available. However this was performed before parsing
the arguments, defaulting to instance 6379. After the connection is
performed it remains stable.

The effect is that if there is an instance running on port 6339,
whatever port you specify is ignored and 6379 is connected to instead.
The right port will be selected only after a reconnection.

Close #3314.
 2016-06-16 17:25:13 +02:00
Jan-Erik Rediger b6007b324b Remove debug printing 2016-06-16 17:18:06 +02:00
antirez f592b4d317 RESTORE: accept RDB dumps with older versions. 
Reference issue #3218.

Checking the code I can't find a reason why the original RESTORE
code was so opinionated about restoring only the current version. The
code in to `rdb.c` appears to be capable as always to restore data from
older versions of Redis, and the only places where it is needed the
current version in order to correctly restore data, is while loading the
opcodes, not the values itself as it happens in the case of RESTORE.

For the above reasons, this commit enables RESTORE to accept older
versions of values payloads.
 2016-06-16 15:56:29 +02:00
oranagra 047ced4473 CLIENT error message was out of date 2016-06-16 12:59:26 +02:00
oranagra 14e04847ac fix georadius returns multiple replies 2016-06-16 12:58:18 +02:00
antirez bd23ea3f9f Minor aesthetic fixes to PR #3264. 
Comment format fixed + local var modified from camel case to underscore
separators as Redis code base normally does (camel case is mostly used
for global symbols like structure names, function names, global vars,
...).
 2016-06-16 12:56:28 +02:00
oranagra 2a3ee58ec7 check WRONGTYPE in BITFIELD before looping on the operations. 
optimization: lookup key only once, and grow at once to the max need
fixes #3259 and #3221, and also an early return if wrongtype is discovered by SET
 2016-06-16 12:56:17 +02:00
oranagra a2e27b810e fix crash in BITFIELD GET on non existing key or wrong type see #3259 
this was a bug in the recent refactoring: bee963c445
 2016-06-16 12:56:17 +02:00
MOON_CLJ 26555f5e00 fix check when can't send the command to the promoted slave 2016-06-15 17:24:43 +02:00
antirez f1c237cb6a Test TOUCH and new TTL / TYPE behavior about object access time. 2016-06-15 17:16:13 +02:00
antirez d4831e3287 GETRANGE: return empty string with negative, inverted start/end. 2016-06-15 16:05:54 +02:00
antirez 9942070f5a Remove additional round brackets from fix for #3282. 2016-06-15 16:05:39 +02:00
wenduo f45fa5d05f bitcount bug:return non-zero value when start > end (both negative) 2016-06-15 16:05:33 +02:00
antirez 0cb86064e6 Regression test for #3282. 2016-06-15 16:04:44 +02:00
antirez b23aa6706a TTL and TYPE LRU access fixed. TOUCH implemented. 2016-06-15 09:18:55 +02:00
antirez 6e4204fec9 redis-cli help.h updated. 2016-06-14 14:45:48 +02:00
antirez bb43f4cab2 Fix GEORADIUS wrong output with radius > Earth radius. 
Close #3266
 2016-06-13 12:11:15 +02:00
antirez 16102bc0af Geo: fix typo in geohashEstimateStepsByRadius(). 
I'm the author of this line but I can't see a good reason for it to
don't be a typo, a step of 26 should be valid with 52 bits per
coordinate, moreover the line was:

    if (step > 26) step = 25;

So a step of 26 was actually already used, except when one of 27 was
computed (which is invalid) only then it was trimmed to 25 instead of
26.

All tests passing after the change.
 2016-06-13 12:11:02 +02:00
antirez 014bf80442 Avoid undefined behavior in BITFIELD implementation. 
Probably there is no compiler that will actaully break the code or raise
a signal for unsigned -> signed overflowing conversion, still it was
apparently possible to write it in a more correct way.

All tests passing.
 2016-06-13 12:10:58 +02:00
antirez b4e5e2ec1a Now that SPOP can be called by scripts use BLPOP on 's' flag test. 2016-06-13 12:10:52 +02:00
antirez df419281a1 Test: run GEO tests by default. 
Thanks to @oranagra for noticing it was missing.
 2016-06-13 12:10:47 +02:00
antirez 40cfe13141 Enable tcp-keepalive by default. 2016-06-13 12:03:19 +02:00
Pierre Chapuis a650aaaf4f fix some compiler warnings 2016-06-10 10:23:17 +02:00
antirez 3fd4baf1e7 Fixed typo in Sentinel compareSlavesForPromotion() comment. 2016-06-10 09:15:37 +02:00
andyli 8d029a5950 fix comment "b>a" to "a > b" 2016-06-10 09:15:21 +02:00
bogdanvlviv 6937f5960d fix pidfile in redis.conf 2016-06-10 09:02:22 +02:00
jspraul b5758cc5ce Include 'fd_set' type name 
Fix an MSYS2-build-breaking error: unknown type name ‘fd_set’
 2016-06-10 09:00:10 +02:00
Itamar Haber 620783e3b5 Allow SPOP from Lua scripts 
The existing `R` flag appears to be sufficient and there's no apparent reason why the command should be blocked.
 2016-05-30 17:48:18 +02:00
oranagra 603234076f minor fixes - mainly signalModifiedKey, and GEORADIUS 2016-05-18 16:52:26 +02:00
antirez 2e6b99499a Code to access object string bytes repeated 3x refactored into 1 function. 2016-05-18 15:37:28 +02:00
oranagra dcaeafc828 fix crash in BITFIELD GET when key is integer encoded 2016-05-18 15:37:25 +02:00
antirez 4ad088818a Clarify that the LOG_STR_SIZE includes null term. 2016-05-18 15:36:44 +02:00
antirez f991cdc3e1 Test for BITFIELD regression #3221. 2016-05-18 15:36:38 +02:00
oranagra cb3e89e2cb reduce struct padding by reordering members 2016-05-18 12:12:20 +02:00
antirez a7f0fb448c Actually use --with-lg-quantum=3 to build jemalloc. 
This change is documented in deps/README.md but was lost in one way or
the other, neutralizing the benefits of 24 bytes size classes (and
others).

Close #3208.
 2016-05-18 12:12:15 +02:00
antirez 029dc0d97f redis-cli: integrate help.h with COMMAND output. 
Use the COMMAND output to fill with partial information the built-in
help. This makes redis-cli able to at least complete commands that are
exported by the Redis server it is connected to, but were not available
in the help.h file when the redis-cli binary was compiled.
 2016-05-08 19:00:07 +02:00
Adam Baldwin 95def3aee0 Removed dofile() from Lua 2016-05-08 18:05:05 +02:00
63 changed files with 3129 additions and 1073 deletions
Expand all files Collapse all files
00-RELEASENOTES
+1223
View File
File diff suppressed because it is too large Load Diff
deps/Makefile
Vendored
+1 -1
View File
@@ -78,7 +78,7 @@ JEMALLOC_LDFLAGS= $(LDFLAGS)
jemalloc: .make-prerequisites
@printf '%b %b\n' $(MAKECOLOR)MAKE$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR)
cd jemalloc && ./configure --with-jemalloc-prefix=je_ --enable-cc-silence CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)"
cd jemalloc && ./configure --with-lg-quantum=3 --with-jemalloc-prefix=je_ --enable-cc-silence CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)"
cd jemalloc && $(MAKE) CFLAGS="$(JEMALLOC_CFLAGS)" LDFLAGS="$(JEMALLOC_LDFLAGS)" lib/libjemalloc.a
.PHONY: jemalloc
deps/geohash-int/geohash.c
Vendored
+2 -2
View File
@@ -1,7 +1,7 @@
/*
* Copyright (c) 2013-2014, yinqiwen <yinqiwen@gmail.com>
* Copyright (c) 2014, Matt Stancliff <matt@genges.com>.
* Copyright (c) 2015, Salvatore Sanfilippo <antirez@gmail.com>.
* Copyright (c) 2015-2016, Salvatore Sanfilippo <antirez@gmail.com>.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -118,7 +118,7 @@ void geohashGetCoordRange(GeoHashRange *long_range, GeoHashRange *lat_range) {
lat_range->min = GEO_LAT_MIN;
}
int geohashEncode(GeoHashRange *long_range, GeoHashRange *lat_range,
int geohashEncode(const GeoHashRange *long_range, const GeoHashRange *lat_range,
double longitude, double latitude, uint8_t step,
GeoHashBits *hash) {
/* Check basic arguments sanity. */
deps/geohash-int/geohash.h
Vendored
+1 -1
View File
@@ -95,7 +95,7 @@ typedef struct {
* -1:failed
*/
void geohashGetCoordRange(GeoHashRange *long_range, GeoHashRange *lat_range);
int geohashEncode(GeoHashRange *long_range, GeoHashRange *lat_range,
int geohashEncode(const GeoHashRange *long_range, const GeoHashRange *lat_range,
double longitude, double latitude, uint8_t step,
GeoHashBits *hash);
int geohashEncodeType(double longitude, double latitude,
deps/geohash-int/geohash_helper.c
Vendored
+61 -39
View File
@@ -1,7 +1,7 @@
/*
* Copyright (c) 2013-2014, yinqiwen <yinqiwen@gmail.com>
* Copyright (c) 2014, Matt Stancliff <matt@genges.com>.
* Copyright (c) 2015, Salvatore Sanfilippo <antirez@gmail.com>.
* Copyright (c) 2015-2016, Salvatore Sanfilippo <antirez@gmail.com>.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -35,6 +35,7 @@
*/
#include "geohash_helper.h"
#include <math.h>
#define D_R (M_PI / 180.0)
#define R_MAJOR 6378137.0
@@ -54,8 +55,8 @@ const double MERCATOR_MIN = -20037726.37;
static inline double deg_rad(double ang) { return ang * D_R; }
static inline double rad_deg(double ang) { return ang / D_R; }
/* You must *ONLY* estimate steps when you are encoding.
* If you are decoding, always decode to GEO_STEP_MAX (26). */
/* This function is used in order to estimate the step (bits precision)
* of the 9 search area boxes during radius queries. */
uint8_t geohashEstimateStepsByRadius(double range_meters, double lat) {
if (range_meters == 0) return 26;
int step = 1;
@@ -63,55 +64,45 @@ uint8_t geohashEstimateStepsByRadius(double range_meters, double lat) {
range_meters *= 2;
step++;
}
step -= 2; /* Make sure range is included in the worst case. */
step -= 2; /* Make sure range is included in most of the base cases. */
/* Wider range torwards the poles... Note: it is possible to do better
* than this approximation by computing the distance between meridians
* at this latitude, but this does the trick for now. */
if (lat > 67 || lat < -67) step--;
if (lat > 80 || lat < -80) step--;
if (lat > 66 || lat < -66) {
step--;
if (lat > 80 || lat < -80) step--;
}
/* Frame to valid range. */
if (step < 1) step = 1;
if (step > 26) step = 25;
if (step > 26) step = 26;
return step;
}
int geohashBitsComparator(const GeoHashBits *a, const GeoHashBits *b) {
/* If step not equal, compare on step. Else, compare on bits. */
return a->step != b->step ? a->step - b->step : a->bits - b->bits;
}
/* Return the bounding box of the search area centered at latitude,longitude
* having a radius of radius_meter. bounds[0] - bounds[2] is the minimum
* and maxium longitude, while bounds[1] - bounds[3] is the minimum and
* maximum latitude. */
int geohashBoundingBox(double longitude, double latitude, double radius_meters,
double *bounds) {
if (!bounds) return 0;
double lonr, latr;
lonr = deg_rad(longitude);
latr = deg_rad(latitude);
double distance = radius_meters / EARTH_RADIUS_IN_METERS;
double min_latitude = latr - distance;
double max_latitude = latr + distance;
/* Note: we're being lazy and not accounting for coordinates near poles */
double min_longitude, max_longitude;
double difference_longitude = asin(sin(distance) / cos(latr));
min_longitude = lonr - difference_longitude;
max_longitude = lonr + difference_longitude;
bounds[0] = rad_deg(min_longitude);
bounds[1] = rad_deg(min_latitude);
bounds[2] = rad_deg(max_longitude);
bounds[3] = rad_deg(max_latitude);
bounds[0] = longitude - rad_deg(radius_meters/EARTH_RADIUS_IN_METERS/cos(deg_rad(latitude)));
bounds[2] = longitude + rad_deg(radius_meters/EARTH_RADIUS_IN_METERS/cos(deg_rad(latitude)));
bounds[1] = latitude - rad_deg(radius_meters/EARTH_RADIUS_IN_METERS);
bounds[3] = latitude + rad_deg(radius_meters/EARTH_RADIUS_IN_METERS);
return 1;
}
/* Return a set of areas (center + 8) that are able to cover a range query
* for the specified position and radius. */
GeoHashRadius geohashGetAreasByRadius(double longitude, double latitude, double radius_meters) {
GeoHashRange long_range, lat_range;
GeoHashRadius radius = { { 0 } };
GeoHashBits hash = { 0 };
GeoHashNeighbors neighbors = { { 0 } };
GeoHashArea area = { { 0 } };
GeoHashRadius radius;
GeoHashBits hash;
GeoHashNeighbors neighbors;
GeoHashArea area;
double min_lon, max_lon, min_lat, max_lat;
double bounds[4];
int steps;
@@ -124,12 +115,43 @@ GeoHashRadius geohashGetAreasByRadius(double longitude, double latitude, double
steps = geohashEstimateStepsByRadius(radius_meters,latitude);
geohashGetCoordRange(&long_range, &lat_range);
geohashEncode(&long_range, &lat_range, longitude, latitude, steps, &hash);
geohashNeighbors(&hash, &neighbors);
geohashGetCoordRange(&long_range, &lat_range);
geohashDecode(long_range, lat_range, hash, &area);
geohashGetCoordRange(&long_range,&lat_range);
geohashEncode(&long_range,&lat_range,longitude,latitude,steps,&hash);
geohashNeighbors(&hash,&neighbors);
geohashDecode(long_range,lat_range,hash,&area);
/* Check if the step is enough at the limits of the covered area.
* Sometimes when the search area is near an edge of the
* area, the estimated step is not small enough, since one of the
* north / south / west / east square is too near to the search area
* to cover everything. */
int decrease_step = 0;
{
GeoHashArea north, south, east, west;
geohashDecode(long_range, lat_range, neighbors.north, &north);
geohashDecode(long_range, lat_range, neighbors.south, &south);
geohashDecode(long_range, lat_range, neighbors.east, &east);
geohashDecode(long_range, lat_range, neighbors.west, &west);
if (geohashGetDistance(longitude,latitude,longitude,north.latitude.max)
< radius_meters) decrease_step = 1;
if (geohashGetDistance(longitude,latitude,longitude,south.latitude.min)
< radius_meters) decrease_step = 1;
if (geohashGetDistance(longitude,latitude,east.longitude.max,latitude)
< radius_meters) decrease_step = 1;
if (geohashGetDistance(longitude,latitude,west.longitude.min,latitude)
< radius_meters) decrease_step = 1;
}
if (steps > 1 && decrease_step) {
steps--;
geohashEncode(&long_range,&lat_range,longitude,latitude,steps,&hash);
geohashNeighbors(&hash,&neighbors);
geohashDecode(long_range,lat_range,hash,&area);
}
/* Exclude the search areas that are useless. */
if (area.latitude.min < min_lat) {
GZERO(neighbors.south);
GZERO(neighbors.south_west);
deps/linenoise/linenoise.c
Vendored
+6 -1
View File
@@ -111,6 +111,7 @@
#include <string.h>
#include <stdlib.h>
#include <ctype.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/ioctl.h>
#include <unistd.h>
@@ -1160,10 +1161,14 @@ int linenoiseHistorySetMaxLen(int len) {
/* Save the history in the specified file. On success 0 is returned
* otherwise -1 is returned. */
int linenoiseHistorySave(const char *filename) {
FILE *fp = fopen(filename,"w");
mode_t old_umask = umask(S_IXUSR|S_IRWXG|S_IRWXO);
FILE *fp;
int j;
fp = fopen(filename,"w");
umask(old_umask);
if (fp == NULL) return -1;
chmod(filename,S_IRUSR|S_IWUSR);
for (j = 0; j < history_len; j++)
fprintf(fp,"%s\n",history[j]);
fclose(fp);
deps/lua/src/lua_cmsgpack.c
Vendored
+10
View File
@@ -385,6 +385,7 @@ void mp_encode_lua_table_as_array(lua_State *L, mp_buf *buf, int level) {
#endif
mp_encode_array(L,buf,len);
luaL_checkstack(L, 1, "in function mp_encode_lua_table_as_array");
for (j = 1; j <= len; j++) {
lua_pushnumber(L,j);
lua_gettable(L,-2);
@@ -400,6 +401,7 @@ void mp_encode_lua_table_as_map(lua_State *L, mp_buf *buf, int level) {
* Lua API, we need to iterate a first time. Note that an alternative
* would be to do a single run, and then hack the buffer to insert the
* map opcodes for message pack. Too hackish for this lib. */
luaL_checkstack(L, 3, "in function mp_encode_lua_table_as_map");
lua_pushnil(L);
while(lua_next(L,-2)) {
lua_pop(L,1); /* remove value, keep key for next iteration. */
@@ -515,10 +517,14 @@ int mp_pack(lua_State *L) {
if (nargs == 0)
return luaL_argerror(L, 0, "MessagePack pack needs input.");
if (!lua_checkstack(L, nargs))
return luaL_argerror(L, 0, "Too many arguments for MessagePack pack.");
buf = mp_buf_new(L);
for(i = 1; i <= nargs; i++) {
/* Copy argument i to top of stack for _encode processing;
* the encode function pops it from the stack when complete. */
luaL_checkstack(L, 1, "in function mp_check");
lua_pushvalue(L, i);
mp_encode_lua_type(L,buf,0);
@@ -547,6 +553,7 @@ void mp_decode_to_lua_array(lua_State *L, mp_cur *c, size_t len) {
int index = 1;
lua_newtable(L);
luaL_checkstack(L, 1, "in function mp_decode_to_lua_array");
while(len--) {
lua_pushnumber(L,index++);
mp_decode_to_lua_type(L,c);
@@ -821,6 +828,9 @@ int mp_unpack_full(lua_State *L, int limit, int offset) {
* subtract the entire buffer size from the unprocessed size
* to get our next start offset */
int offset = len - c.left;
luaL_checkstack(L, 1, "in function mp_unpack_full");
/* Return offset -1 when we have have processed the entire buffer. */
lua_pushinteger(L, c.left == 0 ? -1 : offset);
/* Results are returned with the arg elements still
deps/lua/src/lua_struct.c
Vendored
+29 -25
View File
@@ -1,7 +1,7 @@
/*
** {======================================================
** Library for packing/unpacking structures.
** $Id: struct.c,v 1.4 2012/07/04 18:54:29 roberto Exp $
** $Id: struct.c,v 1.7 2018/05/11 22:04:31 roberto Exp $
** See Copyright Notice at the end of this file
** =======================================================
*/
@@ -15,8 +15,8 @@
** h/H - signed/unsigned short
** l/L - signed/unsigned long
** T - size_t
** i/In - signed/unsigned integer with size `n' (default is size of int)
** cn - sequence of `n' chars (from/to a string); when packing, n==0 means
** i/In - signed/unsigned integer with size 'n' (default is size of int)
** cn - sequence of 'n' chars (from/to a string); when packing, n==0 means
the whole string; when unpacking, n==0 means use the previous
read number as the string length
** s - zero-terminated string
@@ -89,14 +89,12 @@ typedef struct Header {
} Header;
static int getnum (lua_State *L, const char **fmt, int df) {
static int getnum (const char **fmt, int df) {
if (!isdigit(**fmt)) /* no number? */
return df; /* return default value */
else {
int a = 0;
do {
if (a > (INT_MAX / 10) || a * 10 > (INT_MAX - (**fmt - '0')))
luaL_error(L, "integral size overflow");
a = a*10 + *((*fmt)++) - '0';
} while (isdigit(**fmt));
return a;
@@ -117,9 +115,9 @@ static size_t optsize (lua_State *L, char opt, const char **fmt) {
case 'f': return sizeof(float);
case 'd': return sizeof(double);
case 'x': return 1;
case 'c': return getnum(L, fmt, 1);
case 'c': return getnum(fmt, 1);
case 'i': case 'I': {
int sz = getnum(L, fmt, sizeof(int));
int sz = getnum(fmt, sizeof(int));
if (sz > MAXINTSIZE)
luaL_error(L, "integral size %d is larger than limit of %d",
sz, MAXINTSIZE);
@@ -152,7 +150,7 @@ static void controloptions (lua_State *L, int opt, const char **fmt,
case '>': h->endian = BIG; return;
case '<': h->endian = LITTLE; return;
case '!': {
int a = getnum(L, fmt, MAXALIGN);
int a = getnum(fmt, MAXALIGN);
if (!isp2(a))
luaL_error(L, "alignment %d is not a power of 2", a);
h->align = a;
@@ -295,21 +293,26 @@ static int b_unpack (lua_State *L) {
const char *fmt = luaL_checkstring(L, 1);
size_t ld;
const char *data = luaL_checklstring(L, 2, &ld);
size_t pos = luaL_optinteger(L, 3, 1) - 1;
size_t pos = luaL_optinteger(L, 3, 1);
luaL_argcheck(L, pos > 0, 3, "offset must be 1 or greater");
pos--; /* Lua indexes are 1-based, but here we want 0-based for C
* pointer math. */
int n = 0; /* number of results */
defaultoptions(&h);
lua_settop(L, 2);
while (*fmt) {
int opt = *fmt++;
size_t size = optsize(L, opt, &fmt);
pos += gettoalign(pos, &h, opt, size);
luaL_argcheck(L, pos+size <= ld, 2, "data string too short");
luaL_checkstack(L, 1, "too many results");
luaL_argcheck(L, size <= ld && pos <= ld - size,
2, "data string too short");
/* stack space for item + next position */
luaL_checkstack(L, 2, "too many results");
switch (opt) {
case 'b': case 'B': case 'h': case 'H':
case 'l': case 'L': case 'T': case 'i': case 'I': { /* integer types */
int issigned = islower(opt);
lua_Number res = getinteger(data+pos, h.endian, issigned, size);
lua_pushnumber(L, res);
lua_pushnumber(L, res); n++;
break;
}
case 'x': {
@@ -319,25 +322,26 @@ static int b_unpack (lua_State *L) {
float f;
memcpy(&f, data+pos, size);
correctbytes((char *)&f, sizeof(f), h.endian);
lua_pushnumber(L, f);
lua_pushnumber(L, f); n++;
break;
}
case 'd': {
double d;
memcpy(&d, data+pos, size);
correctbytes((char *)&d, sizeof(d), h.endian);
lua_pushnumber(L, d);
lua_pushnumber(L, d); n++;
break;
}
case 'c': {
if (size == 0) {
if (!lua_isnumber(L, -1))
luaL_error(L, "format `c0' needs a previous size");
if (n == 0 || !lua_isnumber(L, -1))
luaL_error(L, "format 'c0' needs a previous size");
size = lua_tonumber(L, -1);
lua_pop(L, 1);
luaL_argcheck(L, pos+size <= ld, 2, "data string too short");
lua_pop(L, 1); n--;
luaL_argcheck(L, size <= ld && pos <= ld - size,
2, "data string too short");
}
lua_pushlstring(L, data+pos, size);
lua_pushlstring(L, data+pos, size); n++;
break;
}
case 's': {
@@ -345,15 +349,15 @@ static int b_unpack (lua_State *L) {
if (e == NULL)
luaL_error(L, "unfinished string in data");
size = (e - (data+pos)) + 1;
lua_pushlstring(L, data+pos, size - 1);
lua_pushlstring(L, data+pos, size - 1); n++;
break;
}
default: controloptions(L, opt, &fmt, &h);
}
pos += size;
}
lua_pushinteger(L, pos + 1);
return lua_gettop(L) - 2;
lua_pushinteger(L, pos + 1); /* next position */
return n + 1;
}
@@ -399,7 +403,7 @@ LUALIB_API int luaopen_struct (lua_State *L) {
/******************************************************************************
* Copyright (C) 2010-2012 Lua.org, PUC-Rio. All rights reserved.
* Copyright (C) 2010-2018 Lua.org, PUC-Rio. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining
* a copy of this software and associated documentation files (the
redis.conf
+33 -3
View File
@@ -117,8 +117,9 @@ timeout 0
# Note that to close the connection the double of the time is needed.
# On other kernels the period depends on the kernel configuration.
#
# A reasonable value for this option is 60 seconds.
tcp-keepalive 0
# A reasonable value for this option is 300 seconds, which is the new
# Redis default starting with Redis 3.2.1.
tcp-keepalive 300
################################# GENERAL #####################################
@@ -146,7 +147,7 @@ supervised no
#
# Creating a pid file is best effort: if Redis is not able to create it
# nothing bad happens, the server will start and run normally.
pidfile /var/run/redis.pid
pidfile /var/run/redis_6379.pid
# Specify the server verbosity level.
# This can be one of:
@@ -434,6 +435,35 @@ slave-priority 100
# By default min-slaves-to-write is set to 0 (feature disabled) and
# min-slaves-max-lag is set to 10.
# A Redis master is able to list the address and port of the attached
# slaves in different ways. For example the "INFO replication" section
# offers this information, which is used, among other tools, by
# Redis Sentinel in order to discover slave instances.
# Another place where this info is available is in the output of the
# "ROLE" command of a masteer.
#
# The listed IP and address normally reported by a slave is obtained
# in the following way:
#
# IP: The address is auto detected by checking the peer address
# of the socket used by the slave to connect with the master.
#
# Port: The port is communicated by the slave during the replication
# handshake, and is normally the port that the slave is using to
# list for connections.
#
# However when port forwarding or Network Address Translation (NAT) is
# used, the slave may be actually reachable via different IP and port
# pairs. The following two options can be used by a slave in order to
# report to its master a specific set of IP and port, so that both INFO
# and ROLE will report those values.
#
# There is no need to use both the options if you need to override just
# the port or the IP address.
#
# slave-announce-ip 5.5.5.5
# slave-announce-port 1234
################################## SECURITY ###################################
# Require clients to issue AUTH <PASSWORD> before processing any other
sentinel.conf
+24
View File
@@ -1,5 +1,21 @@
# Example sentinel.conf
# *** IMPORTANT ***
#
# By default Sentinel will not be reachable from interfaces different than
# localhost, either use the 'bind' directive to bind to a list of network
# interfaces, or disable protected mode with "protected-mode no" by
# adding it to this configuration file.
#
# Before doing that MAKE SURE the instance is protected from the outside
# world via firewalling or other means.
#
# For example you may use one of the following:
#
# bind 127.0.0.1 192.168.1.1
#
# protected-mode no
# port <sentinel-port>
# The port that this sentinel instance will run on
port 26379
@@ -178,4 +194,12 @@ sentinel failover-timeout mymaster 180000
#
# sentinel client-reconfig-script mymaster /var/redis/reconfig.sh
# SECURITY
#
# By default SENTINEL SET will not be able to change the notification-script
# and client-reconfig-script at runtime. This avoids a trivial security issue
# where clients can set the script to anything and trigger a failover in order
# to get the program executed.
sentinel deny-scripts-reconfig yes
src/Makefile
+15 -5
View File
@@ -65,17 +65,27 @@ ifeq ($(uname_S),SunOS)
FINAL_LIBS+= -ldl -lnsl -lsocket -lresolv -lpthread -lrt
else
ifeq ($(uname_S),Darwin)
# Darwin (nothing to do)
# Darwin
FINAL_LIBS+= -ldl
else
ifeq ($(uname_S),AIX)
# AIX
FINAL_LDFLAGS+= -Wl,-bexpall
FINAL_LIBS+= -pthread -lcrypt -lbsd
FINAL_LIBS+=-ldl -pthread -lcrypt -lbsd
else
ifeq ($(uname_S),OpenBSD)
# OpenBSD
FINAL_LIBS+= -lpthread
else
ifeq ($(uname_S),FreeBSD)
# FreeBSD
FINAL_LIBS+= -lpthread
else
# All the other OSes (notably Linux)
FINAL_LDFLAGS+= -rdynamic
FINAL_LIBS+= -pthread
FINAL_LIBS+=-ldl -pthread
endif
endif
endif
endif
endif
@@ -95,7 +105,7 @@ endif
ifeq ($(MALLOC),jemalloc)
DEPENDENCY_TARGETS+= jemalloc
FINAL_CFLAGS+= -DUSE_JEMALLOC -I../deps/jemalloc/include
FINAL_LIBS+= ../deps/jemalloc/lib/libjemalloc.a -ldl
FINAL_LIBS+= ../deps/jemalloc/lib/libjemalloc.a
endif
REDIS_CC=$(QUIET_CC)$(CC) $(FINAL_CFLAGS)
src/ae.c
+41 -7
View File
@@ -158,6 +158,10 @@ void aeDeleteFileEvent(aeEventLoop *eventLoop, int fd, int mask)
aeFileEvent *fe = &eventLoop->events[fd];
if (fe->mask == AE_NONE) return;
/* We want to always remove AE_BARRIER if set when AE_WRITABLE
* is removed. */
if (mask & AE_WRITABLE) mask |= AE_BARRIER;
aeApiDelEvent(eventLoop, fd, mask);
fe->mask = fe->mask & (~mask);
if (fd == eventLoop->maxfd && fe->mask == AE_NONE) {
@@ -402,19 +406,49 @@ int aeProcessEvents(aeEventLoop *eventLoop, int flags)
aeFileEvent *fe = &eventLoop->events[eventLoop->fired[j].fd];
int mask = eventLoop->fired[j].mask;
int fd = eventLoop->fired[j].fd;
int rfired = 0;
int fired = 0; /* Number of events fired for current fd. */
/* note the fe->mask & mask & ... code: maybe an already processed
* event removed an element that fired and we still didn't
* processed, so we check if the event is still valid. */
if (fe->mask & mask & AE_READABLE) {
rfired = 1;
/* Normally we execute the readable event first, and the writable
* event laster. This is useful as sometimes we may be able
* to serve the reply of a query immediately after processing the
* query.
*
* However if AE_BARRIER is set in the mask, our application is
* asking us to do the reverse: never fire the writable event
* after the readable. In such a case, we invert the calls.
* This is useful when, for instance, we want to do things
* in the beforeSleep() hook, like fsynching a file to disk,
* before replying to a client. */
int invert = fe->mask & AE_BARRIER;
/* Note the "fe->mask & mask & ..." code: maybe an already
* processed event removed an element that fired and we still
* didn't processed, so we check if the event is still valid.
*
* Fire the readable event if the call sequence is not
* inverted. */
if (!invert && fe->mask & mask & AE_READABLE) {
fe->rfileProc(eventLoop,fd,fe->clientData,mask);
fired++;
}
/* Fire the writable event. */
if (fe->mask & mask & AE_WRITABLE) {
if (!rfired || fe->wfileProc != fe->rfileProc)
if (!fired || fe->wfileProc != fe->rfileProc) {
fe->wfileProc(eventLoop,fd,fe->clientData,mask);
fired++;
}
}
/* If we have to invert the call, fire the readable event now
* after the writable one. */
if (invert && fe->mask & mask & AE_READABLE) {
if (!fired || fe->wfileProc != fe->rfileProc) {
fe->rfileProc(eventLoop,fd,fe->clientData,mask);
fired++;
}
}
processed++;
}
}
src/ae.h
+9 -4
View File
@@ -38,9 +38,14 @@
#define AE_OK 0
#define AE_ERR -1
#define AE_NONE 0
#define AE_READABLE 1
#define AE_WRITABLE 2
#define AE_NONE 0 /* No events registered. */
#define AE_READABLE 1 /* Fire when descriptor is readable. */
#define AE_WRITABLE 2 /* Fire when descriptor is writable. */
#define AE_BARRIER 4 /* With WRITABLE, never fire the event if the
READABLE event already fired in the same event
loop iteration. Useful when you want to persist
things to disk before sending replies, and want
to do that in a group fashion. */
#define AE_FILE_EVENTS 1
#define AE_TIME_EVENTS 2
@@ -63,7 +68,7 @@ typedef void aeBeforeSleepProc(struct aeEventLoop *eventLoop);
/* File event structure */
typedef struct aeFileEvent {
int mask; /* one of AE_(READABLE|WRITABLE) */
int mask; /* one of AE_(READABLE|WRITABLE|BARRIER) */
aeFileProc *rfileProc;
aeFileProc *wfileProc;
void *clientData;
src/ae_select.c
+1
View File
@@ -29,6 +29,7 @@
*/
#include <sys/select.h>
#include <string.h>
typedef struct aeApiState {
src/anet.c
+3 -2
View File
@@ -462,7 +462,7 @@ static int anetV6Only(char *err, int s) {
static int _anetTcpServer(char *err, int port, char *bindaddr, int af, int backlog)
{
int s, rv;
int s = -1, rv;
char _port[6]; /* strlen("65535") */
struct addrinfo hints, *servinfo, *p;
@@ -486,11 +486,12 @@ static int _anetTcpServer(char *err, int port, char *bindaddr, int af, int backl
goto end;
}
if (p == NULL) {
anetSetError(err, "unable to bind socket");
anetSetError(err, "unable to bind socket, errno: %d", errno);
goto error;
}
error:
if (s != -1) close(s);
s = ANET_ERR;
end:
freeaddrinfo(servinfo);
src/aof.c
+23 -2
View File
@@ -115,6 +115,7 @@ void aofChildWriteDiffData(aeEventLoop *el, int fd, void *privdata, int mask) {
if (nwritten <= 0) return;
memmove(block->buf,block->buf+nwritten,block->used-nwritten);
block->used -= nwritten;
block->free += nwritten;
}
if (block->used == 0) listDelNode(server.aof_rewrite_buf_blocks,ln);
}
@@ -251,7 +252,10 @@ int startAppendOnly(void) {
strerror(errno));
return C_ERR;
}
if (rewriteAppendOnlyFileBackground() == C_ERR) {
if (server.rdb_child_pid != -1) {
server.aof_rewrite_scheduled = 1;
serverLog(LL_WARNING,"AOF was enabled but there is already a child process saving an RDB file on disk. An AOF background was scheduled to start when possible.");
} else if (rewriteAppendOnlyFileBackground() == C_ERR) {
close(server.aof_fd);
serverLog(LL_WARNING,"Redis needs to enable the AOF but can't trigger a background AOF rewrite operation. Check the above logs for more info about the error.");
return C_ERR;
@@ -532,6 +536,22 @@ void feedAppendOnlyFile(struct redisCommand *cmd, int dictid, robj **argv, int a
buf = catAppendOnlyGenericCommand(buf,3,tmpargv);
decrRefCount(tmpargv[0]);
buf = catAppendOnlyExpireAtCommand(buf,cmd,argv[1],argv[2]);
} else if (cmd->proc == setCommand && argc > 3) {
int i;
robj *exarg = NULL, *pxarg = NULL;
/* Translate SET [EX seconds][PX milliseconds] to SET and PEXPIREAT */
buf = catAppendOnlyGenericCommand(buf,3,argv);
for (i = 3; i < argc; i ++) {
if (!strcasecmp(argv[i]->ptr, "ex")) exarg = argv[i+1];
if (!strcasecmp(argv[i]->ptr, "px")) pxarg = argv[i+1];
}
serverAssert(!(exarg && pxarg));
if (exarg)
buf = catAppendOnlyExpireAtCommand(buf,server.expireCommand,argv[1],
exarg);
if (pxarg)
buf = catAppendOnlyExpireAtCommand(buf,server.pexpireCommand,argv[1],
pxarg);
} else {
/* All the other commands don't need translation or need the
* same translation already operated in the command vector
@@ -1261,7 +1281,7 @@ int rewriteAppendOnlyFileBackground(void) {
pid_t childpid;
long long start;
if (server.aof_child_pid != -1) return C_ERR;
if (server.aof_child_pid != -1 || server.rdb_child_pid != -1) return C_ERR;
if (aofCreatePipes() != C_OK) return C_ERR;
start = ustime();
if ((childpid = fork()) == 0) {
@@ -1292,6 +1312,7 @@ int rewriteAppendOnlyFileBackground(void) {
serverLog(LL_WARNING,
"Can't rewrite append only file in background: fork: %s",
strerror(errno));
aofClosePipes();
return C_ERR;
}
serverLog(LL_NOTICE,
src/bitops.c
+81 -42
View File
@@ -215,12 +215,7 @@ void setUnsignedBitfield(unsigned char *p, uint64_t offset, uint64_t bits, uint6
}
void setSignedBitfield(unsigned char *p, uint64_t offset, uint64_t bits, int64_t value) {
uint64_t uv;
if (value >= 0)
uv = value;
else
uv = UINT64_MAX + value + 1;
uint64_t uv = value; /* Casting will add UINT64_MAX + 1 if v is negative. */
setUnsignedBitfield(p,offset,bits,uv);
}
@@ -239,9 +234,21 @@ uint64_t getUnsignedBitfield(unsigned char *p, uint64_t offset, uint64_t bits) {
}
int64_t getSignedBitfield(unsigned char *p, uint64_t offset, uint64_t bits) {
int64_t value = getUnsignedBitfield(p,offset,bits);
int64_t value;
union {uint64_t u; int64_t i;} conv;
/* Converting from unsigned to signed is undefined when the value does
* not fit, however here we assume two's complement and the original value
* was obtained from signed -> unsigned conversion, so we'll find the
* most significant bit set if the original value was negative.
*
* Note that two's complement is mandatory for exact-width types
* according to the C99 standard. */
conv.u = getUnsignedBitfield(p,offset,bits);
value = conv.i;
/* If the top significant bit is 1, propagate it to all the
* higher bits for two complement representation of signed
* higher bits for two's complement representation of signed
* integers. */
if (value & ((uint64_t)1 << (bits-1)))
value |= ((uint64_t)-1) << bits;
@@ -299,7 +306,7 @@ int checkUnsignedBitfieldOverflow(uint64_t value, int64_t incr, uint64_t bits, i
handle_wrap:
{
uint64_t mask = ((int64_t)-1) << bits;
uint64_t mask = ((uint64_t)-1) << bits;
uint64_t res = value+incr;
res &= ~mask;
@@ -342,7 +349,7 @@ int checkSignedBitfieldOverflow(int64_t value, int64_t incr, uint64_t bits, int
handle_wrap:
{
uint64_t mask = ((int64_t)-1) << bits;
uint64_t mask = ((uint64_t)-1) << bits;
uint64_t msb = (uint64_t)1 << (bits-1);
uint64_t a = value, b = incr, c;
c = a+b; /* Perform addition as unsigned so that's defined. */
@@ -476,6 +483,37 @@ robj *lookupStringForBitCommand(client *c, size_t maxbit) {
return o;
}
/* Return a pointer to the string object content, and stores its length
* in 'len'. The user is required to pass (likely stack allocated) buffer
* 'llbuf' of at least LONG_STR_SIZE bytes. Such a buffer is used in the case
* the object is integer encoded in order to provide the representation
* without usign heap allocation.
*
* The function returns the pointer to the object array of bytes representing
* the string it contains, that may be a pointer to 'llbuf' or to the
* internal object representation. As a side effect 'len' is filled with
* the length of such buffer.
*
* If the source object is NULL the function is guaranteed to return NULL
* and set 'len' to 0. */
unsigned char *getObjectReadOnlyString(robj *o, long *len, char *llbuf) {
serverAssert(o->type == OBJ_STRING);
unsigned char *p = NULL;
/* Set the 'p' pointer to the string, that can be just a stack allocated
* array if our string was integer encoded. */
if (o && o->encoding == OBJ_ENCODING_INT) {
p = (unsigned char*) llbuf;
if (len) *len = ll2string(llbuf,LONG_STR_SIZE,(long)o->ptr);
} else if (o) {
p = (unsigned char*) o->ptr;
if (len) *len = sdslen(o->ptr);
} else {
if (len) *len = 0;
}
return p;
}
/* SETBIT key offset bitvalue */
void setbitCommand(client *c) {
robj *o;
@@ -721,21 +759,12 @@ void bitcountCommand(client *c) {
robj *o;
long start, end, strlen;
unsigned char *p;
char llbuf[32];
char llbuf[LONG_STR_SIZE];
/* Lookup, check for type, and return 0 for non existing keys. */
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,o,OBJ_STRING)) return;
/* Set the 'p' pointer to the string, that can be just a stack allocated
* array if our string was integer encoded. */
if (o->encoding == OBJ_ENCODING_INT) {
p = (unsigned char*) llbuf;
strlen = ll2string(llbuf,sizeof(llbuf),(long)o->ptr);
} else {
p = (unsigned char*) o->ptr;
strlen = sdslen(o->ptr);
}
p = getObjectReadOnlyString(o,&strlen,llbuf);
/* Parse start/end range if any. */
if (c->argc == 4) {
@@ -744,6 +773,10 @@ void bitcountCommand(client *c) {
if (getLongFromObjectOrReply(c,c->argv[3],&end,NULL) != C_OK)
return;
/* Convert negative indexes */
if (start < 0 && end < 0 && start > end) {
addReply(c,shared.czero);
return;
}
if (start < 0) start = strlen+start;
if (end < 0) end = strlen+end;
if (start < 0) start = 0;
@@ -775,7 +808,7 @@ void bitposCommand(client *c) {
robj *o;
long bit, start, end, strlen;
unsigned char *p;
char llbuf[32];
char llbuf[LONG_STR_SIZE];
int end_given = 0;
/* Parse the bit argument to understand what we are looking for, set
@@ -795,16 +828,7 @@ void bitposCommand(client *c) {
return;
}
if (checkType(c,o,OBJ_STRING)) return;
/* Set the 'p' pointer to the string, that can be just a stack allocated
* array if our string was integer encoded. */
if (o->encoding == OBJ_ENCODING_INT) {
p = (unsigned char*) llbuf;
strlen = ll2string(llbuf,sizeof(llbuf),(long)o->ptr);
} else {
p = (unsigned char*) o->ptr;
strlen = sdslen(o->ptr);
}
p = getObjectReadOnlyString(o,&strlen,llbuf);
/* Parse start/end range if any. */
if (c->argc == 4 || c->argc == 5) {
@@ -882,6 +906,8 @@ void bitfieldCommand(client *c) {
int j, numops = 0, changes = 0;
struct bitfieldOp *ops = NULL; /* Array of ops to execute at end. */
int owtype = BFOVERFLOW_WRAP; /* Overflow type. */
int readonly = 1;
size_t higest_write_offset = 0;
for (j = 2; j < c->argc; j++) {
int remargs = c->argc-j-1; /* Remaining args other than current. */
@@ -929,8 +955,11 @@ void bitfieldCommand(client *c) {
return;
}
/* INCRBY and SET require another argument. */
if (opcode != BITFIELDOP_GET) {
readonly = 0;
if (higest_write_offset < bitoffset + bits - 1)
higest_write_offset = bitoffset + bits - 1;
/* INCRBY and SET require another argument. */
if (getLongLongFromObjectOrReply(c,c->argv[j+3],&i64,NULL) != C_OK){
zfree(ops);
return;
@@ -950,6 +979,18 @@ void bitfieldCommand(client *c) {
j += 3 - (opcode == BITFIELDOP_GET);
}
if (readonly) {
/* Lookup for read is ok if key doesn't exit, but errors
* if it's not a string. */
o = lookupKeyRead(c->db,c->argv[1]);
if (o != NULL && checkType(c,o,OBJ_STRING)) return;
} else {
/* Lookup by making room up to the farest bit reached by
* this operation. */
if ((o = lookupStringForBitCommand(c,
higest_write_offset)) == NULL) return;
}
addReplyMultiBulkLen(c,numops);
/* Actually process the operations. */
@@ -964,11 +1005,6 @@ void bitfieldCommand(client *c) {
* for simplicity. SET return value is the previous value so
* we need fetch & store as well. */
/* Lookup by making room up to the farest bit reached by
* this operation. */
if ((o = lookupStringForBitCommand(c,
thisop->offset + (thisop->bits-1))) == NULL) return;
/* We need two different but very similar code paths for signed
* and unsigned operations, since the set of functions to get/set
* the integers and the used variables types are different. */
@@ -1035,20 +1071,23 @@ void bitfieldCommand(client *c) {
changes++;
} else {
/* GET */
o = lookupKeyRead(c->db,c->argv[1]);
size_t olen = (o == NULL) ? 0 : sdslen(o->ptr);
unsigned char buf[9];
long strlen = 0;
unsigned char *src = NULL;
char llbuf[LONG_STR_SIZE];
if (o != NULL)
src = getObjectReadOnlyString(o,&strlen,llbuf);
/* For GET we use a trick: before executing the operation
* copy up to 9 bytes to a local buffer, so that we can easily
* execute up to 64 bit operations that are at actual string
* object boundaries. */
memset(buf,0,9);
unsigned char *src = o ? o->ptr : NULL;
int i;
size_t byte = thisop->offset >> 3;
for (i = 0; i < 9; i++) {
if (src == NULL || i+byte >= olen) break;
if (src == NULL || i+byte >= (size_t)strlen) break;
buf[i] = src[i+byte];
}
src/cluster.c
+58 -33
View File
@@ -37,7 +37,6 @@
#include <arpa/inet.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <sys/file.h>
#include <math.h>
@@ -129,7 +128,7 @@ int clusterLoadConfig(char *filename) {
/* Skip blank lines, they can be created either by users manually
* editing nodes.conf or by the config writing process if stopped
* before the truncate() call. */
if (line[0] == '\n') continue;
if (line[0] == '\n' || line[0] == '\0') continue;
/* Split the line into arguments for processing. */
argv = sdssplitargs(line,&argc);
@@ -234,6 +233,7 @@ int clusterLoadConfig(char *filename) {
*p = '\0';
direction = p[1]; /* Either '>' or '<' */
slot = atoi(argv[j]+1);
if (slot < 0 || slot >= CLUSTER_SLOTS) goto fmterr;
p += 3;
cn = clusterLookupNode(p);
if (!cn) {
@@ -253,6 +253,8 @@ int clusterLoadConfig(char *filename) {
} else {
start = stop = atoi(argv[j]);
}
if (start < 0 || start >= CLUSTER_SLOTS) goto fmterr;
if (stop < 0 || stop >= CLUSTER_SLOTS) goto fmterr;
while(start <= stop) clusterAddSlot(n, start++);
}
@@ -1300,13 +1302,15 @@ void clusterProcessGossipSection(clusterMsg *hdr, clusterLink *link) {
clusterNode *node;
sds ci;
ci = representClusterNodeFlags(sdsempty(), flags);
serverLog(LL_DEBUG,"GOSSIP %.40s %s:%d %s",
g->nodename,
g->ip,
ntohs(g->port),
ci);
sdsfree(ci);
if (server.verbosity == LL_DEBUG) {
ci = representClusterNodeFlags(sdsempty(), flags);
serverLog(LL_DEBUG,"GOSSIP %.40s %s:%d %s",
g->nodename,
g->ip,
ntohs(g->port),
ci);
sdsfree(ci);
}
/* Update our state accordingly to the gossip sections */
node = clusterLookupNode(g->nodename);
@@ -2082,7 +2086,7 @@ void clusterReadHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
* from event handlers that will do stuff with the same link later. */
void clusterSendMessage(clusterLink *link, unsigned char *msg, size_t msglen) {
if (sdslen(link->sndbuf) == 0 && msglen != 0)
aeCreateFileEvent(server.el,link->fd,AE_WRITABLE,
aeCreateFileEvent(server.el,link->fd,AE_WRITABLE|AE_BARRIER,
clusterWriteHandler,link);
link->sndbuf = sdscatlen(link->sndbuf, msg, msglen);
@@ -2558,9 +2562,10 @@ void clusterSendFailoverAuthIfNeeded(clusterNode *node, clusterMsg *request) {
}
/* We can vote for this slave. */
clusterSendFailoverAuth(node);
server.cluster->lastVoteEpoch = server.cluster->currentEpoch;
node->slaveof->voted_time = mstime();
clusterDoBeforeSleep(CLUSTER_TODO_SAVE_CONFIG|CLUSTER_TODO_FSYNC_CONFIG);
clusterSendFailoverAuth(node);
serverLog(LL_WARNING, "Failover auth granted to %.40s for epoch %llu",
node->name, (unsigned long long) server.cluster->currentEpoch);
}
@@ -3440,8 +3445,10 @@ int clusterDelNodeSlots(clusterNode *node) {
int deleted = 0, j;
for (j = 0; j < CLUSTER_SLOTS; j++) {
if (clusterNodeGetSlotBit(node,j)) clusterDelSlot(j);
deleted++;
if (clusterNodeGetSlotBit(node,j)) {
clusterDelSlot(j);
deleted++;
}
}
return deleted;
}
@@ -3675,15 +3682,14 @@ static struct redisNodeFlags redisNodeFlagsTable[] = {
/* Concatenate the comma separated list of node flags to the given SDS
* string 'ci'. */
sds representClusterNodeFlags(sds ci, uint16_t flags) {
if (flags == 0) {
ci = sdscat(ci,"noflags,");
} else {
int i, size = sizeof(redisNodeFlagsTable)/sizeof(struct redisNodeFlags);
for (i = 0; i < size; i++) {
struct redisNodeFlags *nodeflag = redisNodeFlagsTable + i;
if (flags & nodeflag->flag) ci = sdscat(ci, nodeflag->name);
}
size_t orig_len = sdslen(ci);
int i, size = sizeof(redisNodeFlagsTable)/sizeof(struct redisNodeFlags);
for (i = 0; i < size; i++) {
struct redisNodeFlags *nodeflag = redisNodeFlagsTable + i;
if (flags & nodeflag->flag) ci = sdscat(ci, nodeflag->name);
}
/* If no flag was added, add the "noflags" special flag. */
if (sdslen(ci) == orig_len) ci = sdscat(ci,"noflags,");
sdsIncrLen(ci,-1); /* Remove trailing comma. */
return ci;
}
@@ -4007,7 +4013,7 @@ void clusterCommand(client *c) {
}
if ((n = clusterLookupNode(c->argv[4]->ptr)) == NULL) {
addReplyErrorFormat(c,"I don't know about node %s",
(char*)c->argv[3]->ptr);
(char*)c->argv[4]->ptr);
return;
}
server.cluster->importing_slots_from[slot] = n;
@@ -4437,7 +4443,7 @@ int verifyDumpPayload(unsigned char *p, size_t len) {
/* Verify RDB version */
rdbver = (footer[1] << 8) | footer[0];
if (rdbver != RDB_VERSION) return C_ERR;
if (rdbver > RDB_VERSION) return C_ERR;
/* Verify CRC64 */
crc = crc64(0,p,len-8);
@@ -4652,13 +4658,13 @@ void migrateCommand(client *c) {
int copy, replace, j;
long timeout;
long dbid;
long long ttl, expireat;
robj **ov = NULL; /* Objects to migrate. */
robj **kv = NULL; /* Key names. */
robj **newargv = NULL; /* Used to rewrite the command as DEL ... keys ... */
rio cmd, payload;
int may_retry = 1;
int write_error = 0;
int argv_rewritten = 0;
/* To support the KEYS option we need the following additional state. */
int first_key = 3; /* Argument index of the first key. */
@@ -4667,7 +4673,6 @@ void migrateCommand(client *c) {
/* Initialization */
copy = 0;
replace = 0;
ttl = 0;
/* Parse additional options */
for (j = 6; j < c->argc; j++) {
@@ -4743,7 +4748,9 @@ try_again:
/* Create RESTORE payload and generate the protocol to call the command. */
for (j = 0; j < num_keys; j++) {
expireat = getExpire(c->db,kv[j]);
long long ttl = 0;
long long expireat = getExpire(c->db,kv[j]);
if (expireat != -1) {
ttl = expireat-mstime();
if (ttl < 1) ttl = 1;
@@ -4841,12 +4848,20 @@ try_again:
goto socket_err; /* A retry is guaranteed because of tested conditions.*/
}
/* On socket errors, close the migration socket now that we still have
* the original host/port in the ARGV. Later the original command may be
* rewritten to DEL and will be too later. */
if (socket_error) migrateCloseSocket(c->argv[1],c->argv[2]);
if (!copy) {
/* Translate MIGRATE as DEL for replication/AOF. */
/* Translate MIGRATE as DEL for replication/AOF. Note that we do
* this only for the keys for which we received an acknowledgement
* from the receiving Redis server, by using the del_idx index. */
if (del_idx > 1) {
newargv[0] = createStringObject("DEL",3);
/* Note that the following call takes ownership of newargv. */
replaceClientCommandVector(c,del_idx,newargv);
argv_rewritten = 1;
} else {
/* No key transfer acknowledged, no need to rewrite as DEL. */
zfree(newargv);
@@ -4855,8 +4870,8 @@ try_again:
}
/* If we are here and a socket error happened, we don't want to retry.
* Just signal the problem to the client, but only do it if we don't
* already queued a different error reported by the destination server. */
* Just signal the problem to the client, but only do it if we did not
* already queue a different error reported by the destination server. */
if (!error_from_target && socket_error) {
may_retry = 0;
goto socket_err;
@@ -4864,7 +4879,11 @@ try_again:
if (!error_from_target) {
/* Success! Update the last_dbid in migrateCachedSocket, so that we can
* avoid SELECT the next time if the target DB is the same. Reply +OK. */
* avoid SELECT the next time if the target DB is the same. Reply +OK.
*
* Note: If we reached this point, even if socket_error is true
* still the SELECT command succeeded (otherwise the code jumps to
* socket_err label. */
cs->last_dbid = dbid;
addReply(c,shared.ok);
} else {
@@ -4874,7 +4893,6 @@ try_again:
sdsfree(cmd.io.buffer.ptr);
zfree(ov); zfree(kv); zfree(newargv);
if (socket_error) migrateCloseSocket(c->argv[1],c->argv[2]);
return;
/* On socket errors we try to close the cached socket and try again.
@@ -4884,7 +4902,12 @@ socket_err:
/* Cleanup we want to perform in both the retry and no retry case.
* Note: Closing the migrate socket will also force SELECT next time. */
sdsfree(cmd.io.buffer.ptr);
migrateCloseSocket(c->argv[1],c->argv[2]);
/* If the command was rewritten as DEL and there was a socket error,
* we already closed the socket earlier. While migrateCloseSocket()
* is idempotent, the host/port arguments are now gone, so don't do it
* again. */
if (!argv_rewritten) migrateCloseSocket(c->argv[1],c->argv[2]);
zfree(newargv);
newargv = NULL; /* This will get reallocated on retry. */
@@ -5187,8 +5210,9 @@ int clusterRedirectBlockedClientIfNeeded(client *c) {
return 1;
}
/* All keys must belong to the same slot, so check first key only. */
di = dictGetIterator(c->bpop.keys);
while((de = dictNext(di)) != NULL) {
if ((de = dictNext(di)) != NULL) {
robj *key = dictGetKey(de);
int slot = keyHashSlot((char*)key->ptr, sdslen(key->ptr));
clusterNode *node = server.cluster->slots[slot];
@@ -5206,6 +5230,7 @@ int clusterRedirectBlockedClientIfNeeded(client *c) {
clusterRedirectClient(c,node,slot,
CLUSTER_REDIR_MOVED);
}
dictReleaseIterator(di);
return 1;
}
}
src/config.c
+41 -20
View File
@@ -549,8 +549,9 @@ void loadServerConfigFromString(char *config) {
unsigned long long hard, soft;
int soft_seconds;
if (class == -1) {
err = "Unrecognized client limit class";
if (class == -1 || class == CLIENT_TYPE_MASTER) {
err = "Unrecognized client limit class: the user specified "
"an invalid one, or 'master' which has no buffer limits.";
goto loaderr;
}
hard = memtoll(argv[2],NULL);
@@ -570,6 +571,16 @@ void loadServerConfigFromString(char *config) {
}
} else if (!strcasecmp(argv[0],"slave-priority") && argc == 2) {
server.slave_priority = atoi(argv[1]);
} else if (!strcasecmp(argv[0],"slave-announce-ip") && argc == 2) {
zfree(server.slave_announce_ip);
server.slave_announce_ip = zstrdup(argv[1]);
} else if (!strcasecmp(argv[0],"slave-announce-port") && argc == 2) {
server.slave_announce_port = atoi(argv[1]);
if (server.slave_announce_port < 0 ||
server.slave_announce_port > 65535)
{
err = "Invalid port"; goto loaderr;
}
} else if (!strcasecmp(argv[0],"min-slaves-to-write") && argc == 2) {
server.repl_min_slaves_to_write = atoi(argv[1]);
if (server.repl_min_slaves_to_write < 0) {
@@ -682,7 +693,7 @@ void loadServerConfig(char *filename, char *options) {
#define config_set_numerical_field(_name,_var,min,max) \
} else if (!strcasecmp(c->argv[2]->ptr,_name)) { \
if (getLongLongFromObject(o,&ll) == C_ERR || ll < 0) goto badfmt; \
if (getLongLongFromObject(o,&ll) == C_ERR) goto badfmt; \
if (min != LLONG_MIN && ll < min) goto badfmt; \
if (max != LLONG_MAX && ll > max) goto badfmt; \
_var = ll;
@@ -824,7 +835,8 @@ void configSetCommand(client *c) {
long val;
if ((j % 4) == 0) {
if (getClientTypeByName(v[j]) == -1) {
int class = getClientTypeByName(v[j]);
if (class == -1 || class == CLIENT_TYPE_MASTER) {
sdsfreesplitres(v,vlen);
goto badfmt;
}
@@ -857,6 +869,9 @@ void configSetCommand(client *c) {
if (flags == -1) goto badfmt;
server.notify_keyspace_events = flags;
} config_set_special_field("slave-announce-ip") {
zfree(server.slave_announce_ip);
server.slave_announce_ip = ((char*)o->ptr)[0] ? zstrdup(o->ptr) : NULL;
/* Boolean fields.
* config_set_bool_field(name,var). */
@@ -895,16 +910,14 @@ void configSetCommand(client *c) {
"timeout",server.maxidletime,0,LONG_MAX) {
} config_set_numerical_field(
"auto-aof-rewrite-percentage",server.aof_rewrite_perc,0,LLONG_MAX){
} config_set_numerical_field(
"auto-aof-rewrite-min-size",server.aof_rewrite_min_size,0,LLONG_MAX) {
} config_set_numerical_field(
"hash-max-ziplist-entries",server.hash_max_ziplist_entries,0,LLONG_MAX) {
} config_set_numerical_field(
"hash-max-ziplist-value",server.hash_max_ziplist_value,0,LLONG_MAX) {
} config_set_numerical_field(
"list-max-ziplist-size",server.list_max_ziplist_size,0,LLONG_MAX) {
"list-max-ziplist-size",server.list_max_ziplist_size,INT_MIN,INT_MAX) {
} config_set_numerical_field(
"list-compress-depth",server.list_compress_depth,0,LLONG_MAX) {
"list-compress-depth",server.list_compress_depth,0,INT_MAX) {
} config_set_numerical_field(
"set-max-intset-entries",server.set_max_intset_entries,0,LLONG_MAX) {
} config_set_numerical_field(
@@ -933,6 +946,8 @@ void configSetCommand(client *c) {
"repl-diskless-sync-delay",server.repl_diskless_sync_delay,0,LLONG_MAX) {
} config_set_numerical_field(
"slave-priority",server.slave_priority,0,LLONG_MAX) {
} config_set_numerical_field(
"slave-announce-port",server.slave_announce_port,0,65535) {
} config_set_numerical_field(
"min-slaves-to-write",server.repl_min_slaves_to_write,0,LLONG_MAX) {
refreshGoodSlavesCount();
@@ -969,6 +984,8 @@ void configSetCommand(client *c) {
}
} config_set_memory_field("repl-backlog-size",ll) {
resizeReplicationBacklog(ll);
} config_set_memory_field("auto-aof-rewrite-min-size",ll) {
server.aof_rewrite_min_size = ll;
/* Enumeration fields.
* config_set_enum_field(name,var,enum_var) */
@@ -1001,7 +1018,7 @@ badfmt: /* Bad format errors */
*----------------------------------------------------------------------------*/
#define config_get_string_field(_name,_var) do { \
if (stringmatch(pattern,_name,0)) { \
if (stringmatch(pattern,_name,1)) { \
addReplyBulkCString(c,_name); \
addReplyBulkCString(c,_var ? _var : ""); \
matches++; \
@@ -1009,7 +1026,7 @@ badfmt: /* Bad format errors */
} while(0);
#define config_get_bool_field(_name,_var) do { \
if (stringmatch(pattern,_name,0)) { \
if (stringmatch(pattern,_name,1)) { \
addReplyBulkCString(c,_name); \
addReplyBulkCString(c,_var ? "yes" : "no"); \
matches++; \
@@ -1017,7 +1034,7 @@ badfmt: /* Bad format errors */
} while(0);
#define config_get_numerical_field(_name,_var) do { \
if (stringmatch(pattern,_name,0)) { \
if (stringmatch(pattern,_name,1)) { \
ll2string(buf,sizeof(buf),_var); \
addReplyBulkCString(c,_name); \
addReplyBulkCString(c,buf); \
@@ -1026,7 +1043,7 @@ badfmt: /* Bad format errors */
} while(0);
#define config_get_enum_field(_name,_var,_enumvar) do { \
if (stringmatch(pattern,_name,0)) { \
if (stringmatch(pattern,_name,1)) { \
addReplyBulkCString(c,_name); \
addReplyBulkCString(c,configEnumGetNameOrUnknown(_enumvar,_var)); \
matches++; \
@@ -1048,6 +1065,7 @@ void configGetCommand(client *c) {
config_get_string_field("unixsocket",server.unixsocket);
config_get_string_field("logfile",server.logfile);
config_get_string_field("pidfile",server.pidfile);
config_get_string_field("slave-announce-ip",server.slave_announce_ip);
/* Numerical values */
config_get_numerical_field("maxmemory",server.maxmemory);
@@ -1090,6 +1108,7 @@ void configGetCommand(client *c) {
config_get_numerical_field("maxclients",server.maxclients);
config_get_numerical_field("watchdog-period",server.watchdog_period);
config_get_numerical_field("slave-priority",server.slave_priority);
config_get_numerical_field("slave-announce-port",server.slave_announce_port);
config_get_numerical_field("min-slaves-to-write",server.repl_min_slaves_to_write);
config_get_numerical_field("min-slaves-max-lag",server.repl_min_slaves_max_lag);
config_get_numerical_field("hz",server.hz);
@@ -1138,12 +1157,12 @@ void configGetCommand(client *c) {
/* Everything we can't handle with macros follows. */
if (stringmatch(pattern,"appendonly",0)) {
if (stringmatch(pattern,"appendonly",1)) {
addReplyBulkCString(c,"appendonly");
addReplyBulkCString(c,server.aof_state == AOF_OFF ? "no" : "yes");
matches++;
}
if (stringmatch(pattern,"dir",0)) {
if (stringmatch(pattern,"dir",1)) {
char buf[1024];
if (getcwd(buf,sizeof(buf)) == NULL)
@@ -1153,7 +1172,7 @@ void configGetCommand(client *c) {
addReplyBulkCString(c,buf);
matches++;
}
if (stringmatch(pattern,"save",0)) {
if (stringmatch(pattern,"save",1)) {
sds buf = sdsempty();
int j;
@@ -1169,7 +1188,7 @@ void configGetCommand(client *c) {
sdsfree(buf);
matches++;
}
if (stringmatch(pattern,"client-output-buffer-limit",0)) {
if (stringmatch(pattern,"client-output-buffer-limit",1)) {
sds buf = sdsempty();
int j;
@@ -1187,14 +1206,14 @@ void configGetCommand(client *c) {
sdsfree(buf);
matches++;
}
if (stringmatch(pattern,"unixsocketperm",0)) {
if (stringmatch(pattern,"unixsocketperm",1)) {
char buf[32];
snprintf(buf,sizeof(buf),"%o",server.unixsocketperm);
addReplyBulkCString(c,"unixsocketperm");
addReplyBulkCString(c,buf);
matches++;
}
if (stringmatch(pattern,"slaveof",0)) {
if (stringmatch(pattern,"slaveof",1)) {
char buf[256];
addReplyBulkCString(c,"slaveof");
@@ -1206,7 +1225,7 @@ void configGetCommand(client *c) {
addReplyBulkCString(c,buf);
matches++;
}
if (stringmatch(pattern,"notify-keyspace-events",0)) {
if (stringmatch(pattern,"notify-keyspace-events",1)) {
robj *flagsobj = createObject(OBJ_STRING,
keyspaceEventsFlagsToString(server.notify_keyspace_events));
@@ -1215,7 +1234,7 @@ void configGetCommand(client *c) {
decrRefCount(flagsobj);
matches++;
}
if (stringmatch(pattern,"bind",0)) {
if (stringmatch(pattern,"bind",1)) {
sds aux = sdsjoin(server.bindaddr,server.bindaddr_count," ");
addReplyBulkCString(c,"bind");
@@ -1768,6 +1787,7 @@ int rewriteConfig(char *path) {
rewriteConfigOctalOption(state,"unixsocketperm",server.unixsocketperm,CONFIG_DEFAULT_UNIX_SOCKET_PERM);
rewriteConfigNumericalOption(state,"timeout",server.maxidletime,CONFIG_DEFAULT_CLIENT_TIMEOUT);
rewriteConfigNumericalOption(state,"tcp-keepalive",server.tcpkeepalive,CONFIG_DEFAULT_TCP_KEEPALIVE);
rewriteConfigNumericalOption(state,"slave-announce-port",server.slave_announce_port,CONFIG_DEFAULT_SLAVE_ANNOUNCE_PORT);
rewriteConfigEnumOption(state,"loglevel",server.verbosity,loglevel_enum,CONFIG_DEFAULT_VERBOSITY);
rewriteConfigStringOption(state,"logfile",server.logfile,CONFIG_DEFAULT_LOGFILE);
rewriteConfigYesNoOption(state,"syslog-enabled",server.syslog_enabled,CONFIG_DEFAULT_SYSLOG_ENABLED);
@@ -1781,6 +1801,7 @@ int rewriteConfig(char *path) {
rewriteConfigStringOption(state,"dbfilename",server.rdb_filename,CONFIG_DEFAULT_RDB_FILENAME);
rewriteConfigDirOption(state);
rewriteConfigSlaveofOption(state);
rewriteConfigStringOption(state,"slave-announce-ip",server.slave_announce_ip,CONFIG_DEFAULT_SLAVE_ANNOUNCE_IP);
rewriteConfigStringOption(state,"masterauth",server.masterauth,NULL);
rewriteConfigYesNoOption(state,"slave-serve-stale-data",server.repl_serve_stale_data,CONFIG_DEFAULT_SLAVE_SERVE_STALE_DATA);
rewriteConfigYesNoOption(state,"slave-read-only",server.repl_slave_ro,CONFIG_DEFAULT_SLAVE_READ_ONLY);
src/db.c
+96 -14
View File
@@ -41,7 +41,10 @@ void slotToKeyFlush(void);
* C-level DB API
*----------------------------------------------------------------------------*/
robj *lookupKey(redisDb *db, robj *key) {
/* Low level key lookup API, not actually called directly from commands
* implementations that should instead rely on lookupKeyRead(),
* lookupKeyWrite() and lookupKeyReadWithFlags(). */
robj *lookupKey(redisDb *db, robj *key, int flags) {
dictEntry *de = dictFind(db->dict,key->ptr);
if (de) {
robj *val = dictGetVal(de);
@@ -49,20 +52,45 @@ robj *lookupKey(redisDb *db, robj *key) {
/* Update the access time for the ageing algorithm.
* Don't do it if we have a saving child, as this will trigger
* a copy on write madness. */
if (server.rdb_child_pid == -1 && server.aof_child_pid == -1)
if (server.rdb_child_pid == -1 &&
server.aof_child_pid == -1 &&
!(flags & LOOKUP_NOTOUCH))
{
val->lru = LRU_CLOCK();
}
return val;
} else {
return NULL;
}
}
robj *lookupKeyRead(redisDb *db, robj *key) {
/* Lookup a key for read operations, or return NULL if the key is not found
* in the specified DB.
*
* As a side effect of calling this function:
* 1. A key gets expired if it reached it's TTL.
* 2. The key last access time is updated.
* 3. The global keys hits/misses stats are updated (reported in INFO).
*
* This API should not be used when we write to the key after obtaining
* the object linked to the key, but only for read only operations.
*
* Flags change the behavior of this command:
*
* LOOKUP_NONE (or zero): no special flags are passed.
* LOOKUP_NOTOUCH: don't alter the last access time of the key.
*
* Note: this function also returns NULL is the key is logically expired
* but still existing, in case this is a slave, since this API is called only
* for read operations. Even if the key expiry is master-driven, we can
* correctly report a key is expired on slaves even if the master is lagging
* expiring our key via DELs in the replication link. */
robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags) {
robj *val;
if (expireIfNeeded(db,key) == 1) {
/* Key expired. If we are in the context of a master, expireIfNeeded()
* returns 0 only when the key does not exist at all, so it's save
* returns 0 only when the key does not exist at all, so it's safe
* to return NULL ASAP. */
if (server.masterhost == NULL) return NULL;
@@ -86,7 +114,7 @@ robj *lookupKeyRead(redisDb *db, robj *key) {
return NULL;
}
}
val = lookupKey(db,key);
val = lookupKey(db,key,flags);
if (val == NULL)
server.stat_keyspace_misses++;
else
@@ -94,9 +122,20 @@ robj *lookupKeyRead(redisDb *db, robj *key) {
return val;
}
/* Like lookupKeyReadWithFlags(), but does not use any flag, which is the
* common case. */
robj *lookupKeyRead(redisDb *db, robj *key) {
return lookupKeyReadWithFlags(db,key,LOOKUP_NONE);
}
/* Lookup a key for write operations, and as a side effect, if needed, expires
* the key if its TTL is reached.
*
* Returns the linked value object if the key exists or NULL if the key
* does not exist in the specified DB. */
robj *lookupKeyWrite(redisDb *db, robj *key) {
expireIfNeeded(db,key);
return lookupKey(db,key);
return lookupKey(db,key,LOOKUP_NONE);
}
robj *lookupKeyReadOrReply(client *c, robj *key, robj *reply) {
@@ -645,7 +684,7 @@ void typeCommand(client *c) {
robj *o;
char *type;
o = lookupKeyRead(c->db,c->argv[1]);
o = lookupKeyReadWithFlags(c->db,c->argv[1],LOOKUP_NOTOUCH);
if (o == NULL) {
type = "none";
} else {
@@ -967,7 +1006,7 @@ void ttlGenericCommand(client *c, int output_ms) {
long long expire, ttl = -1;
/* If the key does not exist at all, return -2 */
if (lookupKeyRead(c->db,c->argv[1]) == NULL) {
if (lookupKeyReadWithFlags(c->db,c->argv[1],LOOKUP_NOTOUCH) == NULL) {
addReplyLongLong(c,-2);
return;
}
@@ -994,21 +1033,26 @@ void pttlCommand(client *c) {
}
void persistCommand(client *c) {
dictEntry *de;
de = dictFind(c->db->dict,c->argv[1]->ptr);
if (de == NULL) {
addReply(c,shared.czero);
} else {
if (lookupKeyWrite(c->db,c->argv[1])) {
if (removeExpire(c->db,c->argv[1])) {
addReply(c,shared.cone);
server.dirty++;
} else {
addReply(c,shared.czero);
}
} else {
addReply(c,shared.czero);
}
}
/* TOUCH key1 [key2 key3 ... keyN] */
void touchCommand(client *c) {
int touched = 0;
for (int j = 1; j < c->argc; j++)
if (lookupKeyRead(c->db,c->argv[j]) != NULL) touched++;
addReplyLongLong(c,touched);
}
/* -----------------------------------------------------------------------------
* API to get key arguments from commands
* ---------------------------------------------------------------------------*/
@@ -1187,6 +1231,44 @@ int *migrateGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkey
return keys;
}
/* Helper function to extract keys from following commands:
* GEORADIUS key x y radius unit [WITHDIST] [WITHHASH] [WITHCOORD] [ASC|DESC]
* [COUNT count] [STORE key] [STOREDIST key]
* GEORADIUSBYMEMBER key member radius unit ... options ... */
int *georadiusGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys) {
int i, num, *keys;
UNUSED(cmd);
/* Check for the presence of the stored key in the command */
int stored_key = -1;
for (i = 5; i < argc; i++) {
char *arg = argv[i]->ptr;
/* For the case when user specifies both "store" and "storedist" options, the
* second key specified would override the first key. This behavior is kept
* the same as in georadiusCommand method.
*/
if ((!strcasecmp(arg, "store") || !strcasecmp(arg, "storedist")) && ((i+1) < argc)) {
stored_key = i+1;
i++;
}
}
num = 1 + (stored_key == -1 ? 0 : 1);
/* Keys in the command come from two places:
* argv[1] = key,
* argv[5...n] = stored key if present
*/
keys = zmalloc(sizeof(int) * num);
/* Add all key positions to keys[] */
keys[0] = 1;
if(num > 1) {
keys[1] = stored_key;
}
*numkeys = num;
return keys;
}
/* Slot to Key API. This is used by Redis Cluster in order to obtain in
* a fast way a key that belongs to a specified hash slot. This is useful
* while rehashing the cluster. */
src/debug.c
+64 -4
View File
@@ -39,6 +39,8 @@
#include <ucontext.h>
#include <fcntl.h>
#include "bio.h"
#include <unistd.h>
#include <dlfcn.h>
#endif /* HAVE_BACKTRACE */
#ifdef __CYGWIN__
@@ -124,7 +126,7 @@ void computeDatasetDigest(unsigned char *final) {
redisDb *db = server.db+j;
if (dictSize(db->dict) == 0) continue;
di = dictGetIterator(db->dict);
di = dictGetSafeIterator(db->dict);
/* hash the DB id, so the same dataset moved in a different
* DB will lead to a different digest */
@@ -671,6 +673,8 @@ static void *getMcontextEip(ucontext_t *uc) {
return (void*) uc->uc_mcontext.gregs[16]; /* Linux 64 */
#elif defined(__ia64__) /* Linux IA64 */
return (void*) uc->uc_mcontext.sc_ip;
#elif defined(__arm__) /* Linux ARM */
return (void*) uc->uc_mcontext.arm_pc;
#endif
#else
return NULL;
@@ -972,6 +976,32 @@ int memtest_test_linux_anonymous_maps(void) {
}
#endif
/* Scans the (assumed) x86 code starting at addr, for a max of `len`
* bytes, searching for E8 (callq) opcodes, and dumping the symbols
* and the call offset if they appear to be valid. */
void dumpX86Calls(void *addr, size_t len) {
size_t j;
unsigned char *p = addr;
Dl_info info;
/* Hash table to best-effort avoid printing the same symbol
* multiple times. */
unsigned long ht[256] = {0};
if (len < 5) return;
for (j = 0; j < len-4; j++) {
if (p[j] != 0xE8) continue; /* Not an E8 CALL opcode. */
unsigned long target = (unsigned long)addr+j+5;
target += *((int32_t*)(p+j+1));
if (dladdr((void*)target, &info) != 0 && info.dli_sname != NULL) {
if (ht[target&0xff] != target) {
printf("Function at 0x%lx is %s\n",target,info.dli_sname);
ht[target&0xff] = target;
}
j += 4; /* Skip the 32 bit immediate. */
}
}
}
void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
ucontext_t *uc = (ucontext_t*) secret;
void *eip = getMcontextEip(uc);
@@ -1022,19 +1052,49 @@ void sigsegvHandler(int sig, siginfo_t *info, void *secret) {
bioKillThreads();
if (memtest_test_linux_anonymous_maps()) {
serverLogRaw(LL_WARNING|LL_RAW,
"!!! MEMORY ERROR DETECTED! Check your memory ASAP !!!");
"!!! MEMORY ERROR DETECTED! Check your memory ASAP !!!\n");
} else {
serverLogRaw(LL_WARNING|LL_RAW,
"Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.");
"Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.\n");
}
#endif
if (eip != NULL) {
Dl_info info;
if (dladdr(eip, &info) != 0) {
serverLog(LL_WARNING|LL_RAW,
"\n------ DUMPING CODE AROUND EIP ------\n"
"Symbol: %s (base: %p)\n"
"Module: %s (base %p)\n"
"$ xxd -r -p /tmp/dump.hex /tmp/dump.bin\n"
"$ objdump --adjust-vma=%p -D -b binary -m i386:x86-64 /tmp/dump.bin\n"
"------\n",
info.dli_sname, info.dli_saddr, info.dli_fname, info.dli_fbase,
info.dli_saddr);
size_t len = (long)eip - (long)info.dli_saddr;
unsigned long sz = sysconf(_SC_PAGESIZE);
if (len < 1<<13) { /* we don't have functions over 8k (verified) */
/* Find the address of the next page, which is our "safety"
* limit when dumping. Then try to dump just 128 bytes more
* than EIP if there is room, or stop sooner. */
unsigned long next = ((unsigned long)eip + sz) & ~(sz-1);
unsigned long end = (unsigned long)eip + 128;
if (end > next) end = next;
len = end - (unsigned long)info.dli_saddr;
serverLogHexDump(LL_WARNING, "dump of function",
info.dli_saddr ,len);
dumpX86Calls(info.dli_saddr,len);
}
}
}
serverLogRaw(LL_WARNING|LL_RAW,
"\n=== REDIS BUG REPORT END. Make sure to include from START to END. ===\n\n"
" Please report the crash by opening an issue on github:\n\n"
" http://github.com/antirez/redis/issues\n\n"
" Suspect RAM error? Use redis-server --test-memory to verify it.\n\n"
);
/* free(messages); Don't call free() with possibly corrupted memory. */
if (server.daemonize && server.supervised == 0) unlink(server.pidfile);
@@ -1055,7 +1115,7 @@ void serverLogHexDump(int level, char *descr, void *value, size_t len) {
unsigned char *v = value;
char charset[] = "0123456789abcdef";
serverLog(level,"%s (hexdump):", descr);
serverLog(level,"%s (hexdump of %zu bytes):", descr, len);
b = buf;
while(len) {
b[0] = charset[(*v)>>4];
src/debugmacro.h
+41
View File
@@ -0,0 +1,41 @@
/* This file contains debugging macros to be used when investigating issues.
*
* -----------------------------------------------------------------------------
*
* Copyright (c) 2016, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of Redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
#include <stdio.h>
#define D(...) \
do { \
FILE *fp = fopen("/tmp/log.txt","a"); \
fprintf(fp,"%s:%s:%d:\t", __FILE__, __func__, __LINE__); \
fprintf(fp,__VA_ARGS__); \
fprintf(fp,"\n"); \
fclose(fp); \
} while (0);
src/dict.c
+15 -12
View File
@@ -871,6 +871,15 @@ unsigned long dictScan(dict *d,
de = de->next;
}
/* Set unmasked bits so incrementing the reversed cursor
* operates on the masked bits */
v |= ~m0;
/* Increment the reverse cursor */
v = rev(v);
v++;
v = rev(v);
} else {
t0 = &d->ht[0];
t1 = &d->ht[1];
@@ -901,22 +910,16 @@ unsigned long dictScan(dict *d,
de = de->next;
}
/* Increment bits not covered by the smaller mask */
v = (((v | m0) + 1) & ~m0) | (v & m0);
/* Increment the reverse cursor not covered by the smaller mask.*/
v |= ~m1;
v = rev(v);
v++;
v = rev(v);
/* Continue while bits covered by mask difference is non-zero */
} while (v & (m0 ^ m1));
}
/* Set unmasked bits so incrementing the reversed cursor
* operates on the masked bits of the smaller table */
v |= ~m0;
/* Increment the reverse cursor */
v = rev(v);
v++;
v = rev(v);
return v;
}
@@ -949,7 +952,7 @@ static unsigned long _dictNextPower(unsigned long size)
{
unsigned long i = DICT_HT_INITIAL_SIZE;
if (size >= LONG_MAX) return LONG_MAX;
if (size >= LONG_MAX) return LONG_MAX + 1LU;
while(1) {
if (i >= size)
return i;
src/geo.c
+57 -12
View File
@@ -1,6 +1,6 @@
/*
* Copyright (c) 2014, Matt Stancliff <matt@genges.com>.
* Copyright (c) 2015, Salvatore Sanfilippo <antirez@gmail.com>.
* Copyright (c) 2015-2016, Salvatore Sanfilippo <antirez@gmail.com>.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -30,6 +30,7 @@
#include "geo.h"
#include "geohash_helper.h"
#include "debugmacro.h"
/* Things exported from t_zset.c only for geo.c, since it is the only other
* part of Redis that requires close zset introspection. */
@@ -156,9 +157,13 @@ double extractDistanceOrReply(client *c, robj **argv,
return -1;
}
if (distance < 0) {
addReplyError(c,"radius cannot be negative");
return -1;
}
double to_meters = extractUnitOrReply(c,argv[1]);
if (to_meters < 0) {
addReplyError(c,"radius cannot be negative");
return -1;
}
@@ -323,6 +328,7 @@ int membersOfGeoHashBox(robj *zobj, GeoHashBits hash, geoArray *ga, double lon,
int membersOfAllNeighbors(robj *zobj, GeoHashRadius n, double lon, double lat, double radius, geoArray *ga) {
GeoHashBits neighbors[9];
unsigned int i, count = 0, last_processed = 0;
int debugmsg = 0;
neighbors[0] = n.hash;
neighbors[1] = n.neighbors.north;
@@ -337,8 +343,26 @@ int membersOfAllNeighbors(robj *zobj, GeoHashRadius n, double lon, double lat, d
/* For each neighbor (*and* our own hashbox), get all the matching
* members and add them to the potential result list. */
for (i = 0; i < sizeof(neighbors) / sizeof(*neighbors); i++) {
if (HASHISZERO(neighbors[i]))
if (HASHISZERO(neighbors[i])) {
if (debugmsg) D("neighbors[%d] is zero",i);
continue;
}
/* Debugging info. */
if (debugmsg) {
GeoHashRange long_range, lat_range;
geohashGetCoordRange(&long_range,&lat_range);
GeoHashArea myarea = {{0,0},{0,0},{0,0}};
geohashDecode(long_range, lat_range, neighbors[i], &myarea);
/* Dump center square. */
D("neighbors[%d]:\n",i);
D("area.longitude.min: %f\n", myarea.longitude.min);
D("area.longitude.max: %f\n", myarea.longitude.max);
D("area.latitude.min: %f\n", myarea.latitude.min);
D("area.latitude.max: %f\n", myarea.latitude.max);
D("\n");
}
/* When a huge Radius (in the 5000 km range or more) is used,
* adjacent neighbors can be the same, leading to duplicated
@@ -347,7 +371,11 @@ int membersOfAllNeighbors(robj *zobj, GeoHashRadius n, double lon, double lat, d
if (last_processed &&
neighbors[i].bits == neighbors[last_processed].bits &&
neighbors[i].step == neighbors[last_processed].step)
{
if (debugmsg)
D("Skipping processing of %d, same as previous\n",i);
continue;
}
count += membersOfGeoHashBox(zobj, neighbors[i], ga, lon, lat, radius);
last_processed = i;
}
@@ -426,13 +454,14 @@ void geoaddCommand(client *c) {
#define SORT_ASC 1
#define SORT_DESC 2
#define RADIUS_COORDS 1
#define RADIUS_MEMBER 2
#define RADIUS_COORDS (1<<0) /* Search around coordinates. */
#define RADIUS_MEMBER (1<<1) /* Search around member. */
#define RADIUS_NOSTORE (1<<2) /* Do not acceot STORE/STOREDIST option. */
/* GEORADIUS key x y radius unit [WITHDIST] [WITHHASH] [WITHCOORD] [ASC|DESC]
* [COUNT count] [STORE key] [STOREDIST key]
* GEORADIUSBYMEMBER key member radius unit ... options ... */
void georadiusGeneric(client *c, int type) {
void georadiusGeneric(client *c, int flags) {
robj *key = c->argv[1];
robj *storekey = NULL;
int storedist = 0; /* 0 for STORE, 1 for STOREDIST. */
@@ -447,11 +476,11 @@ void georadiusGeneric(client *c, int type) {
/* Find long/lat to use for radius search based on inquiry type */
int base_args;
double xy[2] = { 0 };
if (type == RADIUS_COORDS) {
if (flags & RADIUS_COORDS) {
base_args = 6;
if (extractLongLatOrReply(c, c->argv + 2, xy) == C_ERR)
return;
} else if (type == RADIUS_MEMBER) {
} else if (flags & RADIUS_MEMBER) {
base_args = 5;
robj *member = c->argv[2];
if (longLatFromMember(zobj, member, xy) == C_ERR) {
@@ -459,7 +488,7 @@ void georadiusGeneric(client *c, int type) {
return;
}
} else {
addReplyError(c, "unknown georadius search type");
addReplyError(c, "Unknown georadius search type");
return;
}
@@ -496,11 +525,17 @@ void georadiusGeneric(client *c, int type) {
return;
}
i++;
} else if (!strcasecmp(arg, "store") && (i+1) < remaining) {
} else if (!strcasecmp(arg, "store") &&
(i+1) < remaining &&
!(flags & RADIUS_NOSTORE))
{
storekey = c->argv[base_args+i+1];
storedist = 0;
i++;
} else if (!strcasecmp(arg, "storedist") && (i+1) < remaining) {
} else if (!strcasecmp(arg, "storedist") &&
(i+1) < remaining &&
!(flags & RADIUS_NOSTORE))
{
storekey = c->argv[base_args+i+1];
storedist = 1;
i++;
@@ -648,10 +683,20 @@ void georadiusCommand(client *c) {
}
/* GEORADIUSBYMEMBER wrapper function. */
void georadiusByMemberCommand(client *c) {
void georadiusbymemberCommand(client *c) {
georadiusGeneric(c, RADIUS_MEMBER);
}
/* GEORADIUS_RO wrapper function. */
void georadiusroCommand(client *c) {
georadiusGeneric(c, RADIUS_COORDS|RADIUS_NOSTORE);
}
/* GEORADIUSBYMEMBER_RO wrapper function. */
void georadiusbymemberroCommand(client *c) {
georadiusGeneric(c, RADIUS_MEMBER|RADIUS_NOSTORE);
}
/* GEOHASH key ele1 ele2 ... eleN
*
* Returns an array with an 11 characters geohash representation of the
src/help.h
+29 -14
View File
@@ -1,4 +1,4 @@
/* Automatically generated by generate-command-help.rb, do not edit. */
/* Automatically generated by utils/generate-command-help.rb, do not edit. */
#ifndef __REDIS_HELP_H
#define __REDIS_HELP_H
@@ -52,6 +52,11 @@ struct commandHelp {
"Count set bits in a string",
1,
"2.6.0" },
{ "BITFIELD",
"key [GET type offset] [SET type offset value] [INCRBY type offset increment] [OVERFLOW WRAP|SAT|FAIL]",
"Perform arbitrary bitfield integer operations on strings",
1,
"3.2.0" },
{ "BITOP",
"operation destkey key [key ...]",
"Perform bitwise operations between strings",
@@ -83,7 +88,7 @@ struct commandHelp {
9,
"2.6.9" },
{ "CLIENT KILL",
"[ip:port] [ID client-id] [TYPE normal|slave|pubsub] [ADDR ip:port] [SKIPME yes/no]",
"[ip:port] [ID client-id] [TYPE normal|master|slave|pubsub] [ADDR ip:port] [SKIPME yes/no]",
"Kill the connection of a client",
9,
"2.4.0" },
@@ -97,6 +102,11 @@ struct commandHelp {
"Stop processing commands from clients for some time",
9,
"2.9.50" },
{ "CLIENT REPLY",
"ON|OFF|SKIP",
"Instruct the server whether to reply to commands",
9,
"3.2" },
{ "CLIENT SETNAME",
"connection-name",
"Set the current connection name",
@@ -179,7 +189,7 @@ struct commandHelp {
"3.0.0" },
{ "CLUSTER SETSLOT",
"slot IMPORTING|MIGRATING|STABLE|NODE [node-id]",
"Bind an hash slot to a specific node",
"Bind a hash slot to a specific node",
12,
"3.0.0" },
{ "CLUSTER SLAVES",
@@ -321,32 +331,32 @@ struct commandHelp {
"key longitude latitude member [longitude latitude member ...]",
"Add one or more geospatial items in the geospatial index represented using a sorted set",
13,
"" },
"3.2.0" },
{ "GEODIST",
"key member1 member2 [unit]",
"Returns the distance between two members of a geospatial index",
13,
"" },
"3.2.0" },
{ "GEOHASH",
"key member [member ...]",
"Returns members of a geospatial index as standard geohash strings",
13,
"" },
"3.2.0" },
{ "GEOPOS",
"key member [member ...]",
"Returns longitude and latitude of members of a geospatial index",
13,
"" },
"3.2.0" },
{ "GEORADIUS",
"key longitude latitude radius m|km|ft|mi [WITHCOORD] [WITHDIST] [WITHHASH] [COUNT count]",
"key longitude latitude radius m|km|ft|mi [WITHCOORD] [WITHDIST] [WITHHASH] [COUNT count] [ASC|DESC] [STORE key] [STOREDIST key]",
"Query a sorted set representing a geospatial index to fetch members matching a given maximum distance from a point",
13,
"" },
"3.2.0" },
{ "GEORADIUSBYMEMBER",
"key member radius m|km|ft|mi [WITHCOORD] [WITHDIST] [WITHHASH] [COUNT count]",
"key member radius m|km|ft|mi [WITHCOORD] [WITHDIST] [WITHHASH] [COUNT count] [ASC|DESC] [STORE key] [STOREDIST key]",
"Query a sorted set representing a geospatial index to fetch members matching a given maximum distance from a member",
13,
"" },
"3.2.0" },
{ "GET",
"key",
"Get the value of a key",
@@ -528,7 +538,7 @@ struct commandHelp {
1,
"1.0.0" },
{ "MIGRATE",
"host port key destination-db timeout [COPY] [REPLACE]",
"host port key|"" destination-db timeout [COPY] [REPLACE] [KEYS key]",
"Atomically transfer a key from a Redis instance to another one.",
0,
"2.6.0" },
@@ -593,7 +603,7 @@ struct commandHelp {
11,
"2.8.9" },
{ "PING",
"-",
"[message]",
"Ping the server",
8,
"1.0.0" },
@@ -707,6 +717,11 @@ struct commandHelp {
"Get the number of members in a set",
3,
"1.0.0" },
{ "SCRIPT DEBUG",
"YES|SYNC|NO",
"Set the debug mode for executed scripts.",
10,
"3.2.0" },
{ "SCRIPT EXISTS",
"script [script ...]",
"Check existence of scripts in the script cache.",
@@ -768,7 +783,7 @@ struct commandHelp {
1,
"2.2.0" },
{ "SHUTDOWN",
"[NOSAVE] [SAVE]",
"[NOSAVE|SAVE]",
"Synchronously save the dataset to disk and then shut down the server",
9,
"1.0.0" },
src/hyperloglog.c
+3
View File
@@ -598,6 +598,7 @@ int hllSparseToDense(robj *o) {
} else {
runlen = HLL_SPARSE_VAL_LEN(p);
regval = HLL_SPARSE_VAL_VALUE(p);
if ((runlen + idx) > HLL_REGISTERS) break; /* Overflow. */
while(runlen--) {
HLL_DENSE_SET_REGISTER(hdr->registers,idx,regval);
idx++;
@@ -1070,6 +1071,7 @@ int hllMerge(uint8_t *max, robj *hll) {
} else {
runlen = HLL_SPARSE_VAL_LEN(p);
regval = HLL_SPARSE_VAL_VALUE(p);
if ((runlen + i) > HLL_REGISTERS) break; /* Overflow. */
while(runlen--) {
if (regval > max[i]) max[i] = regval;
i++;
@@ -1128,6 +1130,7 @@ int isHLLObjectOrReply(client *c, robj *o) {
if (checkType(c,o,OBJ_STRING))
return C_ERR; /* Error already sent. */
if (!sdsEncodedObject(o)) goto invalid;
if (stringObjectLen(o) < sizeof(*hdr)) goto invalid;
hdr = o->ptr;
src/networking.c
+45 -10
View File
@@ -105,6 +105,7 @@ client *createClient(int fd) {
c->repl_ack_off = 0;
c->repl_ack_time = 0;
c->slave_listening_port = 0;
c->slave_ip[0] = '\0';
c->slave_capa = SLAVE_CAPA_NONE;
c->reply = listCreate();
c->reply_bytes = 0;
@@ -948,11 +949,11 @@ int writeToClient(int fd, client *c, int handler_installed) {
*
* However if we are over the maxmemory limit we ignore that and
* just deliver as much data as it is possible to deliver. */
server.stat_net_output_bytes += totwritten;
if (totwritten > NET_MAX_WRITES_PER_EVENT &&
(server.maxmemory == 0 ||
zmalloc_used_memory() < server.maxmemory)) break;
}
server.stat_net_output_bytes += totwritten;
if (nwritten == -1) {
if (errno == EAGAIN) {
nwritten = 0;
@@ -1008,13 +1009,25 @@ int handleClientsWithPendingWrites(void) {
/* Try to write buffers to the client socket. */
if (writeToClient(c->fd,c,0) == C_ERR) continue;
/* If there is nothing left, do nothing. Otherwise install
* the write handler. */
if (clientHasPendingReplies(c) &&
aeCreateFileEvent(server.el, c->fd, AE_WRITABLE,
/* If after the synchronous writes above we still have data to
* output to the client, we need to install the writable handler. */
if (clientHasPendingReplies(c)) {
int ae_flags = AE_WRITABLE;
/* For the fsync=always policy, we want that a given FD is never
* served for reading and writing in the same event loop iteration,
* so that in the middle of receiving the query, and serving it
* to the client, we'll call beforeSleep() that will do the
* actual fsync of AOF to disk. AE_BARRIER ensures that. */
if (server.aof_state == AOF_ON &&
server.aof_fsync == AOF_FSYNC_ALWAYS)
{
ae_flags |= AE_BARRIER;
}
if (aeCreateFileEvent(server.el, c->fd, ae_flags,
sendReplyToClient, c) == AE_ERR)
{
freeClientAsync(c);
{
freeClientAsync(c);
}
}
}
return processed;
@@ -1268,8 +1281,10 @@ void processInputBuffer(client *c) {
/* CLIENT_CLOSE_AFTER_REPLY closes the connection once the reply is
* written to the client. Make sure to not let the reply grow after
* this flag has been set (i.e. don't process more commands). */
if (c->flags & CLIENT_CLOSE_AFTER_REPLY) break;
* this flag has been set (i.e. don't process more commands).
*
* The same applies for clients we want to terminate ASAP. */
if (c->flags & (CLIENT_CLOSE_AFTER_REPLY|CLIENT_CLOSE_ASAP)) break;
/* Determine request type when unknown. */
if (!c->reqtype) {
@@ -1632,10 +1647,30 @@ void clientCommand(client *c) {
pauseClients(duration);
addReply(c,shared.ok);
} else {
addReplyError(c, "Syntax error, try CLIENT (LIST | KILL ip:port | GETNAME | SETNAME connection-name)");
addReplyError(c, "Syntax error, try CLIENT (LIST | KILL | GETNAME | SETNAME | PAUSE | REPLY)");
}
}
/* This callback is bound to POST and "Host:" command names. Those are not
* really commands, but are used in security attacks in order to talk to
* Redis instances via HTTP, with a technique called "cross protocol scripting"
* which exploits the fact that services like Redis will discard invalid
* HTTP headers and will process what follows.
*
* As a protection against this attack, Redis will terminate the connection
* when a POST or "Host:" header is seen, and will log the event from
* time to time (to avoid creating a DOS as a result of too many logs). */
void securityWarningCommand(client *c) {
static time_t logged_time;
time_t now = time(NULL);
if (labs(now-logged_time) > 60) {
serverLog(LL_WARNING,"Possible SECURITY ATTACK detected. It looks like somebody is sending POST or Host: commands to Redis. This is likely due to an attacker attempting to use Cross Protocol Scripting to compromise your Redis instance. Connection aborted.");
logged_time = now;
}
freeClientAsync(c);
}
/* Rewrite the command vector of the client. All the new objects ref count
* is incremented. The old command vector is freed, and the old objects
* ref count is decremented. */
src/object.c
+5 -12
View File
@@ -241,11 +241,9 @@ void freeStringObject(robj *o) {
}
void freeListObject(robj *o) {
switch (o->encoding) {
case OBJ_ENCODING_QUICKLIST:
if (o->encoding == OBJ_ENCODING_QUICKLIST) {
quicklistRelease(o->ptr);
break;
default:
} else {
serverPanic("Unknown list encoding type");
}
}
@@ -380,10 +378,10 @@ robj *tryObjectEncoding(robj *o) {
if (o->refcount > 1) return o;
/* Check if we can represent this string as a long integer.
* Note that we are sure that a string larger than 21 chars is not
* Note that we are sure that a string larger than 20 chars is not
* representable as a 32 nor 64 bit integer. */
len = sdslen(s);
if (len <= 21 && string2l(s,len,&value)) {
if (len <= 20 && string2l(s,len,&value)) {
/* This object is encodable as a long. Try to use a shared object.
* Note that we avoid using shared integers when maxmemory is used
* because every object needs to have a private LRU field for the LRU
@@ -616,18 +614,13 @@ int getLongDoubleFromObjectOrReply(client *c, robj *o, long double *target, cons
int getLongLongFromObject(robj *o, long long *target) {
long long value;
char *eptr;
if (o == NULL) {
value = 0;
} else {
serverAssertWithInfo(NULL,o,o->type == OBJ_STRING);
if (sdsEncodedObject(o)) {
errno = 0;
value = strtoll(o->ptr, &eptr, 10);
if (isspace(((char*)o->ptr)[0]) || eptr[0] != '\0' ||
errno == ERANGE)
return C_ERR;
if (string2ll(o->ptr,sdslen(o->ptr),&value) == 0) return C_ERR;
} else if (o->encoding == OBJ_ENCODING_INT) {
value = (long)o->ptr;
} else {
src/quicklist.c
+1
View File
@@ -671,6 +671,7 @@ int quicklistReplaceAtIndex(quicklist *quicklist, long index, void *data,
/* quicklistIndex provides an uncompressed node */
entry.node->zl = ziplistDelete(entry.node->zl, &entry.zi);
entry.node->zl = ziplistInsert(entry.node->zl, entry.zi, data, sz);
quicklistNodeUpdateSz(entry.node);
quicklistCompress(quicklist, entry.node);
return 1;
} else {
src/quicklist.h
+1 -1
View File
@@ -92,8 +92,8 @@ typedef struct quicklistEntry {
quicklistNode *node;
unsigned char *zi;
unsigned char *value;
unsigned int sz;
long long longval;
unsigned int sz;
int offset;
} quicklistEntry;
src/rdb.c
+68 -20
View File
@@ -45,13 +45,30 @@
#define RDB_LOAD_ENC (1<<0)
#define RDB_LOAD_PLAIN (1<<1)
#define rdbExitReportCorruptRDB(reason) rdbCheckThenExit(reason, __LINE__);
#define rdbExitReportCorruptRDB(...) rdbCheckThenExit(__LINE__,__VA_ARGS__)
void rdbCheckThenExit(char *reason, int where) {
serverLog(LL_WARNING, "Corrupt RDB detected at rdb.c:%d (%s). "
"Running 'redis-check-rdb %s'",
where, reason, server.rdb_filename);
redis_check_rdb(server.rdb_filename);
extern int rdbCheckMode;
void rdbCheckError(const char *fmt, ...);
void rdbCheckSetError(const char *fmt, ...);
void rdbCheckThenExit(int linenum, char *reason, ...) {
va_list ap;
char msg[1024];
int len;
len = snprintf(msg,sizeof(msg),
"Internal error in RDB reading function at rdb.c:%d -> ", linenum);
va_start(ap,reason);
vsnprintf(msg+len,sizeof(msg)-len,reason,ap);
va_end(ap);
if (!rdbCheckMode) {
serverLog(LL_WARNING, "%s", msg);
char *argv[2] = {"",server.rdb_filename};
redis_check_rdb_main(2,argv);
} else {
rdbCheckError("%s",msg);
}
exit(1);
}
@@ -142,10 +159,14 @@ uint32_t rdbLoadLen(rio *rdb, int *isencoded) {
/* Read a 14 bit len. */
if (rioRead(rdb,buf+1,1) == 0) return RDB_LENERR;
return ((buf[0]&0x3F)<<8)|buf[1];
} else {
} else if (type == RDB_32BITLEN) {
/* Read a 32 bit len. */
if (rioRead(rdb,&len,4) == 0) return RDB_LENERR;
return ntohl(len);
} else {
rdbExitReportCorruptRDB(
"Unknown length encoding %d in rdbLoadLen()",type);
return -1; /* Never reached. */
}
}
@@ -199,7 +220,7 @@ void *rdbLoadIntegerObject(rio *rdb, int enctype, int flags) {
val = (int32_t)v;
} else {
val = 0; /* anti-warning */
rdbExitReportCorruptRDB("Unknown RDB integer encoding type");
rdbExitReportCorruptRDB("Unknown RDB integer encoding type %d",enctype);
}
if (plain) {
char buf[LONG_STR_SIZE], *p;
@@ -298,7 +319,10 @@ void *rdbLoadLzfStringObject(rio *rdb, int flags) {
/* Load the compressed representation and uncompress it to target. */
if (rioRead(rdb,c,clen) == 0) goto err;
if (lzf_decompress(c,clen,val,len) == 0) goto err;
if (lzf_decompress(c,clen,val,len) == 0) {
if (rdbCheckMode) rdbCheckSetError("Invalid LZF compressed string");
goto err;
}
zfree(c);
if (plain)
@@ -406,7 +430,7 @@ void *rdbGenericLoadStringObject(rio *rdb, int flags) {
case RDB_ENC_LZF:
return rdbLoadLzfStringObject(rdb,flags);
default:
rdbExitReportCorruptRDB("Unknown RDB encoding type");
rdbExitReportCorruptRDB("Unknown RDB string encoding type %d",len);
}
}
@@ -560,7 +584,7 @@ ssize_t rdbSaveObject(rio *rdb, robj *o) {
if ((n = rdbSaveLen(rdb,ql->len)) == -1) return -1;
nwritten += n;
do {
while(node) {
if (quicklistNodeIsCompressed(node)) {
void *data;
size_t compress_len = quicklistGetLzf(node, &data);
@@ -570,7 +594,8 @@ ssize_t rdbSaveObject(rio *rdb, robj *o) {
if ((n = rdbSaveRawString(rdb,node->zl,node->sz)) == -1) return -1;
nwritten += n;
}
} while ((node = node->next));
node = node->next;
}
} else {
serverPanic("Unknown list encoding");
}
@@ -895,7 +920,7 @@ int rdbSaveBackground(char *filename) {
pid_t childpid;
long long start;
if (server.rdb_child_pid != -1) return C_ERR;
if (server.aof_child_pid != -1 || server.rdb_child_pid != -1) return C_ERR;
server.dirty_before_bgsave = server.dirty;
server.lastbgsave_try = time(NULL);
@@ -1192,11 +1217,11 @@ robj *rdbLoadObject(int rdbtype, rio *rdb) {
hashTypeConvert(o, OBJ_ENCODING_HT);
break;
default:
rdbExitReportCorruptRDB("Unknown encoding");
rdbExitReportCorruptRDB("Unknown RDB encoding type %d",rdbtype);
break;
}
} else {
rdbExitReportCorruptRDB("Unknown object type");
rdbExitReportCorruptRDB("Unknown RDB encoding type %d",rdbtype);
}
return o;
}
@@ -1562,7 +1587,7 @@ int rdbSaveToSlavesSockets(void) {
long long start;
int pipefds[2];
if (server.rdb_child_pid != -1) return C_ERR;
if (server.aof_child_pid != -1 || server.rdb_child_pid != -1) return C_ERR;
/* Before to fork, create a pipe that will be used in order to
* send back to the parent the IDs of the slaves that successfully
@@ -1666,9 +1691,6 @@ int rdbSaveToSlavesSockets(void) {
exitFromChild((retval == C_OK) ? 0 : 1);
} else {
/* Parent */
server.stat_fork_time = ustime()-start;
server.stat_fork_rate = (double) zmalloc_used_memory() * 1000000 / server.stat_fork_time / (1024*1024*1024); /* GB per second. */
latencyAddSampleIfNeeded("fork",server.stat_fork_time/1000);
if (childpid == -1) {
serverLog(LL_WARNING,"Can't save in background: fork: %s",
strerror(errno));
@@ -1691,6 +1713,10 @@ int rdbSaveToSlavesSockets(void) {
close(pipefds[0]);
close(pipefds[1]);
} else {
server.stat_fork_time = ustime()-start;
server.stat_fork_rate = (double) zmalloc_used_memory() * 1000000 / server.stat_fork_time / (1024*1024*1024); /* GB per second. */
latencyAddSampleIfNeeded("fork",server.stat_fork_time/1000);
serverLog(LL_NOTICE,"Background RDB transfer started by pid %d",
childpid);
server.rdb_save_time_start = time(NULL);
@@ -1717,11 +1743,33 @@ void saveCommand(client *c) {
}
}
/* BGSAVE [SCHEDULE] */
void bgsaveCommand(client *c) {
int schedule = 0;
/* The SCHEDULE option changes the behavior of BGSAVE when an AOF rewrite
* is in progress. Instead of returning an error a BGSAVE gets scheduled. */
if (c->argc > 1) {
if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"schedule")) {
schedule = 1;
} else {
addReply(c,shared.syntaxerr);
return;
}
}
if (server.rdb_child_pid != -1) {
addReplyError(c,"Background save already in progress");
} else if (server.aof_child_pid != -1) {
addReplyError(c,"Can't BGSAVE while AOF log rewriting is in progress");
if (schedule) {
server.rdb_bgsave_scheduled = 1;
addReplyStatus(c,"Background saving scheduled");
} else {
addReplyError(c,
"An AOF log rewriting in progress: can't BGSAVE right now. "
"Use BGSAVE SCHEDULE in order to schedule a BGSAVE whenver "
"possible.");
}
} else if (rdbSaveBackground(server.rdb_filename) == C_OK) {
addReplyStatus(c,"Background saving started");
} else {
src/redis-benchmark.c
+25 -2
View File
@@ -65,6 +65,7 @@ static struct config {
int randomkeys_keyspacelen;
int keepalive;
int pipeline;
int showerrors;
long long start;
long long totlatency;
long long *latency;
@@ -212,6 +213,16 @@ static void readHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
exit(1);
}
if (config.showerrors) {
static time_t lasterr_time = 0;
time_t now = time(NULL);
redisReply *r = reply;
if (r->type == REDIS_REPLY_ERROR && lasterr_time != now) {
lasterr_time = now;
printf("Error from server: %s\n", r->str);
}
}
freeReplyObject(reply);
/* This is an OK for prefix commands such as auth and select.*/
if (c->prefix_pending > 0) {
@@ -227,7 +238,7 @@ static void readHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
c->randptr[j] -= c->prefixlen;
c->prefixlen = 0;
}
continue;
continue;
}
if (config.requests_finished < config.requests)
@@ -518,6 +529,8 @@ int parseOptions(int argc, const char **argv) {
config.loop = 1;
} else if (!strcmp(argv[i],"-I")) {
config.idlemode = 1;
} else if (!strcmp(argv[i],"-e")) {
config.showerrors = 1;
} else if (!strcmp(argv[i],"-t")) {
if (lastarg) goto invalid;
/* We get the list of tests to run as a string in the form
@@ -560,7 +573,7 @@ usage:
" -c <clients> Number of parallel connections (default 50)\n"
" -n <requests> Total number of requests (default 100000)\n"
" -d <size> Data size of SET/GET value in bytes (default 2)\n"
" -dbnum <db> SELECT the specified db number (default 0)\n"
" --dbnum <db> SELECT the specified db number (default 0)\n"
" -k <boolean> 1=keep alive 0=reconnect (default 1)\n"
" -r <keyspacelen> Use random keys for SET/GET/INCR, random values for SADD\n"
" Using this option the benchmark will expand the string __rand_int__\n"
@@ -569,6 +582,8 @@ usage:
" is executed. Default tests use this to hit random keys in the\n"
" specified range.\n"
" -P <numreq> Pipeline <numreq> requests. Default 1 (no pipeline).\n"
" -e If server replies with errors, show them on stdout.\n"
" (no more than 1 error per second is displayed)\n"
" -q Quiet. Just show query/sec values\n"
" --csv Output in CSV format\n"
" -l Loop. Run the tests forever\n"
@@ -649,6 +664,7 @@ int main(int argc, const char **argv) {
config.keepalive = 1;
config.datasize = 3;
config.pipeline = 1;
config.showerrors = 0;
config.randomkeys = 0;
config.randomkeys_keyspacelen = 0;
config.quiet = 0;
@@ -763,6 +779,13 @@ int main(int argc, const char **argv) {
free(cmd);
}
if (test_is_selected("hset")) {
len = redisFormatCommand(&cmd,
"HSET myset:__rand_int__ element:__rand_int__ %s",data);
benchmark("HSET",cmd,len);
free(cmd);
}
if (test_is_selected("spop")) {
len = redisFormatCommand(&cmd,"SPOP myset");
benchmark("SPOP",cmd,len);
src/redis-check-rdb.c
+279 -644
View File
@@ -1,6 +1,5 @@
/*
* Copyright (c) 2009-2012, Pieter Noordhuis <pcnoordhuis at gmail dot com>
* Copyright (c) 2009-2012, Salvatore Sanfilippo <antirez at gmail dot com>
* Copyright (c) 2016, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -28,683 +27,319 @@
* POSSIBILITY OF SUCH DAMAGE.
*/
#include "server.h"
#include "rdb.h"
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/mman.h>
#include "lzf.h"
#include "crc64.h"
#define ERROR(...) { \
serverLog(LL_WARNING, __VA_ARGS__); \
exit(1); \
#include <stdarg.h>
void createSharedObjects(void);
void rdbLoadProgressCallback(rio *r, const void *buf, size_t len);
long long rdbLoadMillisecondTime(rio *rdb);
int rdbCheckMode = 0;
struct {
rio *rio;
robj *key; /* Current key we are reading. */
int key_type; /* Current key type if != -1. */
unsigned long keys; /* Number of keys processed. */
unsigned long expires; /* Number of keys with an expire. */
unsigned long already_expired; /* Number of keys already expired. */
int doing; /* The state while reading the RDB. */
int error_set; /* True if error is populated. */
char error[1024];
} rdbstate;
/* At every loading step try to remember what we were about to do, so that
* we can log this information when an error is encountered. */
#define RDB_CHECK_DOING_START 0
#define RDB_CHECK_DOING_READ_TYPE 1
#define RDB_CHECK_DOING_READ_EXPIRE 2
#define RDB_CHECK_DOING_READ_KEY 3
#define RDB_CHECK_DOING_READ_OBJECT_VALUE 4
#define RDB_CHECK_DOING_CHECK_SUM 5
#define RDB_CHECK_DOING_READ_LEN 6
#define RDB_CHECK_DOING_READ_AUX 7
char *rdb_check_doing_string[] = {
"start",
"read-type",
"read-expire",
"read-key",
"read-object-value",
"check-sum",
"read-len",
"read-aux"
};
char *rdb_type_string[] = {
"string",
"list-linked",
"set-hashtable",
"zset-v1",
"hash-hashtable",
"zset-v2",
"module-value",
"","",
"hash-zipmap",
"list-ziplist",
"set-intset",
"zset-ziplist",
"hash-ziplist",
"quicklist"
};
/* Show a few stats collected into 'rdbstate' */
void rdbShowGenericInfo(void) {
printf("[info] %lu keys read\n", rdbstate.keys);
printf("[info] %lu expires\n", rdbstate.expires);
printf("[info] %lu already expired\n", rdbstate.already_expired);
}
/* data type to hold offset in file and size */
typedef struct {
void *data;
size_t size;
size_t offset;
} pos;
/* Called on RDB errors. Provides details about the RDB and the offset
* we were when the error was detected. */
void rdbCheckError(const char *fmt, ...) {
char msg[1024];
va_list ap;
static unsigned char level = 0;
static pos positions[16];
va_start(ap, fmt);
vsnprintf(msg, sizeof(msg), fmt, ap);
va_end(ap);
#define CURR_OFFSET (positions[level].offset)
/* Hold a stack of errors */
typedef struct {
char error[16][1024];
size_t offset[16];
size_t level;
} errors_t;
static errors_t errors;
#define SHIFT_ERROR(provided_offset, ...) { \
sprintf(errors.error[errors.level], __VA_ARGS__); \
errors.offset[errors.level] = provided_offset; \
errors.level++; \
printf("--- RDB ERROR DETECTED ---\n");
printf("[offset %llu] %s\n",
(unsigned long long) (rdbstate.rio ?
rdbstate.rio->processed_bytes : 0), msg);
printf("[additional info] While doing: %s\n",
rdb_check_doing_string[rdbstate.doing]);
if (rdbstate.key)
printf("[additional info] Reading key '%s'\n",
(char*)rdbstate.key->ptr);
if (rdbstate.key_type != -1)
printf("[additional info] Reading type %d (%s)\n",
rdbstate.key_type,
((unsigned)rdbstate.key_type <
sizeof(rdb_type_string)/sizeof(char*)) ?
rdb_type_string[rdbstate.key_type] : "unknown");
rdbShowGenericInfo();
}
/* Data type to hold opcode with optional key name an success status */
typedef struct {
char* key;
int type;
char success;
} entry;
/* Print informations during RDB checking. */
void rdbCheckInfo(const char *fmt, ...) {
char msg[1024];
va_list ap;
#define MAX_TYPES_NUM 256
#define MAX_TYPE_NAME_LEN 16
/* store string types for output */
static char types[MAX_TYPES_NUM][MAX_TYPE_NAME_LEN];
va_start(ap, fmt);
vsnprintf(msg, sizeof(msg), fmt, ap);
va_end(ap);
/* Return true if 't' is a valid object type. */
static int rdbCheckType(unsigned char t) {
/* In case a new object type is added, update the following
* condition as necessary. */
return
(t >= RDB_TYPE_HASH_ZIPMAP && t <= RDB_TYPE_HASH_ZIPLIST) ||
t <= RDB_TYPE_HASH ||
t >= RDB_OPCODE_EXPIRETIME_MS;
printf("[offset %llu] %s\n",
(unsigned long long) (rdbstate.rio ?
rdbstate.rio->processed_bytes : 0), msg);
}
/* when number of bytes to read is negative, do a peek */
static int readBytes(void *target, long num) {
char peek = (num < 0) ? 1 : 0;
num = (num < 0) ? -num : num;
/* Used inside rdb.c in order to log specific errors happening inside
* the RDB loading internals. */
void rdbCheckSetError(const char *fmt, ...) {
va_list ap;
pos p = positions[level];
if (p.offset + num > p.size) {
return 0;
} else {
memcpy(target, (void*)((size_t)p.data + p.offset), num);
if (!peek) positions[level].offset += num;
}
return 1;
va_start(ap, fmt);
vsnprintf(rdbstate.error, sizeof(rdbstate.error), fmt, ap);
va_end(ap);
rdbstate.error_set = 1;
}
int processHeader(void) {
char buf[10] = "_________";
int dump_version;
/* During RDB check we setup a special signal handler for memory violations
* and similar conditions, so that we can log the offending part of the RDB
* if the crash is due to broken content. */
void rdbCheckHandleCrash(int sig, siginfo_t *info, void *secret) {
UNUSED(sig);
UNUSED(info);
UNUSED(secret);
if (!readBytes(buf, 9)) {
ERROR("Cannot read header");
}
/* expect the first 5 bytes to equal REDIS */
if (memcmp(buf,"REDIS",5) != 0) {
ERROR("Wrong signature in header");
}
dump_version = (int)strtol(buf + 5, NULL, 10);
if (dump_version < 1 || dump_version > 6) {
ERROR("Unknown RDB format version: %d", dump_version);
}
return dump_version;
rdbCheckError("Server crash checking the specified RDB file!");
exit(1);
}
static int loadType(entry *e) {
uint32_t offset = CURR_OFFSET;
void rdbCheckSetupSignals(void) {
struct sigaction act;
/* this byte needs to qualify as type */
unsigned char t;
if (readBytes(&t, 1)) {
if (rdbCheckType(t)) {
e->type = t;
return 1;
} else {
SHIFT_ERROR(offset, "Unknown type (0x%02x)", t);
}
} else {
SHIFT_ERROR(offset, "Could not read type");
}
/* failure */
return 0;
}
static int peekType() {
unsigned char t;
if (readBytes(&t, -1) && (rdbCheckType(t)))
return t;
return -1;
}
/* discard time, just consume the bytes */
static int processTime(int type) {
uint32_t offset = CURR_OFFSET;
unsigned char t[8];
int timelen = (type == RDB_OPCODE_EXPIRETIME_MS) ? 8 : 4;
if (readBytes(t,timelen)) {
return 1;
} else {
SHIFT_ERROR(offset, "Could not read time");
}
/* failure */
return 0;
}
static uint32_t loadLength(int *isencoded) {
unsigned char buf[2];
uint32_t len;
int type;
if (isencoded) *isencoded = 0;
if (!readBytes(buf, 1)) return RDB_LENERR;
type = (buf[0] & 0xC0) >> 6;
if (type == RDB_6BITLEN) {
/* Read a 6 bit len */
return buf[0] & 0x3F;
} else if (type == RDB_ENCVAL) {
/* Read a 6 bit len encoding type */
if (isencoded) *isencoded = 1;
return buf[0] & 0x3F;
} else if (type == RDB_14BITLEN) {
/* Read a 14 bit len */
if (!readBytes(buf+1,1)) return RDB_LENERR;
return ((buf[0] & 0x3F) << 8) | buf[1];
} else {
/* Read a 32 bit len */
if (!readBytes(&len, 4)) return RDB_LENERR;
return (unsigned int)ntohl(len);
}
}
static char *loadIntegerObject(int enctype) {
uint32_t offset = CURR_OFFSET;
unsigned char enc[4];
long long val;
if (enctype == RDB_ENC_INT8) {
uint8_t v;
if (!readBytes(enc, 1)) return NULL;
v = enc[0];
val = (int8_t)v;
} else if (enctype == RDB_ENC_INT16) {
uint16_t v;
if (!readBytes(enc, 2)) return NULL;
v = enc[0]|(enc[1]<<8);
val = (int16_t)v;
} else if (enctype == RDB_ENC_INT32) {
uint32_t v;
if (!readBytes(enc, 4)) return NULL;
v = enc[0]|(enc[1]<<8)|(enc[2]<<16)|(enc[3]<<24);
val = (int32_t)v;
} else {
SHIFT_ERROR(offset, "Unknown integer encoding (0x%02x)", enctype);
return NULL;
}
/* convert val into string */
char *buf;
buf = zmalloc(sizeof(char) * 128);
sprintf(buf, "%lld", val);
return buf;
}
static char* loadLzfStringObject() {
unsigned int slen, clen;
char *c, *s;
if ((clen = loadLength(NULL)) == RDB_LENERR) return NULL;
if ((slen = loadLength(NULL)) == RDB_LENERR) return NULL;
c = zmalloc(clen);
if (!readBytes(c, clen)) {
zfree(c);
return NULL;
}
s = zmalloc(slen+1);
if (lzf_decompress(c,clen,s,slen) == 0) {
zfree(c); zfree(s);
return NULL;
}
zfree(c);
return s;
}
/* returns NULL when not processable, char* when valid */
static char* loadStringObject() {
uint32_t offset = CURR_OFFSET;
int isencoded;
uint32_t len;
len = loadLength(&isencoded);
if (isencoded) {
switch(len) {
case RDB_ENC_INT8:
case RDB_ENC_INT16:
case RDB_ENC_INT32:
return loadIntegerObject(len);
case RDB_ENC_LZF:
return loadLzfStringObject();
default:
/* unknown encoding */
SHIFT_ERROR(offset, "Unknown string encoding (0x%02x)", len);
return NULL;
}
}
if (len == RDB_LENERR) return NULL;
char *buf = zmalloc(sizeof(char) * (len+1));
if (buf == NULL) return NULL;
buf[len] = '\0';
if (!readBytes(buf, len)) {
zfree(buf);
return NULL;
}
return buf;
}
static int processStringObject(char** store) {
unsigned long offset = CURR_OFFSET;
char *key = loadStringObject();
if (key == NULL) {
SHIFT_ERROR(offset, "Error reading string object");
zfree(key);
return 0;
}
if (store != NULL) {
*store = key;
} else {
zfree(key);
}
return 1;
}
static double* loadDoubleValue() {
char buf[256];
unsigned char len;
double* val;
if (!readBytes(&len,1)) return NULL;
val = zmalloc(sizeof(double));
switch(len) {
case 255: *val = R_NegInf; return val;
case 254: *val = R_PosInf; return val;
case 253: *val = R_Nan; return val;
default:
if (!readBytes(buf, len)) {
zfree(val);
return NULL;
}
buf[len] = '\0';
sscanf(buf, "%lg", val);
return val;
}
}
static int processDoubleValue(double** store) {
unsigned long offset = CURR_OFFSET;
double *val = loadDoubleValue();
if (val == NULL) {
SHIFT_ERROR(offset, "Error reading double value");
zfree(val);
return 0;
}
if (store != NULL) {
*store = val;
} else {
zfree(val);
}
return 1;
}
static int loadPair(entry *e) {
uint32_t offset = CURR_OFFSET;
uint32_t i;
/* read key first */
char *key;
if (processStringObject(&key)) {
e->key = key;
} else {
SHIFT_ERROR(offset, "Error reading entry key");
return 0;
}
uint32_t length = 0;
if (e->type == RDB_TYPE_LIST ||
e->type == RDB_TYPE_SET ||
e->type == RDB_TYPE_ZSET ||
e->type == RDB_TYPE_HASH) {
if ((length = loadLength(NULL)) == RDB_LENERR) {
SHIFT_ERROR(offset, "Error reading %s length", types[e->type]);
return 0;
}
}
switch(e->type) {
case RDB_TYPE_STRING:
case RDB_TYPE_HASH_ZIPMAP:
case RDB_TYPE_LIST_ZIPLIST:
case RDB_TYPE_SET_INTSET:
case RDB_TYPE_ZSET_ZIPLIST:
case RDB_TYPE_HASH_ZIPLIST:
if (!processStringObject(NULL)) {
SHIFT_ERROR(offset, "Error reading entry value");
return 0;
}
break;
case RDB_TYPE_LIST:
case RDB_TYPE_SET:
for (i = 0; i < length; i++) {
offset = CURR_OFFSET;
if (!processStringObject(NULL)) {
SHIFT_ERROR(offset, "Error reading element at index %d (length: %d)", i, length);
return 0;
}
}
break;
case RDB_TYPE_ZSET:
for (i = 0; i < length; i++) {
offset = CURR_OFFSET;
if (!processStringObject(NULL)) {
SHIFT_ERROR(offset, "Error reading element key at index %d (length: %d)", i, length);
return 0;
}
offset = CURR_OFFSET;
if (!processDoubleValue(NULL)) {
SHIFT_ERROR(offset, "Error reading element value at index %d (length: %d)", i, length);
return 0;
}
}
break;
case RDB_TYPE_HASH:
for (i = 0; i < length; i++) {
offset = CURR_OFFSET;
if (!processStringObject(NULL)) {
SHIFT_ERROR(offset, "Error reading element key at index %d (length: %d)", i, length);
return 0;
}
offset = CURR_OFFSET;
if (!processStringObject(NULL)) {
SHIFT_ERROR(offset, "Error reading element value at index %d (length: %d)", i, length);
return 0;
}
}
break;
default:
SHIFT_ERROR(offset, "Type not implemented");
return 0;
}
/* because we're done, we assume success */
e->success = 1;
return 1;
}
static entry loadEntry() {
entry e = { NULL, -1, 0 };
uint32_t length, offset[4];
/* reset error container */
errors.level = 0;
offset[0] = CURR_OFFSET;
if (!loadType(&e)) {
return e;
}
offset[1] = CURR_OFFSET;
if (e.type == RDB_OPCODE_SELECTDB) {
if ((length = loadLength(NULL)) == RDB_LENERR) {
SHIFT_ERROR(offset[1], "Error reading database number");
return e;
}
if (length > 63) {
SHIFT_ERROR(offset[1], "Database number out of range (%d)", length);
return e;
}
} else if (e.type == RDB_OPCODE_EOF) {
if (positions[level].offset < positions[level].size) {
SHIFT_ERROR(offset[0], "Unexpected EOF");
} else {
e.success = 1;
}
return e;
} else {
/* optionally consume expire */
if (e.type == RDB_OPCODE_EXPIRETIME ||
e.type == RDB_OPCODE_EXPIRETIME_MS) {
if (!processTime(e.type)) return e;
if (!loadType(&e)) return e;
}
offset[1] = CURR_OFFSET;
if (!loadPair(&e)) {
SHIFT_ERROR(offset[1], "Error for type %s", types[e.type]);
return e;
}
}
/* all entries are followed by a valid type:
* e.g. a new entry, SELECTDB, EXPIRE, EOF */
offset[2] = CURR_OFFSET;
if (peekType() == -1) {
SHIFT_ERROR(offset[2], "Followed by invalid type");
SHIFT_ERROR(offset[0], "Error for type %s", types[e.type]);
e.success = 0;
} else {
e.success = 1;
}
return e;
}
static void printCentered(int indent, int width, char* body) {
char head[256], tail[256];
memset(head, '\0', 256);
memset(tail, '\0', 256);
memset(head, '=', indent);
memset(tail, '=', width - 2 - indent - strlen(body));
serverLog(LL_WARNING, "%s %s %s", head, body, tail);
}
static void printValid(uint64_t ops, uint64_t bytes) {
char body[80];
sprintf(body, "Processed %llu valid opcodes (in %llu bytes)",
(unsigned long long) ops, (unsigned long long) bytes);
printCentered(4, 80, body);
}
static void printSkipped(uint64_t bytes, uint64_t offset) {
char body[80];
sprintf(body, "Skipped %llu bytes (resuming at 0x%08llx)",
(unsigned long long) bytes, (unsigned long long) offset);
printCentered(4, 80, body);
}
static void printErrorStack(entry *e) {
unsigned int i;
char body[64];
if (e->type == -1) {
sprintf(body, "Error trace");
} else if (e->type >= 253) {
sprintf(body, "Error trace (%s)", types[e->type]);
} else if (!e->key) {
sprintf(body, "Error trace (%s: (unknown))", types[e->type]);
} else {
char tmp[41];
strncpy(tmp, e->key, 40);
/* display truncation at the last 3 chars */
if (strlen(e->key) > 40) {
memset(&tmp[37], '.', 3);
}
/* display unprintable characters as ? */
for (i = 0; i < strlen(tmp); i++) {
if (tmp[i] <= 32) tmp[i] = '?';
}
sprintf(body, "Error trace (%s: %s)", types[e->type], tmp);
}
printCentered(4, 80, body);
/* display error stack */
for (i = 0; i < errors.level; i++) {
serverLog(LL_WARNING, "0x%08lx - %s",
(unsigned long) errors.offset[i], errors.error[i]);
}
}
void process(void) {
uint64_t num_errors = 0, num_valid_ops = 0, num_valid_bytes = 0;
entry entry = { NULL, -1, 0 };
int dump_version = processHeader();
/* Exclude the final checksum for RDB >= 5. Will be checked at the end. */
if (dump_version >= 5) {
if (positions[0].size < 8) {
serverLog(LL_WARNING, "RDB version >= 5 but no room for checksum.");
exit(1);
}
positions[0].size -= 8;
}
level = 1;
while(positions[0].offset < positions[0].size) {
positions[1] = positions[0];
entry = loadEntry();
if (!entry.success) {
printValid(num_valid_ops, num_valid_bytes);
printErrorStack(&entry);
num_errors++;
num_valid_ops = 0;
num_valid_bytes = 0;
/* search for next valid entry */
uint64_t offset = positions[0].offset + 1;
int i = 0;
while (!entry.success && offset < positions[0].size) {
positions[1].offset = offset;
/* find 3 consecutive valid entries */
for (i = 0; i < 3; i++) {
entry = loadEntry();
if (!entry.success) break;
}
/* check if we found 3 consecutive valid entries */
if (i < 3) {
offset++;
}
}
/* print how many bytes we have skipped to find a new valid opcode */
if (offset < positions[0].size) {
printSkipped(offset - positions[0].offset, offset);
}
positions[0].offset = offset;
} else {
num_valid_ops++;
num_valid_bytes += positions[1].offset - positions[0].offset;
/* advance position */
positions[0] = positions[1];
}
zfree(entry.key);
}
/* because there is another potential error,
* print how many valid ops we have processed */
printValid(num_valid_ops, num_valid_bytes);
/* expect an eof */
if (entry.type != RDB_OPCODE_EOF) {
/* last byte should be EOF, add error */
errors.level = 0;
SHIFT_ERROR(positions[0].offset, "Expected EOF, got %s", types[entry.type]);
/* this is an EOF error so reset type */
entry.type = -1;
printErrorStack(&entry);
num_errors++;
}
/* Verify checksum */
if (dump_version >= 5) {
uint64_t crc = crc64(0,positions[0].data,positions[0].size);
uint64_t crc2;
unsigned char *p = (unsigned char*)positions[0].data+positions[0].size;
crc2 = ((uint64_t)p[0] << 0) |
((uint64_t)p[1] << 8) |
((uint64_t)p[2] << 16) |
((uint64_t)p[3] << 24) |
((uint64_t)p[4] << 32) |
((uint64_t)p[5] << 40) |
((uint64_t)p[6] << 48) |
((uint64_t)p[7] << 56);
if (crc != crc2) {
SHIFT_ERROR(positions[0].offset, "RDB CRC64 does not match.");
} else {
serverLog(LL_WARNING, "CRC64 checksum is OK");
}
}
/* print summary on errors */
if (num_errors) {
serverLog(LL_WARNING, "Total unprocessable opcodes: %llu",
(unsigned long long) num_errors);
}
sigemptyset(&act.sa_mask);
act.sa_flags = SA_NODEFER | SA_RESETHAND | SA_SIGINFO;
act.sa_sigaction = rdbCheckHandleCrash;
sigaction(SIGSEGV, &act, NULL);
sigaction(SIGBUS, &act, NULL);
sigaction(SIGFPE, &act, NULL);
sigaction(SIGILL, &act, NULL);
}
/* Check the specified RDB file. */
int redis_check_rdb(char *rdbfilename) {
int fd;
off_t size;
struct stat stat;
void *data;
uint64_t dbid;
int type, rdbver;
char buf[1024];
long long expiretime, now = mstime();
FILE *fp;
rio rdb;
fd = open(rdbfilename, O_RDONLY);
if (fd < 1) {
ERROR("Cannot open file: %s", rdbfilename);
if ((fp = fopen(rdbfilename,"r")) == NULL) return C_ERR;
rioInitWithFile(&rdb,fp);
rdbstate.rio = &rdb;
rdb.update_cksum = rdbLoadProgressCallback;
if (rioRead(&rdb,buf,9) == 0) goto eoferr;
buf[9] = '\0';
if (memcmp(buf,"REDIS",5) != 0) {
rdbCheckError("Wrong signature trying to load DB from file");
return 1;
}
if (fstat(fd, &stat) == -1) {
ERROR("Cannot stat: %s", rdbfilename);
} else {
size = stat.st_size;
rdbver = atoi(buf+5);
if (rdbver < 1 || rdbver > RDB_VERSION) {
rdbCheckError("Can't handle RDB format version %d",rdbver);
return 1;
}
if (sizeof(size_t) == sizeof(int32_t) && size >= INT_MAX) {
ERROR("Cannot check dump files >2GB on a 32-bit platform");
startLoading(fp);
while(1) {
robj *key, *val;
expiretime = -1;
/* Read type. */
rdbstate.doing = RDB_CHECK_DOING_READ_TYPE;
if ((type = rdbLoadType(&rdb)) == -1) goto eoferr;
/* Handle special types. */
if (type == RDB_OPCODE_EXPIRETIME) {
rdbstate.doing = RDB_CHECK_DOING_READ_EXPIRE;
/* EXPIRETIME: load an expire associated with the next key
* to load. Note that after loading an expire we need to
* load the actual type, and continue. */
if ((expiretime = rdbLoadTime(&rdb)) == -1) goto eoferr;
/* We read the time so we need to read the object type again. */
rdbstate.doing = RDB_CHECK_DOING_READ_TYPE;
if ((type = rdbLoadType(&rdb)) == -1) goto eoferr;
/* the EXPIRETIME opcode specifies time in seconds, so convert
* into milliseconds. */
expiretime *= 1000;
} else if (type == RDB_OPCODE_EXPIRETIME_MS) {
/* EXPIRETIME_MS: milliseconds precision expire times introduced
* with RDB v3. Like EXPIRETIME but no with more precision. */
rdbstate.doing = RDB_CHECK_DOING_READ_EXPIRE;
if ((expiretime = rdbLoadMillisecondTime(&rdb)) == -1) goto eoferr;
/* We read the time so we need to read the object type again. */
rdbstate.doing = RDB_CHECK_DOING_READ_TYPE;
if ((type = rdbLoadType(&rdb)) == -1) goto eoferr;
} else if (type == RDB_OPCODE_EOF) {
/* EOF: End of file, exit the main loop. */
break;
} else if (type == RDB_OPCODE_SELECTDB) {
/* SELECTDB: Select the specified database. */
rdbstate.doing = RDB_CHECK_DOING_READ_LEN;
if ((dbid = rdbLoadLen(&rdb,NULL)) == RDB_LENERR)
goto eoferr;
rdbCheckInfo("Selecting DB ID %d", dbid);
continue; /* Read type again. */
} else if (type == RDB_OPCODE_RESIZEDB) {
/* RESIZEDB: Hint about the size of the keys in the currently
* selected data base, in order to avoid useless rehashing. */
uint64_t db_size, expires_size;
rdbstate.doing = RDB_CHECK_DOING_READ_LEN;
if ((db_size = rdbLoadLen(&rdb,NULL)) == RDB_LENERR)
goto eoferr;
if ((expires_size = rdbLoadLen(&rdb,NULL)) == RDB_LENERR)
goto eoferr;
continue; /* Read type again. */
} else if (type == RDB_OPCODE_AUX) {
/* AUX: generic string-string fields. Use to add state to RDB
* which is backward compatible. Implementations of RDB loading
* are requierd to skip AUX fields they don't understand.
*
* An AUX field is composed of two strings: key and value. */
robj *auxkey, *auxval;
rdbstate.doing = RDB_CHECK_DOING_READ_AUX;
if ((auxkey = rdbLoadStringObject(&rdb)) == NULL) goto eoferr;
if ((auxval = rdbLoadStringObject(&rdb)) == NULL) goto eoferr;
rdbCheckInfo("AUX FIELD %s = '%s'",
(char*)auxkey->ptr, (char*)auxval->ptr);
decrRefCount(auxkey);
decrRefCount(auxval);
continue; /* Read type again. */
} else {
if (!rdbIsObjectType(type)) {
rdbCheckError("Invalid object type: %d", type);
return 1;
}
rdbstate.key_type = type;
}
/* Read key */
rdbstate.doing = RDB_CHECK_DOING_READ_KEY;
if ((key = rdbLoadStringObject(&rdb)) == NULL) goto eoferr;
rdbstate.key = key;
rdbstate.keys++;
/* Read value */
rdbstate.doing = RDB_CHECK_DOING_READ_OBJECT_VALUE;
if ((val = rdbLoadObject(type,&rdb)) == NULL) goto eoferr;
/* Check if the key already expired. This function is used when loading
* an RDB file from disk, either at startup, or when an RDB was
* received from the master. In the latter case, the master is
* responsible for key expiry. If we would expire keys here, the
* snapshot taken by the master may not be reflected on the slave. */
if (server.masterhost == NULL && expiretime != -1 && expiretime < now)
rdbstate.already_expired++;
if (expiretime != -1) rdbstate.expires++;
rdbstate.key = NULL;
decrRefCount(key);
decrRefCount(val);
rdbstate.key_type = -1;
}
/* Verify the checksum if RDB version is >= 5 */
if (rdbver >= 5 && server.rdb_checksum) {
uint64_t cksum, expected = rdb.cksum;
rdbstate.doing = RDB_CHECK_DOING_CHECK_SUM;
if (rioRead(&rdb,&cksum,8) == 0) goto eoferr;
memrev64ifbe(&cksum);
if (cksum == 0) {
rdbCheckInfo("RDB file was saved with checksum disabled: no check performed.");
} else if (cksum != expected) {
rdbCheckError("RDB CRC error");
} else {
rdbCheckInfo("Checksum OK");
}
}
data = mmap(NULL, size, PROT_READ, MAP_SHARED, fd, 0);
if (data == MAP_FAILED) {
ERROR("Cannot mmap: %s", rdbfilename);
}
/* Initialize static vars */
positions[0].data = data;
positions[0].size = size;
positions[0].offset = 0;
errors.level = 0;
/* Object types */
sprintf(types[RDB_TYPE_STRING], "STRING");
sprintf(types[RDB_TYPE_LIST], "LIST");
sprintf(types[RDB_TYPE_SET], "SET");
sprintf(types[RDB_TYPE_ZSET], "ZSET");
sprintf(types[RDB_TYPE_HASH], "HASH");
/* Object types only used for dumping to disk */
sprintf(types[RDB_OPCODE_EXPIRETIME], "EXPIRETIME");
sprintf(types[RDB_OPCODE_SELECTDB], "SELECTDB");
sprintf(types[RDB_OPCODE_EOF], "EOF");
process();
munmap(data, size);
close(fd);
fclose(fp);
return 0;
eoferr: /* unexpected end of file is handled here with a fatal exit */
if (rdbstate.error_set) {
rdbCheckError(rdbstate.error);
} else {
rdbCheckError("Unexpected EOF reading RDB file");
}
return 1;
}
/* RDB check main: called form redis.c when Redis is executed with the
* redis-check-rdb alias. */
int redis_check_rdb_main(char **argv, int argc) {
* redis-check-rdb alias.
*
* The function never returns, but exits with the status code according
* to success (RDB is sane) or error (RDB is corrupted). */
int redis_check_rdb_main(int argc, char **argv) {
if (argc != 2) {
fprintf(stderr, "Usage: %s <rdb-file-name>\n", argv[0]);
exit(1);
}
serverLog(LL_WARNING, "Checking RDB file %s", argv[1]);
exit(redis_check_rdb(argv[1]));
return 0;
createSharedObjects(); /* Needed for loading. */
server.loading_process_events_interval_bytes = 0;
rdbCheckMode = 1;
rdbCheckInfo("Checking RDB file %s", argv[1]);
rdbCheckSetupSignals();
int retval = redis_check_rdb(argv[1]);
if (retval == 0) {
rdbCheckInfo("\\o/ RDB looks OK! \\o/");
rdbShowGenericInfo();
}
exit(retval);
}
src/redis-cli.c
+107 -29
View File
@@ -130,6 +130,7 @@ static void usage(void);
static void slaveMode(void);
char *redisGitSHA1(void);
char *redisGitDirty(void);
static int cliConnect(int force);
/*------------------------------------------------------------------------------
* Utility functions
@@ -150,20 +151,25 @@ static long long mstime(void) {
}
static void cliRefreshPrompt(void) {
int len;
if (config.eval_ldb) return;
if (config.hostsocket != NULL)
len = snprintf(config.prompt,sizeof(config.prompt),"redis %s",
config.hostsocket);
else
len = anetFormatAddr(config.prompt, sizeof(config.prompt),
config.hostip, config.hostport);
sds prompt = sdsempty();
if (config.hostsocket != NULL) {
prompt = sdscatfmt(prompt,"redis %s",config.hostsocket);
} else {
char addr[256];
anetFormatAddr(addr, sizeof(addr), config.hostip, config.hostport);
prompt = sdscatlen(prompt,addr,strlen(addr));
}
/* Add [dbnum] if needed */
if (config.dbnum != 0 && config.last_cmd_type != REDIS_REPLY_ERROR)
len += snprintf(config.prompt+len,sizeof(config.prompt)-len,"[%d]",
config.dbnum);
snprintf(config.prompt+len,sizeof(config.prompt)-len,"> ");
if (config.dbnum != 0)
prompt = sdscatfmt(prompt,"[%i]",config.dbnum);
/* Copy the prompt in the static buffer. */
prompt = sdscatlen(prompt,"> ",2);
snprintf(config.prompt,sizeof(config.prompt),"%s",prompt);
sdsfree(prompt);
}
/* Return the name of the dotfile for the specified 'dotfilename'.
@@ -238,11 +244,11 @@ static void cliInitHelp(void) {
helpEntry tmp;
helpEntriesLen = len = commandslen+groupslen;
helpEntries = malloc(sizeof(helpEntry)*len);
helpEntries = zmalloc(sizeof(helpEntry)*len);
for (i = 0; i < groupslen; i++) {
tmp.argc = 1;
tmp.argv = malloc(sizeof(sds));
tmp.argv = zmalloc(sizeof(sds));
tmp.argv[0] = sdscatprintf(sdsempty(),"@%s",commandGroups[i]);
tmp.full = tmp.argv[0];
tmp.type = CLI_HELP_GROUP;
@@ -259,6 +265,65 @@ static void cliInitHelp(void) {
}
}
/* cliInitHelp() setups the helpEntries array with the command and group
* names from the help.h file. However the Redis instance we are connecting
* to may support more commands, so this function integrates the previous
* entries with additional entries obtained using the COMMAND command
* available in recent versions of Redis. */
static void cliIntegrateHelp(void) {
if (cliConnect(0) == REDIS_ERR) return;
redisReply *reply = redisCommand(context, "COMMAND");
if(reply == NULL || reply->type != REDIS_REPLY_ARRAY) return;
/* Scan the array reported by COMMAND and fill only the entries that
* don't already match what we have. */
for (size_t j = 0; j < reply->elements; j++) {
redisReply *entry = reply->element[j];
if (entry->type != REDIS_REPLY_ARRAY || entry->elements < 4 ||
entry->element[0]->type != REDIS_REPLY_STRING ||
entry->element[1]->type != REDIS_REPLY_INTEGER ||
entry->element[3]->type != REDIS_REPLY_INTEGER) return;
char *cmdname = entry->element[0]->str;
int i;
for (i = 0; i < helpEntriesLen; i++) {
helpEntry *he = helpEntries+i;
if (!strcasecmp(he->argv[0],cmdname))
break;
}
if (i != helpEntriesLen) continue;
helpEntriesLen++;
helpEntries = zrealloc(helpEntries,sizeof(helpEntry)*helpEntriesLen);
helpEntry *new = helpEntries+(helpEntriesLen-1);
new->argc = 1;
new->argv = zmalloc(sizeof(sds));
new->argv[0] = sdsnew(cmdname);
new->full = new->argv[0];
new->type = CLI_HELP_COMMAND;
sdstoupper(new->argv[0]);
struct commandHelp *ch = zmalloc(sizeof(*ch));
ch->name = new->argv[0];
ch->params = sdsempty();
int args = llabs(entry->element[1]->integer);
if (entry->element[3]->integer == 1) {
ch->params = sdscat(ch->params,"key ");
args--;
}
while(args--) ch->params = sdscat(ch->params,"arg ");
if (entry->element[1]->integer < 0)
ch->params = sdscat(ch->params,"...options...");
ch->summary = "Help not available";
ch->group = 0;
ch->since = "not known";
new->org = ch;
}
freeReplyObject(reply);
}
/* Output command help to stdout. */
static void cliOutputCommandHelp(struct commandHelp *help, int group) {
printf("\r\n \x1b[1m%s\x1b[0m \x1b[90m%s\x1b[0m\r\n", help->name, help->params);
@@ -828,7 +893,7 @@ static int cliSendCommand(int argc, char **argv, int repeat) {
}
/* Setup argument length */
argvlen = malloc(argc*sizeof(size_t));
argvlen = zmalloc(argc*sizeof(size_t));
for (j = 0; j < argc; j++)
argvlen[j] = sdslen(argv[j]);
@@ -851,16 +916,16 @@ static int cliSendCommand(int argc, char **argv, int repeat) {
printf("Entering slave output mode... (press Ctrl-C to quit)\n");
slaveMode();
config.slave_mode = 0;
free(argvlen);
zfree(argvlen);
return REDIS_ERR; /* Error = slaveMode lost connection to master */
}
if (cliReadReply(output_raw) != REDIS_OK) {
free(argvlen);
zfree(argvlen);
return REDIS_ERR;
} else {
/* Store database number when SELECT was successfully executed. */
if (!strcasecmp(command,"select") && argc == 2) {
if (!strcasecmp(command,"select") && argc == 2 && config.last_cmd_type != REDIS_REPLY_ERROR) {
config.dbnum = atoi(argv[1]);
cliRefreshPrompt();
} else if (!strcasecmp(command,"auth") && argc == 2) {
@@ -871,7 +936,7 @@ static int cliSendCommand(int argc, char **argv, int repeat) {
fflush(stdout); /* Make it grep friendly */
}
free(argvlen);
zfree(argvlen);
return REDIS_OK;
}
@@ -1199,6 +1264,7 @@ void cliLoadPreferences(void) {
if (argc > 0) cliSetPreferences(argv,argc,0);
sdsfreesplitres(argv,argc);
}
fclose(fp);
}
sdsfree(rcfile);
}
@@ -1210,6 +1276,11 @@ static void repl(void) {
int argc;
sds *argv;
/* Initialize the help and, if possible, use the COMMAND command in order
* to retrieve missing entries. */
cliInitHelp();
cliIntegrateHelp();
config.interactive = 1;
linenoiseSetMultiLine(1);
linenoiseSetCompletionCallback(completionCallback);
@@ -1235,7 +1306,7 @@ static void repl(void) {
if (argv == NULL) {
printf("Invalid argument(s)\n");
free(line);
linenoiseFree(line);
continue;
} else if (argc > 0) {
if (strcasecmp(argv[0],"quit") == 0 ||
@@ -1264,9 +1335,10 @@ static void repl(void) {
} else {
long long start_time = mstime(), elapsed;
int repeat, skipargs = 0;
char *endptr;
repeat = atoi(argv[0]);
if (argc > 1 && repeat) {
repeat = strtol(argv[0], &endptr, 10);
if (argc > 1 && *endptr == '\0' && repeat) {
skipargs = 1;
} else {
repeat = 1;
@@ -1285,7 +1357,9 @@ static void repl(void) {
}
elapsed = mstime()-start_time;
if (elapsed >= 500) {
if (elapsed >= 500 &&
config.output == OUTPUT_STANDARD)
{
printf("(%.2fs)\n",(double)elapsed/1000);
}
}
@@ -1294,7 +1368,7 @@ static void repl(void) {
sdsfreesplitres(argv,argc);
}
/* linenoise() returns malloc-ed lines like readline() */
free(line);
linenoiseFree(line);
}
exit(0);
}
@@ -1956,8 +2030,13 @@ static void getKeyTypes(redisReply *keys, int *types) {
keys->element[i]->str, context->err, context->errstr);
exit(1);
} else if(reply->type != REDIS_REPLY_STATUS) {
fprintf(stderr, "Invalid reply type (%d) for TYPE on key '%s'!\n",
reply->type, keys->element[i]->str);
if(reply->type == REDIS_REPLY_ERROR) {
fprintf(stderr, "TYPE returned an error: %s\n", reply->str);
} else {
fprintf(stderr,
"Invalid reply type (%d) for TYPE on key '%s'!\n",
reply->type, keys->element[i]->str);
}
exit(1);
}
@@ -2159,7 +2238,7 @@ static char *getInfoField(char *info, char *field) {
n1 = strchr(p,'\r');
n2 = strchr(p,',');
if (n2 && n2 < n1) n1 = n2;
result = malloc(sizeof(char)*(n1-p)+1);
result = zmalloc(sizeof(char)*(n1-p)+1);
memcpy(result,p,(n1-p));
result[n1-p] = '\0';
return result;
@@ -2173,7 +2252,7 @@ static long getLongInfoField(char *info, char *field) {
if (!value) return LONG_MIN;
l = strtol(value,NULL,10);
free(value);
zfree(value);
return l;
}
@@ -2535,7 +2614,6 @@ int main(int argc, char **argv) {
else
config.output = OUTPUT_STANDARD;
config.mb_delim = sdsnew("\n");
cliInitHelp();
firstarg = parseOptions(argc,argv);
argc -= firstarg;
src/redis-trib.rb
+5
View File
@@ -1305,6 +1305,11 @@ class RedisTrib
sleep 1
wait_cluster_join
flush_nodes_config # Useful for the replicas
# Reset the node information, so that when the
# final summary is listed in check_cluster about the newly created cluster
# all the nodes would get properly listed as slaves or masters
reset_nodes
load_cluster_info_from_node(argv[0])
check_cluster
end
src/replication.c
+88 -20
View File
@@ -47,7 +47,7 @@ int cancelReplicationHandshake(void);
/* Return the pointer to a string representing the slave ip:listening_port
* pair. Mostly useful for logging, since we want to log a slave using its
* IP address and it's listening port which is more clear for the user, for
* IP address and its listening port which is more clear for the user, for
* example: "Closing connection with slave 10.1.2.3:6380". */
char *replicationGetSlaveName(client *c) {
static char buf[NET_PEER_ID_LEN];
@@ -55,7 +55,12 @@ char *replicationGetSlaveName(client *c) {
ip[0] = '\0';
buf[0] = '\0';
if (anetPeerToString(c->fd,ip,sizeof(ip),NULL) != -1) {
if (c->slave_ip[0] != '\0' ||
anetPeerToString(c->fd,ip,sizeof(ip),NULL) != -1)
{
/* Note that the 'ip' buffer is always larger than 'c->slave_ip' */
if (c->slave_ip[0] != '\0') memcpy(ip,c->slave_ip,sizeof(c->slave_ip));
if (c->slave_listening_port)
anetFormatAddr(buf,sizeof(buf),ip,c->slave_listening_port);
else
@@ -647,7 +652,7 @@ void syncCommand(client *c) {
} else {
/* No way, we need to wait for the next BGSAVE in order to
* register differences. */
serverLog(LL_NOTICE,"Waiting for next BGSAVE for SYNC");
serverLog(LL_NOTICE,"Can't attach the slave to the current BGSAVE. Waiting for next BGSAVE for SYNC");
}
/* CASE 2: BGSAVE is in progress, with socket target. */
@@ -657,7 +662,7 @@ void syncCommand(client *c) {
/* There is an RDB child process but it is writing directly to
* children sockets. We need to wait for the next BGSAVE
* in order to synchronize. */
serverLog(LL_NOTICE,"Waiting for next BGSAVE for SYNC");
serverLog(LL_NOTICE,"Current BGSAVE has socket target. Waiting for next BGSAVE for SYNC");
/* CASE 3: There is no BGSAVE is progress. */
} else {
@@ -666,12 +671,18 @@ void syncCommand(client *c) {
* replicationCron() since we want to delay its start a
* few seconds to wait for more slaves to arrive. */
if (server.repl_diskless_sync_delay)
serverLog(LL_NOTICE,"Delay next BGSAVE for SYNC");
serverLog(LL_NOTICE,"Delay next BGSAVE for diskless SYNC");
} else {
/* Target is disk (or the slave is not capable of supporting
* diskless replication) and we don't have a BGSAVE in progress,
* let's start one. */
if (startBgsaveForReplication(c->slave_capa) != C_OK) return;
if (server.aof_child_pid == -1) {
startBgsaveForReplication(c->slave_capa);
} else {
serverLog(LL_NOTICE,
"No BGSAVE in progress, but an AOF rewrite is active. "
"BGSAVE for replication delayed");
}
}
}
@@ -711,6 +722,15 @@ void replconfCommand(client *c) {
&port,NULL) != C_OK))
return;
c->slave_listening_port = port;
} else if (!strcasecmp(c->argv[j]->ptr,"ip-address")) {
sds ip = c->argv[j+1]->ptr;
if (sdslen(ip) < sizeof(c->slave_ip)) {
memcpy(c->slave_ip,ip,sdslen(ip)+1);
} else {
addReplyErrorFormat(c,"REPLCONF ip-address provided by "
"slave instance is too long: %zd bytes", sdslen(ip));
return;
}
} else if (!strcasecmp(c->argv[j]->ptr,"capa")) {
/* Ignore capabilities not understood by this master. */
if (!strcasecmp(c->argv[j+1]->ptr,"eof"))
@@ -1365,7 +1385,7 @@ int slaveTryPartialResynchronization(int fd, int read_reply) {
void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
char tmpfile[256], *err = NULL;
int dfd, maxtries = 5;
int dfd = -1, maxtries = 5;
int sockerr = 0, psync_result;
socklen_t errlen = sizeof(sockerr);
UNUSED(el);
@@ -1453,7 +1473,8 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
/* Set the slave port, so that Master's INFO command can list the
* slave listening port correctly. */
if (server.repl_state == REPL_STATE_SEND_PORT) {
sds port = sdsfromlonglong(server.port);
sds port = sdsfromlonglong(server.slave_announce_port ?
server.slave_announce_port : server.port);
err = sendSynchronousCommand(SYNC_CMD_WRITE,fd,"REPLCONF",
"listening-port",port, NULL);
sdsfree(port);
@@ -1473,6 +1494,37 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
"REPLCONF listening-port: %s", err);
}
sdsfree(err);
server.repl_state = REPL_STATE_SEND_IP;
}
/* Skip REPLCONF ip-address if there is no slave-announce-ip option set. */
if (server.repl_state == REPL_STATE_SEND_IP &&
server.slave_announce_ip == NULL)
{
server.repl_state = REPL_STATE_SEND_CAPA;
}
/* Set the slave ip, so that Master's INFO command can list the
* slave IP address port correctly in case of port forwarding or NAT. */
if (server.repl_state == REPL_STATE_SEND_IP) {
err = sendSynchronousCommand(SYNC_CMD_WRITE,fd,"REPLCONF",
"ip-address",server.slave_announce_ip, NULL);
if (err) goto write_error;
sdsfree(err);
server.repl_state = REPL_STATE_RECEIVE_IP;
return;
}
/* Receive REPLCONF ip-address reply. */
if (server.repl_state == REPL_STATE_RECEIVE_IP) {
err = sendSynchronousCommand(SYNC_CMD_READ,fd,NULL);
/* Ignore the error if any, not all the Redis versions support
* REPLCONF listening-port. */
if (err[0] == '-') {
serverLog(LL_NOTICE,"(Non critical) Master does not understand "
"REPLCONF ip-address: %s", err);
}
sdsfree(err);
server.repl_state = REPL_STATE_SEND_CAPA;
}
@@ -1588,6 +1640,7 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) {
error:
aeDeleteFileEvent(server.el,fd,AE_READABLE|AE_WRITABLE);
if (dfd != -1) close(dfd);
close(fd);
server.repl_transfer_s = -1;
server.repl_state = REPL_STATE_CONNECT;
@@ -1740,6 +1793,14 @@ void slaveofCommand(client *c) {
} else {
long port;
if (c->flags & CLIENT_SLAVE)
{
/* If a client is already a replica they cannot run this command,
* because it involves flushing all replicas (including this client) */
addReplyError(c, "Command is not valid when client is a replica.");
return;
}
if ((getLongFromObjectOrReply(c, c->argv[2], &port, NULL) != C_OK))
return;
@@ -1778,12 +1839,16 @@ void roleCommand(client *c) {
listRewind(server.slaves,&li);
while((ln = listNext(&li))) {
client *slave = ln->value;
char ip[NET_IP_STR_LEN];
char ip[NET_IP_STR_LEN], *slaveip = slave->slave_ip;
if (anetPeerToString(slave->fd,ip,sizeof(ip),NULL) == -1) continue;
if (slaveip[0] == '\0') {
if (anetPeerToString(slave->fd,ip,sizeof(ip),NULL) == -1)
continue;
slaveip = ip;
}
if (slave->replstate != SLAVE_STATE_ONLINE) continue;
addReplyMultiBulkLen(c,3);
addReplyBulkCString(c,ip);
addReplyBulkCString(c,slaveip);
addReplyBulkLongLong(c,slave->slave_listening_port);
addReplyBulkLongLong(c,slave->repl_ack_off);
slaves++;
@@ -2304,13 +2369,12 @@ void replicationCron(void) {
replicationScriptCacheFlush();
}
/* If we are using diskless replication and there are slaves waiting
* in WAIT_BGSAVE_START state, check if enough seconds elapsed and
* start a BGSAVE.
/* Start a BGSAVE good for replication if we have slaves in
* WAIT_BGSAVE_START state.
*
* This code is also useful to trigger a BGSAVE if the diskless
* replication was turned off with CONFIG SET, while there were already
* slaves in WAIT_BGSAVE_START state. */
* In case of diskless replication, we make sure to wait the specified
* number of seconds (according to configuration) so that other slaves
* have the time to arrive before we start streaming. */
if (server.rdb_child_pid == -1 && server.aof_child_pid == -1) {
time_t idle, max_idle = 0;
int slaves_waiting = 0;
@@ -2330,9 +2394,13 @@ void replicationCron(void) {
}
}
if (slaves_waiting && max_idle > server.repl_diskless_sync_delay) {
/* Start a BGSAVE. Usually with socket target, or with disk target
* if there was a recent socket -> disk config change. */
if (slaves_waiting &&
(!server.repl_diskless_sync ||
max_idle > server.repl_diskless_sync_delay))
{
/* Start the BGSAVE. The called function may start a
* BGSAVE with socket target or disk target depending on the
* configuration and slaves capabilities. */
startBgsaveForReplication(mincapa);
}
}
src/scripting.c
+4 -2
View File
@@ -443,6 +443,7 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) {
if (j == 10) {
cmdlog = sdscatprintf(cmdlog," ... (%d more)",
c->argc-j-1);
break;
} else {
cmdlog = sdscatlen(cmdlog," ",1);
cmdlog = sdscatsds(cmdlog,c->argv[j]->ptr);
@@ -839,6 +840,8 @@ void luaLoadLibraries(lua_State *lua) {
void luaRemoveUnsupportedFunctions(lua_State *lua) {
lua_pushnil(lua);
lua_setglobal(lua,"loadfile");
lua_pushnil(lua);
lua_setglobal(lua,"dofile");
}
/* This function installs metamethods in the global table _G that prevent
@@ -898,7 +901,6 @@ void scriptingInit(int setup) {
server.lua_caller = NULL;
server.lua_timedout = 0;
server.lua_always_replicate_commands = 0; /* Only DEBUG can change it.*/
server.lua_time_limit = LUA_SCRIPT_TIME_LIMIT;
ldbInit();
}
@@ -2270,7 +2272,7 @@ ldbLog(sdsnew("[e]eval <code> Execute some Lua code (in a different callfr
ldbLog(sdsnew("[r]edis <cmd> Execute a Redis command."));
ldbLog(sdsnew("[m]axlen [len] Trim logged Redis replies and Lua var dumps to len."));
ldbLog(sdsnew(" Specifying zero as <len> means unlimited."));
ldbLog(sdsnew("[a]abort Stop the execution of the script. In sync"));
ldbLog(sdsnew("[a]bort Stop the execution of the script. In sync"));
ldbLog(sdsnew(" mode dataset changes will be retained."));
ldbLog(sdsnew(""));
ldbLog(sdsnew("Debugger functions you can call from Lua scripts:"));
src/sds.c
+4 -4
View File
@@ -55,13 +55,13 @@ static inline int sdsHdrSize(char type) {
}
static inline char sdsReqType(size_t string_size) {
if (string_size < 32)
if (string_size < 1<<5)
return SDS_TYPE_5;
if (string_size < 0xff)
if (string_size < 1<<8)
return SDS_TYPE_8;
if (string_size < 0xffff)
if (string_size < 1<<16)
return SDS_TYPE_16;
if (string_size < 0xffffffff)
if (string_size < 1ll<<32)
return SDS_TYPE_32;
return SDS_TYPE_64;
}
src/sentinel.c
+53 -7
View File
@@ -84,6 +84,7 @@ typedef struct sentinelAddr {
#define SENTINEL_MAX_PENDING_COMMANDS 100
#define SENTINEL_ELECTION_TIMEOUT 10000
#define SENTINEL_MAX_DESYNC 1000
#define SENTINEL_DEFAULT_DENY_SCRIPTS_RECONFIG 1
/* Failover machine different states. */
#define SENTINEL_FAILOVER_STATE_NONE 0 /* No failover in progress. */
@@ -241,6 +242,8 @@ struct sentinelState {
int announce_port; /* Port that is gossiped to other sentinels if
non zero. */
unsigned long simfailure_flags; /* Failures simulation. */
int deny_scripts_reconfig; /* Allow SENTINEL SET ... to change script
paths at runtime? */
} sentinel;
/* A script execution job. */
@@ -468,6 +471,7 @@ void initSentinel(void) {
sentinel.announce_ip = NULL;
sentinel.announce_port = 0;
sentinel.simfailure_flags = SENTINEL_SIMFAILURE_NONE;
sentinel.deny_scripts_reconfig = SENTINEL_DEFAULT_DENY_SCRIPTS_RECONFIG;
memset(sentinel.myid,0,sizeof(sentinel.myid));
}
@@ -1062,11 +1066,18 @@ int sentinelUpdateSentinelAddressInAllMasters(sentinelRedisInstance *ri) {
sentinelRedisInstance *master = dictGetVal(de), *match;
match = getSentinelRedisInstanceByAddrAndRunID(master->sentinels,
NULL,0,ri->runid);
if (match->link->disconnected == 0) {
/* If there is no match, this master does not know about this
* Sentinel, try with the next one. */
if (match == NULL) continue;
/* Disconnect the old links if connected. */
if (match->link->cc != NULL)
instanceLinkCloseConnection(match->link,match->link->cc);
if (match->link->pc != NULL)
instanceLinkCloseConnection(match->link,match->link->pc);
}
if (match == ri) continue; /* Address already updated for it. */
/* Update the address of the matching Sentinel by copying the address
* of the Sentinel object that received the address update. */
releaseSentinelAddr(match->addr);
@@ -1677,6 +1688,12 @@ char *sentinelHandleConfiguration(char **argv, int argc) {
} else if (!strcasecmp(argv[0],"announce-port") && argc == 2) {
/* announce-port <port> */
sentinel.announce_port = atoi(argv[1]);
} else if (!strcasecmp(argv[0],"deny-scripts-reconfig") && argc == 2) {
/* deny-scripts-reconfig <yes|no> */
if ((sentinel.deny_scripts_reconfig = yesnotoi(argv[1])) == -1) {
return "Please specify yes or no for the "
"deny-scripts-reconfig options.";
}
} else {
return "Unrecognized sentinel configuration statement.";
}
@@ -1697,6 +1714,12 @@ void rewriteConfigSentinelOption(struct rewriteConfigState *state) {
line = sdscatprintf(sdsempty(), "sentinel myid %s", sentinel.myid);
rewriteConfigRewriteLine(state,"sentinel",line,1);
/* sentinel deny-scripts-reconfig. */
line = sdscatprintf(sdsempty(), "sentinel deny-scripts-reconfig %s",
sentinel.deny_scripts_reconfig ? "yes" : "no");
rewriteConfigRewriteLine(state,"sentinel",line,
sentinel.deny_scripts_reconfig != SENTINEL_DEFAULT_DENY_SCRIPTS_RECONFIG);
/* For every master emit a "sentinel monitor" config entry. */
di = dictGetIterator(sentinel.masters);
while((de = dictNext(di)) != NULL) {
@@ -1910,6 +1933,7 @@ void sentinelReconnectInstance(sentinelRedisInstance *ri) {
link->cc->errstr);
instanceLinkCloseConnection(link,link->cc);
} else {
link->pending_commands = 0;
link->cc_conn_time = mstime();
link->cc->data = link;
redisAeAttach(server.el,link->cc);
@@ -2571,9 +2595,15 @@ void sentinelSendPeriodicCommands(sentinelRedisInstance *ri) {
/* If this is a slave of a master in O_DOWN condition we start sending
* it INFO every second, instead of the usual SENTINEL_INFO_PERIOD
* period. In this state we want to closely monitor slaves in case they
* are turned into masters by another Sentinel, or by the sysadmin. */
* are turned into masters by another Sentinel, or by the sysadmin.
*
* Similarly we monitor the INFO output more often if the slave reports
* to be disconnected from the master, so that we can have a fresh
* disconnection time figure. */
if ((ri->flags & SRI_SLAVE) &&
(ri->master->flags & (SRI_O_DOWN|SRI_FAILOVER_IN_PROGRESS))) {
((ri->master->flags & (SRI_O_DOWN|SRI_FAILOVER_IN_PROGRESS)) ||
(ri->master_link_down_time != 0)))
{
info_period = 1000;
} else {
info_period = SENTINEL_INFO_PERIOD;
@@ -3313,6 +3343,14 @@ void sentinelSetCommand(client *c) {
changes++;
} else if (!strcasecmp(option,"notification-script")) {
/* notification-script <path> */
if (sentinel.deny_scripts_reconfig) {
addReplyError(c,
"Reconfiguration of scripts path is denied for "
"security reasons. Check the deny-scripts-reconfig "
"configuration directive in your Sentinel configuration");
return;
}
if (strlen(value) && access(value,X_OK) == -1) {
addReplyError(c,
"Notification script seems non existing or non executable");
@@ -3324,6 +3362,14 @@ void sentinelSetCommand(client *c) {
changes++;
} else if (!strcasecmp(option,"client-reconfig-script")) {
/* client-reconfig-script <path> */
if (sentinel.deny_scripts_reconfig) {
addReplyError(c,
"Reconfiguration of scripts path is denied for "
"security reasons. Check the deny-scripts-reconfig "
"configuration directive in your Sentinel configuration");
return;
}
if (strlen(value) && access(value,X_OK) == -1) {
addReplyError(c,
"Client reconfiguration script seems non existing or "
@@ -3874,11 +3920,11 @@ int compareSlavesForPromotion(const void *a, const void *b) {
return (*sa)->slave_priority - (*sb)->slave_priority;
/* If priority is the same, select the slave with greater replication
* offset (processed more data frmo the master). */
* offset (processed more data from the master). */
if ((*sa)->slave_repl_offset > (*sb)->slave_repl_offset) {
return -1; /* a < b */
} else if ((*sa)->slave_repl_offset < (*sb)->slave_repl_offset) {
return 1; /* b > a */
return 1; /* a > b */
}
/* If the replication offset is the same select the slave with that has
@@ -3996,7 +4042,7 @@ void sentinelFailoverSendSlaveOfNoOne(sentinelRedisInstance *ri) {
/* We can't send the command to the promoted slave if it is now
* disconnected. Retry again and again with this state until the timeout
* is reached, then abort the failover. */
if (ri->link->disconnected) {
if (ri->promoted_slave->link->disconnected) {
if (mstime() - ri->failover_state_change_time > ri->failover_timeout) {
sentinelEvent(LL_WARNING,"-failover-abort-slave-timeout",ri,"%@");
sentinelAbortFailover(ri);
src/server.c
+70 -16
View File
@@ -148,7 +148,7 @@ struct redisCommand redisCommandTable[] = {
{"linsert",linsertCommand,5,"wm",0,NULL,1,1,1,0,0},
{"rpop",rpopCommand,2,"wF",0,NULL,1,1,1,0,0},
{"lpop",lpopCommand,2,"wF",0,NULL,1,1,1,0,0},
{"brpop",brpopCommand,-3,"ws",0,NULL,1,1,1,0,0},
{"brpop",brpopCommand,-3,"ws",0,NULL,1,-2,1,0,0},
{"brpoplpush",brpoplpushCommand,4,"wms",0,NULL,1,2,1,0,0},
{"blpop",blpopCommand,-3,"ws",0,NULL,1,-2,1,0,0},
{"llen",llenCommand,2,"rF",0,NULL,1,1,1,0,0},
@@ -163,7 +163,7 @@ struct redisCommand redisCommandTable[] = {
{"smove",smoveCommand,4,"wF",0,NULL,1,2,1,0,0},
{"sismember",sismemberCommand,3,"rF",0,NULL,1,1,1,0,0},
{"scard",scardCommand,2,"rF",0,NULL,1,1,1,0,0},
{"spop",spopCommand,-2,"wRsF",0,NULL,1,1,1,0,0},
{"spop",spopCommand,-2,"wRF",0,NULL,1,1,1,0,0},
{"srandmember",srandmemberCommand,-2,"rR",0,NULL,1,1,1,0,0},
{"sinter",sinterCommand,-2,"rS",0,NULL,1,-1,1,0,0},
{"sinterstore",sinterstoreCommand,-3,"wm",0,NULL,1,-1,1,0,0},
@@ -231,7 +231,7 @@ struct redisCommand redisCommandTable[] = {
{"ping",pingCommand,-1,"tF",0,NULL,0,0,0,0,0},
{"echo",echoCommand,2,"F",0,NULL,0,0,0,0,0},
{"save",saveCommand,1,"as",0,NULL,0,0,0,0,0},
{"bgsave",bgsaveCommand,1,"a",0,NULL,0,0,0,0,0},
{"bgsave",bgsaveCommand,-1,"a",0,NULL,0,0,0,0,0},
{"bgrewriteaof",bgrewriteaofCommand,1,"a",0,NULL,0,0,0,0,0},
{"shutdown",shutdownCommand,-1,"alt",0,NULL,0,0,0,0,0},
{"lastsave",lastsaveCommand,1,"RF",0,NULL,0,0,0,0,0},
@@ -248,6 +248,7 @@ struct redisCommand redisCommandTable[] = {
{"info",infoCommand,-1,"lt",0,NULL,0,0,0,0,0},
{"monitor",monitorCommand,1,"as",0,NULL,0,0,0,0,0},
{"ttl",ttlCommand,2,"rF",0,NULL,1,1,1,0,0},
{"touch",touchCommand,-2,"rF",0,NULL,1,1,1,0,0},
{"pttl",pttlCommand,2,"rF",0,NULL,1,1,1,0,0},
{"persist",persistCommand,2,"wF",0,NULL,1,1,1,0,0},
{"slaveof",slaveofCommand,3,"ast",0,NULL,0,0,0,0,0},
@@ -283,8 +284,10 @@ struct redisCommand redisCommandTable[] = {
{"wait",waitCommand,3,"s",0,NULL,0,0,0,0,0},
{"command",commandCommand,0,"lt",0,NULL,0,0,0,0,0},
{"geoadd",geoaddCommand,-5,"wm",0,NULL,1,1,1,0,0},
{"georadius",georadiusCommand,-6,"w",0,NULL,1,1,1,0,0},
{"georadiusbymember",georadiusByMemberCommand,-5,"w",0,NULL,1,1,1,0,0},
{"georadius",georadiusCommand,-6,"w",0,georadiusGetKeys,1,1,1,0,0},
{"georadius_ro",georadiusroCommand,-6,"r",0,georadiusGetKeys,1,1,1,0,0},
{"georadiusbymember",georadiusbymemberCommand,-5,"w",0,georadiusGetKeys,1,1,1,0,0},
{"georadiusbymember_ro",georadiusbymemberroCommand,-5,"r",0,georadiusGetKeys,1,1,1,0,0},
{"geohash",geohashCommand,-2,"r",0,NULL,1,1,1,0,0},
{"geopos",geoposCommand,-2,"r",0,NULL,1,1,1,0,0},
{"geodist",geodistCommand,-4,"r",0,NULL,1,1,1,0,0},
@@ -293,6 +296,8 @@ struct redisCommand redisCommandTable[] = {
{"pfcount",pfcountCommand,-2,"r",0,NULL,1,-1,1,0,0},
{"pfmerge",pfmergeCommand,-2,"wm",0,NULL,1,-1,1,0,0},
{"pfdebug",pfdebugCommand,-3,"w",0,NULL,0,0,0,0,0},
{"post",securityWarningCommand,-1,"lt",0,NULL,0,0,0,0,0},
{"host:",securityWarningCommand,-1,"lt",0,NULL,0,0,0,0,0},
{"latency",latencyCommand,-2,"aslt",0,NULL,0,0,0,0,0}
};
@@ -663,7 +668,7 @@ int htNeedsResize(dict *dict) {
size = dictSlots(dict);
used = dictSize(dict);
return (size && used && size > DICT_HT_INITIAL_SIZE &&
return (size > DICT_HT_INITIAL_SIZE &&
(used*100/size < HASHTABLE_MIN_FILL));
}
@@ -774,6 +779,11 @@ void activeExpireCycle(int type) {
int dbs_per_call = CRON_DBS_PER_CALL;
long long start = ustime(), timelimit;
/* When clients are paused the dataset should be static not just from the
* POV of clients not being able to write, but also from the POV of
* expires and evictions of keys not being performed. */
if (clientsArePaused()) return;
if (type == ACTIVE_EXPIRE_CYCLE_FAST) {
/* Don't start a fast cycle if the previous cycle did not exited
* for time limt. Also don't repeat a fast cycle for the same period
@@ -1046,12 +1056,15 @@ void databasesCron(void) {
/* Rehash */
if (server.activerehashing) {
for (j = 0; j < dbs_per_call; j++) {
int work_done = incrementallyRehash(rehash_db % server.dbnum);
rehash_db++;
int work_done = incrementallyRehash(rehash_db);
if (work_done) {
/* If the function did some work, stop here, we'll do
* more at the next cron loop. */
break;
} else {
/* If this db didn't need rehash, we'll try the next one. */
rehash_db++;
rehash_db %= server.dbnum;
}
}
}
@@ -1266,8 +1279,8 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
/* Clear the paused clients flag if needed. */
clientsArePaused(); /* Don't check return value, just use the side effect. */
/* Replication cron function -- used to reconnect to master and
* to detect transfer failures. */
/* Replication cron function -- used to reconnect to master,
* detect transfer failures, start background RDB transfers and so forth. */
run_with_period(1000) replicationCron();
/* Run the Redis Cluster cron. */
@@ -1285,6 +1298,22 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
migrateCloseTimedoutSockets();
}
/* Start a scheduled BGSAVE if the corresponding flag is set. This is
* useful when we are forced to postpone a BGSAVE because an AOF
* rewrite is in progress.
*
* Note: this code must be after the replicationCron() call above so
* make sure when refactoring this file to keep this order. This is useful
* because we want to give priority to RDB savings for replication. */
if (server.rdb_child_pid == -1 && server.aof_child_pid == -1 &&
server.rdb_bgsave_scheduled &&
(server.unixtime-server.lastbgsave_try > CONFIG_BGSAVE_RETRY_DELAY ||
server.lastbgsave_status == C_OK))
{
if (rdbSaveBackground(server.rdb_filename) == C_OK)
server.rdb_bgsave_scheduled = 0;
}
server.cronloops++;
return 1000/server.hz;
}
@@ -1514,6 +1543,7 @@ void initServerConfig(void) {
server.migrate_cached_sockets = dictCreate(&migrateCacheDictType,NULL);
server.next_client_id = 1; /* Client IDs, start from 1 .*/
server.loading_process_events_interval_bytes = (1024*1024*2);
server.lua_time_limit = LUA_SCRIPT_TIME_LIMIT;
server.lruclock = getLRUClock();
resetServerSaveParams();
@@ -1537,6 +1567,8 @@ void initServerConfig(void) {
server.repl_diskless_sync = CONFIG_DEFAULT_REPL_DISKLESS_SYNC;
server.repl_diskless_sync_delay = CONFIG_DEFAULT_REPL_DISKLESS_SYNC_DELAY;
server.slave_priority = CONFIG_DEFAULT_SLAVE_PRIORITY;
server.slave_announce_ip = CONFIG_DEFAULT_SLAVE_ANNOUNCE_IP;
server.slave_announce_port = CONFIG_DEFAULT_SLAVE_ANNOUNCE_PORT;
server.master_repl_offset = 0;
/* Replication partial resync backlog */
@@ -1571,6 +1603,8 @@ void initServerConfig(void) {
server.rpopCommand = lookupCommandByCString("rpop");
server.sremCommand = lookupCommandByCString("srem");
server.execCommand = lookupCommandByCString("exec");
server.expireCommand = lookupCommandByCString("expire");
server.pexpireCommand = lookupCommandByCString("pexpire");
/* Slow log */
server.slowlog_log_slower_than = CONFIG_DEFAULT_SLOWLOG_LOG_SLOWER_THAN;
@@ -1757,6 +1791,7 @@ int listenToPort(int port, int *fds, int *count) {
if (server.bindaddr_count == 0) server.bindaddr[0] = NULL;
for (j = 0; j < server.bindaddr_count || j == 0; j++) {
if (server.bindaddr[j] == NULL) {
int unsupported = 0;
/* Bind * for both IPv6 and IPv4, we enter here only if
* server.bindaddr_count == 0. */
fds[*count] = anetTcp6Server(server.neterr,port,NULL,
@@ -1764,19 +1799,27 @@ int listenToPort(int port, int *fds, int *count) {
if (fds[*count] != ANET_ERR) {
anetNonBlock(NULL,fds[*count]);
(*count)++;
} else if (errno == EAFNOSUPPORT) {
unsupported++;
serverLog(LL_WARNING,"Not listening to IPv6: unsupproted");
}
if (*count == 1 || unsupported) {
/* Bind the IPv4 address as well. */
fds[*count] = anetTcpServer(server.neterr,port,NULL,
server.tcp_backlog);
if (fds[*count] != ANET_ERR) {
anetNonBlock(NULL,fds[*count]);
(*count)++;
} else if (errno == EAFNOSUPPORT) {
unsupported++;
serverLog(LL_WARNING,"Not listening to IPv4: unsupproted");
}
}
/* Exit the loop if we were able to bind * on IPv4 and IPv6,
* otherwise fds[*count] will be ANET_ERR and we'll print an
* error and return to the caller with an error. */
if (*count == 2) break;
if (*count + unsupported == 2) break;
} else if (strchr(server.bindaddr[j],':')) {
/* Bind IPv6 address. */
fds[*count] = anetTcp6Server(server.neterr,port,server.bindaddr[j],
@@ -1903,6 +1946,7 @@ void initServer(void) {
server.rdb_child_pid = -1;
server.aof_child_pid = -1;
server.rdb_child_type = RDB_CHILD_TYPE_NONE;
server.rdb_bgsave_scheduled = 0;
aofRewriteBufferReset();
server.aof_buf = sdsempty();
server.lastsave = time(NULL); /* At startup we consider the DB saved. */
@@ -2554,8 +2598,9 @@ int prepareForShutdown(int flags) {
"There is a child rewriting the AOF. Killing it!");
kill(server.aof_child_pid,SIGUSR1);
}
/* Append only file: fsync() the AOF and exit */
/* Append only file: flush buffers and fsync() the AOF at exit */
serverLog(LL_NOTICE,"Calling fsync() on the AOF file.");
flushAppendOnlyFile(1);
aof_fsync(server.aof_fd);
}
@@ -3173,11 +3218,15 @@ sds genRedisInfoString(char *section) {
while((ln = listNext(&li))) {
client *slave = listNodeValue(ln);
char *state = NULL;
char ip[NET_IP_STR_LEN];
char ip[NET_IP_STR_LEN], *slaveip = slave->slave_ip;
int port;
long lag = 0;
if (anetPeerToString(slave->fd,ip,sizeof(ip),&port) == -1) continue;
if (slaveip[0] == '\0') {
if (anetPeerToString(slave->fd,ip,sizeof(ip),&port) == -1)
continue;
slaveip = ip;
}
switch(slave->replstate) {
case SLAVE_STATE_WAIT_BGSAVE_START:
case SLAVE_STATE_WAIT_BGSAVE_END:
@@ -3197,7 +3246,7 @@ sds genRedisInfoString(char *section) {
info = sdscatprintf(info,
"slave%d:ip=%s,port=%d,state=%s,"
"offset=%lld,lag=%ld\r\n",
slaveid,ip,slave->slave_listening_port,state,
slaveid,slaveip,slave->slave_listening_port,state,
slave->repl_ack_off, lag);
slaveid++;
}
@@ -3426,6 +3475,11 @@ int freeMemoryIfNeeded(void) {
int slaves = listLength(server.slaves);
mstime_t latency, eviction_latency;
/* When clients are paused the dataset should be static not just from the
* POV of clients not being able to write, but also from the POV of
* expires and evictions of keys not being performed. */
if (clientsArePaused()) return C_OK;
/* Remove the size of slaves output buffers and AOF buffer from the
* count of used memory. */
mem_used = zmalloc_used_memory();
@@ -3967,7 +4021,7 @@ int main(int argc, char **argv) {
* the program main. However the program is part of the Redis executable
* so that we can easily execute an RDB check on loading errors. */
if (strstr(argv[0],"redis-check-rdb") != NULL)
exit(redis_check_rdb_main(argv,argc));
redis_check_rdb_main(argc,argv);
if (argc >= 2) {
j = 1; /* First option to parse in argv[] */
src/server.h
+32 -16
View File
@@ -110,7 +110,7 @@ typedef long long mstime_t; /* millisecond time type. */
#define CONFIG_DEFAULT_CLUSTER_CONFIG_FILE "nodes.conf"
#define CONFIG_DEFAULT_DAEMONIZE 0
#define CONFIG_DEFAULT_UNIX_SOCKET_PERM 0
#define CONFIG_DEFAULT_TCP_KEEPALIVE 0
#define CONFIG_DEFAULT_TCP_KEEPALIVE 300
#define CONFIG_DEFAULT_PROTECTED_MODE 1
#define CONFIG_DEFAULT_LOGFILE ""
#define CONFIG_DEFAULT_SYSLOG_ENABLED 0
@@ -122,6 +122,8 @@ typedef long long mstime_t; /* millisecond time type. */
#define CONFIG_DEFAULT_REPL_DISKLESS_SYNC_DELAY 5
#define CONFIG_DEFAULT_SLAVE_SERVE_STALE_DATA 1
#define CONFIG_DEFAULT_SLAVE_READ_ONLY 1
#define CONFIG_DEFAULT_SLAVE_ANNOUNCE_IP NULL
#define CONFIG_DEFAULT_SLAVE_ANNOUNCE_PORT 0
#define CONFIG_DEFAULT_REPL_DISABLE_TCP_NODELAY 0
#define CONFIG_DEFAULT_MAXMEMORY 0
#define CONFIG_DEFAULT_MAXMEMORY_SAMPLES 5
@@ -157,7 +159,7 @@ typedef long long mstime_t; /* millisecond time type. */
#define PROTO_REPLY_CHUNK_BYTES (16*1024) /* 16k output buffer */
#define PROTO_INLINE_MAX_SIZE (1024*64) /* Max size of inline reads */
#define PROTO_MBULK_BIG_ARG (1024*32)
#define LONG_STR_SIZE 21 /* Bytes needed for long -> str */
#define LONG_STR_SIZE 21 /* Bytes needed for long -> str + '\0' */
#define AOF_AUTOSYNC_BYTES (1024*1024*32) /* fdatasync every 32MB */
/* When configuring the server eventloop, we setup it so that the total number
@@ -302,13 +304,15 @@ typedef long long mstime_t; /* millisecond time type. */
#define REPL_STATE_RECEIVE_AUTH 5 /* Wait for AUTH reply */
#define REPL_STATE_SEND_PORT 6 /* Send REPLCONF listening-port */
#define REPL_STATE_RECEIVE_PORT 7 /* Wait for REPLCONF reply */
#define REPL_STATE_SEND_CAPA 8 /* Send REPLCONF capa */
#define REPL_STATE_RECEIVE_CAPA 9 /* Wait for REPLCONF reply */
#define REPL_STATE_SEND_PSYNC 10 /* Send PSYNC */
#define REPL_STATE_RECEIVE_PSYNC 11 /* Wait for PSYNC reply */
#define REPL_STATE_SEND_IP 8 /* Send REPLCONF ip-address */
#define REPL_STATE_RECEIVE_IP 9 /* Wait for REPLCONF reply */
#define REPL_STATE_SEND_CAPA 10 /* Send REPLCONF capa */
#define REPL_STATE_RECEIVE_CAPA 11 /* Wait for REPLCONF reply */
#define REPL_STATE_SEND_PSYNC 12 /* Send PSYNC */
#define REPL_STATE_RECEIVE_PSYNC 13 /* Wait for PSYNC reply */
/* --- End of handshake states --- */
#define REPL_STATE_TRANSFER 12 /* Receiving .rdb from master */
#define REPL_STATE_CONNECTED 13 /* Connected to master */
#define REPL_STATE_TRANSFER 14 /* Receiving .rdb from master */
#define REPL_STATE_CONNECTED 15 /* Connected to master */
/* State of slaves from the POV of the master. Used in client->replstate.
* In SEND_BULK and ONLINE state the slave receives new updates
@@ -467,7 +471,7 @@ typedef struct redisObject {
/* Macro used to obtain the current LRU clock.
* If the current resolution is lower than the frequency we refresh the
* LRU clock (as it should be in production servers) we return the
* precomputed value, otherwise we need to resort to a function call. */
* precomputed value, otherwise we need to resort to a system call. */
#define LRU_CLOCK() ((1000/server.hz <= LRU_CLOCK_RESOLUTION) ? server.lruclock : getLRUClock())
/* Macro used to initialize a Redis object allocated on the stack.
@@ -594,7 +598,8 @@ typedef struct client {
copying this slave output buffer
should use. */
char replrunid[CONFIG_RUN_ID_SIZE+1]; /* Master run id if is a master. */
int slave_listening_port; /* As configured with: SLAVECONF listening-port */
int slave_listening_port; /* As configured with: REPLCONF listening-port */
char slave_ip[NET_IP_STR_LEN]; /* Optionally given by REPLCONF ip-address */
int slave_capa; /* Slave capabilities: SLAVE_CAPA_* bitwise OR. */
multiState mstate; /* MULTI/EXEC state */
int btype; /* Type of blocking op if CLIENT_BLOCKED. */
@@ -747,7 +752,8 @@ struct redisServer {
off_t loading_process_events_interval_bytes;
/* Fast pointers to often looked up command */
struct redisCommand *delCommand, *multiCommand, *lpushCommand, *lpopCommand,
*rpopCommand, *sremCommand, *execCommand;
*rpopCommand, *sremCommand, *execCommand, *expireCommand,
*pexpireCommand;
/* Fields used only for stats */
time_t stat_starttime; /* Server start time */
long long stat_numcommands; /* Number of processed commands */
@@ -837,6 +843,7 @@ struct redisServer {
time_t lastbgsave_try; /* Unix time of last attempted bgsave */
time_t rdb_save_time_last; /* Time used by last RDB save run. */
time_t rdb_save_time_start; /* Current RDB save start time. */
int rdb_bgsave_scheduled; /* BGSAVE when possible if true. */
int rdb_child_type; /* Type of save by active child. */
int lastbgsave_status; /* C_OK or C_ERR */
int stop_writes_on_bgsave_err; /* Don't allow writes if can't BGSAVE */
@@ -889,7 +896,9 @@ struct redisServer {
time_t repl_down_since; /* Unix time at which link with master went down */
int repl_disable_tcp_nodelay; /* Disable TCP_NODELAY after SYNC? */
int slave_priority; /* Reported in INFO and used by Sentinel. */
char repl_master_runid[CONFIG_RUN_ID_SIZE+1]; /* Master run id for PSYNC. */
int slave_announce_port; /* Give the master this listening port. */
char *slave_announce_ip; /* Give the master this ip address. */
char repl_master_runid[CONFIG_RUN_ID_SIZE+1]; /* Master run id for PSYNC.*/
long long repl_master_initial_offset; /* Master PSYNC offset. */
/* Replication script cache. */
dict *repl_scriptcache_dict; /* SHA1 all slaves are aware of. */
@@ -1320,7 +1329,6 @@ void serverLogFromHandler(int level, const char *msg);
void usage(void);
void updateDictResizePolicy(void);
int htNeedsResize(dict *dict);
void oom(const char *msg);
void populateCommandTable(void);
void resetCommandTableStats(void);
void adjustOpenFilesLimit(void);
@@ -1395,11 +1403,14 @@ void propagateExpire(redisDb *db, robj *key);
int expireIfNeeded(redisDb *db, robj *key);
long long getExpire(redisDb *db, robj *key);
void setExpire(redisDb *db, robj *key, long long when);
robj *lookupKey(redisDb *db, robj *key);
robj *lookupKey(redisDb *db, robj *key, int flags);
robj *lookupKeyRead(redisDb *db, robj *key);
robj *lookupKeyWrite(redisDb *db, robj *key);
robj *lookupKeyReadOrReply(client *c, robj *key, robj *reply);
robj *lookupKeyWriteOrReply(client *c, robj *key, robj *reply);
robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags);
#define LOOKUP_NONE 0
#define LOOKUP_NOTOUCH (1<<0)
void dbAdd(redisDb *db, robj *key, robj *val);
void dbOverwrite(redisDb *db, robj *key, robj *val);
void setKey(redisDb *db, robj *key, robj *val);
@@ -1425,6 +1436,7 @@ int *zunionInterGetKeys(struct redisCommand *cmd,robj **argv, int argc, int *num
int *evalGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
int *sortGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
int *migrateGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
int *georadiusGetKeys(struct redisCommand *cmd, robj **argv, int argc, int *numkeys);
/* Cluster */
void clusterInit(void);
@@ -1444,7 +1456,7 @@ void sentinelIsRunning(void);
/* redis-check-rdb */
int redis_check_rdb(char *rdbfilename);
int redis_check_rdb_main(char **argv, int argc);
int redis_check_rdb_main(int argc, char **argv);
/* Scripting */
void scriptingInit(int setup);
@@ -1542,6 +1554,7 @@ void pexpireCommand(client *c);
void pexpireatCommand(client *c);
void getsetCommand(client *c);
void ttlCommand(client *c);
void touchCommand(client *c);
void pttlCommand(client *c);
void persistCommand(client *c);
void slaveofCommand(client *c);
@@ -1622,8 +1635,10 @@ void replconfCommand(client *c);
void waitCommand(client *c);
void geoencodeCommand(client *c);
void geodecodeCommand(client *c);
void georadiusByMemberCommand(client *c);
void georadiusbymemberCommand(client *c);
void georadiusbymemberroCommand(client *c);
void georadiusCommand(client *c);
void georadiusroCommand(client *c);
void geoaddCommand(client *c);
void geohashCommand(client *c);
void geoposCommand(client *c);
@@ -1634,6 +1649,7 @@ void pfcountCommand(client *c);
void pfmergeCommand(client *c);
void pfdebugCommand(client *c);
void latencyCommand(client *c);
void securityWarningCommand(client *c);
#if defined(__GNUC__)
void *calloc(size_t count, size_t size) __attribute__ ((deprecated));
src/slowlog.h
+1 -1
View File
@@ -35,7 +35,7 @@ typedef struct slowlogEntry {
robj **argv;
int argc;
long long id; /* Unique entry identifier. */
long long duration; /* Time spent by the query, in nanoseconds. */
long long duration; /* Time spent by the query, in microseconds. */
time_t time; /* Unix time at which the query was executed. */
} slowlogEntry;
src/t_set.c
+6 -3
View File
@@ -359,9 +359,6 @@ void smoveCommand(client *c) {
dbDelete(c->db,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_GENERIC,"del",c->argv[1],c->db->id);
}
signalModifiedKey(c->db,c->argv[1]);
signalModifiedKey(c->db,c->argv[2]);
server.dirty++;
/* Create the destination set when it doesn't exist */
if (!dstset) {
@@ -369,6 +366,10 @@ void smoveCommand(client *c) {
dbAdd(c->db,c->argv[2],dstset);
}
signalModifiedKey(c->db,c->argv[1]);
signalModifiedKey(c->db,c->argv[2]);
server.dirty++;
/* An extra key has changed when ele was successfully added to dstset */
if (setTypeAdd(dstset,ele)) {
server.dirty++;
@@ -554,6 +555,8 @@ void spopWithCountCommand(client *c) {
* the alsoPropagate() API. */
decrRefCount(propargv[0]);
preventCommandPropagation(c);
signalModifiedKey(c->db,c->argv[1]);
server.dirty++;
}
void spopCommand(client *c) {
src/t_string.c
+4
View File
@@ -263,6 +263,10 @@ void getrangeCommand(client *c) {
}
/* Convert negative indexes */
if (start < 0 && end < 0 && start > end) {
addReply(c,shared.emptybulk);
return;
}
if (start < 0) start = strlen+start;
if (end < 0) end = strlen+end;
if (start < 0) start = 0;
src/t_zset.c
+7 -7
View File
@@ -1247,7 +1247,7 @@ void zaddGenericCommand(client *c, int flags) {
/* After the options, we expect to have an even number of args, since
* we expect any number of score-element pairs. */
elements = c->argc-scoreidx;
if (elements % 2) {
if (elements % 2 || !elements) {
addReply(c,shared.syntaxerr);
return;
}
@@ -2148,16 +2148,13 @@ void zunionInterGenericCommand(client *c, robj *dstkey, int op) {
serverPanic("Unknown operator");
}
if (dbDelete(c->db,dstkey)) {
signalModifiedKey(c->db,dstkey);
if (dbDelete(c->db,dstkey))
touched = 1;
server.dirty++;
}
if (dstzset->zsl->length) {
zsetConvertToZiplistIfNeeded(dstobj,maxelelen);
dbAdd(c->db,dstkey,dstobj);
addReplyLongLong(c,zsetLength(dstobj));
if (!touched) signalModifiedKey(c->db,dstkey);
signalModifiedKey(c->db,dstkey);
notifyKeyspaceEvent(NOTIFY_ZSET,
(op == SET_OP_UNION) ? "zunionstore" : "zinterstore",
dstkey,c->db->id);
@@ -2165,8 +2162,11 @@ void zunionInterGenericCommand(client *c, robj *dstkey, int op) {
} else {
decrRefCount(dstobj);
addReply(c,shared.czero);
if (touched)
if (touched) {
signalModifiedKey(c->db,dstkey);
notifyKeyspaceEvent(NOTIFY_GENERIC,"del",dstkey,c->db->id);
server.dirty++;
}
}
zfree(src);
}
src/version.h
+1 -1
View File
@@ -1 +1 @@
#define REDIS_VERSION "3.2.0"
#define REDIS_VERSION "3.2.13"
src/ziplist.c
+23 -15
View File
@@ -180,7 +180,7 @@ typedef struct zlentry {
void ziplistRepr(unsigned char *zl);
/* Return bytes needed to store integer encoded by 'encoding' */
static unsigned int zipIntSize(unsigned char encoding) {
unsigned int zipIntSize(unsigned char encoding) {
switch(encoding) {
case ZIP_INT_8B: return 1;
case ZIP_INT_16B: return 2;
@@ -195,7 +195,7 @@ static unsigned int zipIntSize(unsigned char encoding) {
/* Encode the length 'rawlen' writing it in 'p'. If p is NULL it just returns
* the amount of bytes required to encode such a length. */
static unsigned int zipEncodeLength(unsigned char *p, unsigned char encoding, unsigned int rawlen) {
unsigned int zipEncodeLength(unsigned char *p, unsigned char encoding, unsigned int rawlen) {
unsigned char len = 1, buf[5];
if (ZIP_IS_STR(encoding)) {
@@ -259,7 +259,7 @@ static unsigned int zipEncodeLength(unsigned char *p, unsigned char encoding, un
/* Encode the length of the previous entry and write it to "p". Return the
* number of bytes needed to encode this length if "p" is NULL. */
static unsigned int zipPrevEncodeLength(unsigned char *p, unsigned int len) {
unsigned int zipPrevEncodeLength(unsigned char *p, unsigned int len) {
if (p == NULL) {
return (len < ZIP_BIGLEN) ? 1 : sizeof(len)+1;
} else {
@@ -277,7 +277,7 @@ static unsigned int zipPrevEncodeLength(unsigned char *p, unsigned int len) {
/* Encode the length of the previous entry and write it to "p". This only
* uses the larger encoding (required in __ziplistCascadeUpdate). */
static void zipPrevEncodeLengthForceLarge(unsigned char *p, unsigned int len) {
void zipPrevEncodeLengthForceLarge(unsigned char *p, unsigned int len) {
if (p == NULL) return;
p[0] = ZIP_BIGLEN;
memcpy(p+1,&len,sizeof(len));
@@ -309,14 +309,14 @@ static void zipPrevEncodeLengthForceLarge(unsigned char *p, unsigned int len) {
/* Return the difference in number of bytes needed to store the length of the
* previous element 'len', in the entry pointed to by 'p'. */
static int zipPrevLenByteDiff(unsigned char *p, unsigned int len) {
int zipPrevLenByteDiff(unsigned char *p, unsigned int len) {
unsigned int prevlensize;
ZIP_DECODE_PREVLENSIZE(p, prevlensize);
return zipPrevEncodeLength(NULL, len) - prevlensize;
}
/* Return the total number of bytes used by the entry pointed to by 'p'. */
static unsigned int zipRawEntryLength(unsigned char *p) {
unsigned int zipRawEntryLength(unsigned char *p) {
unsigned int prevlensize, encoding, lensize, len;
ZIP_DECODE_PREVLENSIZE(p, prevlensize);
ZIP_DECODE_LENGTH(p + prevlensize, encoding, lensize, len);
@@ -325,7 +325,7 @@ static unsigned int zipRawEntryLength(unsigned char *p) {
/* Check if string pointed to by 'entry' can be encoded as an integer.
* Stores the integer value in 'v' and its encoding in 'encoding'. */
static int zipTryEncoding(unsigned char *entry, unsigned int entrylen, long long *v, unsigned char *encoding) {
int zipTryEncoding(unsigned char *entry, unsigned int entrylen, long long *v, unsigned char *encoding) {
long long value;
if (entrylen >= 32 || entrylen == 0) return 0;
@@ -352,7 +352,7 @@ static int zipTryEncoding(unsigned char *entry, unsigned int entrylen, long long
}
/* Store integer 'value' at 'p', encoded as 'encoding' */
static void zipSaveInteger(unsigned char *p, int64_t value, unsigned char encoding) {
void zipSaveInteger(unsigned char *p, int64_t value, unsigned char encoding) {
int16_t i16;
int32_t i32;
int64_t i64;
@@ -382,7 +382,7 @@ static void zipSaveInteger(unsigned char *p, int64_t value, unsigned char encodi
}
/* Read integer encoded as 'encoding' from 'p' */
static int64_t zipLoadInteger(unsigned char *p, unsigned char encoding) {
int64_t zipLoadInteger(unsigned char *p, unsigned char encoding) {
int16_t i16;
int32_t i32;
int64_t i64, ret = 0;
@@ -414,7 +414,7 @@ static int64_t zipLoadInteger(unsigned char *p, unsigned char encoding) {
}
/* Return a struct with all information about an entry. */
static void zipEntry(unsigned char *p, zlentry *e) {
void zipEntry(unsigned char *p, zlentry *e) {
ZIP_DECODE_PREVLEN(p, e->prevrawlensize, e->prevrawlen);
ZIP_DECODE_LENGTH(p + e->prevrawlensize, e->encoding, e->lensize, e->len);
@@ -434,7 +434,7 @@ unsigned char *ziplistNew(void) {
}
/* Resize the ziplist. */
static unsigned char *ziplistResize(unsigned char *zl, unsigned int len) {
unsigned char *ziplistResize(unsigned char *zl, unsigned int len) {
zl = zrealloc(zl,len);
ZIPLIST_BYTES(zl) = intrev32ifbe(len);
zl[len-1] = ZIP_END;
@@ -461,7 +461,7 @@ static unsigned char *ziplistResize(unsigned char *zl, unsigned int len) {
*
* The pointer "p" points to the first entry that does NOT need to be
* updated, i.e. consecutive fields MAY need an update. */
static unsigned char *__ziplistCascadeUpdate(unsigned char *zl, unsigned char *p) {
unsigned char *__ziplistCascadeUpdate(unsigned char *zl, unsigned char *p) {
size_t curlen = intrev32ifbe(ZIPLIST_BYTES(zl)), rawlen, rawlensize;
size_t offset, noffset, extra;
unsigned char *np;
@@ -523,7 +523,7 @@ static unsigned char *__ziplistCascadeUpdate(unsigned char *zl, unsigned char *p
}
/* Delete "num" entries, starting at "p". Returns pointer to the ziplist. */
static unsigned char *__ziplistDelete(unsigned char *zl, unsigned char *p, unsigned int num) {
unsigned char *__ziplistDelete(unsigned char *zl, unsigned char *p, unsigned int num) {
unsigned int i, totlen, deleted = 0;
size_t offset;
int nextdiff = 0;
@@ -583,7 +583,7 @@ static unsigned char *__ziplistDelete(unsigned char *zl, unsigned char *p, unsig
}
/* Insert item at "p". */
static unsigned char *__ziplistInsert(unsigned char *zl, unsigned char *p, unsigned char *s, unsigned int slen) {
unsigned char *__ziplistInsert(unsigned char *zl, unsigned char *p, unsigned char *s, unsigned int slen) {
size_t curlen = intrev32ifbe(ZIPLIST_BYTES(zl)), reqlen;
unsigned int prevlensize, prevlen = 0;
size_t offset;
@@ -621,7 +621,12 @@ static unsigned char *__ziplistInsert(unsigned char *zl, unsigned char *p, unsig
/* When the insert position is not equal to the tail, we need to
* make sure that the next entry can hold this entry's length in
* its prevlen field. */
int forcelarge = 0;
nextdiff = (p[0] != ZIP_END) ? zipPrevLenByteDiff(p,reqlen) : 0;
if (nextdiff == -4 && reqlen < 4) {
nextdiff = 0;
forcelarge = 1;
}
/* Store offset because a realloc may change the address of zl. */
offset = p-zl;
@@ -634,7 +639,10 @@ static unsigned char *__ziplistInsert(unsigned char *zl, unsigned char *p, unsig
memmove(p+reqlen,p-nextdiff,curlen-offset-1+nextdiff);
/* Encode this entry's raw length in the next entry. */
zipPrevEncodeLength(p+reqlen,reqlen);
if (forcelarge)
zipPrevEncodeLengthForceLarge(p+reqlen,reqlen);
else
zipPrevEncodeLength(p+reqlen,reqlen);
/* Update offset for tail */
ZIPLIST_TAIL_OFFSET(zl) =
src/zmalloc.c
+3 -2
View File
@@ -413,8 +413,9 @@ size_t zmalloc_get_memory_size(void) {
if (sysctl(mib, 2, &size, &len, NULL, 0) == 0)
return (size_t)size;
return 0L; /* Failed? */
#endif /* sysctl and sysconf variants */
#else
return 0L; /* Unknown method to get the data. */
#endif
#else
return 0L; /* Unknown OS. */
#endif
tests/integration/rdb.tcl
+1 -1
View File
@@ -89,7 +89,7 @@ close $fd
start_server_and_kill_it [list "dir" $server_path] {
test {Server should not start if RDB is corrupted} {
wait_for_condition 50 100 {
[string match {*RDB checksum*} \
[string match {*CRC error*} \
[exec tail -n10 < [dict get $srv stdout]]]
} else {
fail "Server started even if RDB was corrupted!"
tests/integration/replication-psync.tcl
+1 -1
View File
@@ -110,7 +110,7 @@ foreach diskless {no yes} {
test_psync {no reconnection, just sync} 6 1000000 3600 0 {
} $diskless 0
test_psync {ok psync} 6 1000000 3600 0 {
test_psync {ok psync} 6 100000000 3600 0 {
assert {[s -1 sync_partial_ok] > 0}
} $diskless 1
tests/sentinel/tests/05-manual.tcl
+2 -1
View File
@@ -6,7 +6,8 @@ test "Manual failover works" {
set old_port [RI $master_id tcp_port]
set addr [S 0 SENTINEL GET-MASTER-ADDR-BY-NAME mymaster]
assert {[lindex $addr 1] == $old_port}
S 0 SENTINEL FAILOVER mymaster
catch {S 0 SENTINEL FAILOVER mymaster} reply
assert {$reply eq "OK"}
foreach_sentinel_id id {
wait_for_condition 1000 50 {
[lindex [S $id SENTINEL GET-MASTER-ADDR-BY-NAME mymaster] 1] != $old_port
tests/sentinel/tests/07-down-conditions.tcl
+68
View File
@@ -0,0 +1,68 @@
# Test conditions where an instance is considered to be down
source "../tests/includes/init-tests.tcl"
proc ensure_master_up {} {
wait_for_condition 1000 50 {
[dict get [S 4 sentinel master mymaster] flags] eq "master"
} else {
fail "Master flags are not just 'master'"
}
}
proc ensure_master_down {} {
wait_for_condition 1000 50 {
[string match *down* \
[dict get [S 4 sentinel master mymaster] flags]]
} else {
fail "Master is not flagged SDOWN"
}
}
test "Crash the majority of Sentinels to prevent failovers for this unit" {
for {set id 0} {$id < $quorum} {incr id} {
kill_instance sentinel $id
}
}
test "SDOWN is triggered by non-responding but not crashed instance" {
lassign [S 4 SENTINEL GET-MASTER-ADDR-BY-NAME mymaster] host port
ensure_master_up
exec ../../../src/redis-cli -h $host -p $port debug sleep 10 > /dev/null &
ensure_master_down
ensure_master_up
}
test "SDOWN is triggered by crashed instance" {
lassign [S 4 SENTINEL GET-MASTER-ADDR-BY-NAME mymaster] host port
ensure_master_up
kill_instance redis 0
ensure_master_down
restart_instance redis 0
ensure_master_up
}
test "SDOWN is triggered by masters advertising as slaves" {
ensure_master_up
R 0 slaveof 127.0.0.1 34567
ensure_master_down
R 0 slaveof no one
ensure_master_up
}
test "SDOWN is triggered by misconfigured instance repling with errors" {
ensure_master_up
set orig_dir [lindex [R 0 config get dir] 1]
set orig_save [lindex [R 0 config get save] 1]
# Set dir to / and filename to "tmp" to make sure it will fail.
R 0 config set dir /
R 0 config set dbfilename tmp
R 0 config set save "1000000 1000000"
R 0 bgsave
ensure_master_down
R 0 config set save $orig_save
R 0 config set dir $orig_dir
R 0 config set dbfilename dump.rdb
R 0 bgsave
ensure_master_up
}
tests/test_helper.tcl
+2
View File
@@ -46,10 +46,12 @@ set ::all_tests {
unit/scripting
unit/maxmemory
unit/introspection
unit/introspection-2
unit/limits
unit/obuf-limits
unit/bitops
unit/bitfield
unit/geo
unit/memefficiency
unit/hyperloglog
}
tests/unit/bitfield.tcl
+14
View File
@@ -184,4 +184,18 @@ start_server {tags {"bitops"}} {
}
}
}
test {BITFIELD regression for #3221} {
r set bits 1
r bitfield bits get u1 0
} {0}
test {BITFIELD regression for #3564} {
for {set j 0} {$j < 10} {incr j} {
r del mystring
set res [r BITFIELD mystring SET i8 0 10 SET i8 64 10 INCRBY i8 10 99900]
assert {$res eq {0 0 60}}
}
r del mystring
}
}
tests/unit/bitops.tcl
+10
View File
@@ -43,6 +43,16 @@ start_server {tags {"bitops"}} {
r bitcount no-key
} 0
test {BITCOUNT returns 0 with out of range indexes} {
r set str "xxxx"
r bitcount str 4 10
} 0
test {BITCOUNT returns 0 with negative indexes where start > end} {
r set str "xxxx"
r bitcount str -6 -7
} 0
catch {unset num}
foreach vec [list "" "\xaa" "\x00\x00\xff" "foobar" "123"] {
incr num
tests/unit/expire.tcl
+21
View File
@@ -198,4 +198,25 @@ start_server {tags {"expire"}} {
r set foo b
lsort [r keys *]
} {a e foo s t}
test {EXPIRE with empty string as TTL should report an error} {
r set foo bar
catch {r expire foo ""} e
set e
} {*not an integer*}
test {SET - use EX/PX option, TTL should not be reseted after loadaof} {
r config set appendonly yes
r set foo bar EX 100
after 2000
r debug loadaof
set ttl [r ttl foo]
assert {$ttl <= 98 && $ttl > 90}
r set foo bar PX 100000
after 2000
r debug loadaof
set ttl [r ttl foo]
assert {$ttl <= 98 && $ttl > 90}
}
}
tests/unit/geo.tcl
+89 -4
View File
@@ -1,4 +1,4 @@
# Helper functins to simulate search-in-radius in the Tcl side in order to
# Helper functions to simulate search-in-radius in the Tcl side in order to
# verify the Redis implementation with a fuzzy test.
proc geo_degrad deg {expr {$deg*atan(1)*8/360}}
@@ -23,6 +23,44 @@ proc geo_random_point {lonvar latvar} {
set lat [expr {-70 + rand()*140}]
}
# Return elements non common to both the lists.
# This code is from http://wiki.tcl.tk/15489
proc compare_lists {List1 List2} {
set DiffList {}
foreach Item $List1 {
if {[lsearch -exact $List2 $Item] == -1} {
lappend DiffList $Item
}
}
foreach Item $List2 {
if {[lsearch -exact $List1 $Item] == -1} {
if {[lsearch -exact $DiffList $Item] == -1} {
lappend DiffList $Item
}
}
}
return $DiffList
}
# The following list represents sets of random seed, search position
# and radius that caused bugs in the past. It is used by the randomized
# test later as a starting point. When the regression vectors are scanned
# the code reverts to using random data.
#
# The format is: seed km lon lat
set regression_vectors {
{1412 156 149.29737817929004 15.95807862745508}
{441574 143 59.235461856813856 66.269555127373678}
{160645 187 -101.88575239939883 49.061997951502917}
{750269 154 -90.187939661642517 66.615930412251487}
{342880 145 163.03472387745728 64.012747720821181}
{729955 143 137.86663517256579 63.986745399416776}
{939895 151 59.149620271823181 65.204186651485145}
{1412 156 149.29737817929004 15.95807862745508}
{564862 149 84.062063109158544 -65.685403922426232}
}
set rv_idx 0
start_server {tags {"geo"}} {
test {GEOADD create} {
r geoadd nyc -73.9454966 40.747533 "lic market"
@@ -183,16 +221,33 @@ start_server {tags {"geo"}} {
}
test {GEOADD + GEORANGE randomized test} {
set attempt 10
set attempt 30
while {[incr attempt -1]} {
set rv [lindex $regression_vectors $rv_idx]
incr rv_idx
unset -nocomplain debuginfo
set srand_seed [randomInt 1000000]
set srand_seed [clock milliseconds]
if {$rv ne {}} {set srand_seed [lindex $rv 0]}
lappend debuginfo "srand_seed is $srand_seed"
expr {srand($srand_seed)} ; # If you need a reproducible run
r del mypoints
set radius_km [expr {[randomInt 200]+10}]
if {[randomInt 10] == 0} {
# From time to time use very big radiuses
set radius_km [expr {[randomInt 50000]+10}]
} else {
# Normally use a few - ~200km radiuses to stress
# test the code the most in edge cases.
set radius_km [expr {[randomInt 200]+10}]
}
if {$rv ne {}} {set radius_km [lindex $rv 1]}
set radius_m [expr {$radius_km*1000}]
geo_random_point search_lon search_lat
if {$rv ne {}} {
set search_lon [lindex $rv 2]
set search_lat [lindex $rv 3]
}
lappend debuginfo "Search area: $search_lon,$search_lat $radius_km km"
set tcl_result {}
set argv {}
@@ -208,10 +263,40 @@ start_server {tags {"geo"}} {
set res [lsort [r georadius mypoints $search_lon $search_lat $radius_km km]]
set res2 [lsort $tcl_result]
set test_result OK
if {$res != $res2} {
set rounding_errors 0
set diff [compare_lists $res $res2]
foreach place $diff {
set mydist [geo_distance $lon $lat $search_lon $search_lat]
set mydist [expr $mydist/1000]
if {($mydist / $radius_km) > 0.999} {incr rounding_errors}
}
# Make sure this is a real error and not a rounidng issue.
if {[llength $diff] == $rounding_errors} {
set res $res2; # Error silenced
}
}
if {$res != $res2} {
set diff [compare_lists $res $res2]
puts "*** Possible problem in GEO radius query ***"
puts "Redis: $res"
puts "Tcl : $res2"
puts "Diff : $diff"
puts [join $debuginfo "\n"]
foreach place $diff {
if {[lsearch -exact $res2 $place] != -1} {
set where "(only in Tcl)"
} else {
set where "(only in Redis)"
}
lassign [lindex [r geopos mypoints $place] 0] lon lat
set mydist [geo_distance $lon $lat $search_lon $search_lat]
set mydist [expr $mydist/1000]
puts "$place -> [r geopos mypoints $place] $mydist $where"
if {($mydist / $radius_km) > 0.999} {incr rounding_errors}
}
set test_result FAIL
}
unset -nocomplain debuginfo
tests/unit/introspection-2.tcl
+23
View File
@@ -0,0 +1,23 @@
start_server {tags {"introspection"}} {
test {TTL and TYPYE do not alter the last access time of a key} {
r set foo bar
after 3000
r ttl foo
r type foo
assert {[r object idletime foo] >= 2}
}
test {TOUCH alters the last access time of a key} {
r set foo bar
after 3000
r touch foo
assert {[r object idletime foo] < 2}
}
test {TOUCH returns the number of existing keys specified} {
r flushdb
r set key1 1
r set key2 2
r touch key0 key1 key2 key3
} 2
}
tests/unit/scripting.tcl
+1 -1
View File
@@ -142,7 +142,7 @@ start_server {tags {"scripting"}} {
test {EVAL - Scripts can't run certain commands} {
set e {}
catch {r eval {return redis.pcall('spop','x')} 0} e
catch {r eval {return redis.pcall('blpop','x',0)} 0} e
set e
} {*not allowed*}
tests/unit/type/list-3.tcl
+44
View File
@@ -13,6 +13,50 @@ start_server {
assert_equal [r lindex l 1] [lindex $mylist 1]
}
test {Regression for quicklist #3343 bug} {
r del mylist
r lpush mylist 401
r lpush mylist 392
r rpush mylist [string repeat x 5105]"799"
r lset mylist -1 [string repeat x 1014]"702"
r lpop mylist
r lset mylist -1 [string repeat x 4149]"852"
r linsert mylist before 401 [string repeat x 9927]"12"
r lrange mylist 0 -1
r ping ; # It's enough if the server is still alive
} {PONG}
test {Stress tester for #3343-alike bugs} {
r del key
for {set j 0} {$j < 10000} {incr j} {
set op [randomInt 6]
set small_signed_count [expr 5-[randomInt 10]]
if {[randomInt 2] == 0} {
set ele [randomInt 1000]
} else {
set ele [string repeat x [randomInt 10000]][randomInt 1000]
}
switch $op {
0 {r lpush key $ele}
1 {r rpush key $ele}
2 {r lpop key}
3 {r rpop key}
4 {
catch {r lset key $small_signed_count $ele}
}
5 {
set otherele [randomInt 1000]
if {[randomInt 2] == 0} {
set where before
} else {
set where after
}
r linsert key $where $otherele $ele
}
}
}
}
tags {slow} {
test {ziplist implementation: value encoding and backlink} {
if {$::accurate} {set iterations 100} else {set iterations 10}
utils/corrupt_rdb.c
+44
View File
@@ -0,0 +1,44 @@
/* Trivia program to corrupt an RDB file in order to check the RDB check
* program behavior and effectiveness.
*
* Copyright (C) 2016 Salvatore Sanfilippo.
* This software is released in the 3-clause BSD license. */
#include <stdio.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <stdlib.h>
#include <unistd.h>
#include <time.h>
int main(int argc, char **argv) {
struct stat stat;
int fd, cycles;
if (argc != 3) {
fprintf(stderr,"Usage: <filename> <cycles>\n");
exit(1);
}
srand(time(NULL));
cycles = atoi(argv[2]);
fd = open("dump.rdb",O_RDWR);
if (fd == -1) {
perror("open");
exit(1);
}
fstat(fd,&stat);
while(cycles--) {
unsigned char buf[32];
unsigned long offset = rand()%stat.st_size;
int writelen = 1+rand()%31;
int j;
for (j = 0; j < writelen; j++) buf[j] = (char)rand();
lseek(fd,offset,SEEK_SET);
printf("Writing %d bytes at offset %lu\n", writelen, offset);
write(fd,buf,writelen);
}
return 0;
}
utils/create-cluster/README
+4 -4
View File
@@ -2,7 +2,7 @@ Create-custer is a small script used to easily start a big number of Redis
instances configured to run in cluster mode. Its main goal is to allow manual
testing in a condition which is not easy to replicate with the Redis cluster
unit tests, for example when a lot of instances are needed in order to trigger
a give bug.
a given bug.
The tool can also be used just to easily create a number of instances in a
Redis Cluster in order to experiment a bit with the system.
@@ -10,7 +10,7 @@ Redis Cluster in order to experiment a bit with the system.
USAGE
---
To create a cluster, follow this steps:
To create a cluster, follow these steps:
1. Edit create-cluster and change the start / end port, depending on the
number of instances you want to create.
@@ -21,7 +21,7 @@ an actual Redis cluster will be created.
In order to stop a cluster:
1. Use "./craete-cluster stop" to stop all the instances. After you stopped the instances you can use "./create-cluster start" to restart them if you change ideas.
2. Use "./create-cluster clean" to remove all the AOF / log files to restat with a clean environment.
1. Use "./create-cluster stop" to stop all the instances. After you stopped the instances you can use "./create-cluster start" to restart them if you change your mind.
2. Use "./create-cluster clean" to remove all the AOF / log files to restart with a clean environment.
Use the command "./create-cluster help" to get the full list of features.
utils/install_server.sh
+6 -6
View File
@@ -135,13 +135,13 @@ fi
echo "## Generated by install_server.sh ##" > $TMP_FILE
read -r SED_EXPR <<-EOF
s#^port [0-9]{4}\$#port ${REDIS_PORT}#; \
s#^logfile .+\$#logfile ${REDIS_LOG_FILE}#; \
s#^dir .+\$#dir ${REDIS_DATA_DIR}#; \
s#^pidfile .+\$#pidfile ${PIDFILE}#; \
s#^daemonize no\$#daemonize yes#;
s#^port .\+#port ${REDIS_PORT}#; \
s#^logfile .\+#logfile ${REDIS_LOG_FILE}#; \
s#^dir .\+#dir ${REDIS_DATA_DIR}#; \
s#^pidfile .\+#pidfile ${PIDFILE}#; \
s#^daemonize no#daemonize yes#;
EOF
sed -r "$SED_EXPR" $DEFAULT_CONFIG >> $TMP_FILE
sed "$SED_EXPR" $DEFAULT_CONFIG >> $TMP_FILE
#cat $TPL_FILE | while read line; do eval "echo \"$line\"" >> $TMP_FILE; done
cp $TMP_FILE $REDIS_CONFIG_FILE || die "Could not write redis config file $REDIS_CONFIG_FILE"