Compare commits

...
47 Commits
Author SHA1 Message Date
Oran Agra f9470c9a17 Redis 6.2.4
CI / test-ubuntu-latest (push) Has been cancelled
CI / build-ubuntu-old (push) Has been cancelled
CI / build-macos-latest (push) Has been cancelled
CI / build-32bit (push) Has been cancelled
CI / build-libc-malloc (push) Has been cancelled
CI / build-centos7-jemalloc (push) Has been cancelled
2021-06-01 17:03:36 +03:00
Oran Agra e9a1438ac4 Fix integer overflow in STRALGO LCS (CVE-2021-32625) (#9011)
An integer overflow bug in Redis version 6.0 or newer can be exploited using the
STRALGO LCS command to corrupt the heap and potentially result with remote code
execution. This is a result of an incomplete fix by CVE-2021-29477.

(cherry picked from commit 1ddecf1958)
2021-06-01 17:03:36 +03:00
YaacovHazan 5102c0da92 unregister AE_READABLE from the read pipe in backgroundSaveDoneHandlerSocket (#8991)
In diskless replication, we create a read pipe for the RDB, between the child and the parent.
When we close this pipe (fd), the read handler also needs to be removed from the event loop (if it still registered).
Otherwise, next time we will use the same fd, the registration will be fail (panic), because
we will use EPOLL_CTL_MOD (the fd still register in the event loop), on fd that already removed from epoll_ctl

(cherry picked from commit 501d775583)
2021-06-01 17:03:36 +03:00
Wen Hui d68b775df5 fix sentinel test failure (#8983)
fix for recent breakage from #8958, did a mistake when updating the pr.

(cherry picked from commit be6ce8a92a)
2021-06-01 17:03:36 +03:00
Wen Hui ed995ffa03 [SENTINEL] reset sentinel-user/pass to NULL when user config with empty string (#8958)
(cherry picked from commit ae6f58690b)
2021-06-01 17:03:36 +03:00
Madelyn Olson 875a1f07d8 Hide migrate command from slowlog if they include auth (#8859)
Redact commands that include sensitive data from slowlog and monitor

(cherry picked from commit a59e75a475)
2021-06-01 17:03:36 +03:00
patpatbear d070546568 sinterstore: add missing keyspace del event when any source set not exists. (#8949)
this patch fixes sinterstore by add missing keyspace del event when any source set not exists.

Co-authored-by: srzhao <srzhao@sysnew.com>
(cherry picked from commit 46d9f31e94)
2021-06-01 17:03:36 +03:00
Oran Agra 511be70eba Fix crash unlinking a stream with groups rax and no groups (#8932)
When estimating the effort for unlink, we try to compute the effort of
the first group and extrapolate.
If there's a groups rax that's empty, there'a an assertion.

reproduce:
xadd s * a b
xgroup create s bla $
xgroup destroy s bla
unlink s

(cherry picked from commit 97108845e2)
2021-06-01 17:03:36 +03:00
Oran Agra 840f7f61c5 fix redis-benchmark to ignore unsupported configs (#8916)
Redis Enterprise supports the CONFIG GET command, but it replies with am
empty array since the save and appendonly configs are not supported.
before this fix redis-benchmark would segfault for trying to access the
error string on an array type reply.
see #8869

(cherry picked from commit 4d1094e8be)
2021-06-01 17:03:36 +03:00
Wang Yuan 9d69d6e8fb Fix wrong COW memory in log (#8917)
Always 0 MB of memory used by copy-on-write, introduced in #8645.

(cherry picked from commit 81e2d7272b)
2021-06-01 17:03:36 +03:00
yoav-steinberg 15c078df61 Enforce client output buffer soft limit when no traffic. (#8833)
When client breached the output buffer soft limit but then went idle,
we didn't disconnect on soft limit timeout, now we do.
Note this also resolves some sporadic test failures in due to Linux
buffering data which caused tests to fail if during the test we went
back under the soft COB limit.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: sundb <sundbcn@gmail.com>
(cherry picked from commit 152fce5e2c)
2021-06-01 17:03:36 +03:00
Oran Agra e90e5640e7 Redis 6.2.3
CI / test-ubuntu-latest (push) Has been cancelled
CI / build-ubuntu-old (push) Has been cancelled
CI / build-macos-latest (push) Has been cancelled
CI / build-32bit (push) Has been cancelled
CI / build-libc-malloc (push) Has been cancelled
CI / build-centos7-jemalloc (push) Has been cancelled
2021-05-03 22:57:00 +03:00
Oran Agra 2df6695f2b Resolve nonsense static analysis warnings
(cherry picked from commit fd7d51c353)
2021-05-03 22:57:00 +03:00
Oran Agra 92e3b1802f Fix integer overflow in STRALGO LCS (CVE-2021-29477)
An integer overflow bug in Redis version 6.0 or newer could be exploited using
the STRALGO LCS command to corrupt the heap and potentially result with remote
code execution.

(cherry picked from commit f0c5f920d0)
2021-05-03 22:57:00 +03:00
Oran Agra 0463520693 Fix integer overflow in intset (CVE-2021-29478)
An integer overflow bug in Redis 6.2 could be exploited to corrupt the heap and
potentially result with remote code execution.

The vulnerability involves changing the default set-max-intset-entries
configuration value, creating a large set key that consists of integer values
and using the COPY command to duplicate it.

The integer overflow bug exists in all versions of Redis starting with 2.6,
where it could result with a corrupted RDB or DUMP payload, but not exploited
through COPY (which did not exist before 6.2).

(cherry picked from commit 29900d4e6b)
2021-05-03 22:57:00 +03:00
sundb 922e3bf59f Fix memory leak in moduleDefragGlobals (#8853)
(cherry picked from commit 5100ef9f82)
2021-05-03 22:57:00 +03:00
Huang Zhw 42f2ad0516 Improve redis-cli help. When help command, we only match command (#8879)
prefix args not all args. So when we help commands with subcommands,
all subcommands will be output.

(cherry picked from commit 0b1b9edb28)
2021-05-03 22:57:00 +03:00
Binbin 1eab6202ac redis-benchmark: Add zfree(data) and fix lrange size / text mismatch (#8872)
missing zfree(data) in redis-benchmark.

And also correct the wrong size in lrange.
the text mentioned 500, but size was 450, changed to 500

(cherry picked from commit 1eff8564c7)
2021-05-03 22:57:00 +03:00
Binbin d30ee6c44a redis-cli: Do not use hostsocket when we got redirected in cluster mode (#8870)
When redis-cli was used with both -c (cluster) and -s (unix socket),
it would have kept trying to use that unix socket, even if it got
redirected by the cluster (resulting in an infinite loop).

(cherry picked from commit 416f277339)
2021-05-03 22:57:00 +03:00
filipe oliveira 4f2da00e94 redis-benchmark: Error/Warning handling updates. (#8869)
- Immediately exit on errors that are not related to topology updates.
- Deprecates the `-e` option ( retro compatible ) and warns that we now
  exit immediately on errors that are not related to topology updates.
- Fixed wrongfully failing on config fetch error (warning only). This only affects RE.

Bottom line:
- MOVED and ASK errors will not show any warning (unlike the throttled error with `-e` before).
- CLUSTERDOWN still prints an error unconditionally and sleeps for 1 second.
- other errors are fatal.

(cherry picked from commit ef6f902372)
2021-05-03 22:57:00 +03:00
Huang Zhw 34b9a3fa2e Fix potential CONFIG SET bind test failure. (#8875)
Use an invalid IP address to trigger CONFIG SET bind failure, instead of DNS which is not guaranteed to always fail.

(cherry picked from commit 2b22fffc78)
2021-05-03 22:57:00 +03:00
yoav-steinberg 5c7b869e61 Bump freebsd-vm version to fix CI failures (#8876)
Specifically we had issues with NTP sync failure which was resolved here: https://github.com/vmactions/freebsd-vm/commit/457af7345642e154a79d219971a2d4a7c7fe2118

(cherry picked from commit 2e88b06396)
2021-05-03 22:57:00 +03:00
Oran Agra 6cbea7d29b Prevent replicas from sending commands that interact with keyspace (#8868)
This solves an issue reported in #8712 in which a replica would bypass
the client write pause check and cause an assertion due to executing a
write command during failover.

The fact is that we don't expect replicas to execute any command other
than maybe REPLCONF and PING, etc. but matching against the ADMIN
command flag is insufficient, so instead i just block keyspace access
for now.

(cherry picked from commit 46f4ebbe84)
2021-05-03 22:57:00 +03:00
Yossi Gottlieb 8cfa37fc21 Remove redundant -latomic on arm64. (#8867)
(cherry picked from commit ebfbb09109)
2021-05-03 22:57:00 +03:00
Yang Bodong 16c53085f7 When the password is wrong, redis-benchmark should exit (#8855)
(cherry picked from commit 8423b77f14)
2021-05-03 22:57:00 +03:00
Istemi Ekin Akkus 2ccb926314 Modules: Fix RM_GetClusterNodeInfo() to correctly populate the master_id (#8846)
(cherry picked from commit af035c1e1d)
2021-05-03 22:57:00 +03:00
zyxwvu Shi 6384fe3414 Use monotonic clock to check for Lua script timeout. (#8812)
This prevents a case where NTP moves the system clock
forward resulting in a false detection of a busy script.

Signed-off-by: zyxwvu Shi <i@shiyc.cn>
(cherry picked from commit f61c37cec9)
2021-05-03 22:57:00 +03:00
Wang Yuan ef64333e63 Expire key firstly and then notify keyspace event (#8830)
Modules event subscribers may get wrong things in notifyKeyspaceEvent callback,
such as wrong number of keys, or be able to lookup this key.
This commit changes the order to be like the one in evict.c.

Cleanup:
Since we know the key exists (it expires now), db*Delete is sure to return 1,
so there's no need to check it's output (misleading).

(cherry picked from commit 63acfe4b00)
2021-05-03 22:57:00 +03:00
bugwz 0851705304 Print the number of abnormal line in AOF (#8823)
When redis-check-aof finds an error, it prints the line number for faster troubleshooting. 

(cherry picked from commit 761d7d2771)
2021-05-03 22:57:00 +03:00
Huang Zhw b97a4ad7f2 Fix migrateCommand may migrate wrong value. (#8815)
This scene is hard to happen. When first attempt some keys expired,
only kv position is updated not ov. Then socket err happens, second
attempt is taken. This time kv items may be mismatching with ov items.

(cherry picked from commit 080d4579db)
2021-05-03 22:57:00 +03:00
Madelyn Olson d01afe885c Fix memory leak when doing lazyfreeing client tracking table (#8822)
Interior rax pointers were not being freed

(cherry picked from commit c73b4ddfd9)
2021-05-03 22:57:00 +03:00
Oran Agra 959d6035e5 Merge 6.2.2 release
CI / test-ubuntu-latest (push) Has been cancelled
CI / build-ubuntu-old (push) Has been cancelled
CI / build-macos-latest (push) Has been cancelled
CI / build-32bit (push) Has been cancelled
CI / build-libc-malloc (push) Has been cancelled
CI / build-centos7-jemalloc (push) Has been cancelled
Release 6.2.2
2021-04-20 08:03:58 +03:00
Oran Agra aa730ef1ea Redis 6.2.2 2021-04-19 21:39:40 +03:00
Oran Agra f5ca1f9ee9 Merge unstable into 6.2 2021-04-19 21:36:00 +03:00
Oran Agra 92bde124ca Merge Redis 6.2.1
CI / test-ubuntu-latest (push) Has been cancelled
CI / build-ubuntu-old (push) Has been cancelled
CI / build-macos-latest (push) Has been cancelled
CI / build-32bit (push) Has been cancelled
CI / build-libc-malloc (push) Has been cancelled
CI / build-centos7-jemalloc (push) Has been cancelled
Redis 6.2.1
2021-03-02 08:14:39 +02:00
Oran Agra 7018ad69a2 Redis 6.2.1 2021-03-01 21:02:05 +02:00
Oran Agra 46df4db87b Merge unstable into 6.2 2021-03-01 20:54:02 +02:00
Oran Agra 445aa844b9 Merge Redis 6.2.0 GA
CI / test-ubuntu-latest (push) Has been cancelled
CI / build-ubuntu-old (push) Has been cancelled
CI / build-macos-latest (push) Has been cancelled
CI / build-32bit (push) Has been cancelled
CI / build-libc-malloc (push) Has been cancelled
CI / build-centos7-jemalloc (push) Has been cancelled
Redis 6.2.0 GA
2021-02-22 23:23:58 +02:00
Oran Agra f098fe319b Redis 6.2.0 GA 2021-02-22 15:56:55 +02:00
Oran Agra 2ab6fef099 Merge origin/unstable into 6.2 2021-02-22 15:48:42 +02:00
Oran Agra 2dba1e391d Merge 6.2 RC3
CI / test-ubuntu-latest (push) Has been cancelled
CI / build-ubuntu-old (push) Has been cancelled
CI / build-macos-latest (push) Has been cancelled
CI / build-32bit (push) Has been cancelled
CI / build-libc-malloc (push) Has been cancelled
CI / build-centos7-jemalloc (push) Has been cancelled
CI / build-freebsd (push) Has been cancelled
2021-02-01 20:11:42 +02:00
Oran Agra 95338f9cc4 Redis 6.2 RC3 2021-01-31 19:55:20 +02:00
Oran Agra d0854927fc Merge branch 'unstable' into 6.2 2021-01-31 12:20:34 +02:00
Oran Agra ec2d180739 Merge 6.2 RC2
CI / test-ubuntu-latest (push) Has been cancelled
CI / build-ubuntu-old (push) Has been cancelled
CI / build-macos-latest (push) Has been cancelled
CI / build-32bit (push) Has been cancelled
CI / build-libc-malloc (push) Has been cancelled
CI / build-centos7-jemalloc (push) Has been cancelled
2021-01-12 16:21:03 +02:00
Oran Agra 049f2f0805 Redis 6.2 RC2 2021-01-12 16:17:58 +02:00
Oran Agra b34fc03e5e Merge unstable into 6.2 2021-01-12 10:14:04 +02:00
Oran Agra b8c67ce41b Redis 6.2 RC1.
CI / test-ubuntu-latest (push) Has been cancelled
CI / build-ubuntu-old (push) Has been cancelled
CI / build-macos-latest (push) Has been cancelled
CI / build-32bit (push) Has been cancelled
CI / build-libc-malloc (push) Has been cancelled
CI / build-centos7-jemalloc (push) Has been cancelled
2020-12-14 20:54:10 +02:00
38 changed files with 991 additions and 221 deletions
+1 -1
View File
@@ -253,7 +253,7 @@ jobs:
steps:
- uses: actions/checkout@v2
- name: test
uses: vmactions/freebsd-vm@v0.1.2
uses: vmactions/freebsd-vm@v0.1.4
with:
usesh: true
sync: rsync
+586 -11
View File
@@ -1,16 +1,591 @@
Hello! This file is just a placeholder, since this is the "unstable" branch
of Redis, the place where all the development happens.
Redis 6.2 release notes
=======================
There is no release notes for this branch, it gets forked into another branch
every time there is a partial feature freeze in order to eventually create
a new stable release.
--------------------------------------------------------------------------------
Upgrade urgency levels:
Usually "unstable" is stable enough for you to use it in development environments
however you should never use it in production environments. It is possible
to download the latest stable release here:
LOW: No need to upgrade unless there are new features you want to use.
MODERATE: Program an upgrade of the server, but it's not urgent.
HIGH: There is a critical bug that may affect a subset of users. Upgrade!
CRITICAL: There is a critical bug affecting MOST USERS. Upgrade ASAP.
SECURITY: There are security fixes in the release.
--------------------------------------------------------------------------------
http://download.redis.io/releases/redis-stable.tar.gz
================================================================================
Redis 6.2.4 Released Tue July 1 12:00:00 IST 2021
================================================================================
More information is available at https://redis.io
Upgrade urgency: SECURITY, Contains fixes to security issues that affect
authenticated client connections. MODERATE otherwise.
Happy hacking!
Fix integer overflow in STRALGO LCS (CVE-2021-32625)
An integer overflow bug in Redis version 6.0 or newer can be exploited using the
STRALGO LCS command to corrupt the heap and potentially result with remote code
execution. This is a result of an incomplete fix by CVE-2021-29477.
Bug fixes that are only applicable to previous releases of Redis 6.2:
* Fix crash after a diskless replication fork child is terminated (#8991)
* Fix redis-benchmark crash on unsupported configs (#8916)
Other bug fixes:
* Fix crash in UNLINK on a stream key with deleted consumer groups (#8932)
* SINTERSTORE: Add missing keyspace del event when none of the sources exist (#8949)
* Sentinel: Fix CONFIG SET of empty string sentinel-user/sentinel-pass configs (#8958)
* Enforce client output buffer soft limit when no traffic (#8833)
Improvements:
* Hide AUTH passwords in MIGRATE command from slowlog (#8859)
================================================================================
Redis 6.2.3 Released Mon May 3 19:00:00 IST 2021
================================================================================
Upgrade urgency: SECURITY, Contains fixes to security issues that affect
authenticated client connections. LOW otherwise.
Integer overflow in STRALGO LCS command (CVE-2021-29477):
An integer overflow bug in Redis version 6.0 or newer could be exploited using
the STRALGO LCS command to corrupt the heap and potentially result in remote
code execution. The integer overflow bug exists in all versions of Redis
starting with 6.0.
Integer overflow in COPY command for large intsets (CVE-2021-29478):
An integer overflow bug in Redis 6.2 could be exploited to corrupt the heap and
potentially result with remote code execution. The vulnerability involves
changing the default set-max-intset-entries configuration value, creating a
large set key that consists of integer values and using the COPY command to
duplicate it. The integer overflow bug exists in all versions of Redis starting
with 2.6, where it could result with a corrupted RDB or DUMP payload, but not
exploited through COPY (which did not exist before 6.2).
Bug fixes that are only applicable to previous releases of Redis 6.2:
* Fix memory leak in moduleDefragGlobals (#8853)
* Fix memory leak when doing lazy freeing client tracking table (#8822)
* Block abusive replicas from sending command that could assert and crash redis (#8868)
Other bug fixes:
* Use a monotonic clock to check for Lua script timeout (#8812)
* redis-cli: Do not use unix socket when we got redirected in cluster mode (#8870)
Modules:
* Fix RM_GetClusterNodeInfo() to correctly populate master id (#8846)
================================================================================
Redis 6.2.2 Released Mon April 19 19:00:00 IST 2021
================================================================================
Upgrade urgency: HIGH, if you're using ACL and pub/sub, CONFIG REWRITE, or
suffering from performance regression. see below.
Bug fixes for regressions in previous releases of Redis 6.2:
* Fix BGSAVE, AOFRW, and replication slowdown due to child reporting CoW (#8645)
* Fix short busy loop when timer event is about to fire (#8764)
* Fix default user, overwritten and reset users losing pubsub channel permissions (#8723)
* Fix config rewrite with an empty `save` config resulsing in default `save` values (#8719)
* Fix not starting on alpine/libmusl without IPv6 (#8655)
* Fix issues with propagation and MULTI/EXEC in modules (#8617)
Several issues around nested calls and thread safe contexts
Bug fixes that are only applicable to previous releases of Redis 6.2:
* ACL Pub/Sub channels permission handling for save/load scenario (#8794)
* Fix early rejection of PUBLISH inside MULTI-EXEC transaction (#8534)
* Fix missing SLOWLOG records for blocked commands (#8632)
* Allow RESET command during busy scripts (#8629)
* Fix some error replies were not counted on stats (#8659)
Bug fixes:
* Add a timeout mechanism for replicas stuck in fullsync (#8762)
* Process HELLO command even if the default user has no permissions (#8633)
* Client issuing a long running script and using a pipeline, got disconnected (#8715)
* Fix script kill to work also on scripts that use `pcall` (#8661)
* Fix list-compress-depth may compress more node than required (#8311)
* Fix redis-cli handling of rediss:// URL scheme (#8705)
* Cluster: Skip unnecessary check which may prevent failure detection (#8585)
* Cluster: Fix hang manual failover when replica just started (#8651)
* Sentinel: Fix info-refresh time field before sentinel get first response (#8567)
* Sentinel: Fix possible crash on failed connection attempt (#8627)
* Systemd: Send the readiness notification when a replica is ready to accept connections (#8409)
Command behavior changes:
* ZADD: fix wrong reply when INCR used with GT/LT which blocked the update (#8717)
It was responding with the incremented value rather than nil
* XAUTOCLAIM: fix response to return the next available id as the cursor (#8725)
Previous behavior was retuning the last one which was already scanned
* XAUTOCLAIM: fix JUSTID to prevent incrementing delivery_count (#8724)
New config options:
* Add cluster-allow-replica-migration config option (#5285)
* Add replica-announced config option (#8653)
* Add support for plaintext clients in TLS cluster (#8587)
* Add support for reading encrypted keyfiles (#8644)
Improvements:
* Fix performance regression in BRPOP on Redis 6.0 (#8689)
* Avoid adding slowlog entries for config with sensitive data (#8584)
* Improve redis-cli non-binary safe string handling (#8566)
* Optimize CLUSTER SLOTS reply (#8541)
* Handle remaining fsync errors (#8419)
Info fields and introspection changes:
* Strip % sign from current_fork_perc info field (#8628)
* Fix RSS memory info on FreeBSD (#8620)
* Fix client_recent_max_input/output_buffer in 'INFO CLIENTS' when all clients drop (#8588)
* Fix invalid master_link_down_since_seconds in info replication (#8785)
Platform and deployment-related changes:
* Fix FreeBSD <12.x builds (#8603)
Modules:
* Add macros for RedisModule_log logging levels (#4246)
* Add RedisModule_GetAbsExpire / RedisModule_SetAbsExpire (#8564)
* Add a module type for key space notification (#8759)
* Set module eviction context flag only in masters (#8631)
* Fix unusable RedisModule_IsAOFClient API (#8596)
* Fix missing EXEC on modules propagation after failed EVAL execution (#8654)
* Fix edge-case when a module client is unblocked (#8618)
================================================================================
Redis 6.2.1 Released Mon Mar 1 17:51:36 IST 2021
================================================================================
Upgrade urgency: LOW.
Here is a comprehensive list of changes in this release compared to 6.2.0,
each one includes the PR number that added it, so you can get more details
at https://github.com/redis/redis/pull/<number>
Bug fixes:
* Fix sanitize-dump-payload for stream with deleted records (#8568)
* Prevent client-query-buffer-limit config from being set to lower than 1mb (#8557)
Improvements:
* Make port, tls-port and bind config options modifiable at runtime (#8510)
Platform and deployment-related changes:
* Fix compilation error on non-glibc systems if jemalloc is not used (#8533)
* Improved memory consumption and memory usage tracking on FreeBSD (#8545)
* Fix compilation on ARM64 MacOS with jemalloc (#8458)
Modules:
* New Module API for getting user name of a client (#8508)
* Optimize RM_Call by utilizing a shared reusable client (#8516)
* Fix crash running CLIENT INFO via RM_Call (#8560)
================================================================================
Redis 6.2.0 GA Released Tue Feb 22 14:00:00 IST 2021
================================================================================
Upgrade urgency: SECURITY if you use 32bit build of redis (see bellow), MODERATE
if you used earlier versions of Redis 6.2, LOW otherwise.
Integer overflow on 32-bit systems (CVE-2021-21309):
Redis 4.0 or newer uses a configurable limit for the maximum supported bulk
input size. By default, it is 512MB which is a safe value for all platforms.
If the limit is significantly increased, receiving a large request from a client
may trigger several integer overflow scenarios, which would result with buffer
overflow and heap corruption.
Here is a comprehensive list of changes in this release compared to 6.2 RC3,
each one includes the PR number that added it, so you can get more details
at https://github.com/redis/redis/pull/<number>
Bug fixes:
* Avoid 32-bit overflows when proto-max-bulk-len is set high (#8522)
* Fix broken protocol in client tracking tracking-redir-broken message (#8456)
* Avoid unsafe field name characters in INFO commandstats, errorstats, modules (#8492)
* XINFO able to access expired keys during CLIENT PAUSE WRITE (#8436)
* Fix allowed length for REPLCONF ip-address, needed due to Sentinel's support for hostnames (#8517)
* Fix broken protocol in redis-benchmark when used with -a or --dbnum (#8486)
* XADD counts deleted records too when considering switching to a new listpack (#8390)
Bug fixes that are only applicable to previous releases of Redis 6.2:
* Fixes in GEOSEARCH bybox (accuracy and mismatch between width and height) (#8445)
* Fix risk of OOM panic in HRANDFIELD, ZRANDMEMBER commands with huge negative count (#8429)
* Fix duplicate replicas issue in Sentinel, needed due to hostname support (#8481)
* Fix Sentinel configuration rewrite, an improvement of #8271 (#8480)
Command behavior changes:
* SRANDMEMBER uses RESP3 array type instead of set type (#8504)
* EXPIRE, EXPIREAT, SETEX, GETEX: Return error when provided expire time overflows (#8287)
Other behavior changes:
* Remove ACL subcommand validation if fully added command exists. (#8483)
Improvements:
* Optimize sorting in GEORADIUS / GEOSEARCH with COUNT (#8326)
* Optimize HRANDFIELD and ZRANDMEMBER case 4 when ziplist encoded (#8444)
* Optimize in-place replacement of elements in HSET, HINCRBY, LSET (#8493)
* Remove redundant list to store pubsub patterns (#8472)
* Add --insecure option to command line tools (#8416)
Info fields and introspection changes:
* Add INFO fields to track progress of BGSAVE, AOFRW, replication (#8414)
Modules:
* RM_ZsetRem: Delete key if empty, the bug could leave empty zset keys (#8453)
* RM_HashSet: Add COUNT_ALL flag and set errno (#8446)
================================================================================
Redis 6.2 RC3 Released Tue Feb 1 14:00:00 IST 2021
================================================================================
Upgrade urgency LOW: This is the third Release Candidate of Redis 6.2.
Here is a comprehensive list of changes in this release compared to 6.2 RC2,
each one includes the PR number that added it, so you can get more details
at https://github.com/redis/redis/pull/<number>
New commands / args:
* Add HRANDFIELD and ZRANDMEMBER commands (#8297)
* Add FAILOVER command (#8315)
* Add GETEX, GETDEL commands (#8327)
* Add PXAT/EXAT arguments to SET command (#8327)
* Add SYNC arg to FLUSHALL and FLUSHDB, and ASYNC/SYNC arg to SCRIPT FLUSH (#8258)
Sentinel:
* Add hostname support to Sentinel (#8282)
* Prevent file descriptors from leaking into Sentinel scripts (#8242)
* Fix config file line order dependency and config rewrite sequence (#8271)
New configuration options:
* Add set-proc-title config option to disable changes to the process title (#3623)
* Add proc-title-template option to control what's shown in the process title (#8397)
* Add lazyfree-lazy-user-flush config option to control FLUSHALL, FLUSHDB and SCRIPT FLUSH (#8258)
Bug fixes:
* AOF: recover from last write error by turning on/off appendonly config (#8030)
* Exit on fsync error when the AOF fsync policy is 'always' (#8347)
* Avoid assertions (on older kernels) when testing arm64 CoW bug (#8405)
* CONFIG REWRITE should honor umask settings (#8371)
* Fix firstkey,lastkey,step in COMMAND command for some commands (#8367)
Special considerations:
* Fix misleading description of the save configuration directive (#8337)
Improvements:
* A way to get RDB file via replication without excessive replication buffers (#8303)
* Optimize performance of clusterGenNodesDescription for large clusters (#8182)
Info fields and introspection changes:
* SLOWLOG and LATENCY monitor include unblocking time of blocked commands (#7491)
Modules:
* Add modules API for streams (#8288)
* Add event for fork child birth and termination (#8289)
* Add RM_BlockedClientMeasureTime* etc, to track background processing in commandstats (#7491)
* Fix bug in v6.2, wrong value passed to the new unlink callback (#8381)
* Fix bug in v6.2, modules blocked on keys unblock on commands like LPUSH (#8356)
================================================================================
Redis 6.2 RC2 Released Tue Jan 12 16:17:20 IST 2021
================================================================================
Upgrade urgency LOW: This is the second Release Candidate of Redis 6.2.
IMPORTANT: If you're running Redis on ARM64 or a big-endian system, upgrade may
have significant implications. Please be sure to read the notes below.
Here is a comprehensive list of changes in this release compared to 6.2 RC1,
each one includes the PR number that added it, so you can get more details
at https://github.com/redis/redis/pull/<number>
New commands / args:
* Add the REV, BYLEX and BYSCORE arguments to ZRANGE, and the ZRANGESTORE command (#7844)
* Add the XAUTOCLAIM command (#7973)
* Add the MINID trimming strategy and the LIMIT argument to XADD and XTRIM (#8169)
* Add the ANY argument to GEOSEARCH and GEORADIUS (#8259)
* Add the CH, NX, XX arguments to GEOADD (#8227)
* Add the COUNT argument to LPOP and RPOP (#8179)
* Add the WRITE argument to CLIENT PAUSE for pausing write commands exclusively (#8170)
* Change the proto-ver argument of HELLO to optional (#7377)
* Add the CLIENT TRACKINGINFO subcommand (#7309)
Command behavior changes:
* CLIENT TRACKING yields an error when given overlapping BCAST prefixes (#8176)
* SWAPDB invalidates WATCHed keys (#8239)
* SORT command behaves differently when used on a writable replica (#8283)
Other behavior changes:
* Avoid propagating MULTI/EXEC for read-only transactions (#8216)
* Remove the read-only flag from TIME, ECHO, ROLE, LASTSAVE (#8216)
* Fix the command flags of PFDEBUG (#8222)
* Tracking clients will no longer receive unnecessary key invalidation messages after FLUSHDB (#8039)
* Sentinel: Fix missing updates to the config file after SENTINEL SET command (#8229)
Bug fixes with compatibility implications (bugs introduced in Redis 6.0):
* Fix RDB CRC64 checksum on big-endian systems (#8270)
If you're using big-endian please consider the compatibility implications with
RESTORE, replication and persistence.
* Fix wrong order of key/value in Lua's map response (#8266)
If your scripts use redis.setresp() or return a map (new in Redis 6.0), please
consider the implications.
Bug fixes that are only applicable to previous releases of Redis 6.2:
* Resolve rare assertions in active defragmentation while loading (#8284, #8281)
Bug fixes:
* Fix the selection of a random element from large hash tables (#8133)
* Fix an issue where a forked process deletes the parent's pidfile (#8231)
* Fix crashes when enabling io-threads-do-reads (#8230)
* Fix a crash in redis-cli after executing cluster backup (#8267)
* Fix redis-benchmark to use an IP address for the first cluster node (#8154)
* Fix saving of strings larger than 2GB into RDB files (#8306)
Additional improvements:
* Improve replication handshake time (#8214)
* Release client tracking table memory asynchronously in cases where the DB is also freed asynchronously (#8039)
* Avoid wasteful transient memory allocation in certain cases (#8286, #5954)
* Handle binary string values by the 'requirepass' and 'masterauth' configs (#8200)
Platform and deployment-related changes:
* Install redis-check-rdb and redis-check-aof as symlinks to redis-server (#5745)
* Add a check for an ARM64 Linux kernel bug (#8224)
Due to the potential severity of this issue, Redis will refuse to run on
affected platforms by default.
Info fields and introspection changes:
* Add the errorstats section to the INFO command (#8217)
* Add the failed_calls and rejected_calls fields INFO's commandstats section (#8217)
* Report child copy-on-write metrics continuously (#8264)
Module API changes:
* Add the RedisModule_SendChildCOWInfo API (#8264)
* Add the may-replicate command flag (#8170)
================================================================================
Redis 6.2 RC1 Released Mon Dec 14 11:50:00 IST 2020
================================================================================
Upgrade urgency LOW: This is the first Release Candidate of Redis 6.2.
Introduction to the Redis 6.2 release
=====================================
This release is the first significant Redis release managed by the core team
under the new project governance model.
Redis 6.2 includes many new commands and improvements, but no big features. It
mainly makes Redis more complete and addresses issues that have been requested
by many users frequently or for a long time.
Many of these changes were not eligible for 6.0.x for several reasons:
1. They are not backward compatible, which is always the case with new or
extended commands (that cannot be replicated to an older replica).
2. They require a longer release-candidate test cycle.
Here is a comprehensive list of changes in this release compared to 6.0.9,
each one includes the PR number that added it, so you can get more details
at https://github.com/redis/redis/pull/<number>
New commands / args:
* Add SMISMEMBER command that checks multiple members (#7615)
* Add ZMSCORE command that returns an array of scores (#7593)
* Add LMOVE and BLMOVE commands that pop and push arbitrarily (#6929)
* Add RESET command that resets client connection state (#7982)
* Add COPY command that copies keys (#7953)
* Add ZDIFF and ZDIFFSTORE commands (#7961)
* Add ZINTER and ZUNION commands (#7794)
* Add GEOSEARCH/GEOSEARCHSTORE commands for bounding box spatial queries (#8094)
* Add GET parameter to SET command, for more powerful GETSET (#7852)
* Add exclusive range query to XPENDING (#8130)
* Add exclusive range query to X[REV]RANGE (#8072)
* Add GT and LT options to ZADD for conditional score updates (#7818)
* Add CLIENT INFO and CLIENT LIST for specific ids (#8113)
* Add IDLE argument to XPENDING command (#7972)
* Add local address to CLIENT LIST, and a CLIENT KILL filter. (#7913)
* Add NOMKSTREAM option to XADD command (#7910)
* Add command introspection to Sentinel (#7940)
* Add SENTINEL MYID subcommand (#7858)
New features:
* Dump payload sanitization: prevent corrupt payload causing crashes (#7807)
Has flags to enable full O(N) validation (disabled by default).
* ACL patterns for Pub/Sub channels (#7993)
* Support ACL for Sentinel mode (#7888)
* Support getting configuration from both stdin and file at the same time (#7893)
Lets you avoid storing secrets on the disk.
New features in CLI tools:
* redis-cli RESP3 push support (#7609)
* redis-cli cluster import support source and target that require auth (#7994)
* redis-cli URIs able to provide user name in addition to password (#8048)
* redis-cli/redis-benchmark allow specifying the prefered ciphers/ciphersuites (#8005)
* redis-cli add -e option to exit with code when command execution fails (#8136)
Command behavior changes:
* EXISTS should not alter LRU (#8016)
In Redis 5.0 and 6.0 it would have touched the LRU/LFU of the key.
* OBJECT should not reveal logically expired keys (#8016)
Will now behave the same TYPE or any other non-DEBUG command.
* Improve db id range check for SELECT and MOVE (#8085)
Changes the error message text on a wrong db index.
* Modify AUTH / HELLO error message (#7648)
Changes the error message text when the user isn't found or is disabled.
* BITOPS length limited to proto_max_bulk_len rather than 512MB (#8096)
The limit is now configurable like in SETRANGE, and APPEND.
* GEORADIUS[BYMEMBER] can fail with -OOM if Redis is over the memory limit (#8107)
Other behavior changes:
* Optionally (default) fail to start if requested bind address is not available (#7936)
If you rely on Redis starting successfully even if one of the bind addresses
is not available, you'll need to tune the new config.
* Limit the main db dictionaries expansion to prevent key eviction (#7954)
In the past big dictionary rehashing could result in massive data eviction.
Now this rehashing is delayed (up to a limit), which can result in performance
loss due to hash collisions.
* CONFIG REWRITE is atomic and safer, but requires write access to the config file's folder (#7824, #8051)
This change was already present in 6.0.9, but was missing from the release
notes.
* A new incremental eviction mechanism that reduces latency on eviction spikes (#7653)
In pathological cases this can cause memory to grow uncontrolled and may require
specific tuning.
* Not resetting "save" config when Redis is started with command line arguments. (#7092)
In case you provide command line arguments without "save" and count on it
being disabled, Now the defaults "save" config will kick in.
* Update memory metrics for INFO during loading (#7690)
* When "supervised" config is enabled, it takes precedence over "daemonize". (#8036)
* Assertion and panic, print crash log without generating SIGSEGV (#7585)
* Added crash log report on SIGABRT, instead of silently exiting (#8004)
* Disable THP (Transparent Huge Pages) if enabled (#7381)
If you deliberately enabled it, you'll need to config Redis to keep it.
Bug fixes:
* Handle output buffer limits for module blocked clients (#8141)
Could result in a module sending reply to a blocked client to go beyond the
limit.
* Fix setproctitle related crashes. (#8150, #8088)
Caused various crashes on startup, mainly on Apple M1 chips or under
instrumentation.
* A module doing RM_Call could cause replicas to get nested MULTI (#8097).
* Backup/restore cluster mode keys to slots map for repl-diskless-load=swapdb (#8108)
In cluster mode with repl-diskless-load, when loading failed, slot map
wouldn't have been restored.
* Fix oom-score-adj-values range, and bug when used in config file (#8046)
Enabling setting this in the config file in a line after enabling it, would
have been buggy.
* Reset average ttl when empty databases (#8106)
Just causing misleading metric in INFO
* Disable rehash when Redis has child process (#8007)
This could have caused excessive CoW during BGSAVE, replication or AOFRW.
* Further improved ACL algorithm for picking categories (#7966)
Output of ACL GETUSER is now more similar to the one provided by ACL SETUSER.
* Fix bug with module GIL being released prematurely (#8061)
Could in theory (and rarely) cause multi-threaded modules to corrupt memory.
* Fix cluster redirect for module command with no firstkey. (#7539)
* Reduce effect of client tracking causing feedback loop in key eviction (#8100)
* Kill disk-based fork child when all replicas drop and 'save' is not enabled (#7819)
* Rewritten commands (modified for propagation) are logged as their original command (#8006)
* Fix cluster access to unaligned memory (SIGBUS on old ARM) #7958
* If diskless repl child is killed, make sure to reap the child pid (#7742)
* Broadcast a PONG message when slot's migration is over, may reduce MOVED responses (#7571)
Other improvements:
* TLS Support in redis-benchmark (#7959)
* Accelerate diskless master connections, and general re-connections (#6271)
* Run active defrag while blocked / loading (#7726)
* Performance and memory reporting improvement - sds take control of its internal fragmentation (#7875)
* Speedup cluster failover. (#7948)
Platform / toolchain support related improvements:
* Optionally (not by default) use H/W Monotonic clock for faster time sampling (#7644)
* Remove the requirements for C11 and _Atomic supporting compiler (#7707)
This would allow to more easily build and use Redis on older systems and
compilers again.
* Fix crash log registers output on ARM. (#8020)
* Raspberry build fix. (#8095)
* Setting process title support for Haiku. (#8060)
* DragonFlyBSD RSS memory sampling support. (#8023)
New configuration options:
* Enable configuring OpenSSL using the standard openssl.cnf (#8143)
* oom-score-adj-values config can now take absolute values (besides relative ones) (#8046)
* TLS: Add different client cert support. (#8076)
* Note that a few other changes listed above added their config options.
Info fields and introspection changes:
* Add INFO fields to track diskless and disk-based replication progress (#7981)
* Add INFO field for main thread cpu time, and scrape system time. (#8132)
* Add total_forks to INFO STATS (#8155)
* Add maxclients and cluster_connections to INFO CLIENTS (#7979)
* Add tracking bcast flag and client redirection in client list (#7995)
* Fixed INFO client_recent_max_input_buffer includes argv array (#8065, see #7874)
* Note that a few other changes listed above added their info fields.
Module API changes:
* Add CTX_FLAGS_DENY_BLOCKING as a unified the way to know if blocking is allowed (#8025)
* Add data type callbacks for lazy free effort, and unlink (#7912)
* Add data type callback for COPY command (#8112)
* Add callbacks for defrag support. (#8149)
* Add module event for repl-diskless-load swapdb (#8153)
Module related fixes:
* Moved RMAPI_FUNC_SUPPORTED so that it's usable (#8037)
* Improve timer accuracy (#7987)
* Allow '\0' inside of result of RM_CreateStringPrintf (#6260)
Thanks to all the users and developers who made this release possible.
We'll follow up with more RC releases, until the code looks production ready
and we don't get reports of serious issues for a while.
A special thank you for the amount of work put into this release by:
- Oran Agra
- Yossi Gottlieb
- Viktor Söderqvist
- Yang Bodong
- Filipe Oliveira
- Guy Benoish
- Itamar Haber
- Madelyn Olson
- Wang Yuan
- Felipe Machado
- Wen Hui
- Tatsuya Arisawa
- Jonah H. Harris
- Raghav Muddur
- Jim Brunner
- Yaacov Hazan
- Allen Farris
- Chen Yang
- Nitai Caro
- sundb
- Meir Shpilraien
- maohuazhu
- Valentino Geron
- Zhao Zhao
- Qu Chen
- George Prekas
- Tyson Andre
- Uri Yagelnik
- Michael Grunder
- Huang Zw
- alexronke-channeladvisor
- Andy Pan
- Wu Yunlong
- Wei Kukey
- Yoav Steinberg
- Greg Femec
- Uri Shachar
- Nykolas Laurentino de Lima
- xhe
- zhenwei pi
- David CARLIER
Migrating from 6.0 to 6.2
=========================
Redis 6.2 is mostly a strict superset of 6.0, you should not have any problem
upgrading your application from 6.0 to 6.2. However there are some small changes
of behavior listed above, please make sure you are not badly affected by any of
them.
Specifically these sections:
* Command behavior changes
* Other behavior changes
--------------------------------------------------------------------------------
Cheers,
The Redis team
@@ -235,7 +235,7 @@ iget_defrag_hint(tsdn_t *tsdn, void* ptr) {
int free_in_slab = extent_nfree_get(slab);
if (free_in_slab) {
const bin_info_t *bin_info = &bin_infos[binind];
int curslabs = bin->stats.curslabs;
unsigned long curslabs = bin->stats.curslabs;
size_t curregs = bin->stats.curregs;
if (bin->slabcur) {
/* remove slabcur from the overall utilization */
+1 -5
View File
@@ -93,14 +93,10 @@ FINAL_LDFLAGS=$(LDFLAGS) $(REDIS_LDFLAGS) $(DEBUG)
FINAL_LIBS=-lm
DEBUG=-g -ggdb
# Linux ARM needs -latomic at linking time
ifneq (,$(filter aarch64 armv,$(uname_M)))
FINAL_LIBS+=-latomic
else
# Linux ARM32 needs -latomic at linking time
ifneq (,$(findstring armv,$(uname_M)))
FINAL_LIBS+=-latomic
endif
endif
ifeq ($(uname_S),SunOS)
# SunOS
+9 -4
View File
@@ -1892,10 +1892,6 @@ void addACLLogEntry(client *c, int reason, int argpos, sds username) {
void aclCommand(client *c) {
char *sub = c->argv[1]->ptr;
if (!strcasecmp(sub,"setuser") && c->argc >= 3) {
/* Consider information about passwords or permissions
* to be sensitive, which will be the arguments for this
* subcommand. */
preventCommandLogging(c);
sds username = c->argv[2]->ptr;
/* Check username validity. */
if (ACLStringHasSpaces(username,sdslen(username))) {
@@ -1912,6 +1908,12 @@ void aclCommand(client *c) {
user *u = ACLGetUserByName(username,sdslen(username));
if (u) ACLCopyUser(tempu, u);
/* Initially redact all of the arguments to not leak any information
* about the user. */
for (int j = 2; j < c->argc; j++) {
redactClientCommandArgument(c, j);
}
for (int j = 3; j < c->argc; j++) {
if (ACLSetUser(tempu,c->argv[j]->ptr,sdslen(c->argv[j]->ptr)) != C_OK) {
const char *errmsg = ACLSetUserStringError();
@@ -2245,6 +2247,8 @@ void authCommand(client *c) {
addReplyErrorObject(c,shared.syntaxerr);
return;
}
/* Always redact the second argument */
redactClientCommandArgument(c, 1);
/* Handle the two different forms here. The form with two arguments
* will just use "default" as username. */
@@ -2264,6 +2268,7 @@ void authCommand(client *c) {
} else {
username = c->argv[1];
password = c->argv[2];
redactClientCommandArgument(c, 2);
}
if (ACLAuthenticateUser(c,username,password) == C_OK) {
+1 -1
View File
@@ -93,7 +93,7 @@ void sendChildInfoGeneric(childInfoType info_type, size_t keys, double progress,
if (cow) {
serverLog((info_type == CHILD_INFO_TYPE_CURRENT_INFO) ? LL_VERBOSE : LL_NOTICE,
"%s: %zu MB of memory used by copy-on-write",
pname, data.cow / (1024 * 1024));
pname, cow / (1024 * 1024));
}
}
+5 -1
View File
@@ -5361,13 +5361,16 @@ void migrateCommand(client *c) {
}
j++;
password = c->argv[j]->ptr;
redactClientCommandArgument(c,j);
} else if (!strcasecmp(c->argv[j]->ptr,"auth2")) {
if (moreargs < 2) {
addReplyErrorObject(c,shared.syntaxerr);
return;
}
username = c->argv[++j]->ptr;
redactClientCommandArgument(c,j);
password = c->argv[++j]->ptr;
redactClientCommandArgument(c,j);
} else if (!strcasecmp(c->argv[j]->ptr,"keys")) {
if (sdslen(c->argv[3]->ptr) != 0) {
addReplyError(c,
@@ -5465,9 +5468,10 @@ try_again:
if (ttl < 1) ttl = 1;
}
/* Relocate valid (non expired) keys into the array in successive
/* Relocate valid (non expired) keys and values into the array in successive
* positions to remove holes created by the keys that were present
* in the first lookup but are now expired after the second lookup. */
ov[non_expired] = ov[j];
kv[non_expired++] = kv[j];
serverAssertWithInfo(c,NULL,
+1 -1
View File
@@ -726,7 +726,7 @@ void configSetCommand(client *c) {
(config->alias && !strcasecmp(c->argv[2]->ptr,config->alias))))
{
if (config->flags & SENSITIVE_CONFIG) {
preventCommandLogging(c);
redactClientCommandArgument(c,3);
}
if (!config->interface.set(config->data,o->ptr,1,&errstr)) {
goto badfmt;
+8 -5
View File
@@ -1480,7 +1480,7 @@ int keyIsExpired(redisDb *db, robj *key) {
* script execution, making propagation to slaves / AOF consistent.
* See issue #1525 on Github for more information. */
if (server.lua_caller) {
now = server.lua_time_start;
now = server.lua_time_snapshot;
}
/* If we are in the middle of a command execution, we still want to use
* a reference time that does not change: in that case we just use the
@@ -1541,14 +1541,17 @@ int expireIfNeeded(redisDb *db, robj *key) {
if (checkClientPauseTimeoutAndReturnIfPaused()) return 1;
/* Delete the key */
if (server.lazyfree_lazy_expire) {
dbAsyncDelete(db,key);
} else {
dbSyncDelete(db,key);
}
server.stat_expiredkeys++;
propagateExpire(db,key,server.lazyfree_lazy_expire);
notifyKeyspaceEvent(NOTIFY_EXPIRED,
"expired",key,db->id);
int retval = server.lazyfree_lazy_expire ? dbAsyncDelete(db,key) :
dbSyncDelete(db,key);
if (retval) signalModifiedKey(NULL,db,key);
return retval;
signalModifiedKey(NULL,db,key);
return 1;
}
/* -----------------------------------------------------------------------------
+1 -1
View File
@@ -281,7 +281,7 @@ uint32_t intsetLen(const intset *is) {
/* Return intset blob size in bytes. */
size_t intsetBlobLen(intset *is) {
return sizeof(intset)+intrev32ifbe(is->length)*intrev32ifbe(is->encoding);
return sizeof(intset)+(size_t)intrev32ifbe(is->length)*intrev32ifbe(is->encoding);
}
/* Validate the integrity of the data structure.
+3 -4
View File
@@ -39,12 +39,11 @@ void lazyfreeFreeSlotsMap(void *args[]) {
atomicIncr(lazyfreed_objects,len);
}
/* Release the rax mapping Redis Cluster keys to slots in the
* lazyfree thread. */
/* Release the key tracking table. */
void lazyFreeTrackingTable(void *args[]) {
rax *rt = args[0];
size_t len = rt->numele;
raxFree(rt);
freeTrackingRadixTree(rt);
atomicDecr(lazyfree_objects,len);
atomicIncr(lazyfreed_objects,len);
}
@@ -110,7 +109,7 @@ size_t lazyfreeGetFreeEffort(robj *key, robj *obj) {
/* Every consumer group is an allocation and so are the entries in its
* PEL. We use size of the first group's PEL as an estimate for all
* others. */
if (s->cgroups) {
if (s->cgroups && raxSize(s->cgroups)) {
raxIterator ri;
streamCG *cg;
raxStart(&ri,s->cgroups);
+2 -2
View File
@@ -94,8 +94,8 @@ lwCanvas *lwCreateCanvas(int width, int height, int bgcolor) {
lwCanvas *canvas = zmalloc(sizeof(*canvas));
canvas->width = width;
canvas->height = height;
canvas->pixels = zmalloc(width*height);
memset(canvas->pixels,bgcolor,width*height);
canvas->pixels = zmalloc((size_t)width*height);
memset(canvas->pixels,bgcolor,(size_t)width*height);
return canvas;
}
+1 -1
View File
@@ -71,7 +71,7 @@ void memtest_progress_start(char *title, int pass) {
printf("\x1b[H\x1b[2K"); /* Cursor home, clear current line. */
printf("%s [%d]\n", title, pass); /* Print title. */
progress_printed = 0;
progress_full = ws.ws_col*(ws.ws_row-3);
progress_full = (size_t)ws.ws_col*(ws.ws_row-3);
fflush(stdout);
}
+2 -1
View File
@@ -6168,7 +6168,7 @@ int RM_GetClusterNodeInfo(RedisModuleCtx *ctx, const char *id, char *ip, char *m
/* If the information is not available, the function will set the
* field to zero bytes, so that when the field can't be populated the
* function kinda remains predictable. */
if (node->flags & CLUSTER_NODE_MASTER && node->slaveof)
if (node->flags & CLUSTER_NODE_SLAVE && node->slaveof)
memcpy(master_id,node->slaveof->name,REDISMODULE_NODE_ID_LEN);
else
memset(master_id,0,REDISMODULE_NODE_ID_LEN);
@@ -9205,6 +9205,7 @@ long moduleDefragGlobals(void) {
module->defrag_cb(&defrag_ctx);
defragged += defrag_ctx.defragged;
}
dictReleaseIterator(di);
return defragged;
}
+2 -12
View File
@@ -153,8 +153,7 @@ void execCommandAbort(client *c, sds error) {
/* Send EXEC to clients waiting data from MONITOR. We did send a MULTI
* already, and didn't send any of the queued commands, now we'll just send
* EXEC so it is clear that the transaction is over. */
if (listLength(server.monitors) && !server.loading)
replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc);
replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc);
}
void execCommand(client *c) {
@@ -179,7 +178,7 @@ void execCommand(client *c) {
addReply(c, c->flags & CLIENT_DIRTY_EXEC ? shared.execaborterr :
shared.nullarray[c->resp]);
discardTransaction(c);
goto handle_monitor;
return;
}
uint64_t old_flags = c->flags;
@@ -266,15 +265,6 @@ void execCommand(client *c) {
}
server.in_exec = 0;
handle_monitor:
/* Send EXEC to clients waiting data from MONITOR. We do it here
* since the natural order of commands execution is actually:
* MUTLI, EXEC, ... commands inside transaction ...
* Instead EXEC is flagged as CMD_SKIP_MONITOR in the command
* table, and we do it here with correct ordering. */
if (listLength(server.monitors) && !server.loading)
replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc);
}
/* ===================== WATCH (CAS alike for MULTI/EXEC) ===================
+35 -13
View File
@@ -333,7 +333,7 @@ void _addReplyProtoToList(client *c, const char *s, size_t len) {
listAddNodeTail(c->reply, tail);
c->reply_bytes += tail->size;
asyncCloseClientOnOutputBufferLimitReached(c);
closeClientOnOutputBufferLimitReached(c, 1);
}
}
@@ -616,7 +616,7 @@ void setDeferredReply(client *c, void *node, const char *s, size_t length) {
listNodeValue(ln) = buf;
c->reply_bytes += buf->size;
asyncCloseClientOnOutputBufferLimitReached(c);
closeClientOnOutputBufferLimitReached(c, 1);
}
}
@@ -949,7 +949,7 @@ void AddReplyFromClient(client *dst, client *src) {
src->bufpos = 0;
/* Check output buffer limits */
asyncCloseClientOnOutputBufferLimitReached(dst);
closeClientOnOutputBufferLimitReached(dst, 1);
}
/* Copy 'src' client output buffers into 'dst' client output buffers.
@@ -1663,9 +1663,6 @@ void resetClient(client *c) {
c->flags |= CLIENT_REPLY_SKIP;
c->flags &= ~CLIENT_REPLY_SKIP_NEXT;
}
/* Always clear the prevent logging field. */
c->flags &= ~CLIENT_PREVENT_LOGGING;
}
/* This function is used when we want to re-enter the event loop but there
@@ -2967,7 +2964,8 @@ void helloCommand(client *c) {
int moreargs = (c->argc-1) - j;
const char *opt = c->argv[j]->ptr;
if (!strcasecmp(opt,"AUTH") && moreargs >= 2) {
preventCommandLogging(c);
redactClientCommandArgument(c, j+1);
redactClientCommandArgument(c, j+2);
if (ACLAuthenticateUser(c, c->argv[j+1], c->argv[j+2]) == C_ERR) {
addReplyError(c,"-WRONGPASS invalid username-password pair or user is disabled.");
return;
@@ -3054,6 +3052,15 @@ static void retainOriginalCommandVector(client *c) {
}
}
/* Redact a given argument to prevent it from being shown
* in the slowlog. This information is stored in the
* original_argv array. */
void redactClientCommandArgument(client *c, int argc) {
retainOriginalCommandVector(c);
decrRefCount(c->argv[argc]);
c->original_argv[argc] = shared.redacted;
}
/* Rewrite the command vector of the client. All the new objects ref count
* is incremented. The old command vector is freed, and the old objects
* ref count is decremented. */
@@ -3223,18 +3230,33 @@ int checkClientOutputBufferLimits(client *c) {
*
* Note: we need to close the client asynchronously because this function is
* called from contexts where the client can't be freed safely, i.e. from the
* lower level functions pushing data inside the client output buffers. */
void asyncCloseClientOnOutputBufferLimitReached(client *c) {
if (!c->conn) return; /* It is unsafe to free fake clients. */
* lower level functions pushing data inside the client output buffers.
* When `async` is set to 0, we close the client immediately, this is
* useful when called from cron.
*
* Returns 1 if client was (flagged) closed. */
int closeClientOnOutputBufferLimitReached(client *c, int async) {
if (!c->conn) return 0; /* It is unsafe to free fake clients. */
serverAssert(c->reply_bytes < SIZE_MAX-(1024*64));
if (c->reply_bytes == 0 || c->flags & CLIENT_CLOSE_ASAP) return;
if (c->reply_bytes == 0 || c->flags & CLIENT_CLOSE_ASAP) return 0;
if (checkClientOutputBufferLimits(c)) {
sds client = catClientInfoString(sdsempty(),c);
freeClientAsync(c);
serverLog(LL_WARNING,"Client %s scheduled to be closed ASAP for overcoming of output buffer limits.", client);
if (async) {
freeClientAsync(c);
serverLog(LL_WARNING,
"Client %s scheduled to be closed ASAP for overcoming of output buffer limits.",
client);
} else {
freeClient(c);
serverLog(LL_WARNING,
"Client %s closed for overcoming of output buffer limits.",
client);
}
sdsfree(client);
return 1;
}
return 0;
}
/* Helper function used by performEvictions() in order to flush slaves
+1 -1
View File
@@ -836,7 +836,7 @@ size_t objectComputeSize(robj *o, size_t sample_size) {
if (samples) asize += (double)elesize/samples*dictSize(d);
} else if (o->encoding == OBJ_ENCODING_INTSET) {
intset *is = o->ptr;
asize = sizeof(*o)+sizeof(*is)+is->encoding*is->length;
asize = sizeof(*o)+sizeof(*is)+(size_t)is->encoding*is->length;
} else {
serverPanic("Unknown set encoding");
}
+1
View File
@@ -2691,6 +2691,7 @@ static void backgroundSaveDoneHandlerSocket(int exitcode, int bysignal) {
}
if (server.rdb_child_exit_pipe!=-1)
close(server.rdb_child_exit_pipe);
aeDeleteFileEvent(server.el, server.rdb_pipe_read, AE_READABLE);
close(server.rdb_pipe_read);
server.rdb_child_exit_pipe = -1;
server.rdb_pipe_read = -1;
+46 -44
View File
@@ -99,7 +99,6 @@ static struct config {
int randomkeys_keyspacelen;
int keepalive;
int pipeline;
int showerrors;
long long start;
long long totlatency;
const char *title;
@@ -307,7 +306,9 @@ static redisContext *getRedisContext(const char *ip, int port,
fprintf(stderr, "Node %s:%d replied with error:\n%s\n", ip, port, reply->str);
else
fprintf(stderr, "Node %s replied with error:\n%s\n", hostsocket, reply->str);
goto cleanup;
freeReplyObject(reply);
redisFree(ctx);
exit(1);
}
freeReplyObject(reply);
return ctx;
@@ -366,9 +367,16 @@ fail:
fprintf(stderr, "ERROR: failed to fetch CONFIG from ");
if (hostsocket == NULL) fprintf(stderr, "%s:%d\n", ip, port);
else fprintf(stderr, "%s\n", hostsocket);
int abort_test = 0;
if (reply && reply->type == REDIS_REPLY_ERROR &&
(!strncmp(reply->str,"NOAUTH",5) ||
!strncmp(reply->str,"WRONGPASS",9) ||
!strncmp(reply->str,"NOPERM",5)))
abort_test = 1;
freeReplyObject(reply);
redisFree(c);
freeRedisConfig(cfg);
if (abort_test) exit(1);
return NULL;
}
static void freeRedisConfig(redisConfig *cfg) {
@@ -513,44 +521,39 @@ static void readHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
exit(1);
}
redisReply *r = reply;
int is_err = (r->type == REDIS_REPLY_ERROR);
if (is_err && config.showerrors) {
/* TODO: static lasterr_time not thread-safe */
static time_t lasterr_time = 0;
time_t now = time(NULL);
if (lasterr_time != now) {
lasterr_time = now;
if (c->cluster_node) {
printf("Error from server %s:%d: %s\n",
if (r->type == REDIS_REPLY_ERROR) {
/* Try to update slots configuration if reply error is
* MOVED/ASK/CLUSTERDOWN and the key(s) used by the command
* contain(s) the slot hash tag.
* If the error is not topology-update related then we
* immediately exit to avoid false results. */
if (c->cluster_node && c->staglen) {
int fetch_slots = 0, do_wait = 0;
if (!strncmp(r->str,"MOVED",5) || !strncmp(r->str,"ASK",3))
fetch_slots = 1;
else if (!strncmp(r->str,"CLUSTERDOWN",11)) {
/* Usually the cluster is able to recover itself after
* a CLUSTERDOWN error, so try to sleep one second
* before requesting the new configuration. */
fetch_slots = 1;
do_wait = 1;
printf("Error from server %s:%d: %s.\n",
c->cluster_node->ip,
c->cluster_node->port,
r->str);
}
if (do_wait) sleep(1);
if (fetch_slots && !fetchClusterSlotsConfiguration(c))
exit(1);
} else {
if (c->cluster_node) {
printf("Error from server %s:%d: %s\n",
c->cluster_node->ip,
c->cluster_node->port,
r->str);
} else printf("Error from server: %s\n", r->str);
}
}
/* Try to update slots configuration if reply error is
* MOVED/ASK/CLUSTERDOWN and the key(s) used by the command
* contain(s) the slot hash tag. */
if (is_err && c->cluster_node && c->staglen) {
int fetch_slots = 0, do_wait = 0;
if (!strncmp(r->str,"MOVED",5) || !strncmp(r->str,"ASK",3))
fetch_slots = 1;
else if (!strncmp(r->str,"CLUSTERDOWN",11)) {
/* Usually the cluster is able to recover itself after
* a CLUSTERDOWN error, so try to sleep one second
* before requesting the new configuration. */
fetch_slots = 1;
do_wait = 1;
printf("Error from server %s:%d: %s\n",
c->cluster_node->ip,
c->cluster_node->port,
r->str);
}
if (do_wait) sleep(1);
if (fetch_slots && !fetchClusterSlotsConfiguration(c))
exit(1);
}
}
freeReplyObject(reply);
@@ -1293,8 +1296,7 @@ static int fetchClusterSlotsConfiguration(client c) {
atomicGetIncr(config.is_fetching_slots, is_fetching_slots, 1);
if (is_fetching_slots) return -1; //TODO: use other codes || errno ?
atomicSet(config.is_fetching_slots, 1);
if (config.showerrors)
printf("Cluster slots configuration changed, fetching new one...\n");
printf("WARNING: Cluster slots configuration changed, fetching new one...\n");
const char *errmsg = "Failed to update cluster slots configuration";
static dictType dtype = {
dictSdsHash, /* hash function */
@@ -1470,7 +1472,8 @@ int parseOptions(int argc, const char **argv) {
} else if (!strcmp(argv[i],"-I")) {
config.idlemode = 1;
} else if (!strcmp(argv[i],"-e")) {
config.showerrors = 1;
printf("WARNING: -e option has been deprecated. "
"We now immediatly exit on error to avoid false results.\n");
} else if (!strcmp(argv[i],"-t")) {
if (lastarg) goto invalid;
/* We get the list of tests to run as a string in the form
@@ -1573,8 +1576,6 @@ usage:
" is executed. Default tests use this to hit random keys in the\n"
" specified range.\n"
" -P <numreq> Pipeline <numreq> requests. Default 1 (no pipeline).\n"
" -e If server replies with errors, show them on stdout.\n"
" (no more than 1 error per second is displayed)\n"
" -q Quiet. Just show query/sec values\n"
" --precision Number of decimal places to display in latency output (default 0)\n"
" --csv Output in CSV format\n"
@@ -1699,7 +1700,6 @@ int main(int argc, const char **argv) {
config.keepalive = 1;
config.datasize = 3;
config.pipeline = 1;
config.showerrors = 0;
config.randomkeys = 0;
config.randomkeys_keyspacelen = 0;
config.quiet = 0;
@@ -1782,8 +1782,9 @@ int main(int argc, const char **argv) {
} else {
config.redis_config =
getRedisConfig(config.hostip, config.hostport, config.hostsocket);
if (config.redis_config == NULL)
if (config.redis_config == NULL) {
fprintf(stderr, "WARN: could not fetch server CONFIG\n");
}
}
if (config.num_threads > 0) {
pthread_mutex_init(&(config.liveclients_mutex), NULL);
@@ -1946,8 +1947,8 @@ int main(int argc, const char **argv) {
}
if (test_is_selected("lrange") || test_is_selected("lrange_500")) {
len = redisFormatCommand(&cmd,"LRANGE mylist%s 0 449",tag);
benchmark("LRANGE_500 (first 450 elements)",cmd,len);
len = redisFormatCommand(&cmd,"LRANGE mylist%s 0 499",tag);
benchmark("LRANGE_500 (first 500 elements)",cmd,len);
free(cmd);
}
@@ -1974,6 +1975,7 @@ int main(int argc, const char **argv) {
if (!config.csv) printf("\n");
} while(config.loop);
zfree(data);
if (config.redis_config != NULL) freeRedisConfig(config.redis_config);
return 0;
+4 -2
View File
@@ -39,12 +39,14 @@
static char error[1044];
static off_t epos;
static long long line = 1;
int consumeNewline(char *buf) {
if (strncmp(buf,"\r\n",2) != 0) {
ERROR("Expected \\r\\n, got: %02x%02x",buf[0],buf[1]);
return 0;
}
line += 1;
return 1;
}
@@ -201,8 +203,8 @@ int redis_check_aof_main(int argc, char **argv) {
off_t pos = process(fp);
off_t diff = size-pos;
printf("AOF analyzed: size=%lld, ok_up_to=%lld, diff=%lld\n",
(long long) size, (long long) pos, (long long) diff);
printf("AOF analyzed: size=%lld, ok_up_to=%lld, ok_up_to_line=%lld, diff=%lld\n",
(long long) size, (long long) pos, line, (long long) diff);
if (diff > 0) {
if (fix) {
char buf[2];
+1 -1
View File
@@ -250,7 +250,7 @@ int redis_check_rdb(char *rdbfilename, FILE *fp) {
rdbstate.doing = RDB_CHECK_DOING_READ_LEN;
if ((dbid = rdbLoadLen(&rdb,NULL)) == RDB_LENERR)
goto eoferr;
rdbCheckInfo("Selecting DB ID %d", dbid);
rdbCheckInfo("Selecting DB ID %llu", (unsigned long long)dbid);
continue; /* Read type again. */
} else if (type == RDB_OPCODE_RESIZEDB) {
/* RESIZEDB: Hint about the size of the keys in the currently
+7 -5
View File
@@ -663,7 +663,7 @@ static void cliOutputHelp(int argc, char **argv) {
help = entry->org;
if (group == -1) {
/* Compare all arguments */
if (argc == entry->argc) {
if (argc <= entry->argc) {
for (j = 0; j < argc; j++) {
if (strcasecmp(argv[j],entry->argv[j]) != 0) break;
}
@@ -844,7 +844,9 @@ static int cliConnect(int flags) {
cliRefreshPrompt();
}
if (config.hostsocket == NULL) {
/* Do not use hostsocket when we got redirected in cluster mode */
if (config.hostsocket == NULL ||
(config.cluster_mode && config.cluster_reissue_command)) {
context = redisConnect(config.hostip,config.hostport);
} else {
context = redisConnectUnix(config.hostsocket);
@@ -5481,7 +5483,7 @@ static void clusterManagerNodeArrayReset(clusterManagerNodeArray *array) {
static void clusterManagerNodeArrayShift(clusterManagerNodeArray *array,
clusterManagerNode **nodeptr)
{
assert(array->nodes < (array->nodes + array->len));
assert(array->len > 0);
/* If the first node to be shifted is not NULL, decrement count. */
if (*array->nodes != NULL) array->count--;
/* Store the first node to be shifted into 'nodeptr'. */
@@ -5494,7 +5496,7 @@ static void clusterManagerNodeArrayShift(clusterManagerNodeArray *array,
static void clusterManagerNodeArrayAdd(clusterManagerNodeArray *array,
clusterManagerNode *node)
{
assert(array->nodes < (array->nodes + array->len));
assert(array->len > 0);
assert(node != NULL);
assert(array->count < array->len);
array->nodes[array->count++] = node;
@@ -6871,7 +6873,7 @@ void showLatencyDistSamples(struct distsamples *samples, long long tot) {
printf("\033[38;5;0m"); /* Set foreground color to black. */
for (j = 0; ; j++) {
int coloridx =
ceil((float) samples[j].count / tot * (spectrum_palette_size-1));
ceil((double) samples[j].count / tot * (spectrum_palette_size-1));
int color = spectrum_palette[coloridx];
printf("\033[48;5;%dm%c", (int)color, samples[j].character);
samples[j].count = 0;
+1
View File
@@ -377,6 +377,7 @@ void replicationFeedSlavesFromMasterStream(list *slaves, char *buf, size_t bufle
}
void replicationFeedMonitors(client *c, list *monitors, int dictid, robj **argv, int argc) {
if (!(listLength(server.monitors) && !server.loading)) return;
listNode *ln;
listIter li;
int j;
+13 -4
View File
@@ -31,6 +31,7 @@
#include "sha1.h"
#include "rand.h"
#include "cluster.h"
#include "monotonic.h"
#include <lua.h>
#include <lauxlib.h>
@@ -1427,7 +1428,7 @@ sds luaCreateFunction(client *c, lua_State *lua, robj *body) {
/* This is the Lua script "count" hook that we use to detect scripts timeout. */
void luaMaskCountHook(lua_State *lua, lua_Debug *ar) {
long long elapsed = mstime() - server.lua_time_start;
long long elapsed = elapsedMs(server.lua_time_start);
UNUSED(ar);
UNUSED(lua);
@@ -1578,7 +1579,8 @@ void evalGenericCommand(client *c, int evalsha) {
server.in_eval = 1;
server.lua_caller = c;
server.lua_cur_script = funcname + 2;
server.lua_time_start = mstime();
server.lua_time_start = getMonotonicUs();
server.lua_time_snapshot = mstime();
server.lua_kill = 0;
if (server.lua_time_limit > 0 && ldb.active == 0) {
lua_sethook(lua,luaMaskCountHook,LUA_MASKCOUNT,100000);
@@ -1688,6 +1690,9 @@ void evalGenericCommand(client *c, int evalsha) {
}
void evalCommand(client *c) {
/* Explicitly feed monitor here so that lua commands appear after their
* script command. */
replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc);
if (!(c->flags & CLIENT_LUA_DEBUG))
evalGenericCommand(c,0);
else
@@ -1695,6 +1700,9 @@ void evalCommand(client *c) {
}
void evalShaCommand(client *c) {
/* Explicitly feed monitor here so that lua commands appear after their
* script command. */
replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc);
if (sdslen(c->argv[1]->ptr) != 40) {
/* We know that a match is not possible if the provided SHA is
* not the right length. So we return an error ASAP, this way
@@ -2729,7 +2737,7 @@ void luaLdbLineHook(lua_State *lua, lua_Debug *ar) {
/* Check if a timeout occurred. */
if (ar->event == LUA_HOOKCOUNT && ldb.step == 0 && bp == 0) {
mstime_t elapsed = mstime() - server.lua_time_start;
mstime_t elapsed = elapsedMs(server.lua_time_start);
mstime_t timelimit = server.lua_time_limit ?
server.lua_time_limit : 5000;
if (elapsed >= timelimit) {
@@ -2759,6 +2767,7 @@ void luaLdbLineHook(lua_State *lua, lua_Debug *ar) {
lua_pushstring(lua, "timeout during Lua debugging with client closing connection");
lua_error(lua);
}
server.lua_time_start = mstime();
server.lua_time_start = getMonotonicUs();
server.lua_time_snapshot = mstime();
}
}
+10 -8
View File
@@ -3175,11 +3175,13 @@ void sentinelConfigSetCommand(client *c) {
sentinel.announce_port = numval;
} else if (!strcasecmp(o->ptr, "sentinel-user")) {
sdsfree(sentinel.sentinel_auth_user);
sentinel.sentinel_auth_user = sdsnew(val->ptr);
sentinel.sentinel_auth_user = sdslen(val->ptr) == 0 ?
NULL : sdsdup(val->ptr);
drop_conns = 1;
} else if (!strcasecmp(o->ptr, "sentinel-pass")) {
sdsfree(sentinel.sentinel_auth_pass);
sentinel.sentinel_auth_pass = sdsnew(val->ptr);
sentinel.sentinel_auth_pass = sdslen(val->ptr) == 0 ?
NULL : sdsdup(val->ptr);
drop_conns = 1;
} else {
addReplyErrorFormat(c, "Invalid argument '%s' to SENTINEL CONFIG SET",
@@ -4119,16 +4121,16 @@ void sentinelSetCommand(client *c) {
int numargs = j-old_j+1;
switch(numargs) {
case 2:
sentinelEvent(LL_WARNING,"+set",ri,"%@ %s %s",c->argv[old_j]->ptr,
c->argv[old_j+1]->ptr);
sentinelEvent(LL_WARNING,"+set",ri,"%@ %s %s",(char*)c->argv[old_j]->ptr,
(char*)c->argv[old_j+1]->ptr);
break;
case 3:
sentinelEvent(LL_WARNING,"+set",ri,"%@ %s %s %s",c->argv[old_j]->ptr,
c->argv[old_j+1]->ptr,
c->argv[old_j+2]->ptr);
sentinelEvent(LL_WARNING,"+set",ri,"%@ %s %s %s",(char*)c->argv[old_j]->ptr,
(char*)c->argv[old_j+1]->ptr,
(char*)c->argv[old_j+2]->ptr);
break;
default:
sentinelEvent(LL_WARNING,"+set",ri,"%@ %s",c->argv[old_j]->ptr);
sentinelEvent(LL_WARNING,"+set",ri,"%@ %s",(char*)c->argv[old_j]->ptr);
break;
}
}
+32 -23
View File
@@ -706,7 +706,7 @@ struct redisCommand redisCommandTable[] = {
0,NULL,0,0,0,0,0,0},
{"auth",authCommand,-2,
"no-auth no-script ok-loading ok-stale fast no-monitor no-slowlog @connection",
"no-auth no-script ok-loading ok-stale fast @connection",
0,NULL,0,0,0,0,0,0},
/* We don't allow PING during loading since in Redis PING is used as
@@ -749,7 +749,7 @@ struct redisCommand redisCommandTable[] = {
0,NULL,0,0,0,0,0,0},
{"exec",execCommand,1,
"no-script no-monitor no-slowlog ok-loading ok-stale @transaction",
"no-script no-slowlog ok-loading ok-stale @transaction",
0,NULL,0,0,0,0,0,0},
{"discard",discardCommand,1,
@@ -901,17 +901,21 @@ struct redisCommand redisCommandTable[] = {
0,NULL,0,0,0,0,0,0},
{"hello",helloCommand,-1,
"no-auth no-script fast no-monitor ok-loading ok-stale @connection",
"no-auth no-script fast ok-loading ok-stale @connection",
0,NULL,0,0,0,0,0,0},
/* EVAL can modify the dataset, however it is not flagged as a write
* command since we do the check while running commands from Lua. */
* command since we do the check while running commands from Lua.
*
* EVAL and EVALSHA also feed monitors before the commands are executed,
* as opposed to after.
*/
{"eval",evalCommand,-3,
"no-script may-replicate @scripting",
"no-script no-monitor may-replicate @scripting",
0,evalGetKeys,0,0,0,0,0,0},
{"evalsha",evalShaCommand,-3,
"no-script may-replicate @scripting",
"no-script no-monitor may-replicate @scripting",
0,evalGetKeys,0,0,0,0,0,0},
{"slowlog",slowlogCommand,-2,
@@ -1839,6 +1843,7 @@ void clientsCron(void) {
if (clientsCronResizeQueryBuffer(c)) continue;
if (clientsCronTrackExpansiveClients(c, curr_peak_mem_usage_slot)) continue;
if (clientsCronTrackClientsMemUsage(c)) continue;
if (closeClientOnOutputBufferLimitReached(c, 0)) continue;
}
}
@@ -2603,6 +2608,7 @@ void createSharedObjects(void) {
shared.getack = createStringObject("GETACK",6);
shared.special_asterick = createStringObject("*",1);
shared.special_equals = createStringObject("=",1);
shared.redacted = makeObjectShared(createStringObject("(redacted)",10));
for (j = 0; j < OBJ_SHARED_INTEGERS; j++) {
shared.integers[j] =
@@ -3624,12 +3630,6 @@ void preventCommandPropagation(client *c) {
c->flags |= CLIENT_PREVENT_PROP;
}
/* Avoid logging any information about this client's arguments
* since they contain sensitive information. */
void preventCommandLogging(client *c) {
c->flags |= CLIENT_PREVENT_LOGGING;
}
/* AOF specific version of preventCommandPropagation(). */
void preventCommandAOF(client *c) {
c->flags |= CLIENT_PREVENT_AOF_PROP;
@@ -3643,7 +3643,7 @@ void preventCommandReplication(client *c) {
/* Log the last command a client executed into the slowlog. */
void slowlogPushCurrentCommand(client *c, struct redisCommand *cmd, ustime_t duration) {
/* Some commands may contain sensitive data that should not be available in the slowlog. */
if ((c->flags & CLIENT_PREVENT_LOGGING) || (cmd->flags & CMD_SKIP_SLOWLOG))
if (cmd->flags & CMD_SKIP_SLOWLOG)
return;
/* If command argument vector was rewritten, use the original
@@ -3699,15 +3699,6 @@ void call(client *c, int flags) {
server.fixed_time_expire++;
/* Send the command to clients in MONITOR mode if applicable.
* Administrative commands are considered too dangerous to be shown. */
if (listLength(server.monitors) &&
!server.loading &&
!(c->cmd->flags & (CMD_SKIP_MONITOR|CMD_ADMIN)))
{
replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc);
}
/* Initialization: clear the flags that must be set by the command on
* demand, and initialize the array for additional commands propagation. */
c->flags &= ~(CLIENT_FORCE_AOF|CLIENT_FORCE_REPL|CLIENT_PREVENT_PROP);
@@ -3773,6 +3764,14 @@ void call(client *c, int flags) {
if ((flags & CMD_CALL_SLOWLOG) && !(c->flags & CLIENT_BLOCKED))
slowlogPushCurrentCommand(c, real_cmd, duration);
/* Send the command to clients in MONITOR mode if applicable.
* Administrative commands are considered too dangerous to be shown. */
if (!(c->cmd->flags & (CMD_SKIP_MONITOR|CMD_ADMIN))) {
robj **argv = c->original_argv ? c->original_argv : c->argv;
int argc = c->original_argv ? c->original_argc : c->argc;
replicationFeedMonitors(c,server.monitors,c->db->id,argv,argc);
}
/* Clear the original argv.
* If the client is blocked we will handle slowlog when it is unblocked. */
if (!(c->flags & CLIENT_BLOCKED))
@@ -3985,6 +3984,8 @@ int processCommand(client *c) {
return C_OK;
}
int is_read_command = (c->cmd->flags & CMD_READONLY) ||
(c->cmd->proc == execCommand && (c->mstate.cmd_flags & CMD_READONLY));
int is_write_command = (c->cmd->flags & CMD_WRITE) ||
(c->cmd->proc == execCommand && (c->mstate.cmd_flags & CMD_WRITE));
int is_denyoom_command = (c->cmd->flags & CMD_DENYOOM) ||
@@ -4194,7 +4195,7 @@ int processCommand(client *c) {
c->cmd->proc != discardCommand &&
c->cmd->proc != watchCommand &&
c->cmd->proc != unwatchCommand &&
c->cmd->proc != resetCommand &&
c->cmd->proc != resetCommand &&
!(c->cmd->proc == shutdownCommand &&
c->argc == 2 &&
tolower(((char*)c->argv[1]->ptr)[0]) == 'n') &&
@@ -4206,6 +4207,14 @@ int processCommand(client *c) {
return C_OK;
}
/* Prevent a replica from sending commands that access the keyspace.
* The main objective here is to prevent abuse of client pause check
* from which replicas are exempt. */
if ((c->flags & CLIENT_SLAVE) && (is_may_replicate_command || is_write_command || is_read_command)) {
rejectCommandFormat(c, "Replica can't interract with the keyspace");
return C_OK;
}
/* If the server is paused, block the client until
* the pause has ended. Replicas are never paused. */
if (!(c->flags & CLIENT_SLAVE) &&
+6 -5
View File
@@ -279,7 +279,6 @@ extern int configOOMScoreAdjValuesDefaults[CONFIG_OOM_COUNT];
and AOF client */
#define CLIENT_REPL_RDBONLY (1ULL<<42) /* This client is a replica that only wants
RDB without replication buffer. */
#define CLIENT_PREVENT_LOGGING (1ULL<<43) /* Prevent logging of command to slowlog */
/* Client block type (btype field in client structure)
* if CLIENT_BLOCKED flag is set. */
@@ -986,7 +985,7 @@ struct sharedObjectsStruct {
*script, *replconf, *eval, *persist, *set, *pexpireat, *pexpire,
*time, *pxat, *px, *retrycount, *force, *justid,
*lastid, *ping, *setid, *keepttl, *load, *createconsumer,
*getack, *special_asterick, *special_equals, *default_username,
*getack, *special_asterick, *special_equals, *default_username, *redacted,
*select[PROTO_SHARED_SELECT_CMDS],
*integers[OBJ_SHARED_INTEGERS],
*mbulkhdr[OBJ_SHARED_BULKHDR_LEN], /* "*<value>\r\n" */
@@ -1571,7 +1570,8 @@ struct redisServer {
dict *lua_scripts; /* A dictionary of SHA1 -> Lua scripts */
unsigned long long lua_scripts_mem; /* Cached scripts' memory + oh */
mstime_t lua_time_limit; /* Script timeout in milliseconds */
mstime_t lua_time_start; /* Start time of script, milliseconds time */
monotime lua_time_start; /* monotonic timer to detect timed-out script */
mstime_t lua_time_snapshot; /* Snapshot of mstime when script is started */
int lua_write_dirty; /* True if a write command was called during the
execution of the current script. */
int lua_random_dirty; /* True if a random command was called during the
@@ -1864,9 +1864,10 @@ sds getAllClientsInfoString(int type);
void rewriteClientCommandVector(client *c, int argc, ...);
void rewriteClientCommandArgument(client *c, int i, robj *newval);
void replaceClientCommandVector(client *c, int argc, robj **argv);
void redactClientCommandArgument(client *c, int argc);
unsigned long getClientOutputBufferMemoryUsage(client *c);
int freeClientsInAsyncFreeQueue(void);
void asyncCloseClientOnOutputBufferLimitReached(client *c);
int closeClientOnOutputBufferLimitReached(client *c, int async);
int getClientType(client *c);
int getClientTypeByName(char *name);
char *getClientTypeName(int class);
@@ -1911,6 +1912,7 @@ void disableTracking(client *c);
void trackingRememberKeys(client *c);
void trackingInvalidateKey(client *c, robj *keyobj);
void trackingInvalidateKeysOnFlush(int async);
void freeTrackingRadixTree(rax *rt);
void freeTrackingRadixTreeAsync(rax *rt);
void trackingLimitUsedSlots(void);
uint64_t trackingGetTotalItems(void);
@@ -2211,7 +2213,6 @@ void redisOpArrayInit(redisOpArray *oa);
void redisOpArrayFree(redisOpArray *oa);
void forceCommandPropagation(client *c, int flags);
void preventCommandPropagation(client *c);
void preventCommandLogging(client *c);
void preventCommandAOF(client *c);
void preventCommandReplication(client *c);
void slowlogPushCurrentCommand(client *c, struct redisCommand *cmd, ustime_t duration);
+1
View File
@@ -869,6 +869,7 @@ void sinterGenericCommand(client *c, robj **setkeys,
if (dstkey) {
if (dbDelete(c->db,dstkey)) {
signalModifiedKey(c,c->db,dstkey);
notifyKeyspaceEvent(NOTIFY_GENERIC,"del",dstkey,c->db->id);
server.dirty++;
}
addReply(c,shared.czero);
+17 -1
View File
@@ -797,6 +797,12 @@ void stralgoLCS(client *c) {
goto cleanup;
}
/* Detect string truncation or later overflows. */
if (sdslen(a) >= UINT32_MAX-1 || sdslen(b) >= UINT32_MAX-1) {
addReplyError(c, "String too long for LCS");
goto cleanup;
}
/* Compute the LCS using the vanilla dynamic programming technique of
* building a table of LCS(x,y) substrings. */
uint32_t alen = sdslen(a);
@@ -805,9 +811,19 @@ void stralgoLCS(client *c) {
/* Setup an uint32_t array to store at LCS[i,j] the length of the
* LCS A0..i-1, B0..j-1. Note that we have a linear array here, so
* we index it as LCS[j+(blen+1)*j] */
uint32_t *lcs = zmalloc((alen+1)*(blen+1)*sizeof(uint32_t));
#define LCS(A,B) lcs[(B)+((A)*(blen+1))]
/* Try to allocate the LCS table, and abort on overflow or insufficient memory. */
unsigned long long lcssize = (unsigned long long)(alen+1)*(blen+1); /* Can't overflow due to the size limits above. */
unsigned long long lcsalloc = lcssize * sizeof(uint32_t);
uint32_t *lcs = NULL;
if (lcsalloc < SIZE_MAX && lcsalloc / lcssize == sizeof(uint32_t))
lcs = ztrymalloc(lcsalloc);
if (!lcs) {
addReplyError(c, "Insufficient memory");
goto cleanup;
}
/* Start building the LCS table. */
for (uint32_t i = 0; i <= alen; i++) {
for (uint32_t j = 0; j <= blen; j++) {
+2 -2
View File
@@ -1,2 +1,2 @@
#define REDIS_VERSION "255.255.255"
#define REDIS_VERSION_NUM 0x00ffffff
#define REDIS_VERSION "6.2.4"
#define REDIS_VERSION_NUM 0x00060204
+12
View File
@@ -158,6 +158,18 @@ tags {"aof"} {
assert_match "*not valid*" $result
}
test "Short read: Utility should show the abnormal line num in AOF" {
create_aof {
append_to_aof [formatCommand set foo hello]
append_to_aof "!!!"
}
catch {
exec src/redis-check-aof $aof_path
} result
assert_match "*ok_up_to_line=8*" $result
}
test "Short read: Utility should be able to fix the AOF" {
set result [exec src/redis-check-aof --fix $aof_path << "y\n"]
assert_match "*Successfully truncated AOF*" $result
+39
View File
@@ -774,6 +774,45 @@ test "diskless replication child being killed is collected" {
}
}
test "diskless replication read pipe cleanup" {
# In diskless replication, we create a read pipe for the RDB, between the child and the parent.
# When we close this pipe (fd), the read handler also needs to be removed from the event loop (if it still registered).
# Otherwise, next time we will use the same fd, the registration will be fail (panic), because
# we will use EPOLL_CTL_MOD (the fd still register in the event loop), on fd that already removed from epoll_ctl
start_server {tags {"repl"}} {
set master [srv 0 client]
set master_host [srv 0 host]
set master_port [srv 0 port]
set master_pid [srv 0 pid]
$master config set repl-diskless-sync yes
$master config set repl-diskless-sync-delay 0
# put enough data in the db, and slowdown the save, to keep the parent busy at the read process
$master config set rdb-key-save-delay 100000
$master debug populate 20000 test 10000
$master config set rdbcompression no
start_server {} {
set replica [srv 0 client]
set loglines [count_log_lines 0]
$replica config set repl-diskless-load swapdb
$replica replicaof $master_host $master_port
# wait for the replicas to start reading the rdb
wait_for_log_messages 0 {"*Loading DB in memory*"} $loglines 800 10
set loglines [count_log_lines 0]
# send FLUSHALL so the RDB child will be killed
$master flushall
# wait for another RDB child process to be started
wait_for_log_messages -1 {"*Background RDB transfer started by pid*"} $loglines 800 10
# make sure master is alive
$master ping
}
}
}
test {replicaof right after disconnection} {
# this is a rare race condition that was reproduced sporadically by the psync2 unit.
# see details in #7205
+46
View File
@@ -31,6 +31,52 @@ start_server {tags {"introspection"}} {
assert_match {*lua*"set"*"foo"*"bar"*} [$rd read]
}
test {MONITOR supports redacting command arguments} {
set rd [redis_deferring_client]
$rd monitor
$rd read ; # Discard the OK
r migrate [srv 0 host] [srv 0 port] key 9 5000
r migrate [srv 0 host] [srv 0 port] key 9 5000 AUTH user
r migrate [srv 0 host] [srv 0 port] key 9 5000 AUTH2 user password
catch {r auth not-real} _
catch {r auth not-real not-a-password} _
catch {r hello 2 AUTH not-real not-a-password} _
assert_match {*"key"*"9"*"5000"*} [$rd read]
assert_match {*"key"*"9"*"5000"*"(redacted)"*} [$rd read]
assert_match {*"key"*"9"*"5000"*"(redacted)"*"(redacted)"*} [$rd read]
assert_match {*"auth"*"(redacted)"*} [$rd read]
assert_match {*"auth"*"(redacted)"*"(redacted)"*} [$rd read]
assert_match {*"hello"*"2"*"AUTH"*"(redacted)"*"(redacted)"*} [$rd read]
$rd close
}
test {MONITOR correctly handles multi-exec cases} {
set rd [redis_deferring_client]
$rd monitor
$rd read ; # Discard the OK
# Make sure multi-exec statements are ordered
# correctly
r multi
r set foo bar
r exec
assert_match {*"multi"*} [$rd read]
assert_match {*"set"*"foo"*"bar"*} [$rd read]
assert_match {*"exec"*} [$rd read]
# Make sure we close multi statements on errors
r multi
catch {r syntax error} _
catch {r exec} _
assert_match {*"multi"*} [$rd read]
assert_match {*"exec"*} [$rd read]
$rd close
}
test {CLIENT GETNAME should return NIL if name is not assigned} {
r client getname
} {}
+2 -2
View File
@@ -25,7 +25,7 @@ test {CONFIG SET port number} {
test {CONFIG SET bind address} {
start_server {} {
# non-valid address
catch {r CONFIG SET bind "some.wrong.bind.address"} e
catch {r CONFIG SET bind "999.999.999.999"} e
assert_match {*Failed to bind to specified addresses*} $e
# make sure server still bound to the previous address
@@ -33,4 +33,4 @@ test {CONFIG SET bind address} {
$rd PING
$rd close
}
}
}
+59 -52
View File
@@ -18,65 +18,72 @@ start_server {tags {"obuf-limits"}} {
assert {$omem >= 70000 && $omem < 200000}
$rd1 close
}
test {Client output buffer soft limit is not enforced if time is not overreached} {
r config set client-output-buffer-limit {pubsub 0 100000 10}
set rd1 [redis_deferring_client]
$rd1 subscribe foo
set reply [$rd1 read]
assert {$reply eq "subscribe foo 1"}
set omem 0
set start_time 0
set time_elapsed 0
while 1 {
if {$start_time != 0} {
# Slow down loop when omen has reached the limit.
after 10
}
r publish foo [string repeat "x" 1000]
set clients [split [r client list] "\r\n"]
set c [split [lindex $clients 1] " "]
if {![regexp {omem=([0-9]+)} $c - omem]} break
if {$omem > 100000} {
if {$start_time == 0} {set start_time [clock seconds]}
set time_elapsed [expr {[clock seconds]-$start_time}]
if {$time_elapsed >= 5} break
}
foreach {soft_limit_time wait_for_timeout} {3 yes
4 no } {
if $wait_for_timeout {
set test_name "Client output buffer soft limit is enforced if time is overreached"
} else {
set test_name "Client output buffer soft limit is not enforced too early and is enforced when no traffic"
}
assert {$omem >= 100000 && $time_elapsed >= 5 && $time_elapsed <= 10}
$rd1 close
}
test {Client output buffer soft limit is enforced if time is overreached} {
r config set client-output-buffer-limit {pubsub 0 100000 3}
set rd1 [redis_deferring_client]
test $test_name {
r config set client-output-buffer-limit "pubsub 0 100000 $soft_limit_time"
set soft_limit_time [expr $soft_limit_time*1000]
set rd1 [redis_deferring_client]
$rd1 subscribe foo
set reply [$rd1 read]
assert {$reply eq "subscribe foo 1"}
$rd1 client setname test_client
set reply [$rd1 read]
assert {$reply eq "OK"}
set omem 0
set start_time 0
set time_elapsed 0
while 1 {
if {$start_time != 0} {
# Slow down loop when omen has reached the limit.
after 10
$rd1 subscribe foo
set reply [$rd1 read]
assert {$reply eq "subscribe foo 1"}
set omem 0
set start_time 0
set time_elapsed 0
set last_under_limit_time [clock milliseconds]
while 1 {
r publish foo [string repeat "x" 1000]
set clients [split [r client list] "\r\n"]
set c [lsearch -inline $clients *name=test_client*]
if {$start_time != 0} {
set time_elapsed [expr {[clock milliseconds]-$start_time}]
# Make sure test isn't taking too long
assert {$time_elapsed <= [expr $soft_limit_time+3000]}
}
if {$wait_for_timeout && $c == ""} {
# Make sure we're disconnected when we reach the soft limit
assert {$omem >= 100000 && $time_elapsed >= $soft_limit_time}
break
} else {
assert {[regexp {omem=([0-9]+)} $c - omem]}
}
if {$omem > 100000} {
if {$start_time == 0} {set start_time $last_under_limit_time}
if {!$wait_for_timeout && $time_elapsed >= [expr $soft_limit_time-1000]} break
# Slow down loop when omem has reached the limit.
after 10
} else {
# if the OS socket buffers swallowed what we previously filled, reset the start timer.
set start_time 0
set last_under_limit_time [clock milliseconds]
}
}
r publish foo [string repeat "x" 1000]
set clients [split [r client list] "\r\n"]
set c [split [lindex $clients 1] " "]
if {![regexp {omem=([0-9]+)} $c - omem]} break
if {$omem > 100000} {
if {$start_time == 0} {set start_time [clock seconds]}
set time_elapsed [expr {[clock seconds]-$start_time}]
if {$time_elapsed >= 10} break
if {!$wait_for_timeout} {
# After we completely stopped the traffic, wait for soft limit to time out
set timeout [expr {$soft_limit_time+1500 - ([clock milliseconds]-$start_time)}]
wait_for_condition [expr $timeout/10] 10 {
[lsearch [split [r client list] "\r\n"] *name=test_client*] == -1
} else {
fail "Soft limit timed out but client still connected"
}
}
$rd1 close
}
assert {$omem >= 100000 && $time_elapsed < 6}
$rd1 close
}
test {No response for single command if client output buffer hard limit is enforced} {
-3
View File
@@ -266,9 +266,6 @@ start_server {overrides {save ""} tags {"other"}} {
assert_equal [$rd read] "OK"
$rd reset
# skip reset ouptut
$rd read
assert_equal [$rd read] "RESET"
assert_no_match {*flags=O*} [r client list]
+21 -4
View File
@@ -45,18 +45,35 @@ start_server {tags {"slowlog"} overrides {slowlog-log-slower-than 1000000}} {
r config set slowlog-log-slower-than 0
r slowlog reset
r config set masterauth ""
r acl setuser slowlog-test-user
r acl setuser slowlog-test-user +get +set
r config set slowlog-log-slower-than 0
r config set slowlog-log-slower-than 10000
set slowlog_resp [r slowlog get]
# Make sure normal configs work, but the two sensitive
# commands are omitted
assert_equal 2 [llength $slowlog_resp]
assert_equal {slowlog reset} [lindex [lindex [r slowlog get] 1] 3]
# commands are omitted or redacted
assert_equal 4 [llength $slowlog_resp]
assert_equal {slowlog reset} [lindex [lindex [r slowlog get] 3] 3]
assert_equal {config set masterauth (redacted)} [lindex [lindex [r slowlog get] 2] 3]
assert_equal {acl setuser (redacted) (redacted) (redacted)} [lindex [lindex [r slowlog get] 1] 3]
assert_equal {config set slowlog-log-slower-than 0} [lindex [lindex [r slowlog get] 0] 3]
}
test {SLOWLOG - Some commands can redact sensitive fields} {
r config set slowlog-log-slower-than 0
r slowlog reset
r migrate [srv 0 host] [srv 0 port] key 9 5000
r migrate [srv 0 host] [srv 0 port] key 9 5000 AUTH user
r migrate [srv 0 host] [srv 0 port] key 9 5000 AUTH2 user password
r config set slowlog-log-slower-than 10000
# Make sure all 3 commands were logged, but the sensitive fields are omitted
assert_equal 4 [llength [r slowlog get]]
assert_match {* key 9 5000} [lindex [lindex [r slowlog get] 2] 3]
assert_match {* key 9 5000 AUTH (redacted)} [lindex [lindex [r slowlog get] 1] 3]
assert_match {* key 9 5000 AUTH2 (redacted) (redacted)} [lindex [lindex [r slowlog get] 0] 3]
}
test {SLOWLOG - Rewritten commands are logged as their original command} {
r config set slowlog-log-slower-than 0
+11
View File
@@ -395,6 +395,17 @@ start_server {tags {"tracking network"}} {
assert {[lindex msg 2] eq {} }
}
test {Test ASYNC flushall} {
clean_all
r CLIENT TRACKING on REDIRECT $redir_id
r GET key1
r GET key2
assert_equal [s 0 tracking_total_keys] 2
$rd_sg FLUSHALL ASYNC
assert_equal [s 0 tracking_total_keys] 0
assert_equal [lindex [$rd_redirection read] 2] {}
}
# Keys are defined to be evicted 100 at a time by default.
# If after eviction the number of keys still surpasses the limit
# defined in tracking-table-max-keys, we increases eviction