Compare commits

...
22 Commits
Author SHA1 Message Date
Oran Agra 91863dd854 Redis 6.2.14
CI / test-ubuntu-latest (push) Has been cancelled
CI / build-debian-old (push) Has been cancelled
CI / build-macos-latest (push) Has been cancelled
CI / build-32bit (push) Has been cancelled
CI / build-libc-malloc (push) Has been cancelled
CI / build-centos7-jemalloc (push) Has been cancelled
2023-10-18 10:43:10 +03:00
Jachin 2f27ebb414 Fix compile on macOS 13 (#12611)
Use the __MAC_OS_X_VERSION_MIN_REQUIRED macro to detect the
macOS system version instead of using MAC_OS_X_VERSION_10_6.

From MacOSX14.0.sdk, the default definitions of MAC_OS_X_VERSION_xxx have
been removed in usr/include/AvailabilityMacros.h. It includes AvailabilityVersions.h,
where the following condition must be met:
`#if (!defined(_POSIX_C_SOURCE) && !defined(_XOPEN_SOURCE)) || defined(_DARWIN_C_SOURCE)`
Only then will MAC_OS_X_VERSION_xxx be defined.
However, in the project, _DARWIN_C_SOURCE is not defined, which leads to the
loss of the definition for MAC_OS_X_VERSION_10_6.

(cherry picked from commit a2b0701d2c)
2023-10-18 10:43:10 +03:00
sundb 89391e42c8 Skip test for sdsRemoveFreeSpace when mem_allocator is not jemalloc (#11878)
Test `trim on SET with big value` (introduced from #11817) fails under mac m1 with libc mem_allocator.
The reason is that malloc(33000) will allocate 65536 bytes(>42000).
This test still passes under ubuntu with libc mem_allocator.

```
*** [err]: trim on SET with big value in tests/unit/type/string.tcl
Expected [r memory usage key] < 42000 (context: type source line 471 file /Users/iospack/data/redis_fork/tests/unit/type/string.tcl cmd {assert {[r memory usage key] < 42000}} proc ::test)
```

simple test under mac m1 with libc mem_allocator:
```c
void *p = zmalloc(33000);
printf("malloc size: %zu\n", zmalloc_size(p));

# output
malloc size: 65536
```

(cherry picked from commit 3fba3ccd96)
(cherry picked from commit 646069a900)
2023-10-18 10:43:10 +03:00
Yossi Gottlieb 8f30d2a604 Fix issue of listen before chmod on Unix sockets (CVE-2023-45145)
Before this commit, Unix socket setup performed chmod(2) on the socket
file after calling listen(2). Depending on what umask is used, this
could leave the file with the wrong permissions for a short period of
time. As a result, another process could exploit this race condition and
establish a connection that would otherwise not be possible.

We now make sure the socket permissions are set up prior to calling
listen(2).

(cherry picked from commit a11b3bc34a054818f2ac70e50adfc542ca1cba42)
(cherry picked from commit 3ca02a0c541e019d77014424e207123160520cec)
2023-10-18 10:43:10 +03:00
Oran Agra 24c0bf5c5b Redis 6.2.13
CI / test-ubuntu-latest (push) Has been cancelled
CI / build-debian-old (push) Has been cancelled
CI / build-macos-latest (push) Has been cancelled
CI / build-32bit (push) Has been cancelled
CI / build-libc-malloc (push) Has been cancelled
CI / build-centos7-jemalloc (push) Has been cancelled
2023-07-10 14:37:59 +03:00
sundb b313fa4166 Fix alpine tests (partial #12035)
This is a partial cherry-pick from Redis 7.2

1) Fixed `ps -p [pid]`  doesn't output `<defunct>` when using procps 4.x causing `replication
  child dies when parent is killed - diskless` test to fail.

(cherry picked from commit 42c8c61813)
(cherry picked from commit 63fc6c87071d12e3577fc383d5a3f25c60423c8e)
2023-07-10 14:37:59 +03:00
sundb f613873de0 Re-enable hash downsize rehashing during fork (partial #12276)
This was introduced by the recent change in #11692 which prevented a
down-sizing rehashing while there is a fork.

## Solution
1. Fix the rehashing code, so that the same as it allows rehashing for up-sizing
  during fork when the ratio is extreme, it will allow it for down-sizing as well.

Co-authored-by: Oran Agra <oran@redislabs.com>

This is a partial cherry pick of:
(cherry picked from commit b00a235186)
(cherry picked from commit d4c37320382edb342292a3e30250d46896a12016)
2023-07-10 14:37:59 +03:00
Oran AgraandYossi Gottlieb 405d14fd44 Lua cjson and cmsgpack integer overflow issues (CVE-2022-24834)
* Fix integer overflows due to using wrong integer size.
* Add assertions / panic when overflow still happens.
* Deletion of dead code to avoid need to maintain it
* Some changes are not because of bugs, but rather paranoia.
* Improve cmsgpack and cjson test coverage.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2023-07-10 14:37:59 +03:00
Oran Agra 2d4b9401f1 Redis 6.22.12
CI / test-ubuntu-latest (push) Has been cancelled
CI / build-debian-old (push) Has been cancelled
CI / build-macos-latest (push) Has been cancelled
CI / build-32bit (push) Has been cancelled
CI / build-libc-malloc (push) Has been cancelled
CI / build-centos7-jemalloc (push) Has been cancelled
2023-04-17 15:54:26 +03:00
Oran Agra 521e05cb8b fix false valgrind error on new hash test (#11200)
New test fails on valgrind because strtold("+inf") with valgrind returns a non-inf result
same thing is done in incr.tcl.

(cherry picked from commit c3b7bde914)
2023-04-17 15:54:26 +03:00
Binbin 7afd1724dd Fix the bug that CLIENT REPLY OFF|SKIP cannot receive push notifications (#11875)
This bug seems to be there forever, CLIENT REPLY OFF|SKIP will
mark the client with CLIENT_REPLY_OFF or CLIENT_REPLY_SKIP flags.
With these flags, prepareClientToWrite called by addReply* will
return C_ERR directly. So the client can't receive the Pub/Sub
messages and any other push notifications, e.g client side tracking.

In this PR, we adding a CLIENT_PUSHING flag, disables the reply
silencing flags. When adding push replies, set the flag, after the reply,
clear the flag. Then add the flag check in prepareClientToWrite.

Fixes #11874

Note, the SUBSCRIBE command response is a bit awkward,
see https://github.com/redis/redis-doc/pull/2327

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit 416842e6c0)
(cherry picked from commit f8ae7a414c)
2023-04-17 15:54:26 +03:00
Madelyn Olson eee173a64e Always compact nodes in stream listpacks after creating new nodes (#11885)
This change attempts to alleviate a minor memory usage degradation for Redis 6.2 and onwards when using rather large objects (~2k) in streams. Introduced in #6281, we pre-allocate the head nodes of a stream to be 4kb, to limit the amount of unnecessary initial reallocations that are done. However, if we only ever allocate one object because 2 objects exceeds the max_stream_entry_size, we never actually shrink it to fit the single item. This can lead to a lot of excessive memory usage. For smaller item sizes this becomes less of an issue, as the overhead decreases as the items become smaller in size.

This commit also changes the MEMORY USAGE of streams, since it was reporting the lpBytes instead of the allocated size. This introduced an observability issue when diagnosing the memory issue, since Redis reported the same amount of used bytes pre and post change, even though the new implementation allocated more memory.

(cherry picked from commit 2bb29e4aa3)
(cherry picked from commit 17181517ec)
2023-04-17 15:54:26 +03:00
Slava Koyfman 126348536a Disconnect pub-sub subscribers when revoking allchannels permission (#11992)
The existing logic for killing pub-sub clients did not handle the `allchannels`
permission correctly. For example, if you:

    ACL SETUSER foo allchannels

Have a client authenticate as the user `foo` and subscribe to a channel, and then:

    ACL SETUSER foo resetchannels

The subscribed client would not be disconnected, though new clients under that user
would be blocked from subscribing to any channels.

This was caused by an incomplete optimization in `ACLKillPubsubClientsIfNeeded`
checking whether the new channel permissions were a strict superset of the old ones.

(cherry picked from commit f38aa6bfb7)
(cherry picked from commit 9caeadb8661f5fabd8386e3f07d445fe6ca46e7f)
2023-04-17 15:54:26 +03:00
chendianqiang e030e351fd fix hincrbyfloat not to create a key if the new value is invalid (#11149)
Check the validity of the value before performing the create operation,
prevents new data from being generated even if the request fails to execute.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: chendianqiang <chendianqiang@meituan.com>
Co-authored-by: Binbin <binloveplay1314@qq.com>
(cherry picked from commit bc7fe41e58)
(cherry picked from commit 606a385935363ea46c0df4f40f8a949d85f7a20a)
2023-04-17 15:54:26 +03:00
Oran Agra 720ea82eab Redis 6.2.11
CI / test-ubuntu-latest (push) Has been cancelled
CI / build-debian-old (push) Has been cancelled
CI / build-macos-latest (push) Has been cancelled
CI / build-32bit (push) Has been cancelled
CI / build-libc-malloc (push) Has been cancelled
CI / build-centos7-jemalloc (push) Has been cancelled
2023-02-28 18:32:14 +02:00
Oran Agra 0a8a45f94d Integer Overflow in RAND commands can lead to assertion (CVE-2023-25155)
Issue happens when passing a negative long value that greater than
the max positive value that the long can store.

(cherry picked from commit 41430af6a821c551abb862666ef896f2c196dea6)
2023-02-28 18:32:14 +02:00
Tom Levy f44b6a0e9a String pattern matching had exponential time complexity on pathological patterns (CVE-2022-36021)
Authenticated users can use string matching commands with a
specially crafted pattern to trigger a denial-of-service attack on Redis,
causing it to hang and consume 100% CPU time.

(cherry picked from commit e75f92047c22e659d49bba3a083cd0c9935f21e6)
2023-02-28 18:32:14 +02:00
Madelyn Olson 1797c5e374 Backport client pause cron from #9549 2023-02-28 18:32:14 +02:00
Madelyn Olson 683a4ce4b2 Prevent Redis from crashing from key tracking invalidations (#11814)
(cherry picked from commit f7150c45bc5d6f03c8ba86a9a9296d024c6848dc)
2023-02-28 18:32:14 +02:00
zhaozhao.zz 2322f3d87b correct cluster inbound link keepalive time (#11785)
(cherry picked from commit a35e08370a)
(cherry picked from commit 76473f50990e06872d5a08886549815077f5def5)
2023-02-28 18:32:14 +02:00
uriyage f084778cea Optimization: sdsRemoveFreeSpace to avoid realloc on noop (#11766)
In #7875 (Redis 6.2), we changed the sds alloc to be the usable allocation
size in order to:

> reduce the need for realloc calls by making the sds implicitly take over
the internal fragmentation

This change was done most sds functions, excluding `sdsRemoveFreeSpace` and
`sdsResize`, the reason is that in some places (e.g. clientsCronResizeQueryBuffer)
we call sdsRemoveFreeSpace when we see excessive free space and want to trim it.
so if we don't trim it exactly to size, the caller may still see excessive free space and
call it again and again.

However, this resulted in some excessive calls to realloc, even when there's no need
and it's gonna be a no-op (e.g. when reducing 15 bytes allocation to 13).

It turns out that a call for realloc with jemalloc can be expensive even if it ends up
doing nothing, so this PR adds a check using `je_nallocx`, which is cheap to avoid
the call for realloc.

in addition to that this PR unifies sdsResize and sdsRemoveFreeSpace into common
code. the difference between them was that sdsResize would avoid using SDS_TYPE_5,
since it want to keep the string ready to be resized again, while sdsRemoveFreeSpace
would permit using SDS_TYPE_5 and get an optimal memory consumption.
now both methods take a `would_regrow` argument that makes it more explicit.

the only actual impact of that is that in clientsCronResizeQueryBuffer we call both sdsResize
and sdsRemoveFreeSpace for in different cases, and we now prevent the use of SDS_TYPE_5 in both.

The new test that was added to cover this concern used to pass before this PR as well,
this PR is just a performance optimization and cleanup.

Benchmark:
`redis-benchmark -c 100 -t set  -d 512 -P 10  -n  100000000`
on i7-9850H with jemalloc, shows improvement from 1021k ops/sec to 1067k (average of 3 runs).
some 4.5% improvement.

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit 46393f9819)
(cherry picked from commit b12eeccddd9318a5d97a5aee2dad88999dfad53f)
2023-02-28 18:32:14 +02:00
Oran Agra 658f9bf8c4 Make sure that fork child doesn't do incremental rehashing (#11692)
Turns out that a fork child calling getExpire while persisting keys (and
possibly also a result of some module fork tasks) could cause dictFind
to do incremental rehashing in the child process, which is both a waste
of time, and also causes COW harm.

(cherry picked from commit 2bec254d89)
(cherry picked from commit 3e82bdf738)
2023-02-28 18:32:14 +02:00
39 changed files with 882 additions and 208 deletions
+79
View File
@@ -11,6 +11,85 @@ CRITICAL: There is a critical bug affecting MOST USERS. Upgrade ASAP.
SECURITY: There are security fixes in the release.
--------------------------------------------------------------------------------
================================================================================
Redis 6.2.14 Released Wed 18 Oct 2023 10:33:40 IDT
================================================================================
Upgrade urgency SECURITY: See security fixes below.
Security fixes
==============
* (CVE-2023-45145) The wrong order of listen(2) and chmod(2) calls creates a
race condition that can be used by another process to bypass desired Unix
socket permissions on startup.
================================================================================
Redis 6.2.13 Released Mon July 10 12:00:00 IDT 2023
================================================================================
Upgrade urgency SECURITY: See security fixes below.
Security Fixes:
* (CVE-2022-24834) A specially crafted Lua script executing in Redis can trigger
a heap overflow in the cjson and cmsgpack libraries, and result in heap
corruption and potentially remote code execution. The problem exists in all
versions of Redis with Lua scripting support, starting from 2.6, and affects
only authenticated and authorized users.
Bug Fixes
=========
* Re-enable downscale rehashing while there is a fork child (#12276)
================================================================================
Redis 6.2.12 Released Mon Apr 17 16:00:00 IST 2023
================================================================================
Upgrade urgency: SECURITY, contains fixes to security issues.
Security Fixes:
* (CVE-2023-28856) Authenticated users can use the HINCRBYFLOAT command to create
an invalid hash field that will crash Redis on access
Bug Fixes
=========
* Fix CLIENT REPLY OFF|SKIP to not silence push notifications (#11875)
* Disconnect pub-sub subscribers when revoking allchannels permission (#11992)
* Trim excessive memory usage in stream nodes when exceeding `stream-node-max-bytes` (#11885)
================================================================================
Redis 6.2.11 Released Tue Feb 28 12:00:00 IST 2023
================================================================================
Upgrade urgency: SECURITY, contains fixes to security issues.
Security Fixes:
* (CVE-2023-25155) Specially crafted SRANDMEMBER, ZRANDMEMBER, and HRANDFIELD
commands can trigger an integer overflow, resulting in a runtime assertion
and termination of the Redis server process.
* (CVE-2022-36021) String matching commands (like SCAN or KEYS) with a specially
crafted pattern to trigger a denial-of-service attack on Redis, causing it to
hang and consume 100% CPU time.
Bug Fixes
=========
* Fix a crash when reaching the maximum invalidations limit of client-side tracking (#11814)
* Fix cluster inbound link keepalive time (#11785)
* Make sure that fork child doesn't do incremental rehashing (#11692)
Performance and resource utilization improvements
=================================================
* Avoid realloc to reduce size of strings when it is unneeded (#11766)
================================================================================
Redis 6.2.10 Released Mon Jan 17 12:00:00 IST 2023
================================================================================
+7
View File
@@ -2,6 +2,8 @@
uname_S:= $(shell sh -c 'uname -s 2>/dev/null || echo not')
LUA_COVERAGE?=no
CCCOLOR="\033[34m"
LINKCOLOR="\033[34;1m"
SRCCOLOR="\033[33m"
@@ -71,6 +73,11 @@ endif
LUA_CFLAGS+= -O2 -Wall -DLUA_ANSI -DENABLE_CJSON_GLOBAL -DREDIS_STATIC='' -DLUA_USE_MKSTEMP $(CFLAGS)
LUA_LDFLAGS+= $(LDFLAGS)
ifeq ($(LUA_COVERAGE),yes)
LUA_CFLAGS += -fprofile-arcs -ftest-coverage
LUA_LDFLAGS += -fprofile-arcs -ftest-coverage
endif
# lua's Makefile defines AR="ar rcu", which is unusual, and makes it more
# challenging to cross-compile lua (and redis). These defines make it easier
# to fit redis into cross-compilation environments, which typically set AR.
+6 -3
View File
@@ -39,6 +39,7 @@
#include <assert.h>
#include <string.h>
#include <math.h>
#include <stdint.h>
#include <limits.h>
#include "lua.h"
#include "lauxlib.h"
@@ -141,13 +142,13 @@ typedef struct {
typedef struct {
json_token_type_t type;
int index;
size_t index;
union {
const char *string;
double number;
int boolean;
} value;
int string_len;
size_t string_len;
} json_token_t;
static const char *char2escape[256] = {
@@ -473,6 +474,8 @@ static void json_append_string(lua_State *l, strbuf_t *json, int lindex)
* This buffer is reused constantly for small strings
* If there are any excess pages, they won't be hit anyway.
* This gains ~5% speedup. */
if (len > SIZE_MAX / 6 - 3)
abort(); /* Overflow check */
strbuf_ensure_empty_length(json, len * 6 + 2);
strbuf_append_char_unsafe(json, '\"');
@@ -706,7 +709,7 @@ static int json_encode(lua_State *l)
strbuf_t local_encode_buf;
strbuf_t *encode_buf;
char *json;
int len;
size_t len;
luaL_argcheck(l, lua_gettop(l) == 1, 1, "expected 1 argument");
+17 -14
View File
@@ -117,7 +117,9 @@ mp_buf *mp_buf_new(lua_State *L) {
void mp_buf_append(lua_State *L, mp_buf *buf, const unsigned char *s, size_t len) {
if (buf->free < len) {
size_t newsize = (buf->len+len)*2;
size_t newsize = buf->len+len;
if (newsize < buf->len || newsize >= SIZE_MAX/2) abort();
newsize *= 2;
buf->b = (unsigned char*)mp_realloc(L, buf->b, buf->len + buf->free, newsize);
buf->free = newsize - buf->len;
@@ -173,7 +175,7 @@ void mp_cur_init(mp_cur *cursor, const unsigned char *s, size_t len) {
void mp_encode_bytes(lua_State *L, mp_buf *buf, const unsigned char *s, size_t len) {
unsigned char hdr[5];
int hdrlen;
size_t hdrlen;
if (len < 32) {
hdr[0] = 0xa0 | (len&0xff); /* fix raw */
@@ -220,7 +222,7 @@ void mp_encode_double(lua_State *L, mp_buf *buf, double d) {
void mp_encode_int(lua_State *L, mp_buf *buf, int64_t n) {
unsigned char b[9];
int enclen;
size_t enclen;
if (n >= 0) {
if (n <= 127) {
@@ -290,9 +292,9 @@ void mp_encode_int(lua_State *L, mp_buf *buf, int64_t n) {
mp_buf_append(L,buf,b,enclen);
}
void mp_encode_array(lua_State *L, mp_buf *buf, int64_t n) {
void mp_encode_array(lua_State *L, mp_buf *buf, uint64_t n) {
unsigned char b[5];
int enclen;
size_t enclen;
if (n <= 15) {
b[0] = 0x90 | (n & 0xf); /* fix array */
@@ -313,7 +315,7 @@ void mp_encode_array(lua_State *L, mp_buf *buf, int64_t n) {
mp_buf_append(L,buf,b,enclen);
}
void mp_encode_map(lua_State *L, mp_buf *buf, int64_t n) {
void mp_encode_map(lua_State *L, mp_buf *buf, uint64_t n) {
unsigned char b[5];
int enclen;
@@ -790,7 +792,7 @@ void mp_decode_to_lua_type(lua_State *L, mp_cur *c) {
}
}
int mp_unpack_full(lua_State *L, int limit, int offset) {
int mp_unpack_full(lua_State *L, lua_Integer limit, lua_Integer offset) {
size_t len;
const char *s;
mp_cur c;
@@ -802,10 +804,10 @@ int mp_unpack_full(lua_State *L, int limit, int offset) {
if (offset < 0 || limit < 0) /* requesting negative off or lim is invalid */
return luaL_error(L,
"Invalid request to unpack with offset of %d and limit of %d.",
offset, len);
(int) offset, (int) len);
else if (offset > len)
return luaL_error(L,
"Start offset %d greater than input length %d.", offset, len);
"Start offset %d greater than input length %d.", (int) offset, (int) len);
if (decode_all) limit = INT_MAX;
@@ -827,12 +829,13 @@ int mp_unpack_full(lua_State *L, int limit, int offset) {
/* c->left is the remaining size of the input buffer.
* subtract the entire buffer size from the unprocessed size
* to get our next start offset */
int offset = len - c.left;
size_t new_offset = len - c.left;
if (new_offset > LONG_MAX) abort();
luaL_checkstack(L, 1, "in function mp_unpack_full");
/* Return offset -1 when we have have processed the entire buffer. */
lua_pushinteger(L, c.left == 0 ? -1 : offset);
lua_pushinteger(L, c.left == 0 ? -1 : (lua_Integer) new_offset);
/* Results are returned with the arg elements still
* in place. Lua takes care of only returning
* elements above the args for us.
@@ -851,15 +854,15 @@ int mp_unpack(lua_State *L) {
}
int mp_unpack_one(lua_State *L) {
int offset = luaL_optinteger(L, 2, 0);
lua_Integer offset = luaL_optinteger(L, 2, 0);
/* Variable pop because offset may not exist */
lua_pop(L, lua_gettop(L)-1);
return mp_unpack_full(L, 1, offset);
}
int mp_unpack_limit(lua_State *L) {
int limit = luaL_checkinteger(L, 2);
int offset = luaL_optinteger(L, 3, 0);
lua_Integer limit = luaL_checkinteger(L, 2);
lua_Integer offset = luaL_optinteger(L, 3, 0);
/* Variable pop because offset may not exist */
lua_pop(L, lua_gettop(L)-1);
+28 -81
View File
@@ -26,6 +26,7 @@
#include <stdlib.h>
#include <stdarg.h>
#include <string.h>
#include <stdint.h>
#include "strbuf.h"
@@ -38,22 +39,22 @@ static void die(const char *fmt, ...)
va_end(arg);
fprintf(stderr, "\n");
exit(-1);
abort();
}
void strbuf_init(strbuf_t *s, int len)
void strbuf_init(strbuf_t *s, size_t len)
{
int size;
size_t size;
if (len <= 0)
if (!len)
size = STRBUF_DEFAULT_SIZE;
else
size = len + 1; /* \0 terminator */
size = len + 1;
if (size < len)
die("Overflow, len: %zu", len);
s->buf = NULL;
s->size = size;
s->length = 0;
s->increment = STRBUF_DEFAULT_INCREMENT;
s->dynamic = 0;
s->reallocs = 0;
s->debug = 0;
@@ -65,7 +66,7 @@ void strbuf_init(strbuf_t *s, int len)
strbuf_ensure_null(s);
}
strbuf_t *strbuf_new(int len)
strbuf_t *strbuf_new(size_t len)
{
strbuf_t *s;
@@ -81,20 +82,10 @@ strbuf_t *strbuf_new(int len)
return s;
}
void strbuf_set_increment(strbuf_t *s, int increment)
{
/* Increment > 0: Linear buffer growth rate
* Increment < -1: Exponential buffer growth rate */
if (increment == 0 || increment == -1)
die("BUG: Invalid string increment");
s->increment = increment;
}
static inline void debug_stats(strbuf_t *s)
{
if (s->debug) {
fprintf(stderr, "strbuf(%lx) reallocs: %d, length: %d, size: %d\n",
fprintf(stderr, "strbuf(%lx) reallocs: %d, length: %zd, size: %zd\n",
(long)s, s->reallocs, s->length, s->size);
}
}
@@ -113,7 +104,7 @@ void strbuf_free(strbuf_t *s)
free(s);
}
char *strbuf_free_to_string(strbuf_t *s, int *len)
char *strbuf_free_to_string(strbuf_t *s, size_t *len)
{
char *buf;
@@ -131,57 +122,62 @@ char *strbuf_free_to_string(strbuf_t *s, int *len)
return buf;
}
static int calculate_new_size(strbuf_t *s, int len)
static size_t calculate_new_size(strbuf_t *s, size_t len)
{
int reqsize, newsize;
size_t reqsize, newsize;
if (len <= 0)
die("BUG: Invalid strbuf length requested");
/* Ensure there is room for optional NULL termination */
reqsize = len + 1;
if (reqsize < len)
die("Overflow, len: %zu", len);
/* If the user has requested to shrink the buffer, do it exactly */
if (s->size > reqsize)
return reqsize;
newsize = s->size;
if (s->increment < 0) {
if (reqsize >= SIZE_MAX / 2) {
newsize = reqsize;
} else {
/* Exponential sizing */
while (newsize < reqsize)
newsize *= -s->increment;
} else {
/* Linear sizing */
newsize = ((newsize + s->increment - 1) / s->increment) * s->increment;
newsize *= 2;
}
if (newsize < reqsize)
die("BUG: strbuf length would overflow, len: %zu", len);
return newsize;
}
/* Ensure strbuf can handle a string length bytes long (ignoring NULL
* optional termination). */
void strbuf_resize(strbuf_t *s, int len)
void strbuf_resize(strbuf_t *s, size_t len)
{
int newsize;
size_t newsize;
newsize = calculate_new_size(s, len);
if (s->debug > 1) {
fprintf(stderr, "strbuf(%lx) resize: %d => %d\n",
fprintf(stderr, "strbuf(%lx) resize: %zd => %zd\n",
(long)s, s->size, newsize);
}
s->size = newsize;
s->buf = realloc(s->buf, s->size);
if (!s->buf)
die("Out of memory");
die("Out of memory, len: %zu", len);
s->reallocs++;
}
void strbuf_append_string(strbuf_t *s, const char *str)
{
int space, i;
int i;
size_t space;
space = strbuf_empty_length(s);
@@ -197,55 +193,6 @@ void strbuf_append_string(strbuf_t *s, const char *str)
}
}
/* strbuf_append_fmt() should only be used when an upper bound
* is known for the output string. */
void strbuf_append_fmt(strbuf_t *s, int len, const char *fmt, ...)
{
va_list arg;
int fmt_len;
strbuf_ensure_empty_length(s, len);
va_start(arg, fmt);
fmt_len = vsnprintf(s->buf + s->length, len, fmt, arg);
va_end(arg);
if (fmt_len < 0)
die("BUG: Unable to convert number"); /* This should never happen.. */
s->length += fmt_len;
}
/* strbuf_append_fmt_retry() can be used when the there is no known
* upper bound for the output string. */
void strbuf_append_fmt_retry(strbuf_t *s, const char *fmt, ...)
{
va_list arg;
int fmt_len, try;
int empty_len;
/* If the first attempt to append fails, resize the buffer appropriately
* and try again */
for (try = 0; ; try++) {
va_start(arg, fmt);
/* Append the new formatted string */
/* fmt_len is the length of the string required, excluding the
* trailing NULL */
empty_len = strbuf_empty_length(s);
/* Add 1 since there is also space to store the terminating NULL. */
fmt_len = vsnprintf(s->buf + s->length, empty_len + 1, fmt, arg);
va_end(arg);
if (fmt_len <= empty_len)
break; /* SUCCESS */
if (try > 0)
die("BUG: length of formatted string changed");
strbuf_resize(s, s->length + fmt_len);
}
s->length += fmt_len;
}
/* vi:ai et sw=4 ts=4:
*/
+19 -27
View File
@@ -27,15 +27,13 @@
/* Size: Total bytes allocated to *buf
* Length: String length, excluding optional NULL terminator.
* Increment: Allocation increments when resizing the string buffer.
* Dynamic: True if created via strbuf_new()
*/
typedef struct {
char *buf;
int size;
int length;
int increment;
size_t size;
size_t length;
int dynamic;
int reallocs;
int debug;
@@ -44,32 +42,26 @@ typedef struct {
#ifndef STRBUF_DEFAULT_SIZE
#define STRBUF_DEFAULT_SIZE 1023
#endif
#ifndef STRBUF_DEFAULT_INCREMENT
#define STRBUF_DEFAULT_INCREMENT -2
#endif
/* Initialise */
extern strbuf_t *strbuf_new(int len);
extern void strbuf_init(strbuf_t *s, int len);
extern void strbuf_set_increment(strbuf_t *s, int increment);
extern strbuf_t *strbuf_new(size_t len);
extern void strbuf_init(strbuf_t *s, size_t len);
/* Release */
extern void strbuf_free(strbuf_t *s);
extern char *strbuf_free_to_string(strbuf_t *s, int *len);
extern char *strbuf_free_to_string(strbuf_t *s, size_t *len);
/* Management */
extern void strbuf_resize(strbuf_t *s, int len);
static int strbuf_empty_length(strbuf_t *s);
static int strbuf_length(strbuf_t *s);
static char *strbuf_string(strbuf_t *s, int *len);
static void strbuf_ensure_empty_length(strbuf_t *s, int len);
extern void strbuf_resize(strbuf_t *s, size_t len);
static size_t strbuf_empty_length(strbuf_t *s);
static size_t strbuf_length(strbuf_t *s);
static char *strbuf_string(strbuf_t *s, size_t *len);
static void strbuf_ensure_empty_length(strbuf_t *s, size_t len);
static char *strbuf_empty_ptr(strbuf_t *s);
static void strbuf_extend_length(strbuf_t *s, int len);
static void strbuf_extend_length(strbuf_t *s, size_t len);
/* Update */
extern void strbuf_append_fmt(strbuf_t *s, int len, const char *fmt, ...);
extern void strbuf_append_fmt_retry(strbuf_t *s, const char *format, ...);
static void strbuf_append_mem(strbuf_t *s, const char *c, int len);
static void strbuf_append_mem(strbuf_t *s, const char *c, size_t len);
extern void strbuf_append_string(strbuf_t *s, const char *str);
static void strbuf_append_char(strbuf_t *s, const char c);
static void strbuf_ensure_null(strbuf_t *s);
@@ -87,12 +79,12 @@ static inline int strbuf_allocated(strbuf_t *s)
/* Return bytes remaining in the string buffer
* Ensure there is space for a NULL terminator. */
static inline int strbuf_empty_length(strbuf_t *s)
static inline size_t strbuf_empty_length(strbuf_t *s)
{
return s->size - s->length - 1;
}
static inline void strbuf_ensure_empty_length(strbuf_t *s, int len)
static inline void strbuf_ensure_empty_length(strbuf_t *s, size_t len)
{
if (len > strbuf_empty_length(s))
strbuf_resize(s, s->length + len);
@@ -103,12 +95,12 @@ static inline char *strbuf_empty_ptr(strbuf_t *s)
return s->buf + s->length;
}
static inline void strbuf_extend_length(strbuf_t *s, int len)
static inline void strbuf_extend_length(strbuf_t *s, size_t len)
{
s->length += len;
}
static inline int strbuf_length(strbuf_t *s)
static inline size_t strbuf_length(strbuf_t *s)
{
return s->length;
}
@@ -124,14 +116,14 @@ static inline void strbuf_append_char_unsafe(strbuf_t *s, const char c)
s->buf[s->length++] = c;
}
static inline void strbuf_append_mem(strbuf_t *s, const char *c, int len)
static inline void strbuf_append_mem(strbuf_t *s, const char *c, size_t len)
{
strbuf_ensure_empty_length(s, len);
memcpy(s->buf + s->length, c, len);
s->length += len;
}
static inline void strbuf_append_mem_unsafe(strbuf_t *s, const char *c, int len)
static inline void strbuf_append_mem_unsafe(strbuf_t *s, const char *c, size_t len)
{
memcpy(s->buf + s->length, c, len);
s->length += len;
@@ -142,7 +134,7 @@ static inline void strbuf_ensure_null(strbuf_t *s)
s->buf[s->length] = 0;
}
static inline char *strbuf_string(strbuf_t *s, int *len)
static inline char *strbuf_string(strbuf_t *s, size_t *len)
{
if (len)
*len = s->length;
+8 -1
View File
@@ -1284,7 +1284,14 @@ void ACLKillPubsubClientsIfNeeded(user *u, list *upcoming) {
listNode *ln, *lpn;
robj *o;
int kill = 0;
/* If any of the original rule has the all-channels permission, but the new
* one doesn't (verifed by the caller), then the new list is not a strict
* superset of the original, and the next loop can be skipped. */
if (u->flags & USER_FLAG_ALLCHANNELS) {
kill = 1;
}
/* Nothing to kill when the upcoming are a literal super set of the original
* permissions. */
listRewind(u->channels,&li);
+6 -5
View File
@@ -397,13 +397,16 @@ int anetUnixGenericConnect(char *err, const char *path, int flags)
return s;
}
static int anetListen(char *err, int s, struct sockaddr *sa, socklen_t len, int backlog) {
static int anetListen(char *err, int s, struct sockaddr *sa, socklen_t len, int backlog, mode_t perm) {
if (bind(s,sa,len) == -1) {
anetSetError(err, "bind: %s", strerror(errno));
close(s);
return ANET_ERR;
}
if (sa->sa_family == AF_LOCAL && perm)
chmod(((struct sockaddr_un *) sa)->sun_path, perm);
if (listen(s, backlog) == -1) {
anetSetError(err, "listen: %s", strerror(errno));
close(s);
@@ -447,7 +450,7 @@ static int _anetTcpServer(char *err, int port, char *bindaddr, int af, int backl
if (af == AF_INET6 && anetV6Only(err,s) == ANET_ERR) goto error;
if (anetSetReuseAddr(err,s) == ANET_ERR) goto error;
if (anetListen(err,s,p->ai_addr,p->ai_addrlen,backlog) == ANET_ERR) s = ANET_ERR;
if (anetListen(err,s,p->ai_addr,p->ai_addrlen,backlog,0) == ANET_ERR) s = ANET_ERR;
goto end;
}
if (p == NULL) {
@@ -484,10 +487,8 @@ int anetUnixServer(char *err, char *path, mode_t perm, int backlog)
memset(&sa,0,sizeof(sa));
sa.sun_family = AF_LOCAL;
strncpy(sa.sun_path,path,sizeof(sa.sun_path)-1);
if (anetListen(err,s,(struct sockaddr*)&sa,sizeof(sa),backlog) == ANET_ERR)
if (anetListen(err,s,(struct sockaddr*)&sa,sizeof(sa),backlog,perm) == ANET_ERR)
return ANET_ERR;
if (perm)
chmod(sa.sun_path, perm);
return s;
}
+1 -1
View File
@@ -710,7 +710,7 @@ void clusterAcceptHandler(aeEventLoop *el, int fd, void *privdata, int mask) {
}
connNonBlock(conn);
connEnableTcpNoDelay(conn);
connKeepAlive(conn,server.cluster_node_timeout * 2);
connKeepAlive(conn,server.cluster_node_timeout / 1000 * 2);
/* Use non-blocking I/O for cluster messages. */
serverLog(LL_VERBOSE,"Accepting cluster node connection from %s:%d", cip, cport);
+7 -3
View File
@@ -39,8 +39,12 @@
#include <fcntl.h>
#endif
#if defined(__APPLE__) && defined(__MAC_OS_X_VERSION_MAX_ALLOWED) && __MAC_OS_X_VERSION_MAX_ALLOWED >= 1060
#define MAC_OS_10_6_DETECTED
#endif
/* Define redis_fstat to fstat or fstat64() */
#if defined(__APPLE__) && !defined(MAC_OS_X_VERSION_10_6)
#if defined(__APPLE__) && !defined(MAC_OS_10_6_DETECTED)
#define redis_fstat fstat64
#define redis_stat stat64
#else
@@ -79,7 +83,7 @@
#define HAVE_EPOLL 1
#endif
#if (defined(__APPLE__) && defined(MAC_OS_X_VERSION_10_6)) || defined(__FreeBSD__) || defined(__OpenBSD__) || defined (__NetBSD__)
#if (defined(__APPLE__) && defined(MAC_OS_10_6_DETECTED)) || defined(__FreeBSD__) || defined(__OpenBSD__) || defined (__NetBSD__)
#define HAVE_KQUEUE 1
#endif
@@ -248,7 +252,7 @@ void setproctitle(const char *fmt, ...);
#include <kernel/OS.h>
#define redis_set_thread_title(name) rename_thread(find_thread(0), name)
#else
#if (defined __APPLE__ && defined(MAC_OS_X_VERSION_10_7))
#if (defined __APPLE__ && defined(__MAC_OS_X_VERSION_MAX_ALLOWED) && __MAC_OS_X_VERSION_MAX_ALLOWED >= 1070)
int pthread_setname_np(const char *name);
#include <pthread.h>
#define redis_set_thread_title(name) pthread_setname_np(name)
+15 -3
View File
@@ -467,6 +467,8 @@ void debugCommand(client *c) {
" Return the size of different Redis core C structures.",
"ZIPLIST <key>",
" Show low level info about the ziplist encoding of <key>.",
"PAUSE-CRON <0|1>",
" Stop periodic cron job processing.",
NULL
};
addReplyHelp(c, help);
@@ -748,9 +750,16 @@ NULL
* also have a normal reply type after the attribute. */
addReplyBulkCString(c,"Some real reply following the attribute");
} else if (!strcasecmp(name,"push")) {
if (c->resp < 3) {
addReplyError(c,"RESP2 is not supported by this command");
return;
}
uint64_t old_flags = c->flags;
c->flags |= CLIENT_PUSHING;
addReplyPushLen(c,2);
addReplyBulkCString(c,"server-cpu-usage");
addReplyLongLong(c,42);
if (!(old_flags & CLIENT_PUSHING)) c->flags &= ~CLIENT_PUSHING;
/* Push replies are not synchronous replies, so we emit also a
* normal reply in order for blocking clients just discarding the
* push reply, to actually consume the reply and continue. */
@@ -887,6 +896,9 @@ NULL
mallctl_string(c, c->argv+2, c->argc-2);
return;
#endif
} else if (!strcasecmp(c->argv[1]->ptr,"pause-cron") && c->argc == 3) {
server.pause_cron = atoi(c->argv[2]->ptr);
addReply(c,shared.ok);
} else {
addReplySubcommandSyntaxError(c);
return;
@@ -1030,7 +1042,7 @@ static void* getAndSetMcontextEip(ucontext_t *uc, void *eip) {
} \
return old_val; \
} while(0)
#if defined(__APPLE__) && !defined(MAC_OS_X_VERSION_10_6)
#if defined(__APPLE__) && !defined(MAC_OS_10_6_DETECTED)
/* OSX < 10.6 */
#if defined(__x86_64__)
GET_SET_RETURN(uc->uc_mcontext->__ss.__rip, eip);
@@ -1039,7 +1051,7 @@ static void* getAndSetMcontextEip(ucontext_t *uc, void *eip) {
#else
GET_SET_RETURN(uc->uc_mcontext->__ss.__srr0, eip);
#endif
#elif defined(__APPLE__) && defined(MAC_OS_X_VERSION_10_6)
#elif defined(__APPLE__) && defined(MAC_OS_10_6_DETECTED)
/* OSX >= 10.6 */
#if defined(_STRUCT_X86_THREAD_STATE64) && !defined(__i386__)
GET_SET_RETURN(uc->uc_mcontext->__ss.__rip, eip);
@@ -1111,7 +1123,7 @@ void logRegisters(ucontext_t *uc) {
serverLog(LL_WARNING|LL_RAW, "\n------ REGISTERS ------\n");
/* OSX */
#if defined(__APPLE__) && defined(MAC_OS_X_VERSION_10_6)
#if defined(__APPLE__) && defined(MAC_OS_10_6_DETECTED)
/* OSX AMD64 */
#if defined(_STRUCT_X86_THREAD_STATE64) && !defined(__i386__)
serverLog(LL_WARNING,
+24 -16
View File
@@ -47,15 +47,15 @@
#include "zmalloc.h"
#include "redisassert.h"
/* Using dictEnableResize() / dictDisableResize() we make possible to
* enable/disable resizing of the hash table as needed. This is very important
/* Using dictEnableResize() / dictDisableResize() we make possible to disable
* resizing and rehashing of the hash table as needed. This is very important
* for Redis, as we use copy-on-write and don't want to move too much memory
* around when there is a child performing saving operations.
*
* Note that even when dict_can_resize is set to 0, not all resizes are
* prevented: a hash table is still allowed to grow if the ratio between
* the number of elements and the buckets > dict_force_resize_ratio. */
static int dict_can_resize = 1;
static dictResizeEnable dict_can_resize = DICT_RESIZE_ENABLE;
static unsigned int dict_force_resize_ratio = 5;
/* -------------------------- private prototypes ---------------------------- */
@@ -132,7 +132,7 @@ int dictResize(dict *d)
{
unsigned long minimal;
if (!dict_can_resize || dictIsRehashing(d)) return DICT_ERR;
if (dict_can_resize != DICT_RESIZE_ENABLE || dictIsRehashing(d)) return DICT_ERR;
minimal = d->ht[0].used;
if (minimal < DICT_HT_INITIAL_SIZE)
minimal = DICT_HT_INITIAL_SIZE;
@@ -210,7 +210,15 @@ int dictTryExpand(dict *d, unsigned long size) {
* work it does would be unbound and the function may block for a long time. */
int dictRehash(dict *d, int n) {
int empty_visits = n*10; /* Max number of empty buckets to visit. */
if (!dictIsRehashing(d)) return 0;
unsigned long s0 = d->ht[0].size;
unsigned long s1 = d->ht[1].size;
if (dict_can_resize == DICT_RESIZE_FORBID || !dictIsRehashing(d)) return 0;
if (dict_can_resize == DICT_RESIZE_AVOID &&
((s1 > s0 && s1 / s0 < dict_force_resize_ratio) ||
(s1 < s0 && s0 / s1 < dict_force_resize_ratio)))
{
return 0;
}
while(n-- && d->ht[0].used != 0) {
dictEntry *de, *nextde;
@@ -995,10 +1003,12 @@ static int _dictExpandIfNeeded(dict *d)
* table (global setting) or we should avoid it but the ratio between
* elements/buckets is over the "safe" threshold, we resize doubling
* the number of buckets. */
if (d->ht[0].used >= d->ht[0].size &&
(dict_can_resize ||
d->ht[0].used/d->ht[0].size > dict_force_resize_ratio) &&
dictTypeExpandAllowed(d))
if (!dictTypeExpandAllowed(d))
return DICT_OK;
if ((dict_can_resize == DICT_RESIZE_ENABLE &&
d->ht[0].used >= d->ht[0].size) ||
(dict_can_resize != DICT_RESIZE_FORBID &&
d->ht[0].used / d->ht[0].size > dict_force_resize_ratio))
{
return dictExpand(d, d->ht[0].used + 1);
}
@@ -1057,12 +1067,8 @@ void dictEmpty(dict *d, void(callback)(void*)) {
d->pauserehash = 0;
}
void dictEnableResize(void) {
dict_can_resize = 1;
}
void dictDisableResize(void) {
dict_can_resize = 0;
void dictSetResizeEnabled(dictResizeEnable enable) {
dict_can_resize = enable;
}
uint64_t dictGetHash(dict *d, const void *key) {
@@ -1105,7 +1111,9 @@ size_t _dictGetStatsHt(char *buf, size_t bufsize, dictht *ht, int tableid) {
if (ht->used == 0) {
return snprintf(buf,bufsize,
"No stats available for empty dictionaries\n");
"Hash table %d stats (%s):\n"
"No stats available for empty dictionaries\n",
tableid, (tableid == 0) ? "main hash table" : "rehashing target");
}
/* Compute stats. */
+7 -2
View File
@@ -160,6 +160,12 @@ typedef void (dictScanBucketFunction)(void *privdata, dictEntry **bucketref);
#define randomULong() random()
#endif
typedef enum {
DICT_RESIZE_ENABLE,
DICT_RESIZE_AVOID,
DICT_RESIZE_FORBID,
} dictResizeEnable;
/* API */
dict *dictCreate(dictType *type, void *privDataPtr);
int dictExpand(dict *d, unsigned long size);
@@ -186,8 +192,7 @@ void dictGetStats(char *buf, size_t bufsize, dict *d);
uint64_t dictGenHashFunction(const void *key, int len);
uint64_t dictGenCaseHashFunction(const unsigned char *buf, int len);
void dictEmpty(dict *d, void(callback)(void*));
void dictEnableResize(void);
void dictDisableResize(void);
void dictSetResizeEnabled(dictResizeEnable enable);
int dictRehash(dict *d, int n);
int dictRehashMilliseconds(dict *d, int ms);
void dictSetHashFunctionSeed(uint8_t *seed);
+1 -1
View File
@@ -2558,7 +2558,7 @@ int RM_StringTruncate(RedisModuleKey *key, size_t newlen) {
sdssubstr(key->value->ptr,0,newlen);
/* If the string is too wasteful, reallocate it. */
if (sdslen(key->value->ptr) < sdsavail(key->value->ptr))
key->value->ptr = sdsRemoveFreeSpace(key->value->ptr);
key->value->ptr = sdsRemoveFreeSpace(key->value->ptr, 0);
}
}
return REDISMODULE_OK;
+5 -2
View File
@@ -260,8 +260,10 @@ int prepareClientToWrite(client *c) {
/* If CLIENT_CLOSE_ASAP flag is set, we need not write anything. */
if (c->flags & CLIENT_CLOSE_ASAP) return C_ERR;
/* CLIENT REPLY OFF / SKIP handling: don't send replies. */
if (c->flags & (CLIENT_REPLY_OFF|CLIENT_REPLY_SKIP)) return C_ERR;
/* CLIENT REPLY OFF / SKIP handling: don't send replies.
* CLIENT_PUSHING handling: disables the reply silencing flags. */
if ((c->flags & (CLIENT_REPLY_OFF|CLIENT_REPLY_SKIP)) &&
!(c->flags & CLIENT_PUSHING)) return C_ERR;
/* Masters don't receive replies, unless CLIENT_MASTER_FORCE_REPLY flag
* is set. */
@@ -819,6 +821,7 @@ void addReplyAttributeLen(client *c, long length) {
void addReplyPushLen(client *c, long length) {
serverAssert(c->resp >= 3);
serverAssertWithInfo(c, NULL, c->flags & CLIENT_PUSHING);
addReplyAggregateLen(c,length,'>');
}
+5 -3
View File
@@ -430,7 +430,7 @@ void trimStringObjectIfNeeded(robj *o) {
if (o->encoding == OBJ_ENCODING_RAW &&
sdsavail(o->ptr) > sdslen(o->ptr)/10)
{
o->ptr = sdsRemoveFreeSpace(o->ptr);
o->ptr = sdsRemoveFreeSpace(o->ptr, 0);
}
}
@@ -909,7 +909,8 @@ size_t objectComputeSize(robj *o, size_t sample_size) {
size_t lpsize = 0, samples = 0;
while(samples < sample_size && raxNext(&ri)) {
unsigned char *lp = ri.data;
lpsize += lpBytes(lp);
/* Use the allocated size, since we overprovision the node initially. */
lpsize += zmalloc_size(lp);
samples++;
}
if (s->rax->numele <= samples) {
@@ -921,7 +922,8 @@ size_t objectComputeSize(robj *o, size_t sample_size) {
* if there are a few elements in the radix tree. */
raxSeek(&ri,"$",NULL,0);
raxNext(&ri);
asize += lpBytes(ri.data);
/* Use the allocated size, since we overprovision the node initially. */
asize += zmalloc_size(ri.data);
}
raxStop(&ri);
+18
View File
@@ -41,6 +41,8 @@ int clientSubscriptionsCount(client *c);
* to send a special message (for instance an Array type) by using the
* addReply*() API family. */
void addReplyPubsubMessage(client *c, robj *channel, robj *msg) {
uint64_t old_flags = c->flags;
c->flags |= CLIENT_PUSHING;
if (c->resp == 2)
addReply(c,shared.mbulkhdr[3]);
else
@@ -48,12 +50,15 @@ void addReplyPubsubMessage(client *c, robj *channel, robj *msg) {
addReply(c,shared.messagebulk);
addReplyBulk(c,channel);
if (msg) addReplyBulk(c,msg);
if (!(old_flags & CLIENT_PUSHING)) c->flags &= ~CLIENT_PUSHING;
}
/* Send a pubsub message of type "pmessage" to the client. The difference
* with the "message" type delivered by addReplyPubsubMessage() is that
* this message format also includes the pattern that matched the message. */
void addReplyPubsubPatMessage(client *c, robj *pat, robj *channel, robj *msg) {
uint64_t old_flags = c->flags;
c->flags |= CLIENT_PUSHING;
if (c->resp == 2)
addReply(c,shared.mbulkhdr[4]);
else
@@ -62,10 +67,13 @@ void addReplyPubsubPatMessage(client *c, robj *pat, robj *channel, robj *msg) {
addReplyBulk(c,pat);
addReplyBulk(c,channel);
addReplyBulk(c,msg);
if (!(old_flags & CLIENT_PUSHING)) c->flags &= ~CLIENT_PUSHING;
}
/* Send the pubsub subscription notification to the client. */
void addReplyPubsubSubscribed(client *c, robj *channel) {
uint64_t old_flags = c->flags;
c->flags |= CLIENT_PUSHING;
if (c->resp == 2)
addReply(c,shared.mbulkhdr[3]);
else
@@ -73,6 +81,7 @@ void addReplyPubsubSubscribed(client *c, robj *channel) {
addReply(c,shared.subscribebulk);
addReplyBulk(c,channel);
addReplyLongLong(c,clientSubscriptionsCount(c));
if (!(old_flags & CLIENT_PUSHING)) c->flags &= ~CLIENT_PUSHING;
}
/* Send the pubsub unsubscription notification to the client.
@@ -80,6 +89,8 @@ void addReplyPubsubSubscribed(client *c, robj *channel) {
* unsubscribe command but there are no channels to unsubscribe from: we
* still send a notification. */
void addReplyPubsubUnsubscribed(client *c, robj *channel) {
uint64_t old_flags = c->flags;
c->flags |= CLIENT_PUSHING;
if (c->resp == 2)
addReply(c,shared.mbulkhdr[3]);
else
@@ -90,10 +101,13 @@ void addReplyPubsubUnsubscribed(client *c, robj *channel) {
else
addReplyNull(c);
addReplyLongLong(c,clientSubscriptionsCount(c));
if (!(old_flags & CLIENT_PUSHING)) c->flags &= ~CLIENT_PUSHING;
}
/* Send the pubsub pattern subscription notification to the client. */
void addReplyPubsubPatSubscribed(client *c, robj *pattern) {
uint64_t old_flags = c->flags;
c->flags |= CLIENT_PUSHING;
if (c->resp == 2)
addReply(c,shared.mbulkhdr[3]);
else
@@ -101,6 +115,7 @@ void addReplyPubsubPatSubscribed(client *c, robj *pattern) {
addReply(c,shared.psubscribebulk);
addReplyBulk(c,pattern);
addReplyLongLong(c,clientSubscriptionsCount(c));
if (!(old_flags & CLIENT_PUSHING)) c->flags &= ~CLIENT_PUSHING;
}
/* Send the pubsub pattern unsubscription notification to the client.
@@ -108,6 +123,8 @@ void addReplyPubsubPatSubscribed(client *c, robj *pattern) {
* punsubscribe command but there are no pattern to unsubscribe from: we
* still send a notification. */
void addReplyPubsubPatUnsubscribed(client *c, robj *pattern) {
uint64_t old_flags = c->flags;
c->flags |= CLIENT_PUSHING;
if (c->resp == 2)
addReply(c,shared.mbulkhdr[3]);
else
@@ -118,6 +135,7 @@ void addReplyPubsubPatUnsubscribed(client *c, robj *pattern) {
else
addReplyNull(c);
addReplyLongLong(c,clientSubscriptionsCount(c));
if (!(old_flags & CLIENT_PUSHING)) c->flags &= ~CLIENT_PUSHING;
}
/*-----------------------------------------------------------------------------
+76 -16
View File
@@ -287,40 +287,72 @@ sds sdsMakeRoomFor(sds s, size_t addlen) {
*
* After the call, the passed sds string is no longer valid and all the
* references must be substituted with the new pointer returned by the call. */
sds sdsRemoveFreeSpace(sds s) {
sds sdsRemoveFreeSpace(sds s, int would_regrow) {
return sdsResize(s, sdslen(s), would_regrow);
}
/* Resize the allocation, this can make the allocation bigger or smaller,
* if the size is smaller than currently used len, the data will be truncated.
*
* The when the would_regrow argument is set to 1, it prevents the use of
* SDS_TYPE_5, which is desired when the sds is likely to be changed again.
*
* The sdsAlloc size will be set to the requested size regardless of the actual
* allocation size, this is done in order to avoid repeated calls to this
* function when the caller detects that it has excess space. */
sds sdsResize(sds s, size_t size, int would_regrow) {
void *sh, *newsh;
char type, oldtype = s[-1] & SDS_TYPE_MASK;
int hdrlen, oldhdrlen = sdsHdrSize(oldtype);
size_t len = sdslen(s);
size_t avail = sdsavail(s);
sh = (char*)s-oldhdrlen;
/* Return ASAP if there is no space left. */
if (avail == 0) return s;
/* Return ASAP if the size is already good. */
if (sdsalloc(s) == size) return s;
/* Truncate len if needed. */
if (size < len) len = size;
/* Check what would be the minimum SDS header that is just good enough to
* fit this string. */
type = sdsReqType(len);
type = sdsReqType(size);
if (would_regrow) {
/* Don't use type 5, it is not good for strings that are expected to grow back. */
if (type == SDS_TYPE_5) type = SDS_TYPE_8;
}
hdrlen = sdsHdrSize(type);
/* If the type is the same, or at least a large enough type is still
* required, we just realloc(), letting the allocator to do the copy
* only if really needed. Otherwise if the change is huge, we manually
* reallocate the string to use the different header type. */
if (oldtype==type || type > SDS_TYPE_8) {
newsh = s_realloc(sh, oldhdrlen+len+1);
/* If the type is the same, or can hold the size in it with low overhead
* (larger than SDS_TYPE_8), we just realloc(), letting the allocator
* to do the copy only if really needed. Otherwise if the change is
* huge, we manually reallocate the string to use the different header
* type. */
int use_realloc = (oldtype==type || (type < oldtype && type > SDS_TYPE_8));
size_t newlen = use_realloc ? oldhdrlen+size+1 : hdrlen+size+1;
int alloc_already_optimal = 0;
#if defined(USE_JEMALLOC)
/* je_nallocx returns the expected allocation size for the newlen.
* We aim to avoid calling realloc() when using Jemalloc if there is no
* change in the allocation size, as it incurs a cost even if the
* allocation size stays the same. */
alloc_already_optimal = (je_nallocx(newlen, 0) == zmalloc_size(sh));
#endif
if (use_realloc && !alloc_already_optimal) {
newsh = s_realloc(sh, newlen);
if (newsh == NULL) return NULL;
s = (char*)newsh+oldhdrlen;
} else {
newsh = s_malloc(hdrlen+len+1);
} else if (!alloc_already_optimal) {
newsh = s_malloc(newlen);
if (newsh == NULL) return NULL;
memcpy((char*)newsh+hdrlen, s, len+1);
memcpy((char*)newsh+hdrlen, s, len);
s_free(sh);
s = (char*)newsh+hdrlen;
s[-1] = type;
sdssetlen(s, len);
}
sdssetalloc(s, len);
s[len] = 0;
sdssetlen(s, len);
sdssetalloc(s, size);
return s;
}
@@ -1455,6 +1487,34 @@ int sdsTest(int argc, char **argv, int accurate) {
test_cond("sdstemplate() with quoting",
memcmp(x,"v1={value1} {} v2=value2",24) == 0);
sdsfree(x);
/* Test sdsresize - extend */
x = sdsnew("1234567890123456789012345678901234567890");
x = sdsResize(x, 200, 1);
test_cond("sdsrezie() expand len", sdslen(x) == 40);
test_cond("sdsrezie() expand strlen", strlen(x) == 40);
test_cond("sdsrezie() expand alloc", sdsalloc(x) == 200);
/* Test sdsresize - trim free space */
x = sdsResize(x, 80, 1);
test_cond("sdsrezie() shrink len", sdslen(x) == 40);
test_cond("sdsrezie() shrink strlen", strlen(x) == 40);
test_cond("sdsrezie() shrink alloc", sdsalloc(x) == 80);
/* Test sdsresize - crop used space */
x = sdsResize(x, 30, 1);
test_cond("sdsrezie() crop len", sdslen(x) == 30);
test_cond("sdsrezie() crop strlen", strlen(x) == 30);
test_cond("sdsrezie() crop alloc", sdsalloc(x) == 30);
/* Test sdsresize - extend to different class */
x = sdsResize(x, 400, 1);
test_cond("sdsrezie() expand len", sdslen(x) == 30);
test_cond("sdsrezie() expand strlen", strlen(x) == 30);
test_cond("sdsrezie() expand alloc", sdsalloc(x) == 400);
/* Test sdsresize - shrink to different class */
x = sdsResize(x, 4, 1);
test_cond("sdsrezie() crop len", sdslen(x) == 4);
test_cond("sdsrezie() crop strlen", strlen(x) == 4);
test_cond("sdsrezie() crop alloc", sdsalloc(x) == 4);
sdsfree(x);
}
test_report();
return 0;
+2 -1
View File
@@ -265,7 +265,8 @@ sds sdstemplate(const char *template, sdstemplate_callback_t cb_func, void *cb_a
/* Low level functions exposed to the user API */
sds sdsMakeRoomFor(sds s, size_t addlen);
void sdsIncrLen(sds s, ssize_t incr);
sds sdsRemoveFreeSpace(sds s);
sds sdsRemoveFreeSpace(sds s, int would_regrow);
sds sdsResize(sds s, size_t size, int would_regrow);
size_t sdsAllocSize(sds s);
void *sdsAllocPtr(sds s);
+13 -6
View File
@@ -1587,13 +1587,15 @@ int incrementallyRehash(int dbid) {
* as we want to avoid resizing the hash tables when there is a child in order
* to play well with copy-on-write (otherwise when a resize happens lots of
* memory pages are copied). The goal of this function is to update the ability
* for dict.c to resize the hash tables accordingly to the fact we have an
* for dict.c to resize or rehash the tables accordingly to the fact we have an
* active fork child running. */
void updateDictResizePolicy(void) {
if (!hasActiveChildProcess())
dictEnableResize();
if (server.in_fork_child != CHILD_TYPE_NONE)
dictSetResizeEnabled(DICT_RESIZE_FORBID);
else if (hasActiveChildProcess())
dictSetResizeEnabled(DICT_RESIZE_AVOID);
else
dictDisableResize();
dictSetResizeEnabled(DICT_RESIZE_ENABLE);
}
const char *strChildType(int type) {
@@ -1686,7 +1688,7 @@ int clientsCronResizeQueryBuffer(client *c) {
/* Only resize the query buffer if it is actually wasting
* at least a few kbytes. */
if (sdsavail(c->querybuf) > 1024*4) {
c->querybuf = sdsRemoveFreeSpace(c->querybuf);
c->querybuf = sdsRemoveFreeSpace(c->querybuf,1);
}
}
/* Reset the peak again to capture the peak memory usage in the next
@@ -1706,7 +1708,7 @@ int clientsCronResizeQueryBuffer(client *c) {
if(pending_querybuf_size > LIMIT_PENDING_QUERYBUF &&
sdslen(c->pending_querybuf) < (pending_querybuf_size/2))
{
c->pending_querybuf = sdsRemoveFreeSpace(c->pending_querybuf);
c->pending_querybuf = sdsRemoveFreeSpace(c->pending_querybuf,1);
}
}
return 0;
@@ -2070,6 +2072,9 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) {
}
}
/* for debug purposes: skip actual cron work if pause_cron is on */
if (server.pause_cron) return 1000/server.hz;
run_with_period(100) {
long long stat_net_input_bytes, stat_net_output_bytes;
atomicGet(server.stat_net_input_bytes, stat_net_input_bytes);
@@ -2695,6 +2700,7 @@ void initServerConfig(void) {
server.migrate_cached_sockets = dictCreate(&migrateCacheDictType,NULL);
server.next_client_id = 1; /* Client IDs, start from 1 .*/
server.loading_process_events_interval_bytes = (1024*1024*2);
server.pause_cron = 0;
unsigned int lruclock = getLRUClock();
atomicSet(server.lruclock,lruclock);
@@ -5923,6 +5929,7 @@ int redisFork(int purpose) {
server.in_fork_child = purpose;
setOOMScoreAdj(CONFIG_OOM_BGCHILD);
setupChildSignalHandlers();
updateDictResizePolicy();
closeChildUnusedResourceAfterFork();
/* Close the reading part, so that if the parent crashes, the child will
* get a write error and exit. */
+2
View File
@@ -286,6 +286,7 @@ extern int configOOMScoreAdjValuesDefaults[CONFIG_OOM_COUNT];
and AOF client */
#define CLIENT_REPL_RDBONLY (1ULL<<42) /* This client is a replica that only wants
RDB without replication buffer. */
#define CLIENT_PUSHING (1ULL<<43) /* This client is pushing notifications. */
/* Client block type (btype field in client structure)
* if CLIENT_BLOCKED flag is set. */
@@ -1345,6 +1346,7 @@ struct redisServer {
int set_proc_title; /* True if change proc title */
char *proc_title_template; /* Process title template format */
clientBufferLimitsConfig client_obuf_limits[CLIENT_TYPE_OBUF_COUNT];
int pause_cron; /* Don't run cron tasks (debug) */
/* AOF persistence */
int aof_enabled; /* AOF configuration */
int aof_state; /* AOF_(ON|OFF|WAIT_REWRITE) */
+6 -2
View File
@@ -729,6 +729,10 @@ void hincrbyfloatCommand(client *c) {
unsigned int vlen;
if (getLongDoubleFromObjectOrReply(c,c->argv[3],&incr,NULL) != C_OK) return;
if (isnan(incr) || isinf(incr)) {
addReplyError(c,"value is NaN or Infinity");
return;
}
if ((o = hashTypeLookupWriteOrCreate(c,c->argv[1])) == NULL) return;
if (hashTypeGetValue(o,c->argv[2]->ptr,&vstr,&vlen,&ll) == C_OK) {
if (vstr) {
@@ -1192,13 +1196,13 @@ void hrandfieldCommand(client *c) {
ziplistEntry ele;
if (c->argc >= 3) {
if (getLongFromObjectOrReply(c,c->argv[2],&l,NULL) != C_OK) return;
if (getRangeLongFromObjectOrReply(c,c->argv[2],-LONG_MAX,LONG_MAX,&l,NULL) != C_OK) return;
if (c->argc > 4 || (c->argc == 4 && strcasecmp(c->argv[3]->ptr,"withvalues"))) {
addReplyErrorObject(c,shared.syntaxerr);
return;
} else if (c->argc == 4) {
withvalues = 1;
if (l < LONG_MIN/2 || l > LONG_MAX/2) {
if (l < -LONG_MAX/2 || l > LONG_MAX/2) {
addReplyError(c,"value is out of range");
return;
}
+1 -1
View File
@@ -667,7 +667,7 @@ void srandmemberWithCountCommand(client *c) {
dict *d;
if (getLongFromObjectOrReply(c,c->argv[2],&l,NULL) != C_OK) return;
if (getRangeLongFromObjectOrReply(c,c->argv[2],-LONG_MAX,LONG_MAX,&l,NULL) != C_OK) return;
if (l >= 0) {
count = (unsigned long) l;
} else {
+11 -8
View File
@@ -532,22 +532,25 @@ int streamAppendItem(stream *s, robj **argv, int64_t numfields, streamID *added_
* if we need to switch to the next one. 'lp' will be set to NULL if
* the current node is full. */
if (lp != NULL) {
int new_node = 0;
size_t node_max_bytes = server.stream_node_max_bytes;
if (node_max_bytes == 0 || node_max_bytes > STREAM_LISTPACK_MAX_SIZE)
node_max_bytes = STREAM_LISTPACK_MAX_SIZE;
if (lp_bytes + totelelen >= node_max_bytes) {
lp = NULL;
new_node = 1;
} else if (server.stream_node_max_entries) {
unsigned char *lp_ele = lpFirst(lp);
/* Count both live entries and deleted ones. */
int64_t count = lpGetInteger(lp_ele) + lpGetInteger(lpNext(lp,lp_ele));
if (count >= server.stream_node_max_entries) {
/* Shrink extra pre-allocated memory */
lp = lpShrinkToFit(lp);
if (ri.data != lp)
raxInsert(s->rax,ri.key,ri.key_len,lp,NULL);
lp = NULL;
}
if (count >= server.stream_node_max_entries) new_node = 1;
}
if (new_node) {
/* Shrink extra pre-allocated memory */
lp = lpShrinkToFit(lp);
if (ri.data != lp)
raxInsert(s->rax,ri.key,ri.key_len,lp,NULL);
lp = NULL;
}
}
+2 -2
View File
@@ -4238,13 +4238,13 @@ void zrandmemberCommand(client *c) {
ziplistEntry ele;
if (c->argc >= 3) {
if (getLongFromObjectOrReply(c,c->argv[2],&l,NULL) != C_OK) return;
if (getRangeLongFromObjectOrReply(c,c->argv[2],-LONG_MAX,LONG_MAX,&l,NULL) != C_OK) return;
if (c->argc > 4 || (c->argc == 4 && strcasecmp(c->argv[3]->ptr,"withscores"))) {
addReplyErrorObject(c,shared.syntaxerr);
return;
} else if (c->argc == 4) {
withscores = 1;
if (l < LONG_MIN/2 || l > LONG_MAX/2) {
if (l < -LONG_MAX/2 || l > LONG_MAX/2) {
addReplyError(c,"value is out of range");
return;
}
+12 -3
View File
@@ -260,6 +260,9 @@ void trackingRememberKeys(client *c) {
* - Following a flush command, to send a single RESP NULL to indicate
* that all keys are now invalid. */
void sendTrackingMessage(client *c, char *keyname, size_t keylen, int proto) {
uint64_t old_flags = c->flags;
c->flags |= CLIENT_PUSHING;
int using_redirection = 0;
if (c->client_tracking_redirection) {
client *redir = lookupClientByID(c->client_tracking_redirection);
@@ -273,10 +276,14 @@ void sendTrackingMessage(client *c, char *keyname, size_t keylen, int proto) {
addReplyBulkCBuffer(c,"tracking-redir-broken",21);
addReplyLongLong(c,c->client_tracking_redirection);
}
if (!(old_flags & CLIENT_PUSHING)) c->flags &= ~CLIENT_PUSHING;
return;
}
if (!(old_flags & CLIENT_PUSHING)) c->flags &= ~CLIENT_PUSHING;
c = redir;
using_redirection = 1;
old_flags = c->flags;
c->flags |= CLIENT_PUSHING;
}
/* Only send such info for clients in RESP version 3 or more. However
@@ -295,6 +302,7 @@ void sendTrackingMessage(client *c, char *keyname, size_t keylen, int proto) {
* redirecting to another client. We can't send anything to
* it since RESP2 does not support push messages in the same
* connection. */
if (!(old_flags & CLIENT_PUSHING)) c->flags &= ~CLIENT_PUSHING;
return;
}
@@ -305,6 +313,7 @@ void sendTrackingMessage(client *c, char *keyname, size_t keylen, int proto) {
addReplyArrayLen(c,1);
addReplyBulkCBuffer(c,keyname,keylen);
}
if (!(old_flags & CLIENT_PUSHING)) c->flags &= ~CLIENT_PUSHING;
}
/* This function is called when a key is modified in Redis and in the case
@@ -386,10 +395,10 @@ void trackingInvalidateKey(client *c, robj *keyobj, int bcast) {
continue;
}
/* If target is current client, we need schedule key invalidation.
/* If target is current client and it's executing a command, we need schedule key invalidation.
* As the invalidation messages may be interleaved with command
* response and should after command response */
if (target == server.current_client){
* response and should after command response. */
if (target == server.current_client && server.fixed_time_expire) {
incrRefCount(keyobj);
listAddNodeTail(server.tracking_pending_keys, keyobj);
} else {
+23 -4
View File
@@ -45,8 +45,8 @@
#include "sha256.h"
/* Glob-style pattern matching. */
int stringmatchlen(const char *pattern, int patternLen,
const char *string, int stringLen, int nocase)
static int stringmatchlen_impl(const char *pattern, int patternLen,
const char *string, int stringLen, int nocase, int *skipLongerMatches)
{
while(patternLen && stringLen) {
switch(pattern[0]) {
@@ -58,12 +58,25 @@ int stringmatchlen(const char *pattern, int patternLen,
if (patternLen == 1)
return 1; /* match */
while(stringLen) {
if (stringmatchlen(pattern+1, patternLen-1,
string, stringLen, nocase))
if (stringmatchlen_impl(pattern+1, patternLen-1,
string, stringLen, nocase, skipLongerMatches))
return 1; /* match */
if (*skipLongerMatches)
return 0; /* no match */
string++;
stringLen--;
}
/* There was no match for the rest of the pattern starting
* from anywhere in the rest of the string. If there were
* any '*' earlier in the pattern, we can terminate the
* search early without trying to match them to longer
* substrings. This is because a longer match for the
* earlier part of the pattern would require the rest of the
* pattern to match starting later in the string, and we
* have just determined that there is no match for the rest
* of the pattern starting from anywhere in the current
* string. */
*skipLongerMatches = 1;
return 0; /* no match */
break;
case '?':
@@ -165,6 +178,12 @@ int stringmatchlen(const char *pattern, int patternLen,
return 0;
}
int stringmatchlen(const char *pattern, int patternLen,
const char *string, int stringLen, int nocase) {
int skipLongerMatches = 0;
return stringmatchlen_impl(pattern,patternLen,string,stringLen,nocase,&skipLongerMatches);
}
int stringmatch(const char *pattern, const char *string, int nocase) {
return stringmatchlen(pattern,strlen(pattern),string,strlen(string),nocase);
}
+2 -2
View File
@@ -1,2 +1,2 @@
#define REDIS_VERSION "6.2.10"
#define REDIS_VERSION_NUM 0x0006020a
#define REDIS_VERSION "6.2.14"
#define REDIS_VERSION_NUM 0x0006020e
+31 -1
View File
@@ -569,7 +569,7 @@ proc get_child_pid {idx} {
}
proc process_is_alive pid {
if {[catch {exec ps -p $pid} err]} {
if {[catch {exec ps -p $pid -f} err]} {
return 0
} else {
if {[string match "*<defunct>*" $err]} { return 0 }
@@ -800,3 +800,33 @@ proc wait_for_blocked_clients_count {count {maxtries 100} {delay 10}} {
}
}
proc config_set {param value {options {}}} {
set mayfail 0
foreach option $options {
switch $option {
"mayfail" {
set mayfail 1
}
default {
error "Unknown option $option"
}
}
}
if {[catch {r config set $param $value} err]} {
if {!$mayfail} {
error $err
} else {
if {$::verbose} {
puts "Ignoring CONFIG SET $param $value failure: $err"
}
}
}
}
proc config_get_set {param value {options {}}} {
set config [lindex [r config get $param] 1]
config_set $param $value $options
return $config
}
+14
View File
@@ -207,6 +207,20 @@ start_server {tags {"acl"}} {
$rd close
} {0}
test {Subscribers are killed when revoked of allchannels permission} {
set rd [redis_deferring_client]
r ACL setuser psuser allchannels
$rd AUTH psuser pspass
$rd read
$rd CLIENT SETNAME deathrow
$rd read
$rd PSUBSCRIBE foo
$rd read
r ACL setuser psuser resetchannels
assert_no_match {*deathrow*} [r CLIENT LIST]
$rd close
} {0}
test {Subscribers are pardoned if literal permissions are retained and/or gaining allchannels} {
set rd [redis_deferring_client]
r ACL setuser psuser resetchannels &foo:1 &bar:*
+42
View File
@@ -13,6 +13,48 @@ start_server {tags {"introspection"}} {
r client info
} {*addr=*:* fd=* age=* idle=* flags=N db=9 sub=0 psub=0 multi=-1 qbuf=26 qbuf-free=* argv-mem=* obl=0 oll=0 omem=0 tot-mem=* events=r cmd=client*}
test "CLIENT REPLY OFF/ON: disable all commands reply" {
set rd [redis_deferring_client]
# These replies were silenced.
$rd client reply off
$rd ping pong
$rd ping pong2
$rd client reply on
assert_equal {OK} [$rd read]
$rd ping pong3
assert_equal {pong3} [$rd read]
$rd close
}
test "CLIENT REPLY SKIP: skip the next command reply" {
set rd [redis_deferring_client]
# The first pong reply was silenced.
$rd client reply skip
$rd ping pong
$rd ping pong2
assert_equal {pong2} [$rd read]
$rd close
}
test "CLIENT REPLY ON: unset SKIP flag" {
set rd [redis_deferring_client]
$rd client reply skip
$rd client reply on
assert_equal {OK} [$rd read] ;# OK from CLIENT REPLY ON command
$rd ping
assert_equal {PONG} [$rd read]
$rd close
}
test {MONITOR can log executed commands} {
set rd [redis_deferring_client]
$rd monitor
+6
View File
@@ -479,4 +479,10 @@ start_server {tags {"keyspace"}} {
r keys *
r keys *
} {dlskeriewrioeuwqoirueioqwrueoqwrueqw}
test {Regression for pattern matching long nested loops} {
r flushdb
r SET aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 1
r KEYS "a*a*a*a*a*a*a*a*a*a*a*a*a*a*a*a*a*a*a*a*b"
} {}
}
+28
View File
@@ -140,6 +140,30 @@ start_server {tags {"pubsub network"}} {
$rd1 close
}
test "PubSub messages with CLIENT REPLY OFF" {
set rd [redis_deferring_client]
$rd hello 3
$rd read ;# Discard the hello reply
# Test that the subscribe/psubscribe notification is ok
$rd client reply off
assert_equal {1} [subscribe $rd channel]
assert_equal {2} [psubscribe $rd ch*]
# Test that the publish notification is ok
$rd client reply off
assert_equal 2 [r publish channel hello]
assert_equal {message channel hello} [$rd read]
assert_equal {pmessage ch* channel hello} [$rd read]
# Test that the unsubscribe/punsubscribe notification is ok
$rd client reply off
assert_equal {1} [unsubscribe $rd channel]
assert_equal {0} [punsubscribe $rd ch*]
$rd close
}
test "PUNSUBSCRIBE from non-subscribed channels" {
set rd1 [redis_deferring_client]
assert_equal {0 0 0} [punsubscribe $rd1 {foo.* bar.* quux.*}]
@@ -198,6 +222,7 @@ start_server {tags {"pubsub network"}} {
test "Keyspace notifications: we receive keyspace notifications" {
r config set notify-keyspace-events KA
set rd1 [redis_deferring_client]
$rd1 CLIENT REPLY OFF ;# Make sure it works even if replies are silenced
assert_equal {1} [psubscribe $rd1 *]
r set foo bar
assert_equal {pmessage * __keyspace@9__:foo set} [$rd1 read]
@@ -207,6 +232,7 @@ start_server {tags {"pubsub network"}} {
test "Keyspace notifications: we receive keyevent notifications" {
r config set notify-keyspace-events EA
set rd1 [redis_deferring_client]
$rd1 CLIENT REPLY SKIP ;# Make sure it works even if replies are silenced
assert_equal {1} [psubscribe $rd1 *]
r set foo bar
assert_equal {pmessage * __keyevent@9__:set foo} [$rd1 read]
@@ -216,6 +242,8 @@ start_server {tags {"pubsub network"}} {
test "Keyspace notifications: we can receive both kind of events" {
r config set notify-keyspace-events KEA
set rd1 [redis_deferring_client]
$rd1 CLIENT REPLY ON ;# Just coverage
assert_equal {OK} [$rd1 read]
assert_equal {1} [psubscribe $rd1 *]
r set foo bar
assert_equal {pmessage * __keyspace@9__:foo set} [$rd1 read]
+123
View File
@@ -259,6 +259,66 @@ start_server {tags {"scripting"}} {
} 0
} {a b}
test {EVAL - JSON smoke test} {
r eval {
local some_map = {
s1="Some string",
n1=100,
a1={"Some","String","Array"},
nil1=nil,
b1=true,
b2=false}
local encoded = cjson.encode(some_map)
local decoded = cjson.decode(encoded)
assert(table.concat(some_map) == table.concat(decoded))
cjson.encode_keep_buffer(false)
encoded = cjson.encode(some_map)
decoded = cjson.decode(encoded)
assert(table.concat(some_map) == table.concat(decoded))
-- Table with numeric keys
local table1 = {one="one", [1]="one"}
encoded = cjson.encode(table1)
decoded = cjson.decode(encoded)
assert(decoded["one"] == table1["one"])
assert(decoded["1"] == table1[1])
-- Array
local array1 = {[1]="one", [2]="two"}
encoded = cjson.encode(array1)
decoded = cjson.decode(encoded)
assert(table.concat(array1) == table.concat(decoded))
-- Invalid keys
local invalid_map = {}
invalid_map[false] = "false"
local ok, encoded = pcall(cjson.encode, invalid_map)
assert(ok == false)
-- Max depth
cjson.encode_max_depth(1)
ok, encoded = pcall(cjson.encode, some_map)
assert(ok == false)
cjson.decode_max_depth(1)
ok, decoded = pcall(cjson.decode, '{"obj": {"array": [1,2,3,4]}}')
assert(ok == false)
-- Invalid numbers
ok, encoded = pcall(cjson.encode, {num1=0/0})
assert(ok == false)
cjson.encode_invalid_numbers(true)
ok, encoded = pcall(cjson.encode, {num1=0/0})
assert(ok == true)
-- Restore defaults
cjson.decode_max_depth(1000)
cjson.encode_max_depth(1000)
cjson.encode_invalid_numbers(false)
} 0
}
test {EVAL - cmsgpack can pack double?} {
r eval {local encoded = cmsgpack.pack(0.1)
local h = ""
@@ -279,6 +339,68 @@ start_server {tags {"scripting"}} {
} 0
} {d3ffffff0000000000}
test {EVAL - cmsgpack pack/unpack smoke test} {
r eval {
local str_lt_32 = string.rep("x", 30)
local str_lt_255 = string.rep("x", 250)
local str_lt_65535 = string.rep("x", 65530)
local str_long = string.rep("x", 100000)
local array_lt_15 = {1, 2, 3, 4, 5}
local array_lt_65535 = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18}
local array_big = {}
for i=1, 100000 do
array_big[i] = i
end
local map_lt_15 = {a=1, b=2}
local map_big = {}
for i=1, 100000 do
map_big[tostring(i)] = i
end
local some_map = {
s1=str_lt_32,
s2=str_lt_255,
s3=str_lt_65535,
s4=str_long,
d1=0.1,
i1=1,
i2=250,
i3=65530,
i4=100000,
i5=2^40,
i6=-1,
i7=-120,
i8=-32000,
i9=-100000,
i10=-3147483648,
a1=array_lt_15,
a2=array_lt_65535,
a3=array_big,
m1=map_lt_15,
m2=map_big,
b1=false,
b2=true,
n=nil
}
local encoded = cmsgpack.pack(some_map)
local decoded = cmsgpack.unpack(encoded)
assert(table.concat(some_map) == table.concat(decoded))
local offset, decoded_one = cmsgpack.unpack_one(encoded, 0)
assert(table.concat(some_map) == table.concat(decoded_one))
assert(offset == -1)
local encoded_multiple = cmsgpack.pack(str_lt_32, str_lt_255, str_lt_65535, str_long)
local offset, obj = cmsgpack.unpack_limit(encoded_multiple, 1, 0)
assert(obj == str_lt_32)
offset, obj = cmsgpack.unpack_limit(encoded_multiple, 1, offset)
assert(obj == str_lt_255)
offset, obj = cmsgpack.unpack_limit(encoded_multiple, 1, offset)
assert(obj == str_lt_65535)
offset, obj = cmsgpack.unpack_limit(encoded_multiple, 1, offset)
assert(obj == str_long)
assert(offset == -1)
} 0
}
test {EVAL - cmsgpack can pack and unpack circular references?} {
r eval {local a = {x=nil,y=5}
local b = {x=a}
@@ -449,6 +571,7 @@ start_server {tags {"scripting"}} {
}
test {EVAL does not leak in the Lua stack} {
r script flush ;# reset Lua VM
r set x 0
# Use a non blocking client to speedup the loop.
set rd [redis_deferring_client]
+129
View File
@@ -708,6 +708,135 @@ start_server {tags {"tracking network"}} {
assert_equal {} $prefixes
}
test {Regression test for #11715} {
# This issue manifests when a client invalidates keys through the max key
# limit, which invalidates keys to get Redis below the limit, but no command is
# then executed. This can occur in several ways but the simplest is through
# multi-exec which queues commands.
clean_all
r config set tracking-table-max-keys 2
# The cron will invalidate keys if we're above the limit, so disable it.
r debug pause-cron 1
# Set up a client that has listened to 2 keys and start a multi, this
# sets up the crash for later.
$rd HELLO 3
$rd read
$rd CLIENT TRACKING on
assert_match "OK" [$rd read]
$rd mget "1{tag}" "2{tag}"
assert_match "{} {}" [$rd read]
$rd multi
assert_match "OK" [$rd read]
# Reduce the tracking table keys to 1, this doesn't immediately take affect, but
# instead will apply on the next command.
r config set tracking-table-max-keys 1
# This command will get queued, so make sure this command doesn't crash.
$rd ping
$rd exec
# Validate we got some invalidation message and then the command was queued.
assert_match "invalidate *{tag}" [$rd read]
assert_match "QUEUED" [$rd read]
assert_match "PONG" [$rd read]
r debug pause-cron 0
} {OK} {needs:debug}
foreach resp {3 2} {
test "RESP$resp based basic invalidation with client reply off" {
# This entire test is mostly irrelevant for RESP2, but we run it anyway just for some extra coverage.
clean_all
$rd hello $resp
$rd read
$rd client tracking on
$rd read
$rd_sg set foo bar
$rd get foo
$rd read
$rd client reply off
$rd_sg set foo bar2
if {$resp == 3} {
assert_equal {invalidate foo} [$rd read]
} elseif {$resp == 2} { } ;# Just coverage
# Verify things didn't get messed up and no unexpected reply was pushed to the client.
$rd client reply on
assert_equal {OK} [$rd read]
$rd ping
assert_equal {PONG} [$rd read]
}
}
test {RESP3 based basic redirect invalidation with client reply off} {
clean_all
set rd_redir [redis_deferring_client]
$rd_redir hello 3
$rd_redir read
$rd_redir client id
set rd_redir_id [$rd_redir read]
$rd client tracking on redirect $rd_redir_id
$rd read
$rd_sg set foo bar
$rd get foo
$rd read
$rd_redir client reply off
$rd_sg set foo bar2
assert_equal {invalidate foo} [$rd_redir read]
# Verify things didn't get messed up and no unexpected reply was pushed to the client.
$rd_redir client reply on
assert_equal {OK} [$rd_redir read]
$rd_redir ping
assert_equal {PONG} [$rd_redir read]
$rd_redir close
}
test {RESP3 based basic tracking-redir-broken with client reply off} {
clean_all
$rd hello 3
$rd read
$rd client tracking on redirect $redir_id
$rd read
$rd_sg set foo bar
$rd get foo
$rd read
$rd client reply off
$rd_redirection quit
$rd_redirection read
$rd_sg set foo bar2
set res [lsearch -exact [$rd read] "tracking-redir-broken"]
assert_morethan_equal $res 0
# Verify things didn't get messed up and no unexpected reply was pushed to the client.
$rd client reply on
assert_equal {OK} [$rd read]
$rd ping
assert_equal {PONG} [$rd read]
}
$rd_redirection close
$rd_sg close
$rd close
}
+9
View File
@@ -71,6 +71,8 @@ start_server {tags {"hash"}} {
test "HRANDFIELD count overflow" {
r hmset myhash a 1
assert_error {*value is out of range*} {r hrandfield myhash -9223372036854770000 withvalues}
assert_error {*value is out of range*} {r hrandfield myhash -9223372036854775808 withvalues}
assert_error {*value is out of range*} {r hrandfield myhash -9223372036854775808}
} {}
test "HRANDFIELD with <count> against non existing key" {
@@ -799,4 +801,11 @@ start_server {tags {"hash"}} {
set _ $k
} {ZIP_INT_8B 127 ZIP_INT_16B 32767 ZIP_INT_32B 2147483647 ZIP_INT_64B 9223372036854775808 ZIP_INT_IMM_MIN 0 ZIP_INT_IMM_MAX 12}
# On some platforms strtold("+inf") with valgrind returns a non-inf result
if {!$::valgrind} {
test {HINCRBYFLOAT does not allow NaN or Infinity} {
assert_error "*value is NaN or Infinity*" {r hincrbyfloat hfoo field +inf}
assert_equal 0 [r exists hfoo]
}
}
}
+84
View File
@@ -588,6 +588,11 @@ start_server {
r srandmember nonexisting_key 100
} {}
test "SRANDMEMBER count overflow" {
r sadd myset a
assert_error {*value is out of range*} {r srandmember myset -9223372036854775808}
} {}
# Make sure we can distinguish between an empty array and a null response
r readraw 1
@@ -748,6 +753,85 @@ start_server {
}
}
proc is_rehashing {myset} {
set htstats [r debug HTSTATS-KEY $myset]
return [string match {*rehashing target*} $htstats]
}
proc rem_hash_set_top_N {myset n} {
set cursor 0
set members {}
set enough 0
while 1 {
set res [r sscan $myset $cursor]
set cursor [lindex $res 0]
set k [lindex $res 1]
foreach m $k {
lappend members $m
if {[llength $members] >= $n} {
set enough 1
break
}
}
if {$enough || $cursor == 0} {
break
}
}
r srem $myset {*}$members
}
test "SRANDMEMBER with a dict containing long chain" {
set origin_save [config_get_set save ""]
set origin_max_is [config_get_set set-max-intset-entries 0]
set origin_save_delay [config_get_set rdb-key-save-delay 2147483647]
# 1) Create a hash set with 100000 members.
set members {}
for {set i 0} {$i < 100000} {incr i} {
lappend members [format "m:%d" $i]
}
create_set myset $members
# 2) Wait for the hash set rehashing to finish.
while {[is_rehashing myset]} {
r srandmember myset 100
}
# 3) Turn off the rehashing of this set, and remove the members to 500.
r bgsave
rem_hash_set_top_N myset [expr {[r scard myset] - 500}]
assert_equal [r scard myset] 500
# 4) Kill RDB child process to restart rehashing.
set pid1 [get_child_pid 0]
catch {exec kill -9 $pid1}
waitForBgsave r
# 5) Let the set hash to start rehashing
r spop myset 1
assert [is_rehashing myset]
# 6) Verify that when rdb saving is in progress, rehashing will still be performed (because
# the ratio is extreme) by waiting for it to finish during an active bgsave.
r bgsave
while {[is_rehashing myset]} {
r srandmember myset 1
}
if {$::verbose} {
puts [r debug HTSTATS-KEY myset]
}
set pid1 [get_child_pid 0]
catch {exec kill -9 $pid1}
waitForBgsave r
r config set save $origin_save
r config set set-max-intset-entries $origin_max_is
r config set rdb-key-save-delay $origin_save_delay
r save
} {OK}
proc setup_move {} {
r del myset3 myset4
create_set myset1 {1 a b}
+11
View File
@@ -457,6 +457,17 @@ start_server {tags {"string"}} {
assert_equal [string range $bin $_start $_end] [r getrange bin $start $end]
}
}
if {[string match {*jemalloc*} [s mem_allocator]]} {
test {trim on SET with big value} {
# set a big value to trigger increasing the query buf
r set key [string repeat A 100000]
# set a smaller value but > PROTO_MBULK_BIG_ARG (32*1024) Redis will try to save the query buf itself on the DB.
r set key [string repeat A 33000]
# asset the value was trimmed
assert {[r memory usage key] < 42000}; # 42K to count for Jemalloc's additional memory overhead.
}
} ;# if jemalloc
test {Extended SET can detect syntax errors} {
set e {}
+2
View File
@@ -1738,6 +1738,8 @@ start_server {tags {"zset"}} {
test "ZRANDMEMBER count overflow" {
r zadd myzset 0 a
assert_error {*value is out of range*} {r zrandmember myzset -9223372036854770000 withscores}
assert_error {*value is out of range*} {r zrandmember myzset -9223372036854775808 withscores}
assert_error {*value is out of range*} {r zrandmember myzset -9223372036854775808}
} {}
# Make sure we can distinguish between an empty array and a null response