This is only an initial refactoring of the headers, the final goal is to
have hiredis completely independent from the QFork code in order to build
it as a standalone lib that can be used in other projects.
Squashed commit of the following:
commit ba143668b964885ca1b2cf3af927a49bec0f14a4
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jul 7 11:53:46 2015 +0200
[Fix] aeWinQueueAccept wrong return value.
[Change] Variable name acceptsocket changed to acceptfd.
commit 75f6769fd52fe8215bd1a2149f1bd115078ea278
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jul 6 17:26:55 2015 +0200
[Fix] Closing handled before returning.
[Cleanup] Tabs->spaces.
commit 4f22c9be7dee2f34f2cbf5b8a4fd50b87b443937
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jul 6 17:15:20 2015 +0200
[Change] Removed extra space allocated at the end of the mmap file.
[Log] Improved error messages.
commit f306348c243493b192eaeee9aed404ec065bb60c
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Sat Jul 4 12:08:41 2015 +0200
[Fix] RFDMap was not thread safe.
[Cleanup] Removed unused/commented out code, tabs->spaces.
commit 8da060bf484898932f2ecd02c099266eb31d4d80
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Sat Jul 4 12:02:58 2015 +0200
[Fix] UnhandledExceptiontHandler internal exception handling.
commit 99e5e26fba4b48531cd2a544d5a22d6caba0f681
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Thu Jul 2 08:15:11 2015 +0200
[Cleanup] Removed unused include.
commit d812c4ed57cdc30cd817ad2e9181f5bf239dc07b
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jul 1 20:36:13 2015 +0200
[Cleanup] Removed useless platform definition from RedisServer.sln
commit 0401c167120876ca5533beb0bc6054d6401e8b65
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jul 1 20:32:51 2015 +0200
[Cleanup] Changed variable type to match function return value.
checkForSentinelMode() returns an int, not a bool.
commit 6a9ccd3906d5c007d4683d9caea64da78e5ed668
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jul 1 20:27:21 2015 +0200
[Cleanup] Minor change to method signature.
The method input parameter has been changed to a const.
commit e60f617b543edff46ceebb9362f01619e2188594
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jul 1 20:16:00 2015 +0200
[Cleanup] Removed non-existent file from hiredis project.
commit 8d17c3dd052083ec4e47176921337822299167b1
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jul 1 19:58:59 2015 +0200
[WinPort] Explicit cast.
commit 426841dc8c454a2afbc44c8f005cb3bf31b20cd5
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jul 1 17:07:22 2015 +0200
[Change] Move ReleasePackagingTool to its own solution.
[Cleanup] Removed unused code in ReleasePackagingTool.
[Changed] Chocolatey package now imports documents from the same folder where binaries are.
commit 6ed0696c7902d6dc268fd34f55bce95783bc4286
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jul 1 13:58:58 2015 +0200
[Fix] Ignore SymInitialize() return value to make sure Stack Trace is logged
when redis-server is running as a Windows service.
commit f5e18675c9046492e7dd5a2593c2f890627a093d
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jun 30 19:06:59 2015 +0200
[Change] EventLog.dll excluded from NuGet package.
commit 1ae80857b6fe1b5f352d247376c8fbbd94e81dfc
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 29 16:02:25 2015 +0200
Updated scripts to create and push the NuGet and Chocolatey packages.
- [Change] the binaries are now downloaded from the github release.
- [Change] Redis.nuspec: don't copy the files from the "documentation" folder since the documentation is already included with the binaries.
- [Fix] PullBinaries.ps1: replaced hardcoded Redis version number with input parameter.
- [Fix] improved error handling.
commit c8e70630693a4f505177623824dff998602e62b4
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 29 12:21:58 2015 +0200
Fork code for background processing needed fixes and code refactoring.
- [Fix] BeginForkOperation_Aof()/_Rdb()/_Socket() and BeginForkOperation() code
refactoring.
- [Fix] rewriteAppendOnlyFileBackground() code refactoring to minimize the code
changes for WIN32.
- [Fix] rewriteAppendOnlyFileBackground() must update the latency monitor, the
fork stats and replicationScriptCacheFlush().
- [Fix] rdbSaveBackground() code refactoring to minimize the code changes for
WIN32.
- [Fix] rdbSaveBackground() must update the latency monitor and the fork stats.
- [Fix] memory leak in rdbSaveToSlavesSockets().
- [Fix] properly releasing resources in rdbSaveToSlavesSockets().
- [Fix] QForkChildInit() not setting the operationFailed event in case of
exception.
- [Fix] QForkChildInit() AV in catch() statement.
commit f1007fcaad5ceb17d27fcca081b8dbcf8906ee92
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Fri Jun 26 11:54:09 2015 +0200
[Fix] deleting char** correctly (@zeliard)
commit b8009c220a
Author: Alexis Campailla <alexis@janeasystems.com>
Date: Wed Jun 24 23:10:27 2015 +0200
Fix 32-bit build
commit 4f384b9e58
Author: Alexis Campailla <alexis@janeasystems.com>
Date: Wed Jun 24 21:36:57 2015 +0200
Update appveyor version
commit b84d390111
Author: Alexis Campailla <alexis@janeasystems.com>
Date: Wed Jun 24 20:53:17 2015 +0200
Fix child process handle closing
commit fdbf5328c2
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jun 24 17:43:54 2015 +0200
MapViewOfFileEx() error handling, UnhandledExceptiontHandler fixes.
[log] Improved error message if MapViewOfFileEx() fails.
[fix] Child process call exist(1) if MapViewOfFileEx() fails.
[new] GetForkOperationStatus() now detects if the child process is not anymore running.
[fix] Error reporting in UnhandledExceptiontHandler() for "UNKNOWN EXCEPTION".
[new] UnhandledExceptiontHandler() now calls the default C++ unhandled exceptiont handler.
[log] Changed the closing statement of the BUG REPORT.
[comment] Removed commented out code in Win32_dlmalloc.c.
commit 05b20f0e4b
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jun 24 13:05:53 2015 +0200
Change BUG REPORT closing message to point to a different git hub page.
commit 1cb3e84814
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jun 24 11:59:45 2015 +0200
ReleasePackagingTool changes:
[change] the release notes file is not anymore generated from a template
[change] the .md files are not anymore generated from the .docx files
[new] added redis-server.pdb symbols to the zip package file
[fix] create the release package directory if it doesn't exist
commit 8ea708f9e1
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jun 24 01:21:27 2015 +0200
New release notes format (similar to Redis on UNIX)
commit e2df601222
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jun 24 01:16:02 2015 +0200
Fixed broken link in README.
commit 93d00065e4
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jun 24 01:12:49 2015 +0200
Minor update to the README.
commit bd2ee4d2da
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jun 23 23:40:50 2015 +0200
Warning message in config files for the maxmemory flag.
commit 2943ec7db6
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jun 23 21:56:31 2015 +0200
Cleaned up the UnhandledExceptiontHandler() code.
Minor changes.
commit b805f2be89
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jun 23 19:02:03 2015 +0200
Calling abort() in redisOutOfMemoryHandler().
Changed out-of-memory error report.
commit 18a653318d
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jun 23 18:55:53 2015 +0200
Commented out printf calls in dlmalloc.
commit f01052fb5d
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jun 23 18:44:32 2015 +0200
Changed #if defined(_WIN64) to #ifdef _WIN64
It was causing an appveyor build failure.
commit a4865f06a9
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jun 23 18:15:20 2015 +0200
'save' config flag added to the 'incompatibleNoPersistenceCommands' list.
The fix prevents redis-server from crashing if the 'save' flag is set and
'persistence-available' is set to 'no'.
commit 3de4f6cb47
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jun 23 16:54:23 2015 +0200
Change misleading variables/functions names: master->parent, slave->child.
Master and Slave labels were using for the Parent and Child processes.
commit 453b582eed
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jun 23 13:46:45 2015 +0200
Updated authors and description for nuget and chocolatey.
commit e91c9eeef7
Merge: 5c05765dc3285a
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jun 23 13:40:41 2015 +0200
Merge branch '2.8-enrico' of https://github.com/janeasystems/redis-private into 2.8-enrico
commit dc3285af0b
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jun 23 11:42:51 2015 +0200
Update Redis on Windows Release Notes.md
commit 5c05765d9a
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 22 21:23:08 2015 +0200
Typos in warning message.
commit 510cc9bd3e
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 22 19:57:52 2015 +0200
Replaced AddVectoredExceptionHandler with SetUnhandledExceptionFilter
for unhandled exception handling.
Added log warning if the maxmemory flag is not set.
commit 4993487b25
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 22 17:33:41 2015 +0200
Fixed links in README.
commit 710d179f91
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 22 17:25:21 2015 +0200
Added text to links.
commit 04c1ba4c56
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 22 17:15:48 2015 +0200
Updated documentation.
commit 019d317a10
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 22 13:58:40 2015 +0200
Use MEM_TOP_DOWN flag to allocate memory for memtest.
commit 3b4bd2f84e
Author: NickMRamirez <nramirez@hmbnet.com>
Date: Wed Jun 10 15:12:07 2015 -0400
Added logic to prevent firewall exception from being added if the Windows Firewall Windows service is stopped.
(cherry picked from commit 12383540c65236b2657f74178d6c2e4ec0c4d15a)
Conflicts:
msvs/msi/RedisMsi/RedisMsi.wixproj
commit 3935ffb302
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 22 12:35:10 2015 +0200
RejoinCOWPages and background threads should be synchronized (part 2)
commit 87e43c5ce6
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 22 11:23:04 2015 +0200
Changed error message.
commit af7ca8ca28
Author: Alexis Campailla <alexis@janeasystems.com>
Date: Fri Jun 19 21:22:48 2015 +0200
RejoinCOWPages and background threads should be synchronized
RejoinCOWPages is copying dirty pages to a new view
of the memory map.
If another thread modifies the heap between when
RejoincCOWPages copies the data and when the view is
remapped, the modification will be lost, leading to a
memory corruption.
In short, when RejoincCOWPages is running, all other
threads must be stopped.
Fixes: https://github.com/MSOpenTech/redis/issues/244
commit 3cd7d92bf3
Author: Alexis Campailla <alexis@janeasystems.com>
Date: Fri Jun 19 19:31:22 2015 +0200
Build ReleasePackagingTool in Debug for Debug solution configuration
commit 4742811f48
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Fri Jun 19 12:05:32 2015 +0200
Comment wording.
commit 2c8c937325
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Fri Jun 19 00:24:32 2015 +0200
Cleaned up a few #ifdef _WIN32.
commit 2772b6388e
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Fri Jun 19 00:22:32 2015 +0200
Indentation.
commit f8cc98586e
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Fri Jun 19 00:20:19 2015 +0200
StackTrace and info report.
commit d6ede8b354
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Thu Jun 11 17:03:08 2015 +0200
Moved RedisLog.* to Win32_Interop and renamed to Win32_RedisLog.*
Deleted unused file msvs/RedisLog.h
commit 13f36c88e0
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Thu Jun 11 16:05:41 2015 +0200
Added symbols to MSI.
Also updated the release number to 2.8.21.
commit c68970e268
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Thu Jun 11 15:30:04 2015 +0200
Enabled dlmalloc DEBUG mode in the debug build.
commit 1cf1db97ae
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jun 10 23:08:53 2015 +0200
Final portability fixes for 'long' type.
commit 365c4c7559
Merge: fb63c21fbb9d61
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jun 10 12:32:11 2015 +0200
Merge version 2.8.21 from antirez into 2.8
Changes to be committed:
modified: 00-RELEASENOTES
modified: deps/lua/src/ldo.c
modified: deps/lua/src/lua_cmsgpack.c
modified: src/scripting.c
modified: src/version.h
commit fb63c21f81
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jun 10 12:15:57 2015 +0200
Comment.
added a note in case 32bit support will be brought back.
commit bf260ec746
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Wed Jun 10 00:43:46 2015 +0200
Test portability fix.
the 'redis-sentinel' binary is not currently built on Windows, the quick
workaround is to call 'redis-server --sentinel'
commit 2975545e6a
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Tue Jun 9 17:21:56 2015 +0200
Restored support for the command argument --test-memory
along with -h, --help, -v, --version
commit 64c14409d9
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 8 20:40:08 2015 +0200
Added back the RealeasePackaingTool project in the RedisServer.sln build
It was removed by mistake in a previous commit.
commit eaf29cc8aa
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 8 20:35:31 2015 +0200
Truncate AOF in case of unexpected EOF.
The Win32 port was not truncating the AOF file.
commit 91d01bf8fb
Author: Enrico Giordani <enrico.giordani@gmail.com>
Date: Mon Jun 8 16:23:53 2015 +0200
Win32 portability for tests: fix the syntax to invoke 'tail'.
tail -nX file -> tail -X file
commit fbb9d619f7
Author: antirez <antirez@gmail.com>
Date: Thu Jun 4 12:00:21 2015 +0200
Fix 2.8.21 release notes to give full credits.
commit 7f8b865a65
Author: antirez <antirez@gmail.com>
Date: Thu Jun 4 11:32:24 2015 +0200
Redis 2.8.21
commit 700b863f13
Author: Ben Murphy <benmmurphy@gmail.com>
Date: Mon May 11 23:24:37 2015 +0100
hide access to debug table
commit 5a1b22ad7f
Author: Ben Murphy <benmmurphy@gmail.com>
Date: Mon May 11 23:24:24 2015 +0100
disable loading lua bytecode
commit 1eeb9bd714
Author: antirez <antirez@gmail.com>
Date: Wed Jun 3 08:44:43 2015 +0200
Scripting: Lua cmsgpack lib updated to include str8 support
Conflicts:
00-RELEASENOTES
src/version.h
tests/instances.tcl
1. Server unxtime may remain not updated while loading AOF, so ETA is
not updated correctly.
2. Number of processed byte was not initialized.
3. Possible division by zero condition (likely cause of issue #1932).
We need to avoid that a child -> slaves transfer can continue forever.
We use the same timeout used as global replication timeout, which is
documented to also affect I/O operations during bulk transfers.
To perform a socket write() for each RDB rio API write call was
extremely unefficient, so now rio has minimal buffering capabilities.
Writes are accumulated into a buffer and only when a given limit is
reacehd are actually wrote to the N slaves FDs.
Trivia: rio lacked support for buffering since our targets were:
1) Memory buffers.
2) C standard I/O.
Both were buffered already.
We need to remember what is the saving strategy of the current RDB child
process, since the configuration may be modified at runtime via CONFIG
SET and still we'll need to understand, when the child exists, what to
do and for what goal the process was initiated: to create an RDB file
on disk or to write stuff directly to slave's sockets.
When we are blocked and a few events a processed from time to time, it
is smarter to call the event handler a few times in order to handle the
accept, read, write, close cycle of a client in a single pass, otherwise
there is too much latency added for clients to receive a reply while the
server is busy in some way (for example during the DB loading).
Previously, the (!fp) would only catch lack of free space
under OS X. Linux waits to discover it can't write until
it actually writes contents to disk.
(fwrite() returns success even if the underlying file
has no free space to write into. All the errors
only show up at flush/sync/close time.)
Fixesantirez/redis#1604
server.unixtime and server.mstime are cached less precise timestamps
that we use every time we don't need an accurate time representation and
a syscall would be too slow for the number of calls we require.
Such an example is the initialization and update process of the last
interaction time with the client, that is used for timeouts.
However rdbLoad() can take some time to load the DB, but at the same
time it did not updated the time during DB loading. This resulted in the
bug described in issue #1535, where in the replication process the slave
loads the DB, creates the redisClient representation of its master, but
the timestamp is so old that the master, under certain conditions, is
sensed as already "timed out".
Thanks to @yoav-steinberg and Redis Labs Inc for the bug report and
analysis.
The previous fix for false positive timeout detected by master was not
complete. There is another blocking stage while loading data for the
first synchronization with the master, that is, flushing away the
current data from the DB memory.
This commit uses the newly introduced dict.c callback in order to make
some incremental work (to send "\n" heartbeats to the master) while
flushing the old data from memory.
It is hard to write a regression test for this issue unfortunately. More
support for debugging in the Redis core would be needed in terms of
functionalities to simulate a slow DB loading / deletion.
Starting with Redis 2.8 masters are able to detect timed out slaves,
while before 2.8 only slaves were able to detect a timed out master.
Now that timeout detection is bi-directional the following problem
happens as described "in the field" by issue #1449:
1) Master and slave setup with big dataset.
2) Slave performs the first synchronization, or a full sync
after a failed partial resync.
3) Master sends the RDB payload to the slave.
4) Slave loads this payload.
5) Master detects the slave as timed out since does not receive back the
REPLCONF ACK acknowledges.
Here the problem is that the master has no way to know how much the
slave will take to load the RDB file in memory. The obvious solution is
to use a greater replication timeout setting, but this is a shame since
for the 0.1% of operation time we are forced to use a timeout that is
not what is suited for 99.9% of operation time.
This commit tries to fix this problem with a solution that is a bit of
an hack, but that modifies little of the replication internals, in order
to be back ported to 2.8 safely.
During the RDB loading time, we send the master newlines to avoid
being sensed as timed out. This is the same that the master already does
while saving the RDB file to still signal its presence to the slave.
The single newline is used because:
1) It can't desync the protocol, as it is only transmitted all or
nothing.
2) It can be safely sent while we don't have a client structure for the
master or in similar situations just with write(2).
Previously two string encodings were used for string objects:
1) REDIS_ENCODING_RAW: a string object with obj->ptr pointing to an sds
stirng.
2) REDIS_ENCODING_INT: a string object where the obj->ptr void pointer
is casted to a long.
This commit introduces a experimental new encoding called
REDIS_ENCODING_EMBSTR that implements an object represented by an sds
string that is not modifiable but allocated in the same memory chunk as
the robj structure itself.
The chunk looks like the following:
+--------------+-----------+------------+--------+----+
| robj data... | robj->ptr | sds header | string | \0 |
+--------------+-----+-----+------------+--------+----+
| ^
+-----------------------+
The robj->ptr points to the contiguous sds string data, so the object
can be manipulated with the same functions used to manipulate plan
string objects, however we need just on malloc and one free in order to
allocate or release this kind of objects. Moreover it has better cache
locality.
This new allocation strategy should benefit both the memory usage and
the performances. A performance gain between 60 and 70% was observed
during micro-benchmarks, however there is more work to do to evaluate
the performance impact and the memory usage behavior.
When a BGSAVE fails, Redis used to flood itself trying to BGSAVE at
every next cron call, that is either 10 or 100 times per second
depending on configuration and server version.
This commit does not allow a new automatic BGSAVE attempt to be
performed before a few seconds delay (currently 5).
This avoids both the auto-flood problem and filling the disk with
logs at a serious rate.
The five seconds limit, considering a log entry of 200 bytes, will use
less than 4 MB of disk space per day that is reasonable, the sysadmin
should notice before of catastrofic events especially since by default
Redis will stop serving write queries after the first failed BGSAVE.
This fixes issue #849
Enrico Giordani