| Age | Commit message (Collapse) | Author |
|---|
|
We use split generations to detect when readers may be looking at
structures that are replaced by a split. For correctness, we should
only increment the global split generation *after* a split becomes
public. Only then can we safely check that no thread is still reading
with the old generation.
Previously, a split could increment the global split generation, then a
thread could start reading with the new split generation but see the old
index structure.
This issue was introduced by WT 3088, where we wanted a way to ensure
that newly-allocated pages don't split until it is safe. That is solved
here by having the split code pin a split generation in the ordinary way
(without allocating a new one) for the duration that splits of new
pages need to be prevented.
(cherry picked from commit 51d22616094e0a0d34997d26aec925adf949fbdf)
|
|
* Recheck for existence after acquiring write lock when creating a new dhandle.
* Add a wtperf workload that reproduced the original failure.
(cherry picked from commit b77f9cc3b7fe7c15445c13df9bef74f1dd39b991)
|
|
|
|
proof. (#3309)
src/evict/evict_lru.c:1683:2: error: default label in switch which covers all enumeration values [-Werror,-Wcovered-switch-default]
|
|
Choosing a random point isn't very efficient in append only workloads.
|
|
A NULL page could be encountered when traversing a tree that is being used by exclusive access.
|
|
* WT-3187 Ramp up aggressiveness in reducing cache pool usage
We could get into situations where no participants looked like
good candidates.
Also put a failsafe into the balance loop, to ensure future failures
to reduce usage won't lead to hang on shutdown.
* KNF and wordsmithing.
|
|
|
|
|
|
(#3303)
In this case, we must use the entire (raw) key to duplicate the position, instead of truncating to the visible part.
|
|
|
|
Eviction walks don't need to start on leaf pages: just try to descend
through the tree and as soon as we can't swap to a child page, start
the walk from the parent.
|
|
Replaced a fprintf call, and cleaned up a call to access system call.
|
|
hold the fs lock while manipulating the list of file handles.
|
|
rather than the subtest. Disable core files for the subtest, as they
are rarely interesting. Fix some uint64 values/parameters that were
declared as uint32.
|
|
(#3296)
FreeBSD's backtrace is the outlier, everybody else (OS X, Solaris, Linux)
is using int types, not size_t.
|
|
(#3295)
* WT-2909 Create automatable test verifying checkpoint integrity after errors
Make gcc 4.7 work again.
* Linux (Red Hat 5.3.1-6) declares backtrace(3) to return an int, FreeBSD
(10.3-RELEASE-p11) declares it to return a size_t.
* Remove repeated #include files, check for error returns from a few
functions.
* The Linux/FreeBSD backtrace() calls are fundamentally incompatible,
add an #ifdef.
|
|
integrity (#3272)
Implement a custom file system, and use it via a test case to add validate checkpoint integrity in the face of file-system level errors.
|
|
Only choose a random point when there is no saved walk point.
Fixes to random search as well - noticed search termination conditions when sampling the search page vs. walking it sequentially weren't the same. Changed that, which caused the
test_compact02 test to fail.
There's an underlying bug in this code, if we return WT_NOTFOUND, we can
lose a hazard pointer on the page of the tree we unsucessfully searched.
Add a page-release in the case of returning not-found.
|
|
page (#3289)
* If random descent through the tree fails, fallback to skipping through the tree's pages; if skipping through the tree's pages fails, fallback to a random entry from the first page in the tree that contains anything at all.
* Add tests that create a tree with enough data for multiple pages, reopens the connection so we have a real tree, then truncates most / all of the tree and makes sure random lookups find data / fail (respectively). That way we're testing WT_REF_DELETED, not just empty pages.
* Fix a documentation error, we never implemented a next_random_sample_percent configuration.
|
|
|
|
Reverts part of 7f5d0f9981214c723f2ed90cf4533887ed406176.
Fixes a deadlock in diagnostic mode. Also revert a change that could
cause diagnostic code to read pages into cache: we don't want diagnostic
adding cache pressure and we already verify pages as they are evicted.
|
|
* WT-3174 Coverity/lint cleanup
clang38 complaints:
wt3135_search_near_collator/main.c:75:22: error: implicit conversion loses
integer precision: 'int64_t' (aka 'long') to 'int'
[-Werror,-Wshorten-64-to-32]
*cmp = compare_int(pkey1, pkey2);
~~~~~~~~~~~ ^~~~~
wt3135_search_near_collator/main.c:75:29: error: implicit conversion loses
integer precision: 'int64_t' (aka 'long') to 'int'
[-Werror,-Wshorten-64-to-32]
*cmp = compare_int(pkey1, pkey2);
~~~~~~~~~~~ ^~~~~
* Coverity complains in __split_root():
dead_error_condition: The switch value complete cannot be WT_ERR_PANIC.
CID 1371132 (#1 of 1): Logically dead code (DEADCODE)
dead_error_begin: Execution cannot reach this statement: case WT_ERR_PANIC:.
Revert a minor part of 7f5d0f9, don't switch to benign error mode
(setting WT_ERR_IGNORE) until after the split has been verified in
DIAGNOSTIC mode. That makes sense and should make Coverity happy.
* Fix type-casting, sizeof()/sizeof() is a size_t.
|
|
variable length. (#3254)
* For checkpoint logging, use a format that ends in 'u' to be compatible
with previously created log files. In previous WT versions, these formats
end in 'U', and a final 'U' does have a prefixed size. Now, a 'U' in any
position has a prefixed size.
|
|
* Change how eviction walk point is saved during walk.
* After 0b9e453, we no longer need to do any non-DIAGNOSTIC work after completing the split (previously, we had changes to make the newly created split pages evictable, but now they are initially given a generation number which will prevent their eviction until it's OK).
Rename __split_ref_step2() to be __split_verify_intl(), and change it to verify all of the internal pages involved in the split.
Previously, we only verified the pages we had to read and update anyway. Now we don't have to update any pages and we're only reading pages in DIAGNOSTIC mode, verify all of them.
Don't release the hazard pointer explicitly, use the more standard __wt_page_release() call (it should make no difference, it's just a bit more consistent).
Rename __split_ref_step1() to be __split_ref_prepare(), there's no longer a step #2.
* We don't need to publish WT_BTREE.evict_ref, or use a barrier: in one we're guaranteed that only the writing thread will check the assertion in the discard code (that we're not discarding the eviction's reference), and in the other case we're doing hazard-pointer coupling, which implies
there is a barrier in the code path before the page can possibly be discarded by any thread.
* Review barriers use in splits. (#3288). In all cases, use the pattern "Update the page index, which includes a barrier to make the split live, switch to benign error mode, then verify the pages involved in the split are correct."
|
|
|
|
WT-3088 bug: WiredTiger can evict the tree's current eviction walk point
|
|
|
|
It is not possible to continue without risking data loss.
|
|
* More aggressive error handling.
* Alternative checkpoint cleanup. (#3281)
|
|
Always print an error message if a WT_SESSION method fails, we don't
know if the WiredTiger library printed out a message or not.
Free memory allocated by util_uri() in some cases where we either didn't
have the necessary free call, or simply returned without freeing memory.
Try and be more consistent with error messages, use the leading call as
the first string, and any arguments to that call as the second.
Replace some of the places we're writing to stderr explicitly with the
utility error handlers.
Initialize the return variable from util_uri() in all cases.
Change error messages that referenced WT_SESSION.open to reference
WT_SESSION.open_cursor.
|
|
|
|
|
|
|
|
|
|
* Enhance wtperf to support periodic table scans
* Implement scans as read_range.
* Use a random cursor to set key in table properly.
* Don't allow insert workload with table specifier.
* Reset the rand cursor so it isn't positioned.
* Make wtperf pre_load_data an option.
|
|
In particular, don't have the eviction server give up all walks each
time it is interrupted, and only wait for requesting threads to make
progress: don't go to sleep.
|
|
It was a spinlock, but most acquirers only need shared access and it can be a contention point in many-table workloads.
Split uses of the handle list lock into small operations. In particular, only hold the handle list lock to get the "next" handle, not for loops over all the handles in the system.
Update statistics around handle list lock and corresponding doc.
|
|
|
|
* Add run-time flags checking to __wt_cond_wait_signal(), and its wrappers (__wt_cond_wait(), __wt_cond_auto_wait_signal() and __wt_cond_auto_wait()) so callers of those functions can configure a check that ensures that if the waiting thread races with a waking thread that's turned off flags so the waiting thread quits, the waiting thread returns immediately.
* Rework the WT_SESSION.transaction_sync code to wait for the entire time it's configured to wait, it will be awoken if the log reaches stability before that.
* Assert we're not waiting longer than a second if not checking the run status.
* Set/Clear WT_CONN_LOG_SERVER_RUN in __wt_logmgr_open/__wt_logmgr_destroy rather than in the connection open code. (It's the only server-run flag that gets set in the connection-open code, and I can't see any reason for that exception.)
|
|
|
|
Lint:
Don't print int32_t's with %d.
WT_LOGSLOT.slot_error is an int, not an int32_t.
Don't print off_t's as 32-bits, use the maximum size unsigned object.
|
|
|
|
|
|
|
|
extension (#3261)
This commit represents fixes for Coverity errors, LeakSanitizer errors, and additional cleanup:
* pread/pwrite return value is -1 on error, but the error is in errno.
* Convert size_t and off_t to uintmax_t/PRIuMAX, not uint64_t/PRIu64.
* Coverity ID 1369085 (#1 of 1): Extra sizeof expression (SIZEOF_MISMATCH)
suspicious_pointer_arithmetic: Adding allocated * 8UL /* sizeof (char
*) */ to pointer entries of type char ** is suspicious because adding
an integral value to this pointer automatically scales that value by the
size, 8 bytes, of the pointed-to type, char *. Most likely, the
multiplication by sizeof (char *) in this expression is extraneous and
should be eliminated.
* CID 1369084 (#1 of 1): Resource leak (RESOURCE_LEAK) 9. overwrite_var:
Overwriting handle ret in ret = 12 leaks the handle.
* CID 1369083 (#1 of 1): Logically dead code (DEADCODE) dead_error_line: Execution cannot reach this statement: while (count > 0U) null:
At condition entries != NULL, the value of entries must be NULL. dead_error_condition: The condition entries != NULL cannot be true.
* Custom filesystems have to configure early-load, otherwise we'll have already configured a default filesystem by the time the extension is loaded.
* Add early-load configuration to the wt3120_filesys test.
* Add code to WiredTiger that fails if a custom filesystem is configured after we've already configured a default filesystem.
|
|
page.
clang 3.8 complains descent might be left uninitialized in some case. I don't think that's possible, but it's a simple change.
|
|
There may be empty pages in the tree, and they're useless to us when trying to find random samples. If we don't find a non-empty page in "entries" random guesses, take the first non-empty page in the tree. If the search page contains nothing other than empty pages, restart from the root some number of times before giving up.
|
|
(#3258)
|
|
loaded via shared lib (#3239)
Also add fail_fs extension, as well as a simple test for it.
|
