diff options
Diffstat (limited to 'src/docs')
| -rw-r--r-- | src/docs/Doxyfile | 2 | ||||
| -rw-r--r-- | src/docs/backup.dox | 44 | ||||
| -rw-r--r-- | src/docs/cursor-join.dox | 25 | ||||
| -rw-r--r-- | src/docs/custom-file-systems.dox | 25 | ||||
| -rw-r--r-- | src/docs/error-handling.dox | 3 | ||||
| -rw-r--r-- | src/docs/examples.dox | 6 | ||||
| -rw-r--r-- | src/docs/in-memory.dox | 12 | ||||
| -rw-r--r-- | src/docs/programming.dox | 2 | ||||
| -rw-r--r-- | src/docs/spell.ok | 5 | ||||
| -rw-r--r-- | src/docs/tune-cache.dox | 4 |
10 files changed, 107 insertions, 21 deletions
diff --git a/src/docs/Doxyfile b/src/docs/Doxyfile index 4c1682de6eb..69e9716b425 100644 --- a/src/docs/Doxyfile +++ b/src/docs/Doxyfile @@ -1570,6 +1570,8 @@ PREDEFINED = DOXYGEN \ __wt_event_handler:=WT_EVENT_HANDLER \ __wt_extension_api:=WT_EXTENSION_API \ __wt_extractor:=WT_EXTRACTOR \ + __wt_file_handle:=WT_FILE_HANDLE \ + __wt_file_system:=WT_FILE_SYSTEM \ __wt_item:=WT_ITEM \ __wt_lsn:=WT_LSN \ __wt_session:=WT_SESSION \ diff --git a/src/docs/backup.dox b/src/docs/backup.dox index 7742e698ac8..45edc85d6a5 100644 --- a/src/docs/backup.dox +++ b/src/docs/backup.dox @@ -42,6 +42,12 @@ Copying the database files for a backup does not require any special alignment or block size (specifically, Linux or Windows filesystems that do not support read/write isolation can be safely read for backups). +The database file may grow in size during the copy, and the file copy +should not consider that an error. Blocks appended to the file after the +copy starts can be safely ignored, that is, it is correct for the copy +to determine an initial size of the file and then copy that many bytes, +ignoring any bytes appended after the backup cursor was opened. + The cursor must not be closed until all of the files have been copied, however, there is no requirement the files be copied in any order or in any relationship to the WT_CURSOR::next calls, only that all files have @@ -98,29 +104,35 @@ and removing log files from the original database home: 1. Perform a full backup of the database (as described above). -2. Perform a full database checkpoint. - -3. Open a cursor on the \c "backup:" data source, with the - \c "target=(\"log:\\")" target specified, which begins the - process of an incremental backup. +2. Open a cursor on the \c "backup:" data source, configured with the + \c "target=(\"log:\\")" target specified, which begins the process + of an incremental backup. -4. Copy each log file returned by the WT_CURSOR::next method to the backup +3. Copy each log file returned by the WT_CURSOR::next method to the backup directory. It is not an error to copy a log file which has been copied before, but care should be taken to ensure each log file is completely copied - as the most recent log file may change in size while being copied. + as the most recent log file may grow in size while being copied. -5. If all log files have been successfully copied, archive the log +4. If all log files have been successfully copied, archive the log files by calling the WT_SESSION::truncate method with the URI - <code>log:</code> and specifying the backup cursor as the - start cursor to that method. + <code>log:</code> and specifying the backup cursor as the start + cursor to that method. (Note there is no requirement backups be + coordinated with database checkpoints, however, an incremental backup + will repeatedly copy the same files, and will not make additional log + files available for archival, unless there was a checkpoint after the + previous incremental backup.) + +5. Close the backup cursor. + +Steps 2-5 can be repeated any number of times before step 1 is repeated. +Full and incremental backups may be repeated as long as the backup +database directory has not been opened and recovery run. Once recovery +has run in a backup directory, you can no longer back up to that +database directory. -6. Close the backup cursor. +An example of opening the backup data source for an incremental backup: -Steps 2-6 can be repeated any number of times before step 1 is -repeated. These steps can be repeated as long as the backup database -directory has not been opened, recovery run and become live. Once -the database becomes live, you must repeat all steps 1-6 to another, -different backup database directory. +@snippet ex_all.c incremental backup @section backup_o_direct Backup and O_DIRECT diff --git a/src/docs/cursor-join.dox b/src/docs/cursor-join.dox index 51da6b174bf..5ea064a250b 100644 --- a/src/docs/cursor-join.dox +++ b/src/docs/cursor-join.dox @@ -14,6 +14,31 @@ Here is an example using join cursors: Joins support various comparison operators: \c "eq", \c "gt", \c "ge", \c "lt", \c "le". Ranges with lower and upper bounds can also be specified, by joining two cursors on the same index, for example, one with \c "compare=ge" and another \c "compare=lt". In addition to joining indices, the main table can be joined so that a range of primary keys can be specified. +By default, a join cursor returns a conjunction, that is, all keys that +satisfy all the joined comparisons. By specifying a configuration with \c +"operation=or", a join cursor will return a disjunction, or all keys that +satisfy at least one of the joined comparisons. More complex joins can be +composed by specifying another join cursor as the reference cursor in a join +call. + +Here is an example using these concepts to show a conjunction of a disjunction: + +@snippet ex_schema.c Complex join cursors + All the joins should be done on the join cursor before WT_CURSOR::next is called. Calling WT_CURSOR::next on a join cursor for the first time populates any bloom filters and performs other initialization. The join cursor's key is the primary key (the key for the main table), and its value is the entire set of values of the main table. A join cursor can be created with a projection by appending \c "(col1,col2,...)" to the URI if a different set of values is needed. +Keys returned from the join cursor are ordered according to the +first reference cursor joined. For example, if an index cursor was joined +first, that index determines the order of results. If the join cursor +uses disjunctions, then the ordering of all joins determines the order. +The first join in a conjunctive join, or all joins in a disjunctive join, +are distinctive in that they are iterated internally as the cursor join +returns values in order. Any bloom filters specified on the +joins that are used for iteration are not useful, and are silently ignored. + +When disjunctions are used where the sets of keys overlap on these 'iteration +joins', a join cursor will return duplicates. A join cursor never returns +duplicates unless \c "operation=or" is used in a join configuration, or unless +the first joined cursor is itself a join cursor that would return duplicates. + */ diff --git a/src/docs/custom-file-systems.dox b/src/docs/custom-file-systems.dox new file mode 100644 index 00000000000..4b012952e15 --- /dev/null +++ b/src/docs/custom-file-systems.dox @@ -0,0 +1,25 @@ +/*! @page custom_file_systems Custom File Systems + +Applications can provide a custom file system implementation that will be +used by WiredTiger to interact with the I/O subsystem using the +WT_FILE_SYSTEM and WT_FILE_HANDLE interfaces. + +It is not necessary for all file system providers to implement all methods +in the WT_FILE_SYSTEM and WT_FILE_HANDLE structures. The documentation for +those structures indicate which methods are optional. Methods which are not +provided should be set to NULL. Generally the function pointers should not +be changed once a handle is created. There is one exception to this, which +are the fallocate and fallocate_nolock - for an example of how fallocate +can be changed after create see the WiredTiger POSIX file system +implementation. + +WT_FILE_SYSTEM and WT_FILE_HANDLE methods which fail but not fatally +(for example, a file truncation call which fails because the file is +currently mapped into memory), should return EBUSY. + +Unless explicitly stated otherwise, WiredTiger may invoke methods on the +WT_FILE_SYSTEM and WT_FILE_HANDLE interfaces from multiple threads +concurrently. It is the responsibility of the implementation to protect +any shared data. + +*/ diff --git a/src/docs/error-handling.dox b/src/docs/error-handling.dox index d91a126ee21..62be498fc15 100644 --- a/src/docs/error-handling.dox +++ b/src/docs/error-handling.dox @@ -52,6 +52,9 @@ This error indicates an underlying problem that requires the application exit an @par <code>WT_RUN_RECOVERY</code> This error is generated when wiredtiger_open is configured to return an error if recovery is required to use the database. +@par <code>WT_CACHE_FULL</code> +This error is only generated when wiredtiger_open is configured to run in-memory, and an insert or update operation requires more than the configured cache size to complete. The operation may be retried; if a transaction is in progress, it should be rolled back and the operation retried in a new transaction. + @if IGNORE_BUILT_BY_API_ERR_END @endif diff --git a/src/docs/examples.dox b/src/docs/examples.dox index 3ed7357b52c..c5a106a00c9 100644 --- a/src/docs/examples.dox +++ b/src/docs/examples.dox @@ -9,9 +9,6 @@ Show how to configure and use asynchronous operations. A more complex schema based on a call center example, showing how to map some SQL constructs onto the WiredTiger API. -@example ex_config.c -Shows how to configure some properties of the database and tables. - @example ex_cursor.c Shows some common cursor types and operations. @@ -55,4 +52,7 @@ Shows how to access the database log files. @example ex_thread.c Shows how to access a database with multiple threads. +@example ex_file_system.c +Shows how to extend WiredTiger with a custom file-system implementation. + */ diff --git a/src/docs/in-memory.dox b/src/docs/in-memory.dox new file mode 100644 index 00000000000..df221dc34d6 --- /dev/null +++ b/src/docs/in-memory.dox @@ -0,0 +1,12 @@ +/*! @m_page{{c,java},in_memory,In-memory databases} + +The ::wiredtiger_open \c in_memory configuration changes WiredTiger to +run in cache without writing to a backing disk. Data is limited to the +configured cache size. + +If \c in_memory is configured, WT_CURSOR::insert and WT_CURSOR::update +methods may return an additional error, ::WT_CACHE_FULL, indicating the +insert or update operation requires more than the configured cache size +to complete. If a transaction is in progress, it should be rolled back. + + */ diff --git a/src/docs/programming.dox b/src/docs/programming.dox index f717f4ed1fe..81e612e8ee8 100644 --- a/src/docs/programming.dox +++ b/src/docs/programming.dox @@ -41,6 +41,7 @@ each of which is ordered by one or more columns. - @subpage compact - @subpage checkpoint - @subpage durability +- @subpage in_memory - @subpage cursor_join - @subpage cursor_log - @ref transaction_named_snapshots @@ -55,6 +56,7 @@ each of which is ordered by one or more columns. - @subpage custom_collators - @subpage custom_extractors - @subpage custom_data_sources +- @subpage custom_file_systems - @subpage helium @m_endif diff --git a/src/docs/spell.ok b/src/docs/spell.ok index efc306568cd..d197b5517f2 100644 --- a/src/docs/spell.ok +++ b/src/docs/spell.ok @@ -25,6 +25,7 @@ EBUSY ECMA EINVAL ENCRYPTOR +ENOTSUP EmpId Encryptors Facebook @@ -178,6 +179,8 @@ desc destructor destructors dev +disjunction +disjunctions distclean dl dll @@ -207,6 +210,7 @@ erlang errno exe fadvise +fallocate failchk fd's fdatasync @@ -331,6 +335,7 @@ nocase nocasecoll nodup noflush +nolock nolocking nommap nop diff --git a/src/docs/tune-cache.dox b/src/docs/tune-cache.dox index c9603085905..505da436277 100644 --- a/src/docs/tune-cache.dox +++ b/src/docs/tune-cache.dox @@ -11,9 +11,9 @@ The cache size for the database is normally configured by setting the function. The cache size can be adjusted after the open call with WT_CONNECTION::reconfigure. -An example of setting a cache size to 500MB: +An example of setting a cache size to 5GB: -@snippet ex_config.c configure cache size +@snippet ex_all.c Open a connection The effectiveness of the chosen cache size can be measured by reviewing the page eviction statistics for the database. |