Package org.h2.mvstore

Class MVStore

  • All Implemented Interfaces:
    java.lang.AutoCloseable
    public final class MVStore
extends java.lang.Object
implements java.lang.AutoCloseable
    A persistent storage for maps.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  MVStore.Builder
      A builder for an MVStore.
      static class  MVStore.TxCounter
      Class TxCounter is a simple data structure to hold version of the store along with the counter of open transactions, which are still operating on this version.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()
      Close the file and the store.
      void close​(int allowedCompactionTime)
      Close the store.
      void closeImmediately()
      Close the file and the store, without writing anything.
      long commit()
      Commit the changes.
      boolean compact​(int targetFillRate, int write)
      Try to increase the fill rate by re-writing partially full chunks.
      static void compact​(java.lang.String fileName, boolean compress, FileStore<?> fileStore)
      Compact database file by copying only live pages.
      static void compact​(java.lang.String sourceFileName, java.lang.String targetFileName, boolean compress, FileStore<?> fileStore)
      Copy all live pages from the source store to the target store.
      void compactFile​(int maxCompactTime)
      Compact store file, that is, compact blocks that have a low fill rate, and move chunks next to each other.
      void countNewPage​(boolean leaf)  
      boolean decrementVersionUsageCounter​(MVStore.TxCounter txCounter)
      De-register (close) completed operation (transaction).
      void deregisterVersionUsage​(MVStore.TxCounter txCounter)
      De-register (close) completed operation (transaction).
      void executeFilestoreOperation​(java.lang.Runnable operation)  
      int getAutoCommitDelay()
      Get the auto-commit delay.
      int getAutoCommitMemory()
      Get the maximum memory (in bytes) used for unsaved pages.
      int getCacheSize()
      Get the maximum cache size, in MB.
      int getCacheSizeUsed()
      Get the amount of memory used for caching, in MB.
      long getCurrentVersion()
      Get the current version of the data.
      FileStore<?> getFileStore()
      Get the file store.
      int getFillRate()  
      int getKeysPerPage()  
      java.util.Map<java.lang.String,​java.lang.String> getLayoutMap()
      Get this store's layout map.
      <K,​V>
      MVMap<K,​V>      		 getMap​(int id)
      Get map by id.
      java.lang.String getMapName​(int id)
      Get the name of the given map.
      java.util.Set<java.lang.String> getMapNames()
      Get the set of all map names.
      long getMaxPageSize()  
      MVMap<java.lang.String,​java.lang.String> getMetaMap()
      Get the metadata map.
      MVStoreException getPanicException()  
      int getRetentionTime()  
      java.util.Map<java.lang.String,​java.lang.Object> getStoreHeader()
      Get the store header.
      int getStoreVersion()
      Get the store version.
      long getTimeAbsolute()  
      int getUnsavedMemory()
      Get the estimated memory (in bytes) of unsaved data.
      int getVersionsToKeep()
      Get the oldest version to retain in memory (for in-memory stores).
      boolean hasData​(java.lang.String name)
      Check whether a given map exists and has data.
      boolean hasMap​(java.lang.String name)
      Check whether a given map exists.
      boolean hasUnsavedChanges()
      Check whether there are any unsaved changes.
      boolean isClosed()
      Determine that store is open, or wait for it to be closed (by other thread)
      boolean isPersistent()
      Indicates whether this MVStore is backed by FileStore, and therefore it's data will survive this store closure (but not necessary process termination in case of in-memory store).
      boolean isReadOnly()
      Whether the store is read-only.
      boolean isSpaceReused()  
      boolean isVersioningRequired()
      Indicates whether store versions are rolling.
      static MVStore open​(java.lang.String fileName)
      Open a store in exclusive mode.
      <M extends MVMap<K,​V>,​K,​V>
      M      		 openMap​(int id, MVMap.MapBuilder<M,​K,​V> builder)
      Open an existing map with the given builder.
      <K,​V>
      MVMap<K,​V>      		 openMap​(java.lang.String name)
      Open a map with the default settings.
      <M extends MVMap<K,​V>,​K,​V>
      M      		 openMap​(java.lang.String name, MVMap.MapBuilder<M,​K,​V> builder)
      Open a map with the given builder.
      MVMap<java.lang.String,​java.lang.String> openMetaMap()  
      void panic​(java.lang.Throwable e)  
      void panic​(MVStoreException exception)  
      void populateInfo​(java.util.function.BiConsumer<java.lang.String,​java.lang.String> consumer)  
      void registerUnsavedMemory​(int memory)
      Adjust amount of "unsaved memory" meaning amount of RAM occupied by pages not saved yet to the file.
      MVStore.TxCounter registerVersionUsage()
      Register opened operation (transaction).
      void removeMap​(java.lang.String name)
      Remove map by name.
      void removeMap​(MVMap<?,​?> map)
      Prepare map removal from the store.
      void renameMap​(MVMap<?,​?> map, java.lang.String newName)
      Rename a map.
      void rollback()
      Revert to the beginning of the current version, reverting all uncommitted changes.
      void rollbackTo​(long version)
      Revert to the beginning of the given version.
      void setAutoCommitDelay​(int millis)
      Set the maximum delay in milliseconds to auto-commit changes.
      void setCacheSize​(int kb)
      Set the maximum memory to be used by the cache.
      void setOldestVersionTracker​(java.util.function.LongConsumer callback)  
      void setRetentionTime​(int ms)
      How long to retain old, persisted chunks, in milliseconds.
      void setReuseSpace​(boolean reuseSpace)
      Whether empty space in the file should be re-used.
      void setStoreVersion​(int version)
      Update the store version.
      void setVersionsToKeep​(int count)
      How many versions to retain for in-memory stores.
      void sync()
      Force all stored changes to be written to the storage.
      long tryCommit()
      Unlike regular commit this method returns immediately if there is commit in progress on another thread, otherwise it acts as regular commit.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • backgroundExceptionHandler

        public final java.lang.Thread.UncaughtExceptionHandler backgroundExceptionHandler

    • Method Detail

      • openMetaMap

        public MVMap<java.lang.String,​java.lang.String> openMetaMap()

      • compact

        public static void compact​(java.lang.String fileName,
                           boolean compress,
                           FileStore<?> fileStore)
        Compact database file by copying only live pages.
        Parameters:
        fileName - to compact
        compress - whether new database file should use compression
        fileStore - open filestore instance based on fileName provided, if database is not encrypted it can be null, and the standard fileStore opening procedure will be used, but for encrypted files that fileStore is used as a factory with the knowledge of encryption key, and it must be supplied. This fileStore will be closed here.

      • compact

        public static void compact​(java.lang.String sourceFileName,
                           java.lang.String targetFileName,
                           boolean compress,
                           FileStore<?> fileStore)
        Copy all live pages from the source store to the target store.
        Parameters:
        sourceFileName - the name of the source store
        targetFileName - the name of the target store
        compress - whether to compress the data
        fileStore - open filestore instance based on sourceFileName provided, if database is not encrypted it can be null, and the standard fileStore opening procedure will be used, but for encrypted files that fileStore is used as a factory with the knowledge of encryption key, and it must be supplied. This fileStore will be closed here.

      • panic

        public void panic​(java.lang.Throwable e)

      • open

        public static MVStore open​(java.lang.String fileName)
        Open a store in exclusive mode. For a file-based store, the parent directory must already exist.
        Parameters:
        fileName - the file name (null for in-memory)
        Returns:
        the store

      • openMap

        public <K,​V> MVMap<K,​V> openMap​(java.lang.String name)
        Open a map with the default settings. The map is automatically create if it does not yet exist. If a map with this name is already open, this map is returned.
        Type Parameters:
        K - the key type
        V - the value type
        Parameters:
        name - the name of the map
        Returns:
        the map

      • openMap

        public <M extends MVMap<K,​V>,​K,​V> M openMap​(java.lang.String name,
                                                              MVMap.MapBuilder<M,​K,​V> builder)
        Open a map with the given builder. The map is automatically create if it does not yet exist. If a map with this name is already open, this map is returned.
        Type Parameters:
        M - the map type
        K - the key type
        V - the value type
        Parameters:
        name - the name of the map
        builder - the map builder
        Returns:
        the map

      • openMap

        public <M extends MVMap<K,​V>,​K,​V> M openMap​(int id,
                                                              MVMap.MapBuilder<M,​K,​V> builder)
        Open an existing map with the given builder.
        Type Parameters:
        M - the map type
        K - the key type
        V - the value type
        Parameters:
        id - the map id
        builder - the map builder
        Returns:
        the map

      • getMap

        public <K,​V> MVMap<K,​V> getMap​(int id)
        Get map by id.
        Type Parameters:
        K - the key type
        V - the value type
        Parameters:
        id - map id
        Returns:
        Map

      • getMapNames

        public java.util.Set<java.lang.String> getMapNames()
        Get the set of all map names.
        Returns:
        the set of names

      • getLayoutMap

        public java.util.Map<java.lang.String,​java.lang.String> getLayoutMap()
        Get this store's layout map. This data is for informational purposes only. The data is subject to change in future versions.

        The data in this map should not be modified (changing system data may corrupt the store).

        The layout map contains the following entries: 

         chunk.{chunkId} = {chunk metadata}
 root.{mapId} = {root position}
        Returns:
        the metadata map

      • getMetaMap

        public MVMap<java.lang.String,​java.lang.String> getMetaMap()
        Get the metadata map. This data is for informational purposes only. The data is subject to change in future versions.

        The data in this map should not be modified (changing system data may corrupt the store).

        The metadata map contains the following entries: 

         name.{name} = {mapId}
 map.{mapId} = {map metadata}
 setting.storeVersion = {version}
        Returns:
        the metadata map

      • hasMap

        public boolean hasMap​(java.lang.String name)
        Check whether a given map exists.
        Parameters:
        name - the map name
        Returns:
        true if it exists

      • hasData

        public boolean hasData​(java.lang.String name)
        Check whether a given map exists and has data.
        Parameters:
        name - the map name
        Returns:
        true if it exists and has data.

      • close

        public void close()
        Close the file and the store. Unsaved changes are written to disk first.
        Specified by:
        close in interface java.lang.AutoCloseable

      • close

        public void close​(int allowedCompactionTime)
        Close the store. Pending changes are persisted. If time is allocated for housekeeping, chunks with a low fill rate are compacted, and some chunks are put next to each other. If time is unlimited then full compaction is performed, which uses different algorithm - opens alternative temp store and writes all live data there, then replaces this store with a new one.
        Parameters:
        allowedCompactionTime - time (in milliseconds) allotted for file compaction activity, 0 means no compaction, -1 means unlimited time (full compaction)

      • closeImmediately

        public void closeImmediately()
        Close the file and the store, without writing anything. This will try to stop the background thread (without waiting for it). This method ignores all errors.

      • isPersistent

        public boolean isPersistent()
        Indicates whether this MVStore is backed by FileStore, and therefore it's data will survive this store closure (but not necessary process termination in case of in-memory store).
        Returns:
        true if persistent

      • tryCommit

        public long tryCommit()
        Unlike regular commit this method returns immediately if there is commit in progress on another thread, otherwise it acts as regular commit.

        This method may return BEFORE this thread changes are actually persisted!

        Returns:
        the new version (incremented if there were changes) or -1 if there were no commit

      • commit

        public long commit()
        Commit the changes.

        This method does nothing if there are no unsaved changes, otherwise it increments the current version and stores the data (for file based stores).

        It is not necessary to call this method when auto-commit is enabled (the default setting), as in this case it is automatically called from time to time or when enough changes have accumulated. However, it may still be called to flush all changes to disk.

        At most one store operation may run at any time.

        Returns:
        the new version (incremented if there were changes) or -1 if there were no commit

      • getTimeAbsolute

        public long getTimeAbsolute()

      • hasUnsavedChanges

        public boolean hasUnsavedChanges()
        Check whether there are any unsaved changes.
        Returns:
        if there are any changes

      • executeFilestoreOperation

        public void executeFilestoreOperation​(java.lang.Runnable operation)

      • sync

        public void sync()
        Force all stored changes to be written to the storage. The default implementation calls FileChannel.force(true).

      • compactFile

        public void compactFile​(int maxCompactTime)
        Compact store file, that is, compact blocks that have a low fill rate, and move chunks next to each other. This will typically shrink the file. Changes are flushed to the file, and old chunks are overwritten.
        Parameters:
        maxCompactTime - the maximum time in milliseconds to compact

      • compact

        public boolean compact​(int targetFillRate,
                       int write)
        Try to increase the fill rate by re-writing partially full chunks. Chunks with a low number of live items are re-written.

        If the current fill rate is higher than the target fill rate, nothing is done.

        Please note this method will not necessarily reduce the file size, as empty chunks are not overwritten.

        Only data of open maps can be moved. For maps that are not open, the old chunk is still referenced. Therefore, it is recommended to open all maps before calling this method.

        Parameters:
        targetFillRate - the minimum percentage of live entries
        write - the minimum number of bytes to write
        Returns:
        if any chunk was re-written

      • getFillRate

        public int getFillRate()

      • getKeysPerPage

        public int getKeysPerPage()

      • getMaxPageSize

        public long getMaxPageSize()

      • getCacheSize

        public int getCacheSize()
        Get the maximum cache size, in MB. Note that this does not include the page chunk references cache, which is 25% of the size of the page cache.
        Returns:
        the cache size

      • getCacheSizeUsed

        public int getCacheSizeUsed()
        Get the amount of memory used for caching, in MB. Note that this does not include the page chunk references cache, which is 25% of the size of the page cache.
        Returns:
        the amount of memory used for caching

      • setCacheSize

        public void setCacheSize​(int kb)
        Set the maximum memory to be used by the cache.
        Parameters:
        kb - the maximum size in KB

      • isSpaceReused

        public boolean isSpaceReused()

      • setReuseSpace

        public void setReuseSpace​(boolean reuseSpace)
        Whether empty space in the file should be re-used. If enabled, old data is overwritten (default). If disabled, writes are appended at the end of the file.

        This setting is specially useful for online backup. To create an online backup, disable this setting, then copy the file (starting at the beginning of the file). In this case, concurrent backup and write operations are possible (obviously the backup process needs to be faster than the write operations).

        Parameters:
        reuseSpace - the new value

      • getRetentionTime

        public int getRetentionTime()

      • setRetentionTime

        public void setRetentionTime​(int ms)
        How long to retain old, persisted chunks, in milliseconds. Chunks that are older may be overwritten once they contain no live data.

        The default value is 45000 (45 seconds) when using the default file store. It is assumed that a file system and hard disk will flush all write buffers within this time. Using a lower value might be dangerous, unless the file system and hard disk flush the buffers earlier. To manually flush the buffers, use MVStore.getFile().force(true), however please note that according to various tests this does not always work as expected depending on the operating system and hardware.

        The retention time needs to be long enough to allow reading old chunks while traversing over the entries of a map.

        This setting is not persisted.

        Parameters:
        ms - how many milliseconds to retain old chunks (0 to overwrite them as early as possible)

      • isVersioningRequired

        public boolean isVersioningRequired()
        Indicates whether store versions are rolling.
        Returns:
        true if versions are rolling, false otherwise

      • setVersionsToKeep

        public void setVersionsToKeep​(int count)
        How many versions to retain for in-memory stores. If not set, 5 old versions are retained.
        Parameters:
        count - the number of versions to keep

      • getVersionsToKeep

        public int getVersionsToKeep()
        Get the oldest version to retain in memory (for in-memory stores).
        Returns:
        the version

      • setOldestVersionTracker

        public void setOldestVersionTracker​(java.util.function.LongConsumer callback)

      • registerUnsavedMemory

        public void registerUnsavedMemory​(int memory)
        Adjust amount of "unsaved memory" meaning amount of RAM occupied by pages not saved yet to the file. This is the amount which triggers auto-commit.
        Parameters:
        memory - adjustment

      • getStoreVersion

        public int getStoreVersion()
        Get the store version. The store version is usually used to upgrade the structure of the store after upgrading the application. Initially the store version is 0, until it is changed.
        Returns:
        the store version

      • setStoreVersion

        public void setStoreVersion​(int version)
        Update the store version.
        Parameters:
        version - the new store version

      • rollback

        public void rollback()
        Revert to the beginning of the current version, reverting all uncommitted changes.

      • rollbackTo

        public void rollbackTo​(long version)
        Revert to the beginning of the given version. All later changes (stored or not) are forgotten. All maps that were created later are closed. A rollback to a version before the last stored version is immediately persisted. Rollback to version 0 means all data is removed.
        Parameters:
        version - the version to revert to

      • getCurrentVersion

        public long getCurrentVersion()
        Get the current version of the data. When a new store is created, the version is 0.
        Returns:
        the version

      • getFileStore

        public FileStore<?> getFileStore()
        Get the file store.
        Returns:
        the file store

      • getStoreHeader

        public java.util.Map<java.lang.String,​java.lang.Object> getStoreHeader()
        Get the store header. This data is for informational purposes only. The data is subject to change in future versions. The data should not be modified (doing so may corrupt the store).
        Returns:
        the store header

      • renameMap

        public void renameMap​(MVMap<?,​?> map,
                      java.lang.String newName)
        Rename a map.
        Parameters:
        map - the map
        newName - the new name

      • removeMap

        public void removeMap​(MVMap<?,​?> map)
        Prepare map removal from the store. This includes removal of its references by name and by id from meta map. It's still held in maps, so actual removal is delayed until all of its usages (iterators etc.) are completed. This is controlled by store versions, and actual removal is done by MVStore.setWriteVersion() and MVMap,setWriteVersion() when removal from maps and from layout is done.
        Parameters:
        map - the map to remove

      • removeMap

        public void removeMap​(java.lang.String name)
        Remove map by name.
        Parameters:
        name - the map name

      • getMapName

        public java.lang.String getMapName​(int id)
        Get the name of the given map.
        Parameters:
        id - the map id
        Returns:
        the name, or null if not found

      • populateInfo

        public void populateInfo​(java.util.function.BiConsumer<java.lang.String,​java.lang.String> consumer)

      • isClosed

        public boolean isClosed()
        Determine that store is open, or wait for it to be closed (by other thread)
        Returns:
        true if store is open, false otherwise

      • setAutoCommitDelay

        public void setAutoCommitDelay​(int millis)
        Set the maximum delay in milliseconds to auto-commit changes.

        To disable auto-commit, set the value to 0. In this case, changes are only committed when explicitly calling commit.

        The default is 1000, meaning all changes are committed after at most one second.

        Parameters:
        millis - the maximum delay

      • getAutoCommitDelay

        public int getAutoCommitDelay()
        Get the auto-commit delay.
        Returns:
        the delay in milliseconds, or 0 if auto-commit is disabled.

      • getAutoCommitMemory

        public int getAutoCommitMemory()
        Get the maximum memory (in bytes) used for unsaved pages. If this number is exceeded, unsaved changes are stored to disk.
        Returns:
        the memory in bytes

      • getUnsavedMemory

        public int getUnsavedMemory()
        Get the estimated memory (in bytes) of unsaved data. If the value exceeds the auto-commit memory, the changes are committed.

        The returned value is an estimation only.

        Returns:
        the memory in bytes

      • isReadOnly

        public boolean isReadOnly()
        Whether the store is read-only.
        Returns:
        true if it is

      • registerVersionUsage

        public MVStore.TxCounter registerVersionUsage()
        Register opened operation (transaction). This would increment usage counter for the current version. This version (and all after it) should not be dropped until all transactions involved are closed and usage counter goes to zero.
        Returns:
        TxCounter to be decremented when operation finishes (transaction closed).

      • deregisterVersionUsage

        public void deregisterVersionUsage​(MVStore.TxCounter txCounter)
        De-register (close) completed operation (transaction). This will decrement usage counter for the corresponding version. If counter reaches zero, that version (and all unused after it) can be dropped immediately.
        Parameters:
        txCounter - to be decremented, obtained from registerVersionUsage()

      • decrementVersionUsageCounter

        public boolean decrementVersionUsageCounter​(MVStore.TxCounter txCounter)
        De-register (close) completed operation (transaction). This will decrement usage counter for the corresponding version.
        Parameters:
        txCounter - to be decremented, obtained from registerVersionUsage()
        Returns:
        true if counter reaches zero, which indicates that version is no longer in use, false otherwise.

      • countNewPage

        public void countNewPage​(boolean leaf)