Changelog

7.1

Features

  • New public AFS connectors

    • AWS S3 and DynamoDB

    • Azure Storage

    • Google Cloud Firestore

    • Hazelcast

    • Kafka

    • Oracle Coherence, DB and Cloud Object Store

    • Redis

    • Hana DB

  • Finalized storage garbage collector, activated by default [402]

  • Improve CDI integration [377]

  • CDI Integration: Allow customization of the EmbeddedStorageFoundation before StorageManager is created [388][398]

  • Spring Boot integration: Add interceptor logic to store marked objects at end of method [384]

  • Clarification around cache expiry when using StorageManager [387]

  • Store reference inside Lazy when Lazy is marked (CDI integration) [386]

  • Enable microstream cache [351]

  • Validate Channel Folders to Channel Count [192]

Bugfixes

  • Remove @Store from CDI integration [397][394]

  • Fix CDI integration cache configuration keys [395]

  • Active the test in the CDI extension [354]

7.0

Features

Bugfixes

  • Stop LazyrefenceManager automatically [343]

  • Check for cache clear after loading storage entities [326]

  • Reduce memory usage [323]

  • Fixed storage stalls in case of errors during the housekeeping phase [303]

  • Eliminate null in StorageConfiguration.toString() [205]

  • Fixed exclusive file access conflicts when restarting storage [282]

Migration guide

From version 6 no actions are necessary, for older versions see below.

6.1

Bugfix release.

Bugfixes

  • Fixed various bugs in backup [194][272][273][278][280]

  • Fixed storage restart error with same configuration instance [279]

  • Fixed Javadoc warnings [281]

6.0

Features

Bugfixes

  • Fixed swallowed exceptions in storage channels [265]

  • Fixed failing import with multiple channels [254]

  • Fixed zero byte blocks in storage [255]

  • Fixed faulty loading behaviour in persisted cache [226]

  • Fixed possible deadlock in persistence manager [240]

  • Fixed bug in type handler for PriorityQueue [234]

Migration guide

From version 5 no actions are necessary, for older versions see below.

5.0.2

Bugfix release.

Bugfixes

  • Fixed buffer flush regression [225]

5.0.1

Bugfix release.

Bugfixes

  • Fixed broken build for microstream-storage-restclient-app

5.0

This release introduces some breaking changes. To fully support the Java Module System (Jigsaw) and to meet Maven’s naming standards, we had to rename some packages and our artifact’s IDs. But don’t worry. Just change the dependency configuration and organize some imports, and you are good to go. Existing storages are not affected and are fully compatible with this version. For detailed information, see the migration guide below.

Features

  • General purpose configuration layer on which the storage and cache configuration are now based on, with following key features:

    • Tree like structure to support sub-configurations

    • Input from various sources (ini, xml, yaml, …​)

    • Typed getters for properties

    • Extendable: custom property parsers and configuration loaders

    • Adaptable: easy adaption to other framework’s configurations, like e.g. Spring Boot

  • Storage configuration now supports different file systems for live storage and backup

  • GraalVM support: MicroStream’s optimized memory adapter is now used automatically when running on GraalVM

  • Restructured MicroStream’s module and package structure to fully support the Java Module System and adopt Maven naming standards, see Migration Guide

Bugfixes

  • Fixed sporadic index out of bounds exception when storage and backup had different storage targets with a channel count greater than one

Migration guide

In order to fully support the Java Module System and adopt Maven naming standards we had to introduce some changes:

Artifacts and Packages

These are the changed artifacts and their packages.

Old (< 5) New (>= 5)

base

microstream-base

AFS and AFS/NIO were extracted from base module:

base

microstream-afs

one.microstream.afs

one.microstream.afs.types, one.microstream.afs.exceptions

base

microstream-afs-nio

one.microstream.afs.nio

one.microstream.afs.nio.types

cache

microstream-cache

one.microstream.cache

one.microstream.cache.types, one.microstream.cache.exceptions

cache.hibernate

microstream-cache-hibernate

one.microstream.cache.hibernate

one.microstream.cache.hibernate.types

communication

microstream-communication

one.microstream.com

one.microstream.communication.types

communication.binary

microstream-communication-binary

one.microstream.com.binary

one.microstream.communication.binary.types

AWS was split into AWS-DynamoDB and AWS-S3:

filesystem.aws

microstream-enterprise-afs-aws-dynamodb

one.microstream.afs.aws.dynamodb

one.microstream.enterprise.afs.aws.dynamodb.types

filesystem.aws

microstream-enterprise-afs-aws-s3

one.microstream.afs.aws.s3

one.microstream.enterprise.afs.aws.s3.types

filesystem.azure

microstream-enterprise-afs-azure-storage

one.microstream.afs.azure.storage

one.microstream.enterprise.afs.azure.storage.types

filesystem.blobstore

microstream-enterprise-afs-blobstore

one.microstream.afs.blobstore

one.microstream.enterprise.afs.blobstore.types

filesystem.coherence

microstream-enterprise-afs-oracle-coherence

one.microstream.afs.coherence

one.microstream.enterprise.afs.oracle.coherence.types

filesystem.hazelcast

microstream-enterprise-afs-hazelcast

one.microstream.afs.hazelcast

one.microstream.enterprise.afs.hazelcast.types

filesystem.jpa.hibernate

microstream-enterprise-afs-hibernate

one.microstream.afs.jpa.hibernate

one.microstream.afs.hibernate.types

filesystem.kafka

microstream-enterprise-afs-kafka

one.microstream.afs.kafka

one.microstream.enterprise.afs.kafka.types

filesystem.mongodb

microstream-enterprise-afs-mongodb

one.microstream.afs.mongodb

one.microstream.enterprise.afs.mongodb.types

filesystem.oraclecloud

microstream-enterprise-afs-oraclecloud-objectstorage

one.microstream.afs.oracle.cloud.objectstorage

one.microstream.enterprise.afs.oraclecloud.objectstorage.types

filesystem.oraclenosql

microstream-enterprise-afs-oracle-nosql

one.microstream.afs.oracle.nosql

one.microstream.enterprise.afs.oracle.nosql.types

filesystem.redis

microstream-enterprise-afs-redis

one.microstream.afs.redis

one.microstream.enterprise.afs.redis.types

SQL was split into SQL and Enterprise-SQL:

filesystem.sql

microstream-afs-sql

on.microstream.afs.sql

one.microstream.afs.sql.types

filesystem.sql

microstream-enterprise-afs-sql

on.microstream.afs.sql

one.microstream.enterprise.afs.sql.types

persistence

microstream-persistence

persistence.binary

microstream-persistence-binary

one.microstream.collections

one.microstream.persistence.binary.one.microstream.collections

one.microstream.entity

one.microstream.persistence.binary.one.microstream.entity

one.microstream.persistence.types

one.microstream.persistence.binary.one.microstream.persistence.types

one.microstream.reference

one.microstream.persistence.binary.one.microstream.reference

one.microstream.util

one.microstream.persistence.binary.one.microstream.util

one.microstream.java.*

one.microstream.persistence.binary.java.*

persistence.binary.jdk8

microstream-persistence-binary-jdk8

one.microstream.jdk8.java.util

one.microstream.persistence.binary.jdk8.java.util

one.microstream.memory.sun

one.microstream.persistence.binary.jdk8.types

storage

microstream-storage

storage.embedded

microstream-storage-embedded

one.microstream.storage.types

one.microstream.storage.embedded.types

storage.embedded.configuration

microstream-storage-embedded-configuration

one.microstream.storage.configuration

one.microstream.storage.embedded.configuration.types

storage.restadapter

microstream-storage-restadapter

one.microstream.storage.restadapter

one.microstream.storage.restadapter.types, one.microstream.storage.restadapter.exceptions

storage.restclient

microstream-storage-restclient

one.microstream.storage.restclient

one.microstream.storage.restclient.types, one.microstream.storage.restclient.exceptions

storage.restclient.app

microstream-storage-restclient-app

one.microstream.storage.restclient.app

one.microstream.storage.restclient.app.types

storage.restclient.jersey

microstream-storage-restclient-jersey

one.microstream.storage.restclient.jersey

one.microstream.storage.restclient.jersey.types

storage.restservice

microstream-storage-restservice

one.microstream.storage.restservice

one.microstream.storage.restservice.types, one.microstream.storage.restservice.exceptions

storage.restservice.sparkjava

microstream-storage-restservice-sparkjava

one.microstream.storage.restservice.sparkjava

one.microstream.storage.restservice.sparkjava.types, one.microstream.storage.restservice.sparkjava.exceptions

Configuration

The storage and cache configuration layers are now utilizing the newly introduced general purpose configuration layer.

Cache

The configuration property names have changed from camelCase to the dash-notation.

Old (< 5) New (>= 5)

keyType

key-type

valueType

value-type

storageConfigurationResourceName

storage-configuration-resource-name

cacheLoaderFactory

cache-loader-factory

cacheWriterFactory

cache-writer-factory

expiryPolicyFactory

expiry-policy-factory

evictionManagerFactory

eviction-manager-factory

readThrough

read-through

writeThrough

write-through

storeByValue

store-by-value

statisticsEnabled

statistics-enabled

managementEnabled

management-enabled
Storage

The old storage configuration API is deprecated and should be migrated to the new one.

The configuration property names have changed from camelCase to the dash-notation.

Old (< 5) New (>= 5)

baseDirectory

storage-directory

deletionDirectory

deletion-directory

truncationDirectory

truncation-directory

backupDirectory

backup-directory

channelCount

channel-count

channelDirectoryPrefix

channel-directory-prefix

dataFilePrefix

data-file-prefix

dataFileSuffix

data-file-suffix

transactionFilePrefix

transaction-file-prefix

transactionFileSuffix

transaction-file-suffix

typeDictionaryFilename

type-dictionary-file-name

rescuedFileSuffix

rescued-file-suffix

lockFileName

lock-file-name

houseKeepingIntervalMs

housekeeping-interval

housekeepingTimeBudgetNs

housekeeping-time-budget

entityCacheThreshold

entity-cache-threshold

entityCacheTimeoutMs

entity-cache-timeout

dataFileMinimumSize

data-file-minimum-size

dataFileMaximumSize

data-file-maximum-size

dataFileMinimumUseRatio

data-file-minimum-use-ratio

dataFileCleanupHeadFile

data-file-cleanup-head-file

4.1

Features

  • API to create full backup additional to continuous backup

  • Simplified usage of other NIO based file systems, e.g. in-memory file systems for unit testing

  • OSGi Headers are added in released jar files and a p2 update site is available next to the maven repository

  • Additional storage targets

    • AWS S3 and DynamoDB

    • Azure Storage

    • Hazelcast

    • Kafka

    • MongoDB

    • Redis

    • Plain JDBC: SAP HANA, PostgreSQL, MariaDB

Bugfixes

  • Fixed type handler incompatibilities with JDK 11

  • Removed wrongfully initial values in configuration for deletion and truncation directories

Migration guide

From 4.1 no actions are necessary, for older versions see below.

4.0

Features

  • File System Abstraction to support a variety of storage targets.

  • Exporter API for storage configuration.

Bugfixes

  • Better error handling in housekeeping, to avoid storage channel thread interruptions.

  • Fixed XML loading bug under Android.

Migration guide

All configuration code, which includes file paths, may be rewritten to use the new Abstract File System.
For example see Storage Files or Backup.

3.0.1

Bugfix release for projects using an old non public version.

Bugfixes

  • Fixed bug in type dictionary parser for old legacy type handlers

3.0

Features

  • Added a convenience layer for defining custom type handlers

  • Fully automatic lazy reference managing for clearing older Lazy references as free memory gets lower.

  • Completed optimized persistence handling for remaining types in JDK that are reasonable to appear in entity graphs. E.g. Currency, Locale, URI, URL, Path

  • Compatibility with projects using java modules ("jigsaw")

  • Added JavaDoc for common types like StorageConnection, StorageManager, StorageController, etc.

  • Auto-recognition for CSV separator character in configuration files (e.g. legacy type mapping)

  • JSR-107 compatible JCache provider with additional Hibernate cache region factory

  • Storage REST Service and Client are available to access the storage data via REST, a Java-Wrapper thereof and a Web-UI

Bugfixes

  • Fixed a potential race condition during loading

  • Fixed a potential race condition and robustness issue during storing

  • StorageConnectionFoundation instance is now properly created on demand.

  • Removed unnecessary memory consumption exponential to channel count.

  • Improved exception messages on invalid configuration values (file sizes etc.)

  • Workarounded a JDK bug regarding file locking when importing files (JDK exception says another process is locking the file which is outright wrong)

  • Fixed type resolving problems when different ClassLoaders are involved.

  • Fixed a bug that caused loading of zero-length arrays to throw an exception despite everything being fine.

  • Various smaller bugfixes for unnecessary exceptions in special cases and state robustness after exceptions.

Migration guide

From 2.2 no actions are necessary, for older versions see below.

2.2

Features

  • Removed SelfStoring without replacement since it could not be used recursively and has no advantages over just creating a static storing utility method for a certain entity.

  • Added state validation of value type objects (e.g. String, Integer, BigDecimal, etc.) upon loading. This is hardly relevant in practice, but not having it can lead to confusing hello-world-like test applications.

  • EmbeddedStorageManager now implements java.lang.AutoClosable.

  • Replaced all provisional RuntimeExceptions with either PersistenceException or StorageException, depending on the architectural level the corresponding source code it located.

  • The two technically different root handling concepts ("default" and "custom") have been consolidated in a way that they are the same thing on the API level and interchangeable, meaning no more confusion with those root exception messages.

  • All entity fields of type transient EmbeddedStorageManager now get a reference to the used EmbeddedStorageManager instance set upon loading/updating.

  • The interfaces around storage managing have been enhanced so that it is now equally valid to just write StorageManager instead of EmbeddedStorageManager. (An EmbeddedStorageManager "is a" StorageManager)

  • Slight technical consolidation of Lazy reference handling caused the type Lazy to be moved from the package one.microstream.persistence.lazy to one.microstream.reference. The reason is that the lazy handling has actually no inherent connection to persistence or storage. It’s actually just a generic concept that can be used by those layers. See Migration Guide below on how to adjust existing projects.

Bugfixes

  • Fixed an off-heap memory leak when restarting the storage multiple times in the same process.

  • Fixed a bug where changing the fields of an entity type caused an exception. This was a regression bug from fixing a similar problem for another case in version 2.1. Now, both cases work correctly.

Migration Guide

All occurrences in user code of one.microstream.persistence.lazy.Lazy have to be refactored to one.microstream.reference.Lazy. Modern IDEs provide a functionality to "auto-import" missing types or automatically "organize imports", so this should be resolved with a proverbial push of a button.

2.1

Features

  • Android support MicroStream is now Java-wise fully compatible with Android.

  • Replaced all usages of java.util.File with java.nio.file.Path to allow using custom file implementations.

  • Improved skipping functionality of Storers (see EmbeddedStorageManager#createStorer and Storer#skip).

  • The class Lazy is now an interface to allow custom implementations. See Migration guide below.

Bugfixes

  • Fixed a few minor bugs in the skipping functionality of Storers.

  • Fixed a bug where files remained locked after the storage was shut down.

  • Fixed a bug where files remained locked after an exception in storage initialization.

  • Enums defining an abstract method are now handled correctly.

  • By default, all threads created by MicroStream now start with the prefix "MicroStream-". This can be customized by the new interface StorageThreadNameProvider.

  • Fixed a NullPointerException in import.

  • Fixed a bug that caused enums with a certain field layout to be loaded inconsistently.

  • java.util.Locale is now persisted and created using Locale’s #toLanguageTag and #forLanguageTag.

Migration Guide

In the directory of an existing storage, in the TypeDictionary file (default name "PersistenceTypeDictionary.ptd"), all occurrences of "one.microstream.persistence.lazy.Lazy" must be replaced with "one.microstream.persistence.lazy.Lazy$Default".