Changelog

5.0.2

Bugfix release.

Bugfixes

  • Fixed buffer flush regression [225]

5.0.1

Bugfix release.

Bugfixes

  • Fixed broken build for microstream-storage-restclient-app

5.0

This release introduces some breaking changes. To fully support the Java Module System (Jigsaw) and to meet Maven’s naming standards, we had to rename some packages and our artifact’s IDs. But don’t worry. Just change the dependency configuration and organize some imports, and you are good to go. Existing storages are not affected and are fully compatible with this version. For detailed information, see the migration guide below.

Features

  • General purpose configuration layer on which the storage and cache configuration are now based on, with following key features:

    • Tree like structure to support sub-configurations

    • Input from various sources (ini, xml, yaml, …​)

    • Typed getters for properties

    • Extendable: custom property parsers and configuration loaders

    • Adaptable: easy adaption to other framework’s configurations, like e.g. Spring Boot

  • Storage configuration now supports different file systems for live storage and backup

  • GraalVM support: MicroStream’s optimized memory adapter is now used automatically when running on GraalVM

  • Restructured MicroStream’s module and package structure to fully support the Java Module System and adopt Maven naming standards, see Migration Guide

Bugfixes

  • Fixed sporadic index out of bounds exception when storage and backup had different storage targets with a channel count greater than one

Migration guide

Artifacts

First you must adjust the artifacts:

Old (< 5) New (>= 5)

base

microstream-base, microstream-afs and microstream-afs-nio (AFS and AFS/NIO) was extracted from base module)

cache

microstream-cache

cache.hibernate

microstream-cache-hibernate

communication

microstream-communication

communication.binary

microstream-communication-binary

filesystem.aws

microstream-enterprise-afs-aws-dynamodb and microstream-enterprise-afs-aws-s3

filesystem.azure

microstream-enterprise-afs-azure-storage

filesystem.blobstore

microstream-enterprise-afs-blobstore

filesystem.coherence

microstream-enterprise-afs-oracle-coherence

filesystem.hazelcast

microstream-enterprise-afs-hazelcast

filesystem.jpa.hibernate

microstream-enterprise-afs-hibernate

filesystem.kafka

microstream-enterprise-afs-kafka

filesystem.mongodb

microstream-enterprise-afs-mongodb

filesystem.oraclecloud

microstream-enterprise-afs-oraclecloud-objectstorage

filesystem.oraclenosql

microstream-enterprise-afs-oracle-nosql

filesystem.redis

microstream-enterprise-afs-redis

filesystem.sql

microstream-afs-sql and microstream-enterprise-afs-sql

persistence

microstream-persistence

persistence.binary

microstream-persistence-binary

persistence.binary.jdk8

microstream-persistence-binary-jdk8

storage

microstream-storage

storage.embedded

microstream-storage-embedded

storage.embedded.configuration

microstream-storage-embedded-configuration

storage.restadapter

microstream-storage-restadapter

storage.restclient

microstream-storage-restclient

storage.restclient.app

microstream-storage-restclient-app

storage.restclient.jersey

microstream-storage-restclient-jersey

storage.restservice

microstream-storage-restservice

storage.restservice.sparkjava

microstream-storage-restservice-sparkjava

Packages

These package names have changed:

Old (< 5) [artifact:package] New (>= 5) [artifact:package]

base:one.microstream.afs

microstream-afs:one.microstream.afs.types,exceptions

base:one.microstream.afs.nio

microstream-afs-nio:one.microstream.afs.nio.types

cache:one.microstream.cache

microstream-cache:one.microstream.cache.types,exceptions

cache.hibernate:one.microstream.cache.hibernate

microstream-cache-hibernate:one.microstream.cache.hibernate.types

communication:one.microstream.com

microstream-communication:one.microstream.communication.types

communication.binary:one.microstream.com.binary

microstream-communication-binary:one.microstream.communication.binary.types

filesystem.aws:one.microstream.afs.aws.dynamodb

microstream-enterprise-afs-aws-dynamodb:one.microstream.enterprise.afs.aws.dynamodb.types

filesystem.aws:one.microstream.afs.aws.s3

microstream-enterprise-afs-aws-s3:one.microstream.enterprise.afs.aws.s3.types

filesystem.azure:one.microstream.afs.azure.storage

microstream-enterprise-afs-azure-storage:one.microstream.enterprise.afs.azure.storage.types

filesystem.blobstore:one.microstream.afs.blobstore

microstream-enterprise-afs-blobstore:one.microstream.enterprise.afs.blobstore.types

filesystem.coherence:one.microstream.afs.coherence

microstream-enterprise-afs-oracle-coherence:one.microstream.enterprise.afs.oracle.coherence.types

filesystem.hazelcast:one.microstream.afs.hazelcast

microstream-enterprise-afs-hazelcast:one.microstream.enterprise.afs.hazelcast.types

filesystem.jpa.hibernate:one.microstream.afs.jpa.hibernate

microstream-enterprise-afs-hibernate:one.microstream.afs.hibernate.types

filesystem.kafka:one.microstream.afs.kafka

microstream-enterprise-afs-kafka:one.microstream.enterprise.afs.kafka.types

filesystem.mongodb:one.microstream.afs.mongodb

microstream-enterprise-afs-mongodb:one.microstream.enterprise.afs.mongodb.types

filesystem.oraclecloud:one.microstream.afs.oracle.cloud.objectstorage

microstream-enterprise-afs-oraclecloud-objectstorage:one.microstream.enterprise.afs.oraclecloud.objectstorage.types

filesystem.redis:one.microstream.afs.redis

microstream-enterprise-afs-redis:one.microstream.enterprise.afs.redis.types

filesystem.sql

microstream-afs-sql:one.microstream.afs.sql.types and microstream-enterprise-afs-sql:one.microstream.enterprise.afs.sql.types

persistence.binary:one.microstream.collections

persistence-binary:one.microstream.persistence.binary.one.microstream.collections

persistence.binary:one.microstream.entity

persistence-binary:one.microstream.persistence.binary.one.microstream.entity

persistence.binary:one.microstream.persistence.types

persistence-binary:one.microstream.persistence.binary.one.microstream.persistence.types

persistence.binary:one.microstream.reference

persistence-binary:one.microstream.persistence.binary.one.microstream.reference

persistence.binary:one.microstream.util

persistence-binary:one.microstream.persistence.binary.one.microstream.util

persistence.binary:one.microstream.java.*

persistence-binary:one.microstream.persistence.binary.java.*

persistence.binary.jdk8:one.microstream.jdk8.java.util

microstream-persistence-binary-jdk8:one.microstream.persistence.binary.jdk8.java.util

persistence.binary.jdk8:one.microstream.memory.sun

microstream-persistence-binary-jdk8:one.microstream.persistence.binary.jdk8.types

storage.embedded:one.microstream.storage.types

microstream-storage-embedded:one.microstream.storage.embedded.types

storage.embedded.configuration:one.microstream.storage.configuration

microstream-storage-embedded-configuration:one.microstream.storage.embedded.configuration.types

storage.restadapter:one.microstream.storage.restadapter

microstream-storage-restadapter:one.microstream.storage.restadapter.types,exceptions

storage.restclient:one.microstream.storage.restclient

microstream-storage-restclient:one.microstream.storage.restclient.types,exceptions

storage.restclient.app:one.microstream.storage.restclient.app

microstream-storage-restclient-app:one.microstream.storage.restclient.app.types

storage.restclient.jersey:one.microstream.storage.restclient.jersey

microstream-storage-restclient-jersey:one.microstream.storage.restclient.jersey.types

storage.restservice:one.microstream.storage.restservice

microstream-storage-restservice:one.microstream.storage.restservice.types,exceptions

storage.restservice.sparkjava:one.microstream.storage.restservice.sparkjava

microstream-storage-restservice-sparkjavaone.microstream.storage.restservice.sparkjava.types,exceptions

Configuration

The storage and cache configuration layers are now utilizing the newly introduced general purpose configuration layer.

Cache

The configuration property names have changed from camelCase to the dash-notation.

Old (< 5) New (>= 5)

keyType

key-type

valueType

value-type

storageConfigurationResourceName

storage-configuration-resource-name

cacheLoaderFactory

cache-loader-factory

cacheWriterFactory

cache-writer-factory

expiryPolicyFactory

expiry-policy-factory

evictionManagerFactory

eviction-manager-factory

readThrough

read-through

writeThrough

write-through

storeByValue

store-by-value

statisticsEnabled

statistics-enabled

managementEnabled

management-enabled

Storage

The old storage configuration API is deprecated and should be migrated to the new one.

The configuration property names have changed from camelCase to the dash-notation.

Old (< 5) New (>= 5)

baseDirectory

storage-directory

deletionDirectory

deletion-directory

truncationDirectory

truncation-directory

backupDirectory

backup-directory

channelCount

channel-count

channelDirectoryPrefix

channel-directory-prefix

dataFilePrefix

data-file-prefix

dataFileSuffix

data-file-suffix

transactionFilePrefix

transaction-file-prefix

transactionFileSuffix

transaction-file-suffix

typeDictionaryFilename

type-dictionary-file-name

rescuedFileSuffix

rescued-file-suffix

lockFileName

lock-file-name

houseKeepingIntervalMs

housekeeping-interval

housekeepingTimeBudgetNs

housekeeping-time-budget

entityCacheThreshold

entity-cache-threshold

entityCacheTimeoutMs

entity-cache-timeout

dataFileMinimumSize

data-file-minimum-size

dataFileMaximumSize

data-file-maximum-size

dataFileMinimumUseRatio

data-file-minimum-use-ratio

dataFileCleanupHeadFile

data-file-cleanup-head-file

4.1

Features

  • API to create full backup additional to continuous backup

  • Simplified usage of other NIO based file systems, e.g. in-memory file systems for unit testing

  • OSGi Headers are added in released jar files and a p2 update site is available next to the maven repository

  • Additional storage targets

    • AWS S3 and DynamoDB

    • Azure Storage

    • Hazelcast

    • Kafka

    • MongoDB

    • Redis

    • Plain JDBC: SAP HANA, PostgreSQL, MariaDB

Bugfixes

  • Fixed type handler incompatibilities with JDK 11

  • Removed wrongfully initial values in configuration for deletion and truncation directories

Migration guide

From 4.1 no actions are necessary, for older versions see below.

4.0

Features

  • File System Abstraction to support a variety of storage targets.

  • Exporter API for storage configuration.

Bugfixes

  • Better error handling in housekeeping, to avoid storage channel thread interruptions.

  • Fixed XML loading bug under Android.

Migration guide

All configuration code, which includes file paths, may be rewritten to use the new Abstract File System.
For example see Storage Files or Backup.

3.0.1

Bugfix release for projects using an old non public version.

Bugfixes

  • Fixed bug in type dictionary parser for old legacy type handlers

3.0

Features

  • Added a convenience layer for defining custom type handlers

  • Fully automatic lazy reference managing for clearing older Lazy references as free memory gets lower.

  • Completed optimized persistence handling for remaining types in JDK that are reasonable to appear in entity graphs. E.g. Currency, Locale, URI, URL, Path

  • Compatibility with projects using java modules ("jigsaw")

  • Added JavaDoc for common types like StorageConnection, StorageManager, StorageController, etc.

  • Auto-recognition for CSV separator character in configuration files (e.g. legacy type mapping)

  • JSR-107 compatible JCache provider with additional Hibernate cache region factory

  • Storage REST Service and Client are available to access the storage data via REST, a Java-Wrapper thereof and a Web-UI

Bugfixes

  • Fixed a potential race condition during loading

  • Fixed a potential race condition and robustness issue during storing

  • StorageConnectionFoundation instance is now properly created on demand.

  • Removed unnecessary memory consumption exponential to channel count.

  • Improved exception messages on invalid configuration values (file sizes etc.)

  • Workarounded a JDK bug regarding file locking when importing files (JDK exception says another process is locking the file which is outright wrong)

  • Fixed type resolving problems when different ClassLoaders are involved.

  • Fixed a bug that caused loading of zero-length arrays to throw an exception despite everything being fine.

  • Various smaller bugfixes for unnecessary exceptions in special cases and state robustness after exceptions.

Migration guide

From 2.2 no actions are necessary, for older versions see below.

2.2

Features

  • Removed SelfStoring without replacement since it could not be used recursively and has no advantages over just creating a static storing utility method for a certain entity.

  • Added state validation of value type objects (e.g. String, Integer, BigDecimal, etc.) upon loading. This is hardly relevant in practice, but not having it can lead to confusing hello-world-like test applications.

  • EmbeddedStorageManager now implements java.lang.AutoClosable.

  • Replaced all provisional RuntimeExceptions with either PersistenceException or StorageException, depending on the architectural level the corresponding source code it located.

  • The two technically different root handling concepts ("default" and "custom") have been consolidated in a way that they are the same thing on the API level and interchangeable, meaning no more confusion with those root exception messages.

  • All entity fields of type transient EmbeddedStorageManager now get a reference to the used EmbeddedStorageManager instance set upon loading/updating.

  • The interfaces around storage managing have been enhanced so that it is now equally valid to just write StorageManager instead of EmbeddedStorageManager. (An EmbeddedStorageManager "is a" StorageManager)

  • Slight technical consolidation of Lazy reference handling caused the type Lazy to be moved from the package one.microstream.persistence.lazy to one.microstream.reference. The reason is that the lazy handling has actually no inherent connection to persistence or storage. It’s actually just a generic concept that can be used by those layers. See Migration Guide below on how to adjust existing projects.

Bugfixes

  • Fixed an off-heap memory leak when restarting the storage multiple times in the same process.

  • Fixed a bug where changing the fields of an entity type caused an exception. This was a regression bug from fixing a similar problem for another case in version 2.1. Now, both cases work correctly.

Migration Guide

All occurrences in user code of one.microstream.persistence.lazy.Lazy have to be refactored to one.microstream.reference.Lazy. Modern IDEs provide a functionality to "auto-import" missing types or automatically "organize imports", so this should be resolved with a proverbial push of a button.

2.1

Features

  • Android support MicroStream is now Java-wise fully compatible with Android.

  • Replaced all usages of java.util.File with java.nio.file.Path to allow using custom file implementations.

  • Improved skipping functionality of Storers (see EmbeddedStorageManager#createStorer and Storer#skip).

  • The class Lazy is now an interface to allow custom implementations. See Migration guide below.

Bugfixes

  • Fixed a few minor bugs in the skipping functionality of Storers.

  • Fixed a bug where files remained locked after the storage was shut down.

  • Fixed a bug where files remained locked after an exception in storage initialization.

  • Enums defining an abstract method are now handled correctly.

  • By default, all threads created by MicroStream now start with the prefix "MicroStream-". This can be customized by the new interface StorageThreadNameProvider.

  • Fixed a NullPointerException in import.

  • Fixed a bug that caused enums with a certain field layout to be loaded inconsistently.

  • java.util.Locale is now persisted and created using Locale’s #toLanguageTag and #forLanguageTag.

Migration Guide

In the directory of an existing storage, in the TypeDictionary file (default name "PersistenceTypeDictionary.ptd"), all occurrences of "one.microstream.persistence.lazy.Lazy" must be replaced with "one.microstream.persistence.lazy.Lazy$Default".