There is a bug in Mirth Connect and derivative forks that might delete all configuration entries for a single plugin when they are stored in the database under specific circumstances.

Bug Prerequisites #

This bug will only occur when an additional database is deliberately set up to act as a read replica of a main write database, and Mirth Connect is also configured to use this read replica for performance improvements (i.e., a read/write split configuration).

The bug manifests when there is replication latency between the main write database and the read replica at the time a plugin’s configuration is being saved. Because it relies on this latency, the bug is unpredictable.

Important Note

• Only the configuration entries of plugins (both built-in and third-party) are affected by this bug, and only when those entries are changed/updated.
• This bug should never result in the loss of business data, which should be stored in other tables or databases.

⚙️ Configuration References #

• How to configure a read/write split configuration in PostgreSQL (Example)
• How to configure Mirth Connect to use a split database

Affected Versions #

Explanation of the Mirth Connect bug #

This a step by step explanation of the Mirth Connect bug (a Read/Write Split Race Condition) that may lead to the loss of a plugin configuration:

• When a plugin stores its properties, it executes the setPluginProperties (1) method in DefaultExtensionController.java.
  • The implementation of this method first deletes all existing properties for the plugin from the database (DELETE operation). (2)
  • Next, it proceeds to save the current properties again (INSERT or UPDATE operations). (3)

• A race condition occurs when the read-only replica suffers latency and has not yet registered the deletion from (2), meaning it still holds the old configuration values.
• The saveProperty (4) method (executed for each property) first attempts to check if the property already exists using the getProperty (5) method.
  • The getProperty method incorrectly checks the read-only replica database (which is configured to provide fast lookups). (7)
  • The read-only replica reports that the property does exist (due to the replication latency).
  • The system then attempts an UPDATE operation on the main write database. (6)
  • This UPDATE fails because the property was successfully deleted from the main write database in (2).
• The result is that the new configuration values are not saved after the deletion, leaving the plugin with zero configuration entries until the operation is performed again successfully.

Additional notes about the bug #

• Configuration save operations for one plugin do not interfere with other plugins. Only the configurations for the currently modified plugin can be lost.
• The setPluginProperties method offers a mergeProperties parameter that, when set to true, simply updates existing properties without first deleting them. This mechanism theoretically avoids the configuration loss bug.
• The Meditecs ASTM extension sets mergeProperties to true, so it should not be affected by this bug.

Solutions for the Mirth Connect bug #

The core issue is allowing read operations for configuration lookups to hit the potentially stale read-only replica. Solutions to this Mirth Connect bug involve forcing these specific operations to the main write database.

• BridgeLink Fix (Version 4.6): The solution was simply to use the normal session manager (main write database) instead of relying on the read-only database for these specific configuration lookups.
  • Reference: https://github.com/Innovar-Healthcare/BridgeLink/commit/f7908060e6e5fdc3f860d09501f4c6a78a038b60

• Open Integration Engine Fix: This solution forces the save operation by performing an unconditional UPDATE first, followed by an INSERT if the UPDATE was not successful. This bypasses the need to check for existence via a potentially stale read replica.
  • Reference: https://github.com/OpenIntegrationEngine/engine/pull/195
• Alternative Fix (Architectural): An alternative approach is to create a new method for obtaining a property that specifically uses the main write database. Then, modify saveProperty to use this new method instead of the standard getProperty. This maintains the original, faster getProperty method for other non-critical read operations.

Migrate to an Actively Supported Integration Engine: Meditecs Smart Connect #

As the open-source version of Mirth Connect is not actively maintained, it is highly recommended to migrate to a supported integration engine.

At Meditecs, we actively maintain Meditecs Smart Connect, providing ongoing support, security patches, and fixes for critical issues like the one described above.

Learn More:

• Meditecs Smart Connect
• Smart Migration from Mirth Connect