YAML confg migration tool
Nowadays YAML configs are everywhere. And often such configs must be updated over time. In many cases, it is possible to simply override the entire config file, but what if not?
Here is the way for updating YAML configs preserving comments and all current values.
Preparing the tool
The tool could be used in different ways, but I'll show the most universal command line usage.
Download yaml-config-updater-cli-1.2.0-all.jar from maven central.
wget https://repo1.maven.org/maven2/ru/vyarus/yaml-config-updater-cli/1.2.0/yaml-config-updater-cli-1.2.0-all.jar
Java 8 or above must be installed in the target environment.
Adding missed properties
Suppose your config is:
section:
# Important description
# Value was manually updated (must be preserved)
prop1: 12
In the new version, new property was added:
section:
# Important description
prop1: # value must be put manually after installation
# New description
prop2: 10
Performing migration:
java -jar yaml-config-updater-cli-1.2.0-all.jar config.yml update.yml
Output report:
Updating configuration: /home/user/test/config.yml
Configuration: /home/user/test/config.yml (105 bytes, 5 lines)
Updated from source of 137 bytes, 6 lines
Resulted in 93 bytes, 7 lines
Added from new file:
section/prop2 6 | prop2: 10
Backup created: config.yml.20210911140958
As a result, the new property would be added to the current config:
section:
# Important description
prop1: 12
# New description
prop2: 10
Note that the prop1
comment was also changed, as comment from updating file may contain important information updates. Only in-line comments (after value) survive merging.
If you don't trust the report you can easily build diff with the created backup file (original config):
diff config.yml.20210911140958 config.yml
5a6,8
> # New description
> prop2: 10
>
The trust
Good question: why should you trust the tool?
The tool is very young and most likely there are still bugs and uncovered cases.
And still, I can guarantee that the resulted file would be correct:
- Internally, time-prooved snakeyaml parser used for all files validation. This way "comments" parser can be sure to always work with the correct YAML.
- But that's not all. Parsed trees from comments and snakeyaml parsers are compared to eagerly detect incorrect parsing (self-validation)
- Merge result is read by snakeyaml to make sure the file is correct.
- Merged file values are carefully checked: all old values must be preserved and all new values added (using snakyaml trees to operate exact values).
So you can be sure that the resulted config would be correct. If anything goes wrong, the configuration file would not be touched at all.
Removing values
Adding new properties is not the only case: some properties might be removed in the actual configuration.
By default, the tool would not remove properties, so the merged file would still contain them. In order to remove a property, you must explicitly specify it.
For example, suppose in config:
prop1: 1
sub:
foo: 2
bar: 3
prop1
and foo
are no longer needed.
java -jar yaml-config-updater-cli-1.2.0-all.jar config.yml update.yml -d prop1 sub.foo
Updating configuration: /home/user/test/config.yml
Configuration: /home/user/test/config.yml (36 bytes, 6 lines)
Updated from source of 11 bytes, 2 lines
Resulted in 18 bytes, 4 lines
Removed from old file:
prop1 1 | prop1: 1
sub/foo 4 | foo: 2
Backup created: config.yml.20210911144139
This may be also used for replacing values: removed property would be replaced with a property from updating config with the default value.
You can remove entire sub-trees this way.
Config-template
Another common need is configuration "adoption" for the target environment. In this case, updating configuration become "a template" with placeholders to replace by environment-specific values:
prop1: #{one}
prop2: #{two}
prop3: #{three}
java -jar yaml-config-updater-cli-1.2.0-all.jar config.yml update.yml -e one=1 two=2
Also, this time, updating configuration does not exist yet (first installation case).
Updating configuration: /home/user/test/config.yml
Not exising configuration: /home/user/test/config.yml
Updated from source of 33 bytes, 3 lines
Resulted in 34 bytes, 3 lines
Applied variables:
two = 2
one = 1
New configuration copied as-is
The resulted config would be:
prop1: 1
prop2: 2
prop3: #{three}
Note that the third variable was not processed (as the value was not specified), but it is not a problem as any YAML parser would treat this placeholder as an in-line comment.
Environment variables might also be specified in a property file (and multiple files could be specified). By default, system environment variables are all provided (use -i
option to see the list of all available variables during processing).
Links
This was just a brief introduction. More info could be found in github repository: