We have just released v3.4.8 — our last feature update before the next major release. Although this is a backwards compatible update release, it is packed with a set of useful features. Please let me share a few of them here. As always, you can find the full change log in the SpecSync Documentation.

After this release we will start the preparation for our next major release, v3.5, which is expected to be finalized in Q1 2024. During this period, the v3.4 line will only receive bug fixes and minor improvements. We have great plans for v3.5, including the complex Test Suite hierarchy synchronization solution, but we are planning another exciting feature for dealing with test results that we hope to reveal soon. Please check out our updated roadmap.

Keep synchronization conflicts under control

Most of the teams using SpecSync maintain a strict update policy. For example, they always change the scenarios in the source control only and never modify the synchronized Test Cases in Azure DevOps manually. With such rules, the synchronization is easier as they reduce the chance of conflicts.

For more complex development workflows, the modification of the tests might be required both in the file system (“local”) and in Azure DevOps (“remote”). In these situations more fine-grained control might be required for handling synchronization conflicts.

So far, interactive conflict resolution has only been available for the SpecSync “pull” command, but in this release we have introduced five different conflict resolution methods (force override, interactive, error, skip, ignore) and you can configure the desired method separately for “push” and “pull”.

This change introduces interactive conflict resolution for “push”, but with the other methods, you can also make your process safer even if you have a simple change policy. For example by setting the conflict resolution method to “error” for push, your pipeline will fail in the case there was an accidental modification on one of the synchronized Test Case. In such cases you can resolve the conflict and accept the local version by running the push command with the “–force” flag.

Please check out our Synchronization conflict resolution documentation page for further details.

Identify your Test Suites

Although the full-blown Test Suite hierarchy synchronization feature is only planned for v3.5, even the current version can do a lot with Test Suites: you can use them as “remote scope”, you can publish test results to a suite or even to multiple suites and you can also add synchronized Test Cases to various Test Suites based on tag or file path conditions.

For all these features you need to identify the suite in the SpecSync configuration. This might be tricky because Test Suite names are not expected to be unique. You can have multiple suites with the same name even within the same Test Plan. So far you have been able to identify the suites with name (if they were unique) or ID only and you could limit the search with a Test Plan ID only.

The v3.4.8 release introduces a new “testPlan” setting that can either contain a name (e.g. “Regression Plan”) or a Test Plan ID (e.g. “#4567”).

Also now you can identify your Test Suites within a Test Plan with a suite “path”. The path is like a file path of the nested suites starting from the Test Plan root. E.g. if you have multiple suites named “Card Payment”, but you would like to use the one that is within the Test Suite “Ordering”, you can use the path specifier “Ordering/Card Payment”.

When you use the new path specifier for the remote scope suite or with the “Add Test Cases to Suites” customization, SpecSync will create the suite hierarchy for you if it does not exist. So if you need to transfer your tests to a new Test Plan, you don’t need to worry about reconstructing the suite hierarchy manually.

The new “testPlan” and “path” specifiers can be used in all configuration sections where you need to specify a Test Suite.

Secrets of VSTest task

The Visual Studio Test task from Azure DevOps pipelines is commonly known as “VSTest” task. It’s an old hand in Azure DevOps: it has become somewhat obsolete since “dotnet test” arrived, but it still has a few unique features that is not provided by the new kid.

One of these unique features is the ability to re-run failed tests. It is very useful for system and integration tests depending on external resources that might not be available temporarily.

Another feature that I has not even been aware of until recently is the ability to run the tests distributed to multiple test agents with minimal configuration.

SpecSync can help you with publishing the test results even if you use VSTest, but it has to be configured slightly differently to overcome some of the strange things of VSTest, for example that it deletes all the local TRX files in some situations.

In v3.4.8, we have even introduced a new feature that is able to (re)publish tests results from another Test Run published by VSTest. This has been required to provide a complete support for the distributed execution scenario of VSTest.

We also have now a consolidated guide in our documentation that describes how to use SpecSync with VSTest in different situations.

Have you got an idea?

SpecSync has been growing out of the needs of the customers and we keep focusing on these needs with the improvements. So if you have an idea how SpecSync could do a better job or a feedback about any of the existing features, please let us know.

Happy synching!