Why does the MettleCI Agent Host require Microsoft Windows?
MettleCI Agent Host Responsibilities
The MettleCI documentation outlines the requirements for a Microsoft Windows-based host, known as the MettleCI Agent Host. This host provides essential capabilities for your CI/CD pipelines, enabling them to execute necessary steps. These capabilities include, but are not limited to:
Exporting and importing assets in DataStage repositories
Committing and pulling assets in Git repositories
Performing intelligent deployments of assets to target environments
Compiling DataStage jobs in a target environment
Executing DataStage jobs and unit tests
Running Compliance Rules in both fatal and non-fatal execution modes
Generating JUnit formatted XML test results as a by-product of compilation, unit test execution, and both compliance execution modes
Transferring test results back to your build platform, allowing it to provide user feedback on the CI/CD pipeline results and adjust its behavior accordingly (e.g., failing a CI/CD pipeline when test errors occur).
Many of the capabilities mentioned above are also available in non-Windows environments:
The DataStage Engine tier offers command line tools that fulfill some build pipeline capabilities.
Almost all MettleCI commands can be executed in non-Windows environments.
Git interactions are facilitated by your chosen build agent, which operates on non-Windows platforms.
However, some key capabilities are exclusively provided by the DataStage Client tier or by MettleCI commands that require DataStage Client tier capabilities. For these reasons, the MettleCI Agent Host necessitates the following components:
A build agent/runner for your selected CI/CD build platform (e.g., a Jenkins build agent)
(In certain scenarios, you may opt to utilize multiple MettleCI Agent Hosts.)
What about the Flow Designer REST API?
Users of DataStage v11.7.x may wonder if the Flow Designer REST API provides a viable alternative to the DataStage Client tier. The REST API does describe a non-client job compilation capability, for instance. However, there are several reasons why you should not consider building a CI/CD pipeline using this interface: The REST API:
Does not support Server jobs
Supports a large subset of DataStage v11.7 Parallel stages, but not all
Does not support all features/properties of the stages it claims to support
Does not produce the same output (i.e., OSH) as the Client tier compiler, which could result in compiled jobs that behave differently from those compiled with the Client tier compiler
Offers no capability to compile more than one job at a time
Is, to the best of our knowledge, no longer supported and is unlikely to receive any updates, enhancements, or fixes in the future
Provides a set of deprecated Git interfaces that we strongly recommend users avoid.
Other challenges
When considering alternatives to a Windows-based MettleCI Agent Host there are several pitfalls to avoid:
Your deployments would miss out on the incremental capabilities of the MettleCI command line, potentially leading to misconfigured target environments or, at best, significantly longer deployment times.
CCMT (the Connector Migration Tool, for use in upgrade scenarios) is only available on Windows.
You would need to design and construct a solution for the 'remote' execution of unit and compliance tests, providing functional equivalents of the MettleCI import, export, and deploy commands.
While you could execute unit and compliance tests, you would need to devise a solution for retrieving their JUnit XML responses from the execution environment and exposing those to your build tool.