Skip to content

Add code coverage reporting to public pipeline#1586

Merged
David-Engel merged 23 commits intodevfrom
david/codecov
Mar 20, 2026
Merged

Add code coverage reporting to public pipeline#1586
David-Engel merged 23 commits intodevfrom
david/codecov

Conversation

@David-Engel
Copy link
Copy Markdown
Collaborator

@David-Engel David-Engel commented Mar 13, 2026

This pull request introduces major improvements to code coverage collection, merging, and reporting in the CI pipeline for both Linux and Windows builds. The changes replace the previous Coveralls-based approach with Cobertura XML reports, implement artifact staging and publishing for coverage results, and add a new cross-platform coverage merging script. Several legacy diagnostic steps are removed for clarity and efficiency.

Code coverage collection and reporting improvements:

  • Switched from Coveralls to Cobertura XML coverage reporting for both Linux and Windows builds. On Linux, coverage is collected using gcovr, and on Windows, coverage is collected using OpenCppCoverage. [1] [2]
  • Added steps to stage and publish coverage artifacts for both platforms as pipeline artifacts, making coverage data available for downstream processing. [1] [2]
  • Introduced a new MergeCoverage job that downloads coverage artifacts from both platforms, merges them using the new buildscripts/merge_coverage.py script, publishes the merged coverage report, and uploads it to Codecov. [1] [2]
  • Note: Code coverage isn't collected on macOS. (1) It would slow down the already slow macOS run. (2) There is zero driver code difference between Linux and macOS, which means the coverage result wouldn't change.

Build and installation enhancements:

  • Improved ODBC driver installation on Linux by pinning unixODBC packages to prefer Ubuntu versions, avoiding conflicts with Microsoft repositories.
  • Updated ODBC Driver 18 download URL for Windows to use the latest link, ensuring up-to-date driver installation.
  • Refined Windows build steps to copy both DLLs and PDBs for sqlsrv and pdo_sqlsrv drivers, improving debugging and artifact completeness.

Cleanup and simplification:

  • Removed legacy Coveralls steps and several diagnostic scripts (such as PHP checks and ODBC driver version displays) to streamline the pipeline and reduce noise. [1] [2] [3] [4] [5]

New script for coverage merging:

  • Added buildscripts/merge_coverage.py, which normalizes file paths, deduplicates shared files, and merges Cobertura XML coverage reports from both Linux and Windows into a single unified report.

These changes modernize the CI pipeline's coverage reporting, improve artifact management, and simplify build steps, resulting in clearer, more actionable coverage metrics across platforms.

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.36%. Comparing base (e1eb5aa) to head (5d35e34).
⚠️ Report is 97 commits behind head on dev.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##              dev    #1586      +/-   ##
==========================================
+ Coverage   83.45%   85.36%   +1.91%     
==========================================
  Files          22       23       +1     
  Lines        7851     7128     -723     
==========================================
- Hits         6552     6085     -467     
+ Misses       1299     1043     -256     

see 45 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@David-Engel David-Engel changed the title Testing code coverage Add code coverage reporting to public pipeline Mar 17, 2026
@David-Engel David-Engel marked this pull request as ready for review March 18, 2026 21:06
@David-Engel David-Engel requested a review from jahnvi480 March 18, 2026 21:06
Comment thread azure-pipelines.yml
Comment on lines +565 to +572
# 1ES images' Microsoft repos have priority 1001, causing apt to prefer
# unixodbc 2.3.7 from Microsoft over 2.3.9 from Ubuntu. Pin unixODBC
# packages to prefer Ubuntu's versions.
printf '%s\n' \
'Package: unixodbc unixodbc-dev unixodbc-common libodbc1 libodbc2 libodbcinst2 odbcinst odbcinst1debian2' \
'Pin: origin packages.microsoft.com' \
'Pin-Priority: 100' \
| sudo tee /etc/apt/preferences.d/unixodbc-pin
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block isn't strictly necessary against the Azure Pipelines pool. I switched to a 1ES pool for Linux/Windows for quicker turnaround times when other teams were using all our Azure Pipelines agent slots (limit of 20). The 1ES images prioritize the MS package repo over Ubuntu, causing our lower unixodbc version to be preferred. Figured I would leave this in if we ever want to switch back to our own pool (say our public CI usage increased).

Copy link
Copy Markdown
Contributor

@jahnvi480 jahnvi480 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@David-Engel David-Engel merged commit 6b2c33e into dev Mar 20, 2026
11 checks passed
@David-Engel David-Engel deleted the david/codecov branch March 20, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants