ci: introduce towncrier to generate changelog from fragments

[MikeGoldsmith]

Description

Replace manual CHANGELOG.md editing with towncrier fragment-based changelog management. Each PR adds a small text file in .changelog/ instead of editing the changelog directly, eliminating merge conflicts.

This supersedes #4382 (PoC by @emdneto) — builds on that foundation with updates for current main, additional features informed by how pip, pytest, attrs, and Twisted use towncrier.

Fixes #4307
Fixes #4309

What changed

Core towncrier setup:

• pyproject.toml: towncrier config with 5 fragment types (added, changed, deprecated, removed, fixed)
• scripts/changelog_template.j2: custom Jinja2 template matching existing changelog format
• CHANGELOG.md: <!-- changelog start --> marker added above existing ## Unreleased section; existing entries preserved as-is and can be migrated to fragments at the next release

CI & workflows:

• .github/workflows/changelog.yml: uses towncrier check (per Twisted's approach) instead of grepping for CHANGELOG changes; rejects direct CHANGELOG.md edits; suggests tox -e new-changelog on failure
• .github/workflows/prepare-release-branch.yml: sed replaced with towncrier build
• .github/workflows/prepare-patch-release.yml: same + adds backport step to sync changelog to main

Developer experience:

• tox -e new-changelog -- PR_NUMBER TYPE "Description" to create a fragment
• tox -e changelog to preview generated changelog
• tox -e precommit checks for missing fragments via scripts/check_changelog_fragment.py
• CONTRIBUTING.md: changelog section with style guidance

Improvements over #4382

• Added deprecated and removed categories (original only had added/changed/fixed)
• Uses towncrier check (Twisted's approach) instead of manual shell script for CI validation
• Pre-commit integration for local fragment validation
• Style guidance in CONTRIBUTING.md (imperative tone, package prefix, <80 chars)
• towncrier added to dev dependency group
• Updated bot references to otelbot[bot]
• tox commands for fragment creation and changelog preview
• Based on current main (original PR was months stale)

Type of change

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How Has This Been Tested?

• towncrier build --draft --version "Unreleased" — verified fragment parsing and categorized output
• towncrier build --yes --version "1.42.0/0.63b0" — verified full build inserts into CHANGELOG.md at marker, deletes fragments
• towncrier check --compare-with upstream/main — verified branch-level fragment detection
• Manually verified scripts/check_changelog_fragment.py logic for main vs feature branch cases

Does This PR Require a Contrib Repo Change?

• Yes. — Contrib repo uses the same changelog workflow and should adopt towncrier too (tracked separately, this PR is core-first)

Checklist:

• Followed the style guidelines of this project
• Changelogs have been updated
• Unit tests have been added
• Documentation has been updated

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: emdneto / name: Emídio Neto (f664c36)
• ✅ login: MikeGoldsmith / name: Mike Goldsmith (0bd5867, 21c4873, 5ddece8, 988c2e7, 9f096aa, b9306e7, f664c36, fd3eef6)

[emdneto]

Awesome. Thanks for resurrecting this 👍🏻 Left some comments

[aabmass]

Awesome thanks Mike! I think we discussed it on slack but I can't remember the pro/cons of town crier vs chloggen.

[MikeGoldsmith]

Awesome thanks Mike! I think we discussed it on slack but I can't remember the pro/cons of town crier vs chloggen.

Both tools solve the same problem in a similar way - they replace a single CHANGELOG.md that causes merge conflicts with per-PR fragment files that get compiled at release time.

	towncrier	chloggen
Language	Python (pip install)	Go binary
Fragment format	Plain text file	Structured YAML (change_type, component, note, issues, subtext)
File naming	<PR#>.<type>	Auto-generated from git branch name
Validation	towncrier check (branch-level)	chloggen validate (validates against component list)
Used by	pip, pytest, attrs, Twisted	OTel Collector, Collector Contrib
Install in CI	pip install towncrier	Requires Go toolchain or pre-built binary

I suggested Towncrier because it's built for use with with python projects, chloggen is a custom OTel tool used by the collector and requires a go binary.

• Native to the Python ecosystem — pip install in CI, no Go toolchain needed
• Simpler contributor experience — create a text file with one line, vs filling out a YAML template with multiple fields
• Already widely adopted by major Python projects, so many contributors will already be familiar with the workflow
• chloggen's component validation is valuable for collector-contrib (200+ components) but less useful for core Python where all packages release together

I'm not against chloggen, they are very similar. chloggen would tie back into more OTel and has a stricter fragment strucure which would be nice.

[emdneto]

I'm really happy to see that we now have an agreement on this!

My suggestion is we wait for the next release and prioritize merging this ASAP so we can start with a very fresh changelog, and you won't need to map every already merged PR to an entry here. Also, I believe we need at least the same setup for -contrib so the contributor experience is the same in both repos.

[emdneto]

Can you add a new step here to build and print the changelog? Just for ourselves to see when we want, and to save time from running the command locally to see how it renders?