NUT Maintainer Guide
____________________
:Author: Arnaud_Quette_and_Jim_Klimov
:Author Initials: AQ & JK

Introduction
============

//////////////////////////////////////////////////////////////////////////////
NOTE: This file is currently not delivered in tarballs nor spellchecked.
//////////////////////////////////////////////////////////////////////////////

...

Mailing lists administration
============================

NUT provides various
link:https://alioth.debian.org/mail/?group_id=30602[mailing list], to support
users and developers. These can administered at the following addresses:

- link:http://lists.alioth.debian.org/cgi-bin/mailman/admin/nut-upsuser[Nut-upsuser]
- link:http://lists.alioth.debian.org/cgi-bin/mailman/admin/nut-upsdev[Nut-upsdev]
- nut-tracker
- nut-packaging

The password is the same for all administrators, and is provided to any new
NUT admin.

Best moderation practices
-------------------------

These are the general rules that apply to mailing list moderation:

- non subscribed:
  - ACCEPT: complete entry submission (ie no need for further mail)
  - ACCEPT: foreign reply to cross mailing list post (ex: reply from FreeIPMI
    Al Chu to a mail from Arnaud on upsdev...)
  - ACCEPT: people that are already members, with another address.

- spams: always DISCARD, not REJECT, and apply a general ban to this address.

- big messages:
  - source code commits: forward the header, which mentions the changes, but
    not the actual changes details.
  - others (users feedback): ask for attachment compression, or using a link
    to store the file(s).

- other (ie report or request requiring more than 1 mail): REJECT with a
  message explaining the reason. The following can serve as a base:

	Dear XXX,
	Your message to the nut-upsXXX mailing was rejected because you must
	suscribe to the mailing list. This is just to eradicate spam noise from
	the mailing list.

	Use the following link to subscribe to this mailing list:
	https://lists.alioth.debian.org/mailman/listinfo/nut-upsXXX

	where 'XXX' can be replaced by 'user', 'dev' or 'packaging'.

	NUT maintainers


//////////////////////////////////////////////////////////////////////////////
!! DRAFT !!

Release process
===============

New process:
- we will only work on the trunk for the day to day bugfixing and
  standard modifications (what was mostly happening in Testing
  currently),
- the trunk will be used to generate the testing releases (only using
  the tags, after a small freeze period),
- bigger changes, invasive modifications and cutting edge developments
  will have to be addressed in separate branches, until stabilization.
  When things are ok and validated to enter the trunk, merging these
  branches into the trunk can happen.

I insist on the *validation* to enter the trunk, since some changes
might have to wait for major releases, to match our current release
process.

MAINTAINER SANDBOX (to be completed and pushed)

* be sure to proceed with the below steps on a system provisioned with all
  possible build prerequisites for NUT, especially regarding documentation
  and nut-website building and checking, and with maintainer GPG keys in
  the chain

NOTE: DO NOT RUSH to issue the release right away. There are always some
last-minute discoveries and days/weeks of delays before a perfect release
does ultimately happen. Use `vX.Y.Z-rcN` tags until completely certain that
a release is final (perhaps by tagging the last RC with `vX.Y.Z` later).
Note that some third-party tools like PyPI repo do not permit replacing
artifacts associated with a particular release version/tag, and changing
Git tags on public repositories is generally considered as a bad practice.
So best to avoid it in the first place.

NOTE: Take time to push release candidates also as one of `fightwarn*` git
branches (maybe with QEMU scenarios too), and maybe post a `FTY` branch
update (while it is not all upstreamed), to gather more insights into code
and recipe quality from scenarios that do not often run. Also revise if
CodeQL or similar tools on GitHub have posted any relevant suggestions
about security or other matters.

* clean up "in-development" bits from files (which we would revert after
  release), e.g.:
** DO NOT yet remove "PLANNED:" from the new release section title
** revise (remove if not implemented) the TODO etc. entries referring to
   planned future in the `NEWS.adoc` and `UPDATING.adoc` files -- except
   for what was actually completed the upcoming release
** comment away the top-most (auto-resolved, templated like `@PACKAGE_VERSION@`)
   NUT version and build date in `docs/docinfo.xml.in` -- DO NOT add (or at
   least commit) an entry for the actual fixed release version and date just yet
** NOTE: The `docs/docinfo.xml.sh` script can help update the list for
   existing git tags, if some were skipped before. This can be used to
   cheat a bit about adding the new entry (with a draft tag).
** commit this change (to add a revert-commit after the release tag), e.g.:
+
----
:; git commit -sm 'NEWS.adoc, UPGRADING.adoc, docs/docinfo.xml.in: finalize text before NUT v2.8.0 release'
----

* revise the lists of symbols exported by shared libraries delivered by NUT:
  some changes may be seen right in the respective `Makefile.am`, but since
  we use regular expressions, the set of method names may change between
  releases implicitly. Use `nm` or similar tools to review symbols exposed
  on an older build and the newer ones. For version bump algorithm consult
  https://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
  (since we are likely to change common source code between releases, the
  "revision" is likely to increment always, to reflect that the library was
  rebuilt from different sources -- unless we bump "current" instead and
  reset "revision" to 0 due to new symbols).
+
TODO: Maintain proper library scripts (maybe in Git) with exact lists of
symbol names and API versions, similar to what `GNUC` or `SUNW` libraries do.

* revise the contents of `NEWS.adoc` and `UPDATING.adoc` files; verify that
  any recent changes to drivers (including `main.c` and other major impact
  from common code) and sub-drivers (`*-hid.c` for `usbhid-ups`, `*-mib.c`
  for `snmp-ups`, `nutdrv_qx_*` etc.) have been reflected in bumps to their
  `DRIVER_VERSION` or equivalent macros
** ideally maintained during development, as features are getting merged for
   community testing and future development baseline in the master branch
** for the last pre-release check, this can help match changes in code and
   metadata (note that not all changes are worth reflecting in a version bump,
   e.g. non-functional code style and comment improvements):
+
----
:; git diff v2.8.4 drivers/*.c 2>&1 | egrep '(^diff |_VERSION)'
----
** this is the good time to remove the `PLANNED` status from the upcoming
   release info section title and commit it separately, e.g.:
+
----
:; git commit -sm 'NEWS.adoc: remove "PLANNED" from section title before NUT v2.8.0 release'
----
* NOTE that the `ChangeLog` file is currently not tracked in SCM
* update this document: `docs/maintainer-guide.txt` as it inevitably requires
* commit these finishing touches

* add an entry in `docs/docinfo.xml.in` for the actual fixed release version
   and date, focusing on what was changed since last release documentation-wise;
+
----
:; git commit -sm 'docs/docinfo.xml.in: add docs increment summary for NUT v2.8.0 release'
----

* bump the release identification (consider committing as one commit later on):
** see `docs/nut-versioning.adoc` for details about NUT SEMVER spec and practice
** revise `.github/workflows/08-PyNUTClient.yml` for fallback `TAG_NAME` naming
** revise `appveyor.yml` for branch naming
** revise `scripts/Windows/build-mingw-nut.sh` for fallback value of `VER_OPT`
** revise `scripts/obs/debian.changelog` (at least a dummy entry)
** revise `scripts/obs/nut.dsc` and `scripts/obs/nut.spec` for their `Version`
   fields
** update the fallback `NUT_DEFAULT_VERSION` in `tools/gitlog2version.sh` to
   <exact incremented version> (ex: `2.8.0`)
*** DOUBLE-CHECK that the `NUT_DEFAULT_VERSION` in `tools/gitlog2version.sh`
    is an `X.Y.Z` triplet, without the fourth `.1` suffix, for the release!
*** Note that since NUT v2.8.3 this script dictates the version propagated by
    the `configure` script, so direct changes of `AC_INIT` there are no longer
    needed
** commit with a relevant release message, e.g.:
+
----
:; git commit -sm 'Update versions for release of NUT v2.8.0'
----
** provide the `VERSION_FORCED` and `VERSION_FORCED_SEMVER` files (to same
   effect as fallback in `tools/gitlog2version.sh` but more explicitly and
   visibly) in the local workspace to be added to the `dist` archive tarball
   file (DO NOT add them to git to avoid confusion later, or revert any such
   addition after the release -- still has a chance to confuse e.g. branches
   made from release tag later):
*** Automation was introduced to help maintain these files, e.g.:
+
----
# Current commit MUST be git-tagged for *-(rc|alpha|beta) for magic to work:
:; git tag v2.8.3-rc7

:; NUT_VERSION_QUERY="UPDATE_FILE_GIT_RELEASE" ./tools/gitlog2version.sh

# Output sample wrapped for readability:
SEMVER=2.8.3; TRUNK='upstream/master'; BASE='735451f1f21556f2a7b8443053e8962c2184239f';
DESC='v2.8.2-2881+gbdade6241' => TAG='v2.8.2' + SUFFIX='-2881+gbdade6241+v2.8.3+rc7'
 => VER5='2.8.2.2878.3' => DESC5='2.8.2.2878.3-2881+gbdade6241+v2.8.3+rc7'
 => VER50='2.8.2.2878.3' => DESC50='2.8.2.2878.3-2881+gbdade6241+v2.8.3+rc7'

# Note it assigns SEMVER from the pre-release tag, but other git info from
# real git-describe details

# You can verify current contents of these files with:
:; grep . VER*
VERSION_FORCED:NUT_VERSION_FORCED='2.8.2.2878.3-2881+gbdade6241+v2.8.3+rc7'
VERSION_FORCED_SEMVER:NUT_VERSION_FORCED_SEMVER='2.8.3'
----
*** An older, possibly more reliable, way would go like this:
+
----
# Do not let them be easily deleted by e.g. `git clean -fdX` while you
# would inevitably iterate the release attempts - keep in a subdir, and
# juggle hardlinks (so they are archived into the tarball as real files):
:; rm -rf .relver
:; mkdir -p .relver

# Generally may be the nuanced NUT SEMVER; is a triplet for releases
# or suffixed for release candidates
:; echo 'NUT_VERSION_FORCED="2.8.3"' > .relver/VERSION_FORCED
:; echo 'NUT_VERSION_FORCED="2.8.3-rc1"' > .relver/VERSION_FORCED

# Should always be a triplet
:; echo 'NUT_VERSION_FORCED_SEMVER="2.8.3"' > .relver/VERSION_FORCED_SEMVER

# Do not let them be easily lost; later you can just re-create these links
# if file names in your top_srcdir are removed:
:; ln .relver/* .
----

* last-minute update against possible master-branch changes (and be sure to
  apply the release-version changes described above to your local copy of
  the `master` branch, even if originally staged in another):
+
----
:; git fetch --all && git rebase upstream/master -i --autosquash
----
* if you had prepared the documentation changes listed above in advance,
  revise the `docs/docinfo.xml.in` file to enter the actual release date
* run the last-minute build to be sure the release is sane (no typos in any
  recently changed document files, log-processing code behaves sanely when
  current commit is *the* release tag, etc.) with `./ci_build.sh` or full
  ritual; keep an eye on the build progress to catch any warnings or recipe
  errors:
+
----
:; rm -f Makefile configure VERSION_DEFAULT
:; ./autogen.sh && \
   ./configure --with-all --with-dev --with-doc --enable-spellcheck \
      --enable-docs-man-for-progs-built-only=no                     \
      --enable-warnings --enable-Werror --enable-maintainer-mode && \
   make -j 8 all && \
   make -j 8 spellcheck && \
   make -j 8 distcheck && \
   make -j 8 distcheck-completeness
----
* create an annotated GPG-signed tag v<incremented version> (ex: `v2.8.0-rc1`):
+
----
:; git tag -asm 'Release NUT v2.8.0' v2.8.0-rc1
----
* push the tag to a developer's private repository first; wait for CI
  builds to complete without hiccup (test-PyPI push, GHA scanning, etc.)
** in case of second thoughts, `git tag -d v2.8.0-rc1` and retry later
** try to avoid adding signed tags later (ex. v2.8.0-signed) to avoid the
   mess in GitHub release URLs (or do amend that post-factum), for more
   details see e.g. https://github.com/networkupstools/nut/issues/1971
* only when the tagged builds seem viable, push the RC tag to upstream
  NUT repository; eventually create the final release tag like `v2.8.0`
* don't forget to push not only the code, but also the tag:
+
----
:; git push upstream && git push upstream v2.8.0
----
** avoid `git push --tags` so your experimental tags do not pollute common
   NUT repository

* `make dist-files` (especially if you did not `make distcheck` above,
  or had some changes since then) to store the source tarball, checksum
  and signature files
** Bonus feature: `make dist-docs` (if on a system configured to build all
   documentation types)

* post-release update of the "in-development" codebase:
** start a feature branch to return the master branch into development state,
   e.g. `git checkout master && git checkout -b post-release-2.8.0`
** update the fallback `NUT_DEFAULT_VERSION` in `tools/gitlog2version.sh` to
   `<incremented version>.1` (ex: `2.8.0.1`) and commit this, e.g.:
+
----
:; git commit -sm 'tools/gitlog2version.sh: for development, set fallback NUT_VERSION_DEFAULT=2.8.0.1'
----
** remove the `VERSION_FORCED` and `VERSION_FORCED_SEMVER` files from your
   workspace
** `git revert --signoff` the commit which cleaned up "in-development" bits
   above (the one with `...finalize text before NUT v2.8.0 release` comment)
*** Possibly resolve relevant merge conflicts for the changed context,
   e.g. the changed "PLANNED" status of the now-issued release info
*** If there are merge conflicts, be sure to sign off the ultimate commit
    when you fix those
** Ensure that `NEWS.adoc` has a `PLANNED: Release notes for...` section
   with a new URL to GitHub "milestone" for the next release, and unfinished
   "expected" points pushed back from the previous release (if any)
** Ensure that `UPDATING.adoc` has a `Changes from ... to ...` section for
   the new couple of release numbers
** `git commit` if there were any changes after that `git revert`
** Initiate a PR to `Post-release 2.8.0 rituals: begin next cycle` with this
   branch, but do not merge it yet

* Update `nut-website`:
** rendering should auto-update by NUT CI farm based on source changes,
   but in case of urgency -- can be expedited with a prepared workstation
   (see README of that project) or from the NUT CI farm agent directly
** add an entry to `news.txt`
** update `nut` and `ddl` submodules in nut-website/ to refer to latest info
   as of current release (this should update the website's version as well).
   NOTE: For `nut` submodule be sure to refer to the tagged commit, not to
   the subsequent "in-development" codebase.
** in `source` submodule add a copy of tarball, checksum and hash files for
   download: `make dist{,-hash,-sig}` (or since NUT v2.8.3, `make dist-files`)
*** for a `new-X.Y.Z.txt` file use the `NEWS.adoc` with a stripped title
    and prominent `link:` asciidoc markup (see existing files), e.g. strip
    `link:UPGRADING.adoc[]` markup => `UPGRADING.adoc`, or generally speaking:
+
----
:; sed -e 's,link:\([^ ]*\)\[\],`\1`,g' -i new-2.8.0.txt
----
*** Check that the `new-X.Y.Z.txt` file DOES NOT include the `PLANNED:...`
    title and TODO list for the future release, if restored in the steps above
*** Copy over the `ChangeLog` file (original variant, not `.adoc`) and fix the
    commit range in title to end with the new release tag name (not `HEAD`)
** update `historic/index.txt` with an entry for the new release tag and date
** commit content changes and tag the release in submodules: `source`, `package`
   and `nut-website` repos likely touched above, and `ddl` for good measure:
+
----
:; git commit -sm 'NUT v2.8.0 released'
   git tag -asm 'NUT v2.8.0 released' v2.8.0 \
   && git push \
   && git push origin v2.8.0
----
** add and commit sub-module references (pointing to their `master` branches
   checked out and prepared for the release):
+
----
:; for R in nut ddl source package ; do
       (cd $R && git checkout v2.8.0) || break
       git add -p $R
   done \
   && git commit -sm 'NUT v2.8.0 released - update submodule references'
----
** tag the `nut-website` release, e.g.:
+
----
:; git tag -asm 'NUT v2.8.0 released' v2.8.0 && git push origin v2.8.0

# NOTE: Maybe `git push upstream v2.8.0` for nut-website directory itself
----
** generate and publish a "historic" sub-site snapshot (currently manually,
   check comments in nut-website's `ci_build.sh` script and/or its README file)
+
----
# Verify against nut-website README actual when publishing, last time it was:
:; CI_AUTOCOMMIT=true CI_AUTOPUSH=true NUT_HISTORIC_RELEASE=v2.8.5 ./ci_build.sh
----
*** Note that the historic site generation routine modifies `docs/` contents
    in the `nut` workspace (to inject "historic vs. development" notes)
** update `nut` submodule to current commit ("in-development" codebase), and
   the `historic/index.txt` to refer to the snapshot for reference (users of
   specific-version packages)
** generate and publish the usual website revision (by CI or manually)
** Note that it may be needed to manually copy the new tarball-related files,
   `new-X.Y.Z.txt`, change log, and maintainer keys (if updated), into the
   rendered website repository, even if using the CI scripted publication,
   then `git commit` and `git push` these changes.

* check that the website renders properly (a few minutes after publication)
** re-check the tarball signature according to current `security.txt` document
   on an empty VM/container/chroot/user-home setup, to make sure that the GPG
   instructions fit the reality (needed maintainer keys are published, etc.)
** `git tag` the published website release

* draft and publish a GitHub release based on the signed tag
** attach *the same copy of* the tarball, checksum and hash files as published
   on nut-website (prepared by `make dist-files` above)
** when https://ci.appveyor.com/project/nut-travis/nut is done building the
   release commit (takes about an hour, more if the build queue is congested),
   download and rename the 7z archive, e.g.:
+
----
:; mv NUT-for-Windows-x86_64-SNAPSHOT-2.8.3.3028-master.7z \
      NUT-for-Windows-x86_64-RELEASE-2.8.3.7z
----
+
and attach it to the GitHub release artifacts, and to nut-website via
   the https://github.com/networkupstools/nut-package-archive.git repository
   (its `index.html` will get updated as part of subsequent nut-website build,
   a templated reference from NUT Downloads page is already in place).
+
FIXME: get our AppVeyor recipe to name the release archive as such, the scripts
should already know everything needed for this.  Explore git-tag based builds?
+
FIXME: eventually get real installer files made, publish them then.

* Make sure that the PyNUT module for the tagged release got published to
  the main PyPI repository, as a build associated with the NUT release
  version. This *should* be ensured by the GitHub actions, but the curent
  workflow definition uses both `paths` and `branches`/`tags` triggers and
  this may cause a release to be skipped if the PyNUT sources did not change.
  The `scripts/python/module/Makefile.am` recipes automate the needed actions
  to happen from a prepared maintainer's operating environment.

* (After CI builds complete the exact release commit's build)
  Push/Merge the updated commits for development to master branch;
  until then can cache it as a side branch and PR though, merge later.

* announce on mailing list, IRC, etc.

* update https://en.wikipedia.org/wiki/Network_UPS_Tools

* create a GitHub issue label for the new release, e.g. at
  https://github.com/networkupstools/nut/labels?q=impact :
----
impacts-release-2.8.0
Issues reported against NUT release 2.8.0 (maybe vanilla or with minor packaging tweaks)
----

* Merge the PR to initiate the new release cycle (done last, as we probably
  would update this document along the way a few times)
** Do this before updating and merging any other PR's that might be queued
   and waiting for the new release (e.g. late-coming high-risk code changes)

//////////////////////////////////////////////////////////////////////////////
