share staging infrastructure

[kevinjqliu]

Uses .asf.yaml to autobuild and autostage pelican builds site/* branches.

This way, multiple branches can stage previews at the same time.

For example:

• Push branch site/foo to autobuild, the output goes to site/foo-staging.
• site/foo-staging matches site/* in the autostage command.
• The site is staged as datafusion-foo.staged.apache.org.

Here's the code reference for options:

• pelican
• staging

Note, .github/workflows/stage-site.yml is no longer used. Staging is handled by .asf.yaml instead of the github action.

[kevinjqliu]

ok CI now fails with

remote: Permission to apache/datafusion-site.git denied to github-actions[bot].
fatal: unable to access 'https://github.com/apache/datafusion-site/': The requested URL returned error: 403
Error: Process completed with exit code 128.

good thing is this failed at the last step to push to repo

[kevinjqliu]

Looking at past successful runs are all from either andygrove, alamb, or timsaucer, i wonder if theres a permission element

The CI log's "Set up job" step shows the following

GITHUB_TOKEN Permissions
  Actions: read
  Attestations: read
  Checks: read
  Contents: read
  Deployments: read
  Discussions: read
  Issues: read
  Metadata: read
  Models: read
  Packages: read
  Pages: read
  PullRequests: read
  RepositoryProjects: read
  SecurityEvents: read
  Statuses: read

So i think I dont have permission to push to the asf-staging branch, even in CI

[kevinjqliu]

@alamb could you maybe push an empty commit to this branch? Or if its easier, add a github suggestion and apply it. That should trigger CI using your github user

[alamb]

@alamb could you maybe push an empty commit to this branch? Or if its easier, add a github suggestion and apply it. That should trigger CI using your github user

Done -- pushed 0a54840

[alamb]

I wonder if we could use github pages or something in the local fork. That would require setup for the other users but we wouldn't have permission problems 🤔

[kevinjqliu]

I found an interesting way to stage multiple branches to https://datafusion.staged.apache.org

The folks at asf infra has already thought of this
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405038#Git.asf.yamlfeatures-Automaticallystagingnewbrancheswithadynamicprofile
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=127405038#Git.asf.yamlfeatures-Automaticallybuildingnewbranches

it looks like the site/foo branch will be staged at datafusion-foo.staged.apache.org

This still requires branch prefixed with site/ to be pushed to the apache repo, which only the committer are allowed to do. But at least multiple branches can be hosted at the same time.