Skip to content

Commit 6fd01eb

Browse files
tidousttidoust
authored
Bind auto-publish jobs to actual spec updates (#825)
Each time an update was made to any of the specs in the repository, all 16 specs got re-published because the auto-publish workflow did not know what was changed. This took time for no good reason. The [`paths` filter](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#onpushpull_requestpull_request_targetpathspaths-ignore) allows to run a workflow conditionally. Unfortunately, it only applies at the workflow level (and cannot be set at the job level within a workflow in particular). This adds an auto-publish workflow template and a companion script to generate auto-publish workflows for each and every spec in the repository, that only run when needed, meaning only when the spec source did change. The script only needs to run once in a while when new specs get added or when the workflows need to change. The new auto-publish workflows created by the script replace the previous `auto-publish.yml` workflow. Note: The `auto-publish.yml` workflow still had a `BUILD_FAIL_ON: nothing` property, which was meant to be a temporary measure and should no longer be needed: adfb63c This update switches back to `BUILD_FAIL_ON: warning`. This is the same approach as that done for w3c/encrypted-media: w3c/encrypted-media@cefbd73
1 parent dd799db commit 6fd01eb

19 files changed

Lines changed: 990 additions & 124 deletions

‎.github/auto-publish-template.yml‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
######################################################################
2+
# This is the template used to generate auto-publication workflows for
3+
# the different documents in the repository. Check generate script for
4+
# details. Edit this file and run the script again to update the
5+
# workflows.
6+
######################################################################
7+
8+
name: Auto publish {{shortname}}
9+
10+
on:
11+
# Worflow runs on pull requests where it makes sure that the spec can still be
12+
# generated, that markup is valid and that there are no broken links, as
13+
# well as on pushes to the default branch where it also deploys the generated
14+
# spec to the gh-pages branch and publishes the result to /TR.
15+
# The "workflow_dispatch" hook allows admins to also trigger the workflow
16+
# manually from GitHub's UI.
17+
pull_request:
18+
paths:
19+
- '{{source}}'{{additionalPaths}}
20+
push:
21+
branches: [main]
22+
paths:
23+
- '{{source}}'{{additionalPaths}}
24+
workflow_dispatch:
25+
26+
jobs:
27+
main:
28+
runs-on: ubuntu-latest
29+
steps:
30+
# See doc at https://github.com/actions/checkout#checkout-v2
31+
- name: Checkout repository
32+
uses: actions/checkout@v4
33+
34+
# See doc at https://w3c.github.io/spec-prod/
35+
# The action only deploys the generated spec to the gh-pages branch when
36+
# the workflow was triggered by a push to the default branch.
37+
- name: Build and validate spec, push to gh-pages branch and deploy to /TR if needed
38+
uses: w3c/spec-prod@v2
39+
with:
40+
TOOLCHAIN: bikeshed
41+
BUILD_FAIL_ON: warning
42+
SOURCE: {{source}}
43+
DESTINATION: {{destination}}
44+
GH_PAGES_BRANCH: gh-pages
45+
W3C_ECHIDNA_TOKEN: ${{ secrets.{{tokenName}} }}
46+
W3C_WG_DECISION_URL: https://github.com/w3c/media-wg/issues/27
47+
W3C_BUILD_OVERRIDE: |
48+
status: {{publicationStatus}}

‎.github/generate-auto-publish-workflows.mjs‎

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
/**********************************************************************
2+
* Companion script to create individual auto publication workflows for
3+
* specifications contained in the repository. Run the script any time
4+
* the auto-publish-template.yml is updated or when the list of specs
5+
* changes below.
6+
**********************************************************************/
7+
8+
/**
9+
* List of properties that can appear to describe a spec.
10+
*/
11+
const properties = [
12+
// Spec shortname (without level), e.g., 'webcodecs'
13+
// (Shortname does not *have* to be the actual published shortname, just
14+
// an identifier used to tell specs apart internally)
15+
// The shortname is derived from the source file name if not provided,
16+
// replacing "_" with "-"
17+
'shortname',
18+
19+
// Spec status on /TR, e.g., 'WD'
20+
'publicationStatus',
21+
22+
// Relative path to source file
23+
'source',
24+
25+
// Relative path to destination file in gh-pages branch (Optional)
26+
// If not provided, the destination file is computed from the source by
27+
// removing `.src`.
28+
'destination',
29+
30+
// Name of the repository secret that contains the publication token for
31+
// Echidna (Optional).
32+
// If not provided, the name is computed from the shortname, e.g.,
33+
// `ECHIDNA_TOKEN_WEBCODECS` for `webcodecs`
34+
'tokenName',
35+
36+
// Additional paths that should trigger the auto-publish script if changed
37+
// on top of the actual source (Optional). Glob patterns may be used.
38+
'additionalPaths'
39+
];
40+
41+
42+
/**
43+
* List of specs for which an auto-publish script needs to be created.
44+
*/
45+
const specs = [
46+
{
47+
shortname: 'webcodecs',
48+
source: 'index.src.html',
49+
publicationStatus: 'WD'
50+
},
51+
{
52+
source: 'codec_registry.src.html',
53+
publicationStatus: 'DRY'
54+
},
55+
{
56+
source: 'avc_codec_registration.src.html',
57+
publicationStatus: 'NOTE-WD'
58+
},
59+
{
60+
source: 'vorbis_codec_registration.src.html',
61+
publicationStatus: 'NOTE-WD'
62+
},
63+
{
64+
source: 'mp3_codec_registration.src.html',
65+
publicationStatus: 'NOTE-WD'
66+
},
67+
{
68+
source: 'aac_codec_registration.src.html',
69+
publicationStatus: 'NOTE-WD'
70+
},
71+
{
72+
source: 'flac_codec_registration.src.html',
73+
publicationStatus: 'NOTE-WD'
74+
},
75+
{
76+
source: 'opus_codec_registration.src.html',
77+
publicationStatus: 'NOTE-WD'
78+
},
79+
{
80+
source: 'av1_codec_registration.src.html',
81+
publicationStatus: 'NOTE-WD'
82+
},
83+
{
84+
source: 'vp9_codec_registration.src.html',
85+
publicationStatus: 'NOTE-WD'
86+
},
87+
{
88+
source: 'vp8_codec_registration.src.html',
89+
publicationStatus: 'NOTE-WD'
90+
},
91+
{
92+
source: 'pcm_codec_registration.src.html',
93+
publicationStatus: 'NOTE-WD'
94+
},
95+
{
96+
source: 'alaw_codec_registration.src.html',
97+
publicationStatus: 'NOTE-WD'
98+
},
99+
{
100+
source: 'ulaw_codec_registration.src.html',
101+
publicationStatus: 'NOTE-WD'
102+
},
103+
{
104+
source: 'hevc_codec_registration.src.html',
105+
publicationStatus: 'NOTE-WD'
106+
},
107+
{
108+
source: 'video_frame_metadata_registry.src.html',
109+
publicationStatus: 'DRY',
110+
tokenName: 'ECHIDNA_TOKEN_VIDEOFRAMEMETADATA_REGISTRY'
111+
}
112+
];
113+
114+
115+
/**
116+
* Main loop, create a workflow per spec
117+
*/
118+
import assert from 'node:assert';
119+
import { readFile, writeFile } from 'node:fs/promises';
120+
import { dirname, join } from 'node:path';
121+
import { fileURLToPath } from 'node:url';
122+
123+
const scriptPath = dirname(fileURLToPath(import.meta.url));
124+
let template = await readFile(join(scriptPath, 'auto-publish-template.yml'), 'utf8');
125+
template = template.replace(/#{5,}.*#{5,}/s,
126+
`######################################################################
127+
# IMPORTANT: Do not edit this file directly!
128+
#
129+
# This workflow was automatically generated through the
130+
# generate-auto-publish-workflows.mjs script. To update the workflow,
131+
# make changes to the template file and run the script again.
132+
######################################################################`);
133+
134+
for (const spec of specs) {
135+
if (!spec.shortname) {
136+
spec.shortname = spec.source.split('.')[0].replace(/_/g, '-');
137+
}
138+
if (!spec.destination) {
139+
spec.destination = spec.source.replace(/\.src/, '');
140+
}
141+
if (!spec.tokenName) {
142+
spec.tokenName = 'ECHIDNA_TOKEN_' +
143+
spec.shortname.toUpperCase()
144+
.replace(/-/g, '_')
145+
.replace(/_CODEC_/, '_'); // Tokens don't have "_CODEC_" in their name
146+
}
147+
if (spec.additionalPaths) {
148+
spec.additionalPaths = spec.additionalPaths.map(path => `\n - '${path}'`).join('');
149+
}
150+
151+
let content = template;
152+
for (const prop of properties) {
153+
content = content.replace(new RegExp('{{' + prop + '}}', 'g'), spec[prop] ?? '');
154+
}
155+
156+
const filename = join(scriptPath, 'workflows', `auto-publish-${spec.shortname}.yml`);
157+
await writeFile(filename, content, 'utf8');
158+
}

‎.github/workflows/auto-publish-aac-codec-registration.yml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
######################################################################
2+
# IMPORTANT: Do not edit this file directly!
3+
#
4+
# This workflow was automatically generated through the
5+
# generate-auto-publish-workflows.mjs script. To update the workflow,
6+
# make changes to the template file and run the script again.
7+
######################################################################
8+
9+
name: Auto publish aac-codec-registration
10+
11+
on:
12+
# Worflow runs on pull requests where it makes sure that the spec can still be
13+
# generated, that markup is valid and that there are no broken links, as
14+
# well as on pushes to the default branch where it also deploys the generated
15+
# spec to the gh-pages branch and publishes the result to /TR.
16+
# The "workflow_dispatch" hook allows admins to also trigger the workflow
17+
# manually from GitHub's UI.
18+
pull_request:
19+
paths:
20+
- 'aac_codec_registration.src.html'
21+
push:
22+
branches: [main]
23+
paths:
24+
- 'aac_codec_registration.src.html'
25+
workflow_dispatch:
26+
27+
jobs:
28+
main:
29+
runs-on: ubuntu-latest
30+
steps:
31+
# See doc at https://github.com/actions/checkout#checkout-v2
32+
- name: Checkout repository
33+
uses: actions/checkout@v4
34+
35+
# See doc at https://w3c.github.io/spec-prod/
36+
# The action only deploys the generated spec to the gh-pages branch when
37+
# the workflow was triggered by a push to the default branch.
38+
- name: Build and validate spec, push to gh-pages branch and deploy to /TR if needed
39+
uses: w3c/spec-prod@v2
40+
with:
41+
TOOLCHAIN: bikeshed
42+
BUILD_FAIL_ON: warning
43+
SOURCE: aac_codec_registration.src.html
44+
DESTINATION: aac_codec_registration.html
45+
GH_PAGES_BRANCH: gh-pages
46+
W3C_ECHIDNA_TOKEN: ${{ secrets.ECHIDNA_TOKEN_AAC_REGISTRATION }}
47+
W3C_WG_DECISION_URL: https://github.com/w3c/media-wg/issues/27
48+
W3C_BUILD_OVERRIDE: |
49+
status: NOTE-WD

‎.github/workflows/auto-publish-alaw-codec-registration.yml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
######################################################################
2+
# IMPORTANT: Do not edit this file directly!
3+
#
4+
# This workflow was automatically generated through the
5+
# generate-auto-publish-workflows.mjs script. To update the workflow,
6+
# make changes to the template file and run the script again.
7+
######################################################################
8+
9+
name: Auto publish alaw-codec-registration
10+
11+
on:
12+
# Worflow runs on pull requests where it makes sure that the spec can still be
13+
# generated, that markup is valid and that there are no broken links, as
14+
# well as on pushes to the default branch where it also deploys the generated
15+
# spec to the gh-pages branch and publishes the result to /TR.
16+
# The "workflow_dispatch" hook allows admins to also trigger the workflow
17+
# manually from GitHub's UI.
18+
pull_request:
19+
paths:
20+
- 'alaw_codec_registration.src.html'
21+
push:
22+
branches: [main]
23+
paths:
24+
- 'alaw_codec_registration.src.html'
25+
workflow_dispatch:
26+
27+
jobs:
28+
main:
29+
runs-on: ubuntu-latest
30+
steps:
31+
# See doc at https://github.com/actions/checkout#checkout-v2
32+
- name: Checkout repository
33+
uses: actions/checkout@v4
34+
35+
# See doc at https://w3c.github.io/spec-prod/
36+
# The action only deploys the generated spec to the gh-pages branch when
37+
# the workflow was triggered by a push to the default branch.
38+
- name: Build and validate spec, push to gh-pages branch and deploy to /TR if needed
39+
uses: w3c/spec-prod@v2
40+
with:
41+
TOOLCHAIN: bikeshed
42+
BUILD_FAIL_ON: warning
43+
SOURCE: alaw_codec_registration.src.html
44+
DESTINATION: alaw_codec_registration.html
45+
GH_PAGES_BRANCH: gh-pages
46+
W3C_ECHIDNA_TOKEN: ${{ secrets.ECHIDNA_TOKEN_ALAW_REGISTRATION }}
47+
W3C_WG_DECISION_URL: https://github.com/w3c/media-wg/issues/27
48+
W3C_BUILD_OVERRIDE: |
49+
status: NOTE-WD

‎.github/workflows/auto-publish-av1-codec-registration.yml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
######################################################################
2+
# IMPORTANT: Do not edit this file directly!
3+
#
4+
# This workflow was automatically generated through the
5+
# generate-auto-publish-workflows.mjs script. To update the workflow,
6+
# make changes to the template file and run the script again.
7+
######################################################################
8+
9+
name: Auto publish av1-codec-registration
10+
11+
on:
12+
# Worflow runs on pull requests where it makes sure that the spec can still be
13+
# generated, that markup is valid and that there are no broken links, as
14+
# well as on pushes to the default branch where it also deploys the generated
15+
# spec to the gh-pages branch and publishes the result to /TR.
16+
# The "workflow_dispatch" hook allows admins to also trigger the workflow
17+
# manually from GitHub's UI.
18+
pull_request:
19+
paths:
20+
- 'av1_codec_registration.src.html'
21+
push:
22+
branches: [main]
23+
paths:
24+
- 'av1_codec_registration.src.html'
25+
workflow_dispatch:
26+
27+
jobs:
28+
main:
29+
runs-on: ubuntu-latest
30+
steps:
31+
# See doc at https://github.com/actions/checkout#checkout-v2
32+
- name: Checkout repository
33+
uses: actions/checkout@v4
34+
35+
# See doc at https://w3c.github.io/spec-prod/
36+
# The action only deploys the generated spec to the gh-pages branch when
37+
# the workflow was triggered by a push to the default branch.
38+
- name: Build and validate spec, push to gh-pages branch and deploy to /TR if needed
39+
uses: w3c/spec-prod@v2
40+
with:
41+
TOOLCHAIN: bikeshed
42+
BUILD_FAIL_ON: warning
43+
SOURCE: av1_codec_registration.src.html
44+
DESTINATION: av1_codec_registration.html
45+
GH_PAGES_BRANCH: gh-pages
46+
W3C_ECHIDNA_TOKEN: ${{ secrets.ECHIDNA_TOKEN_AV1_REGISTRATION }}
47+
W3C_WG_DECISION_URL: https://github.com/w3c/media-wg/issues/27
48+
W3C_BUILD_OVERRIDE: |
49+
status: NOTE-WD

0 commit comments

Comments
 (0)