-
Notifications
You must be signed in to change notification settings - Fork 26
build(analyze-wasm, redact-wasm): use jco-transpile
#5102
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Merging to
|
|
Ah, Next/Webpack fails on some code that’s generated: 🤔 |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
|
The transpile options today are: So we could investigate whether we could use more of those |
analyze-wasm/build.ts
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't dug into this in-depth yet but my feeling was that we'd put this into our pipeline as a Rollup plugin.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’m a bit 🤷 on rollup: it currently doesn’t do much, it helps with WebAssembly, injects the current version, and builds .d.ts files. We could do more there. Or, perhaps we can do without it, and cut those dependencies?
In this case, these 2 duplicate scripts only needs to run when generating new WebAssembly, right?
So it could be a single script in the top-level monorepo?
Generating the version in a file could also be a simple node script.
And TS can generate types too (and we soon wouldn’t need it for our tests)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rollup will continue to be used because it is the basis of the zero-dependency philosophy. We will remove all non-arcjet packages and then we will produce a single bundle for each SDK/Adapter that will contain the packages in this repo. Trying to move things out of the rollup pipeline will make this harder in the future.
blaine-arcjet
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's build this as a rollup plugin to work towards the architectural goal of bundling adapters as a zero dependency package
34736c5 to
75a32d4
Compare
75a32d4 to
b1618ad
Compare
blaine-arcjet
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the approach here. After reviewing the implementation, I wanted to look at the codegen since we've been behind and I think I found some non-starters. We will likely need to contribute fixes upstream before we can land this change if we don't want to apply more workarounds after transpile but before file-write.
Notably:
- It looks like wasi-p2 shim is being applied. We'll need a way to disable.
- The
_debugLoglooks atprocess.envwhich will need to be removed. Promiseis shimmed, which would modify Arcjet users' runtimes.- Titus mentioned the types are not generated as well as they used to be.
- ...Maybe other stuff I missed
| } | ||
|
|
||
| if (!Promise.withResolvers) { | ||
| Promise.withResolvers = () => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we want to shim Promise in Arcjet user's runtime. This may be a non-starter unless we fix upstream.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I don’t see a reason why a library should do this.
It could be turned into a ponyfill function. Or alternatively, an option to turn it off?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| const _debugLog = (...args) => { | ||
| if (!globalThis?.process?.env?.JCO_DEBUG) { return; } | ||
| console.debug(...args); | ||
| }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is going to cause warnings in Next.js targeting the Edge runtime. We had a problem before where static analysis was warning on any process.env access.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There seems to not be no option to turn debug logging off.
That could be a way to go.
Another one is, there is an option to turn off/on node compat. Defaulting to off, it injects this code when on.
Perhaps, when rendering the debug log, that option could be looked at, to not inject process.env stuff?
package-lock.json
Outdated
| "bin": { | ||
| "wizer-win32-x64": "wizer" | ||
| } | ||
| "node_modules/@bytecodealliance/jco-transpile/node_modules/@bytecodealliance/preview2-shim": { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The wasi-p2 shim is still pulled in as a dependency so I worry that this is actually shimming WASI which we don't want to expose in our components since WASI allows for exiting the sandbox (which would be a security risk).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Passing wasiShim: false does not affect the output.
That is likely because it seems to be off by default: https://github.com/bytecodealliance/jco/blob/b3bc6266cfa521093afe8e563670a0b39d35715d/packages/jco-transpile/src/transpile.ts#L236
Since there is now a `jco-transpile` package with few dependencies (for 2 weeks), and that we’ve been stuck on an older `jco` to prevent package bloat, this PR attempts to unblock us by switching to the new package. It seems to work! Their package is at `0.0.4` though, and the types a bit broken, but things pass. All options except for `no-wasi-shim` that were used are also in `jco-transpile. I think that’s something that is wrapped around `transpile`. It could be some shim is injected though, I do not know enough about webassembly to verify whether some shim is in here. Related-to: bytecodealliance/jco#500. Related-to: GH-3446. Related-to: GH-4342. Closes GH-5101.
b1618ad to
3435f21
Compare
Since there is now a
jco-transpilepackage with few dependencies (for 2 weeks), and that we’ve been stuck on an olderjcoto prevent package bloat, this PR attempts to unblock us by switching to the new package.It seems to work! Their package is at
0.0.4though, and the types a bit broken, but things pass. All options except forno-wasi-shimthat were used are also injco-transpile. I think that’s something that is wrapped aroundtranspile. It could be some shim is injected though, I do not know enough about webassembly to verify whether some shim is in here.Related-to: bytecodealliance/jco#500.
Related-to: GH-3446.
Related-to: GH-4342.
Closes GH-5101.