When zx first appeared, it was a tiny esm script that just proposed a new idea for how child_process.spawn API could be enhanced with string template literals.
const result = await $`call smth --with any ${args}`
As we now know, this was exactly what JS folks had been looking for for many years :-). The main goal was to improve the developer experience, so we focused on finding the best combination of utils to solve everyday problems, expand the area of application.
Short history
The project went through 8 major releases, but in the context of the assembly issues, there are only 3 significant ones.
Packed as CommonJS
Despite the fact that the future belongs to ESM, many users have to remain in the notation of their legacy codebase (without any irony). So we have added compatibility with the require API for them.
https://github.com/google/zx/commit/303feea3cd0d95e8d733ee72b96b3316ea5827aa
Dropped CJS support
On the other hand, there were libraries on which zx depend: and only the latest modern major versions received patches, including vulnerabilities fixes. In fact we inherited CVE issues transitively, which was completely unacceptable. And we switched back to ESM.
https://github.com/google/zx/commit/5578d67ebae2049d430bf6489784f6a2aa47b0b1
Migrated to TypeScript
As the code base grew, it became harder for us to ensure consistency of internal contracts. Adding typing was the solution, but it also made us dependent on transpilation and we could no longer control the package artifact in detail.
https://github.com/google/zx/commit/8c2aea89fe7c65767d29c52bfb48f6a46af01864
Problems
Size
Filling the zx with new functionality, we continued to expand the dependency tree, which required their own dependencies, etc. Installation time increased significantly. And at some point it became impossible to ignore: zx#712, zx#669
| 1.15.2 | 2.1.0 | 3.1.0 | 4.3.0 | 5.3.0 | 6.2.5 | 7.2.3 |
|---|---|---|---|---|---|---|
| 1.38 MB | 3.01 MB | 3.01 MB | 3.89 MB | 10.5 MB | 10.7 MB | 16.3 MB |
Compatibility
When we define node engine requirement we limit not users, but ourselves in what platform API we can use. But this in no way prevents our dependencies from pursuing their own policies which affects zx.
Stability & Security
Reproducible setups are important. Especially for utilities of our type. We can try to pin down versions, but if there is a caret (^, ~) or an asterisk (*) at the secondary level, this introduces node_modules variability. Lockfiles can mitigate the problem, but not for npx/yarn dlx execution case.
Bundle
At first we wanted to just get rid of all the helper utilities. Keep only the kernel, but this would mean a loss of backward compatibility. We needed some efficient code processing instead with recomposition and tree-shaking. We needed a bundler. But which one? Our testing approach relies on targets, not sources. We rebuilt the project frequently, speed was critical requirement. In essence, we chose a solution from a couple of among all available alternatives: esbuild and parcel. Esbuild won. Specifically in our case, it proved to be more productive and customizable.
js
Customizable means that we found a way to adapt the tool for our tasks. And a lot had to be improved, including writing our own plugins:
Esbuild provides code-splitting with dynamic imports to reuse common code chunks. But not is static form. The manual chunks API is missing.
esbuild-plugin-entry-chunksBy default, esbuild injects helpers into each cjs module. It's fine, when you have just a few, but definitely not when there are many.
esbuild-plugin-extract-helpersWe also wanted to bring back cjs support, but at the same time avoid duplicating code in two versions of the bundles.
esbuild-plugin-hybrid-exportEsbuild and its plugins mostly focus on sources processing, but sometimes additional modification is also necessary for dependencies or bundles:
- Polyfill injects: esbuild#2840, esbuild#3517, esbuild#3099
- Custom patches: esbuild#3360
- Dynamic banners: esbuild#3291
- Importable helpers: esbuild#1230.
d.ts
While we were fighting against the modules, we forgot one small detail - their built-in typings. Esbuild can't do this at all yet. Unbelievable, but the tsc, native TS compiler, also does not provide a typings concat feature. Got around this problem: we've introduced a utility to combine typings of zx own code, and applied some monkey patches for external libdefs squashed via dts-bundle-generator.
Results
There is no silver bullet, all tools and approaches have limitations. Summary:
- Install size reduced: 16Mb → 1Mb
- Reproducible npx invocations
- Hybrid package: ESM and CJS
- Wide range of supported Node.js versions 12...22
- Easy to audit: complete code in one place
Perhaps our efforts are not commensurate with the profit. But we gained invaluable experience. Now we understand the entire zx code in detail down to each char.
zx
#!/usr/bin/env zx
await $`cat package.json | grep name`
let branch = await $`git branch --show-current`
await $`dep deploy --branch=${branch}`
await Promise.all([
$`sleep 1; echo 1`,
$`sleep 2; echo 2`,
$`sleep 3; echo 3`,
])
let name = 'foo bar'
await $`mkdir /tmp/${name}`
Bash is great, but when it comes to writing more complex scripts,
many people prefer a more convenient programming language.
JavaScript is a perfect choice, but the Node.js standard library
requires additional hassle before using. The zx package provides
useful wrappers around child_process, escapes arguments and
gives sensible defaults.
Install
npm install zx
Documentation
Read documentation on google.github.io/zx.
License
Disclaimer: This is not an officially supported Google product.
Top comments (0)