Snowpack with Fluid Framework
I first heard about Snowpack as a faster Webpack, but later as part of a large toolchain ecosystem. I vaguely knew that Snowpack was making a big bet on ES modules.
Little did I know, my friend Fred Schott built the tool! Mind blown. I chatted with him after writing this and he offered a few thoughts. I'll list them at the end.
I was particularly interested in how Snowpack and Skypack, a related project, could be used to make the Fluid Framework's Container and Code Loading model easier to reason over and easier to develop. I also suspected Snowpack would speed up our developer inner loop, which is currently affected by some classic monorepo sluggishness. (Curtis Man even made us our own “build:fast” command, which may seem like overkill but works quite nicely).
If you just want the code, go here.
Why Snowpack?
I believe in Snowpack because I believe in JavaScript standards (as defined by Ecma) and because I believe in ES6 modules. ES modules (ESM) deserve a whole post of their own, but luckily other people have written about it extensively.
Until 2015, JavaScript did not have an official standard for code reuse. We were all sitting around relying on a mountain of hacks like AMD modules, CommonJS, and UMD just to be able to import packages. (UMD is particularly funny. I implore you to read the code.) These formats and the challenge of getting people to agree on a standard are, to some degree, the reason configuring Webpack is so complicated. Webpack takes in packages with all sorts of module types and outputs them in one module type (UMD by default.)
Well, in 2015 this changed. ECMAScript () defined ESM to be the definitive and native way of importing and exporting code. By 2019 for browsers and 2020 for NodeJS, the standard has become common place. ESM has a number of benefits: a universal syntax, direct JavaScript runtime support, fewer build tools, tree shaking through static analysis, etc. The critical take away is that the mountain of hacks is gone and your browser can directly understand modules.
Snowpack's build creates a single ES module for each dependency in your project. This is an incredibly powerful, ECMAScript standards based approach that allows you to build dependencies once. Remember, the browser can directly understand ES modules. Once the ES module is built, other files can take a dependency on that module indefinitely without recompiling the entire dependency tree. Neat.
Let's dive in.
Getting Started
The create-snowpack-app bootstrapper is very good. I decided to start with the @snowpack/app-template-blank-typescript because I vastly prefer typescript.
npx create-snowpack-app snowpack-test --template @snowpack/app-template-blank-typescript
I was sitting in my kitchen, which has relatively slow WiFi speeds, but the install was reasonable and the CLI dialogue was useful. Success. Hello World!
Heeelloooo World!
So what do we get?
The directory structure is somewhat bare. We have a src/index.ts which fetches a div from the root index.html file; a public with html, css, etc; and a decent number of configuration files.
Neat
There’s also a curious /types folder, which I hadn’t seen in other create-react-app style templaters, but seems designed to let me extend .d.ts files natively or import common file types natively instead of using Webpack style plugins.
/* Use this file to declare any custom file extensions for importing */
/* Use this folder to also add/extend a package d.ts file, if needed. */
I think this could use some documentation because it felt like an unusual solution to pre-type common file types. I could see these typings being useful in the snowpack scenario. All dependencies get imported individually, so Snowpack better assume that we’ll use css-loader style semantics! The docs make this clear.
“Snowpack supports basic CSS imports inside of your JavaScript files. When you import a CSS file via the import keyword, Snowpack will automatically apply those styles to the page. This works for CSS and compile-to-CSS languages like Sass & Less.”
In fact, Snowpack supports the following file types by default.
- JavaScript (.js, .mjs)
- TypeScript (.ts, .tsx)
- JSX (.jsx, .tsx)
- CSS (.css)
- CSS Modules (.module.css)
- Images (.svg, .jpg, .png, etc.)
Although I could have used some clearer documentation, this is in line with the type definitions from the .d.ts file and makes sense to me. The tsconfig file does make some of this clear. It imports the src and types directories. The tsconfig also references an unusual directory, “web_modules,” as a path.
Web Modules: The ES Modules Story
What is this doing here?
"paths": { "*": ["web_modules/.types/*"] }
Ah. Right. ES modules. Skypack. It's all coming back.
While I didn't verify with Fred, my assumption is that we're calling these web modules because they're part of the native ECMA (web) standard. We can take the modules from web_modules and use them directly on the web! It seems extra relevant because I know about Skypack, Snowpack’s npmjs-like (unpkg-like) cousin.
I was particularly curious to look in web_modules and my build directory. What does a ES6 module look like?? I'm so accustomed to my messy, bundled Webpack builds.
Build directory of our finished Fluid + Snowpack project
Good god is it clean! The directory structure makes sense and the compiled code is even better. As expected, my index.js is importing static ESMs from my web_modules directory using ES6 syntax. Completely readable. Very fast. Very elegant.
Look how reasonable!
import {getDefaultObjectFromContainer} from "../web_modules/@fluidframework/aqueduct.js";
import {getTinyliciousContainer} from "../web_modules/@fluidframework/get-tinylicious-container.js";
import {DiceRollerContainerRuntimeFactory} from "./containerCode.js";
import {renderDiceRoller} from "./view.js";
Anyway, these files are all accessible from the dev server on npm run start
. For example, in the create-snowpack-app template, I can see the canvas-confetti.js module at http://localhost:8080/web_modules/canvas-confetti.js. This makes sense and I suggest reading a bit of the compiled code to get a feel for 'bare metal' ES6 syntax.
npm run start
Even for a such a simple project the npm start
felt unusually snappy, although I didn’t do any experiments. If we can trust the CLI, npm start
in the “Hello World” template typically builds in 1-2 milliseconds (in watch mode.)
2ms!!
Before we get started adding the @fluid-example/hello-world to this project, I did want to examine the snowpack.config.js file and the extension model.
Extensions & Plugins
The @fluid-example/hello-world
uses HtmlWebpackPlugin by default to bind in HTML and the ts-loader to compile Typescript. Based on our earlier examination of the .d.ts and our assumptions about imports, I assume we don’t need to install an HTML parsing plugin. I couldn’t find documentation specifically indicating this, but the Snowpack.config.js file does use the mount
keyword and reference a few folders. I assume there’s some automagic happening here to pull in the HTML.
The snowpack.config.js does include the @snowpack/plugin-typescript
plugin. This feels pretty natural and the docs indicate there are a bunch of plugins. Many of them are officially supported. Very convenient having spent some time finagling community supported Webpack plugins. I also like that “plugins” can just be CLI commands (This is worth digging into, but it allows you to use tools like postcss, eslint, etc directly from Snowpack!). That feels like an obviously useful syntax for ad hoc definition of plugins.
I took a quick glance at the documentation for snowpack.config.js. There is a decent amount that can be done here, but I didn’t take the time to understand it. I am curious about the install, proxy, and alias configurations in particular. I can’t immediately imagine what I’d want to install at the snowpack level rather than the npm level, but this may be my misunderstanding of what’s actually going on under the hood. Either way, the template I used leaves most of these blank except mount
and plugins
as previously described.
With a basic understanding of Snowpack, I started porting over the Fluid Hello World.
Step 1. Install dependencies. Nothing to see here, this worked as expected.
Step 2... Copy and paste the typescript files in.
While we have previously done some weird things with the tsconfig, the template tsconfig and Fluid Hello World tsconfigs are pretty aligned. So I figure this should work.
And it mostly does seem to work. Snowpack does a much better job handling new files than Webpack does. Webpack dev server always seemed to break when new files were added to the src dir.
My only errors are around importing types, which we do a lot. It seems like I could modify the importsNotUsedAsValues flag in the tsconfig or just be more explicit in my imports. I chose the latter and resolved the five errors.†
Errors resolved, I’m not quite out of the gate. I’m mishandling a dependency from Aqueduct. Aqueduct is Fluid’s convenience library (get it 😉?)
I know it's becoming standard, but I do like this error dialogue. While we’ll have to see how clear the instructions are, the error and source are both clear, I can click through the possible errors, and I can clear the error dialogue! What a feature! If I click the X, I see the default HTML of the page.
I’m also getting a ton of polyfill errors. This is a known usability issue with Fluid and one that we’re addressing. Snowpack made the resolution pretty easy, although I still had lingering issues with assert. Luckily @christianGo just fixed this last month and an upgrade to 0.29 Fluid libraries resolved the issue.
With all CLI and pop up errors gone, I tried to load again and hit a pernicious issue with our nifty TypedEventEmitter binding to scope. For some reason, the parent here doesn’t have the off
parameter, so binding it to my scope fails.
Interestingly, I could try to catch and ignore this error because off
is just an alias for removeListener. My guess is this is an artifact from the polyfill and a quick Google confirms this. I opened a PR with a solution, but saved the polyfill locally to continue on with the experiment.
npm install --save-dev ~/Code/rollup-plugin-node-polyfills
And imported the rollup plugin from my dev dependencies.
module.exports = {
mount: {
public: '/',
src: '/_dist_',
},
plugins: ['@snowpack/plugin-typescript'],
install: [/* ... */],
installOptions: {
installTypes: true,
rollup: {
// @ts-ignore
plugins: [require('rollup-plugin-node-polyfills')(
{
events: true,
url: true,
assert:true,
querystring: true,
}
)],
},
},
devOptions: {/* ... */},
buildOptions: {/* ... */},
proxy: {/* ... */},
alias: {/* ... */},
};
Now past the initial polyfill issue, I quickly hit another. This time, my global variable is undefined in the buffer polyfill. Jeeze!
While I am generally curious why I’m hitting this issue, I’m eager to get my familiar Fluid Hello World up and running, so I hard coded const global = {}
, which would bypass the error message.‡
† Fred mentioned that the importsNotUsedAsValues flag is optional and mentioned in the docs. I missed it!
‡ Fred's comment, "Snowpack definitely rewards ESM packages that don't use a lot of Node.js dependencies by default. And those polyfills could really be improved... we may want to take over maintaining a fork just for our own use case." PR on the fork.
Success!
Boom. Dice Roller!
Review
All up... It took me about 25 minutes to get integrate the Fluid Hello World with the create-snowpack-app. This is a tad longer than I'd hoped, but felt reasonable for a build toolchain swap.
Snowpack is impressive and interesting. This limited exploration piqued my interest in the ecosystem (Skypack + Fluid Containers next?) and redoubled my interest in ES modules. I was delighted with the clarity and simplicity of Snowpack's build directory. I'm a strong believer in adopting standards based technology and Snowpack proves the point. It's simply a faster and more elegant build tool.
As more developers realize the performance benefits of the switch, I expect the OSS community will do what it does best and the cost of adoption will continue to drop.
The full code for this demo is available here.†
† I should have named the repo 'Ice Water'.