Calling Babel from Django for React+JSX

leave this field blank to prove your humanity

Article title*

Article revisions must have a non-empty title

Article body*

Sharing my frustrations so you can enjoy them too... :)

As of writing, this website is mostly "over the wire" and has very little frontend. It's nice because the server sends everything you need in one neat bundle. Fast and efficient. It has a little hand written JS to handle posting comments, fetching notifications, update timestamps and load continuous scrolling lists.

I was curious to try [React](https://react.dev/), partly to see what the fuss was about and maybe it could help reduce bugs and repetitiveness in the dynamic features of some pages.

Rather frustratingly, the first results for using React with Django tell me to install a node.js webserver 🤦. Actually some start with CORS permissiosn and routing between docker containers for running both. So that smells. Django and node are two entirely separate things that do the same job. It makes no sense to run a node server whos only job is to forward API calls to Django. Maybe they have their reasons but I didn't hang around to find out. Anyway, this highlights **what even is React and how could it relate to Django?**

# What is React?

A few terms, if I'm understanding correctly. Take this with a grain of salt as I'm definitely on the noob side of web devs.

- **Static website** straight HTML/CSS with declaritive "what to display" UI
- **Dynamic website** are stateful and can show different content at the same URL, e.g. an online shop or wiki.
- **Responsive pages** (unrelated) change UI nicely depending on the screen size, e.g. for mobile devices.
- **Reactive pages** can have dynamic content without loading or reloading a page by using javascript to update the existing page.

**React** (again IIUC) is a javascript library that tries to make dynamic content more declaritive. It has reusable *components* with restrictive data flow to encourage reusability in the design. While it can be used without, it's commonly used with [JSX](https://legacy.reactjs.org/docs/introducing-jsx.html), javascript extension language with templating to inline HTML. Since browsers don't run JSX, it needs to be "transpiled" to regular javascript, like TypeScript.

Anyway, this page is about integrating react code with Django. Integrating react would be as easy as adding `<script/>` tags for the [React CDN](https://legacy.reactjs.org/docs/cdn-links.html) but I want JSX too. JSX needs to be transpiled into javascript before it can be sent to the client's browser by the webserver. Before diving into Django, I think it's important to cover how it's normally done - with `npm`/`yarn`, `node`, `webpack` and `babel`.

- [`npm`](https://docs.npmjs.com/about-npm) / [`yarn`](https://engineering.fb.com/2016/10/11/web/yarn-a-new-package-manager-for-javascript/) - these are javascript package managers, just like `pip` or `apt` but for javascript, with similar features to python's `virtualenv` and `requirements.txt`
- [Node.js](https://nodejs.org/en/about) - the javascript runtime, just like the `python` executable runs *.py python scripts
- `node` - a javascript webserver that runs on Node.js. Frustratingly, lines between Node.js and the webserver are blurred, as is evident by the [Node.js docs](https://nodejs.org/en/docs/guides/anatomy-of-an-http-transaction) diving straight into discussions about HTTP and webservers like it's just assumed that's why you're using Node.js
- [`webpack`](https://webpack.js.org/) - a "bundler", a tiny bit like [django-compressor](https://github.com/django-compressor/django-compressor/) that handles `import { something } from '...'` statements. IMO this is vaguely like a linker that stitches compiled C object files into an executable or shared library, ready to be given to people to run.
- [`babel`](https://babeljs.io/docs) - a "transpiler" with support for multiple languages including React. It consumes the higher level languages and produces raw javascript

Putting this all together, `npm` or `yarn` installs the other packages and ideally their dependencies. You write some React code in a `.jsx` file, run `webpack` which in turn runs `babel` and packages everything into a big javascript blob and finally give that to `node` to serve to the client. `webpack` reads a config file in your project that will say how to run `babel` and with what plugins and presets, e.g. `@babel/preset-react`. For starting out, people could skip all the setup with [`create-react-app`](https://create-react-app.dev/).

So that's the expected usage, but I already have a webserver and I really only want `babel`.

# Babel and Django

Put simply, I want `babel` to transpile JSX so that I can include it in my Django project's javascript payload, along with React libraries. There is actually [PyReact](https://pypi.org/project/PyReact/) but after seeing its 8 year old commits, my instinct was that `babel` and its react plugin would be the way to go. For installing `babel`, see the [next section](#installing-babel).

I'm already using [django-compressor](https://github.com/django-compressor/django-compressor/), which groups and minifies all the separate .js files, so I don't think I want to complicate things by adding `webpack` to the mix just yet. It does mean adding each transpiled jsx source by hand and missing out on including jsx files from each other. If needed I expect I could find a way to call `webpack` instead of babel and pass its single output file to django-compressor so it could mix in existing javascript from my Django project. I learned a lot from and would recomend [Modern JavaScript for Django Developers](https://www.saaspegasus.com/guides/modern-javascript-for-django-developers/), which does use `webpack`. The initial discussion on server-first, client-first and the hybrid architecture is a great read.

## django-compressor

Now the question is, how can I get Django to call babel. I found that django-compressor actually supports a `COMPRESS_PRECOMPILERS` setting that could add a transpile step to each of my sources. For example, from [SO](https://stackoverflow.com/questions/41467708/precompile-jsx-for-react-in-django-compressor) I got this to run:

COMPRESS_PRECOMPILERS = (
       ('text/jsx', '< "{infile}" babel --plugins @babel/plugin-transform-react-jsx-source --presets @babel/preset-env,@babel/preset-react > "{outfile}"'),
    )

Then django-compressor just needs to find a `type="text/javascript"` attribute in a `<script/>` tag such as the following:

I had trouble getting this to automatically re-run babel after changing the source file when running in `DEBUG` mode. That said, I later found that Django was enabling its [cached template loader](https://nickjanetakis.com/blog/django-4-1-html-templates-are-cached-by-default-with-debug-true) so it might have just been that.

## django-static-precompiler

An alternative is [django-static-precompiler](https://github.com/andreyfedoseev/django-static-precompiler) which basically does the same thing but before django-compressor compresses the files.

STATIC_PRECOMPILER_COMPILERS = (
        (
            "static_precompiler.compilers.Babel",
            {
                "executable": "babel",
                "sourcemap_enabled": True,
                #"plugins": "@babel/plugin-transform-react-jsx", # from the docs
                "plugins": "@babel/plugin-transform-react-jsx-source",
                "presets": "@babel/preset-env,@babel/preset-react",
            },
        ),
    )

The catch is that it doesn't have access to the `type="..."` attribute and its Babel backend only runs for `.es6` files.

I was getting errors from the browser parsing an unknown `exports` variable that was [fixed](https://stackoverflow.com/questions/49260209/how-to-use-babel-without-webpack) by using `@babel/plugin-transform-react-jsx-source` instead of `@babel/plugin-transform-react-jsx`. I guess this changes is needed because I'm skipping webpack and adding sources by hand as separate script tags.

## React library

With the above taking care of transpiling JSX I just needed to include the React library itself. I'm guessing if I used `webpack` it could be included already. I could also host it separately myself, but using the [CDN](https://legacy.reactjs.org/docs/cdn-links.html) works too.

{% if debug %}
    <script crossorigin src="https://unpkg.com/react@18/umd/react.development.js"></script>
    <script crossorigin src="https://unpkg.com/react-dom@18/umd/react-dom.development.js"></script>
    {% else %}
    <script crossorigin src="https://unpkg.com/react@18/umd/react.production.min.js"></script>
    <script crossorigin src="https://unpkg.com/react-dom@18/umd/react-dom.production.min.js"></script>
    {% endif %}

# Installing Babel

I already have Django running in a debian based docker image. There is no node to speak of so it needs adding. The super easy way for me is to use `apt` and skip `npm`/`yarn` entirely. I need the babel command line interface and the plugins/presets for JSX.

apt update
    apt install -y \
        node-babel-cli \
        node-babel-preset-env \
        node-babel-preset-react \
        node-babel-plugin-transform-react-jsx

This installs the executable `/usr/bin/babeljs` but the version that came with the distro I was using was rather old. When using `"sourcemap_enabled": True,` (`babel -s`) I would get the following error with all but the simplest jsx files:

Error: original.line and original.column are not numbers ...

I am guessing that was [this bug](https://github.com/webpack-contrib/babel-minify-webpack-plugin/issues/68) and I could either update the OS to try with a newer version from `apt` or get the most recent by trying `npm` and `yarn`.

## npm and yarn

The following commands install the equivalent of the above (maybe with some unnecessary extras, I'm not sure). I found that I needed to add all packages to the one command line otherwise version conflicts could occur. I guess the expected use case is to list all the packages in a `package.json` file. I actually used `npm -g ...` and `yarn global ...` in a docker container so it was a clean environment each time. Globally installing packages is [not a good idea](https://stackoverflow.com/a/43901681/1888983) otherwise. Installing `yarn` with `apt` [isn't advised right now](https://stackoverflow.com/questions/50627636/global-file-or-directory-not-found-for-yarn). Oddly, I saw [python 2.7](https://www.python.org/doc/sunset-python-2/) being installed as a dependency with `apt install npm`.

# npm
    apt install npm
    npm install \
        @babel/core \
        @babel/runtime \
        @babel/cli \
        @babel/preset-env \
        @babel/preset-react \
        @babel/plugin-syntax-jsx \
        @babel/plugin-transform-runtime \
        @babel/plugin-transform-react-jsx \
        @babel/plugin-transform-react-jsx-source

# alternatively with yarn,
    npm install yarn
    yarn add \
        @babel/core \
        @babel/runtime \
        @babel/cli \
        @babel/preset-env \
        @babel/preset-react \
        @babel/plugin-syntax-jsx \
        @babel/plugin-transform-runtime \
        @babel/plugin-transform-react-jsx \
        @babel/plugin-transform-react-jsx-source

Both of these did give me a `babel` binary at `/usr/local/bin/babel` but when running it I'd get something along the lines of:

Error: Cannot find module '@babel/plugin-transform-react-jsx-source'

... or any other names I tried throwing at it. This is stupid - I literally just installed these packages; how are they missing? Actually it was searching for the difference between [`npm` and `npx`](https://stackoverflow.com/questions/50605219/difference-between-npx-and-npm) that hinted to the problem here. Also running `find / -name \*babel\*`.

My project has no `node_modules` directory and I was installing everything globally in the docker container. I guess `babel` (or the Node.js context it runs itself in) was not set up with default module search paths, like `PYTHONHOME`.

$ npm root -g
    /usr/local/lib/node_modules

$ yarn global dir
    /usr/local/share/.config/yarn/global

Note the output from `yarn` does not include `node_modules`. Armed with this information I could finally run one of:

NODE_PATH=$(npm root -g) babel --plugins @babel/plugin-transform-react-jsx --presets @babel/preset-env,@babel/preset-react -s -o helloworld.js helloworld.jsx

NODE_PATH=$(yarn global dir)/node_modules babel --plugins @babel/plugin-transform-react-jsx --presets @babel/preset-env,@babel/preset-react -s -o helloworld.js helloworld.jsx

Then it was a matter of routing this environment variable into Django for django-compressor or django-static-precompiler to run babel with. For this I just set `ENV NODE_PATH /usr/local/share/.config/yarn/global/node_modules` in the Dockerfile. There's probably a better way. Or maybe in the future, Node.js executables will automatically add their own path...

$ realpath /usr/local/bin/babel
    /usr/local/share/.config/yarn/global/node_modules/@babel/cli/bin/babel.js
    $ cat /usr/local/bin/../share/.config/yarn/global/node_modules/.bin/../@babel/cli/bin/babel.js
    #!/usr/bin/env node
    
    require("../lib/babel");
    $  file /usr/bin/node
    /usr/bin/node: ELF 64-bit LSB executable ...

In this case `yarn` put `node` at `/usr/bin/node`. I wonder if it could instead put it under `/usr/local/share/.config/yarn/global`, have a symlink at `/usr/bin/node` and then node could infer `NODE_PATH` from the binary's location.

Toggle Preview

Edit message*

A description of the changes made

Calling Babel from Django for React+JSX

Flag the thing you clicked for moderator attention.

Error