bxroberts ) ¶

Twitter announce ¶

https://fediverse.org/bxroberts/status/1549055945069568000?s=20&t=qImii6Um9LHBfJXf54DI2Q

Ello! Today I published some notes on two technologies I’ve been using to write one-off web tools to speed up my investigations: Htmx and Hyperscript.

Sharing some snippets and ways to write small things fast.

Introduction ¶

Htmx and Hyperscript are two different, but related web languages/tools .

Together, they allow programmers to write browser-based interfaces with much less code than if one had chosen a single-page application framework like React.

When I work on data heavy investigations, I often need to build one-off web apps that automate manual tasks. This is especially true when machine learning or NLP are part of my workflow.

Just last month I needed to seamlessly review PDFs and split the pages up into groups of sub-documents. It wasn’t really feasible to do it by hand—I had thousands of files, many of them quite large.

Time spent waiting for things to load or going between the mouse and keyboard gets frustrating quick. I needed a tool.

Conventional wisdom contends that I should build a single page app (SPA) from the ground up, using something like React or whatever. Timesucks when writing SPAs include: handling file uploads, writing page navigation and menus, user auth, configuration, little UI transitions, managing intermediate state, data serialization—I could increase this list ad-infinitum. Writing SPAs is tedious.

I’ve grown tired of it. I end up spending far too much time working out features not directly related to the predicament at hand. In the case of my previously mentioned PDF tool, nothing was unique or new, all I needed was the ability to:

Display a list of documents + some metadata (name, page count, status)
Display all the page images of a selected document
Visualize page sub-document groups somehow
Create, remove or change the groups

This list isn’t long, but some parts are deceptively tricky. The thought of just powering through with yet another SPA was too much to accept, so I decided to try something different.

Recently, I came across two technologies: Hyperscript and Htmx .

They let programmers write UIs in a declarative style that embraces some unusual philosophies. I find them to be well suited for quickly building the one-off tools that I use in my investigations.

Htmx: Declarative Programming for the Web ¶

Htmx takes the concept of a declarative language engine (the query planner in SQL and the solver in Prolog) and brings it to web applications programming.

Htmx provides a small JavaScript (JS) engine that operates using special HTML attributes. All the common things that web applications do, like fetch URLs, render responses, and event handling can be done in a declarative style. This drastically reduces the upfront labor in building UIs .

An illustrative example is a button that displays a list of results when clicked. To handle the click event in the SPA style, one would need to write a bunch of JS, asynchronously fetch an API page, parse the data, and render the data as HTML. This could easily be a hundred lines of code. With htmx, this is possible with just a couple tags:

           <span class="button"
   hx-get="/pages/10"
   hx-target="#doc-10-pages-area"
   hx-swap="innerHTML">
Load Pages
</span>
<div id="doc-10-pages-area"></div>

          

Above, when the “Load Pages” span is clicked, htmx fetches the /pages/10 URL and writes the results into the div. This greatly simplifies building UIs. One catch here is that htmx expects the backend to return htmx or HTML, not JSON. This is in stark contrast to the JSON REST API backend assumed by most SPA frameworks.

At first, using htmx was definitely weird, but once it “clicked” I realized I could throw all kinds of web applications together with barely any effort. At times it was so easy it felt like cheating.

Building a simple htmx backend with Django ¶

When I’m managing large scale inquiries, I like to keep track of all the agencies I’ve requested from and the records I’ve received from them. This way I already have all the metadata for my PDFs in a Django application. I continue to use this system as I convert, clean and process the data.

The home page of the tool simply lists all the agencies from which I have PDFs in need of review. Home page view code:

           def GET_segmentable_home(request):
    """
    Renders a htmx-powered home page for segmenting
    PDFs from agencies with responsive records. This
    page simply lists the agencies, and allows the user
    to click each agency, rendering the agency page
    (which lists each document).
    """
    context = {
        "agencies": Agency.objects.all(),
    }
    return render(
        request,
        "segmentable_home.htmx",
        context=context
    )

          

The template, segmentable_home.htmx, loops over each agency and renders them using the agency.htmx template. This is going to important later.

           <body>
  <div class="main">
  <section class="segmentable-home">
    {% for agency in agencies %}
      {% include 'agency.htmx' with agency=agency %}
    {% endfor %}
  </section>
  <!-- half the screen: full PDF page -->
  <section id="zoomed-page">
  </section>
</body>

          

Here’s the agency.htmx template itself

           <div id="agency-{{ agency.id }}"
         class="agency">
  <h2>{{ agency.name }}</h2>
  <p class="info complete">
    Total Segmented: {{ agency.segmented_docs }}
  </p>
  <p class="info total">
    Total Docs: {{ agency.total_segmentable_docs }}
  </p>
  <p class="button"
     hx-get="{% url 'docs' agency.id %}"
     hx-target="#agency-{{ agency.id }}"
     _="on click add .opened to #agency-{{ agency.id }}">
     Show Documents
  </p>
</div>

          

This template reveals two key htmx concepts .

First , each item (agency) in the list is its own template making the open/close logic really simple. Opening an agency is accomplished with the Show documents button. When clicked, htmx fetches the ‘docs’ endpoint and replaces everything inside the div tag with the result. This should reveal that agency’s list of PDF documents (docs.htmx). Closing the list reverses this process by replacing the div tag with the original agency.htmx. Here’s the close button:

           <p class="button"
   hx-get="{% url 'agency.htmx' agency.pk %}"
   hx-target="#agency-{{ agency.pk }}"
   hx-swap="outerHTML">
   Close Documents List
</p>

          

Clicking the close button gets us back to the initial state.

The second principle at play is the use of hyperscript to trigger UI transitions. When the user clicks an agency documents list to view, the opened class gets added to the agency. I use this to make the agency item more prominent on the screen. When the user closes the documents list, the class will be removed because the original agency div element no longer has the opened class added.

There’s a circular nature to writing htmx tools that can be unintuitive at first. But, once it is familiar, writing code in this way is really quick and highly re-usable. I was able to implement this strategy to quickly build the agencies, documents list and document pages list views.

Simplifying complex UX with Hyperscript ¶

The second part of this app that could be complicated is the page grouping stuff. I need a grid of PDF pages that shows which ones are in a group. Ideally, clicking a page would add or remove it from the adjoining group without sub-menus or follow-up clicks required.

This is the page-segmentation UI built using the htmx+hypersciprt techniques described here. Clicking on one of the pages causes it to join or separate from the previous page. The borders display the page groups. You can see an enlarged version of each page by hovering over it with your mouse.

First, I came up with a simple way to encode the page groups:

           [[group0_1st_page, group0_last_page],
  ...,
  [groupN_1st_page, groupN_last_page]]

          

The above is zero-indexed and inclusive. It assumes that pages in a group are always next to each other (which is true in my data). A PDF with 5 pages, where the first and second pages are in a group and the rest in a second group would look like this:

           <div id="doc-{{ doc.id }}-pages">
  {% for img in images %}
  <div id="pg-img-{{ doc.id }}-{{ loop.index }}"
       class="{% if loop.index in pg_group_ends[{{doc.id}}] %}
                end
              {% endif %} pg-img"
        _="on click
              toggle .end then call saveSegments({{ doc.id }})">
    <img src="{{ img.url }}"
         alt="Page {{ img.page }}" />
    <caption>Page {{ img.page }}</caption>
  </div>
  {% endfor %}
</div>

          

Conclusion + Future Work ¶

I am really impressed by the abilities of htmx and hyperscript .

With careful crafting of data structures and thinking through my workflow, it was possible to do a lot with very little. In the time since I’ve written this PDF page grouping tool, I’ve used htmx+hyperscript on several other projects where I needed small interactive tools.

In particular, I’ve been using it to flesh out plugins inside the Datasette ecosystem with really good results.

I’m a big proponent of using the right tool for the job. Every tool has its place and I believe htmx+hyperscript are extremely well suited to small projects—the kind typically needed during investigative projects.

That said, I’d caution against using them on large, complex projects. The tradeoffs that allow you to build small things fast come at a cost.

There are still some rough edges around using htmx—the main one being that there’s no backend that directly supports the kind of thinking that it expects. Using Django works because of how easily its template system is mixed with htmx code. But, I quickly realized I had stumbled upon a boilerplate pattern of grabbing objects, rendering the htmx/jinja template, and adding the open/close logic described above. Even if it’s not complicated or verbose, it gets repetitive.

In the future, I’d like to try building a htmx-specific ModelView for Django or maybe even a custom htmx serializer for the Django Rest Framework.