Hi there! I want to show off a little feature I made using hanami, htmx and a little bit of redis + sidekiq.
It is a popular pattern to show a progress bar when you're doing some longer-running task, like uploading a file, or processing some data. I wanted to show you how to do it in hanami, while making the progress bar fill smoothly, and not just jump from 0 to 100% when the task is done.
The app and the task
I have an app running Hanami 2.1, it has persistence setup with ROM and is using hanami-views for rendering, hanami-assets for providing CSS. Pretty much everything that we need for our task is in a slice called main.
Sidekiq is already configured along with assets, tailwindsCSS.
Our app is going to be a personal library management system, where users can track their books, their placement on a shelf, racks, status of borrowing etc.
The main page has an input for ISBN number. That input sends a GET request to the server, which then fetches the book data from google books API, parses the data to fit our own books relation in ROM, checks if we have it saved in our DB, saves it if not, or just returns the finished status if we already have it.
In the meantime, users sees a progress bar that fills up smoothly, and when the task is done, the progress bar disappears and the book data is shown.
The resulting code will have to use some sleep statements to show the "animation" on the progress bar since normally it is too fast to notice.
Redis
Redis setup is super easy thanks to providers
#config/providers/redis.rb
Hanami.app.register_provider(:redis) do
prepare do
require "redis"
end
start do
client ||= ConnectionPool::Wrapper.new do
Redis.new(url: target["settings"].redis_url)
end
register "redis", client
end
end
#config/settings.rb
setting :redis_url, default: "redis://localhost:6379", constructor: Types::String
Now when we do include Deps['redis] we get a redis client that we can use to store/read our progress data.
HTMX
HTMX has a couple of ways to install. I chose npm because I already used it in the project for tailwindCSS and I have hanami-assets configured for assets.
//slices/main/assets/js/app.js
import "../css/app.css";
import 'htmx.org';
import './htmx.js'
//slices/main/assets/js/htmx.js
window.htmx = require('htmx.org');
And this is all it takes to have HTMX working on your hanami project. If you want to see a quick test example to make sure it works, check out this commit on the demo repo.
Actions and Views
We need few blocks in place: downloading the data, parsing it, saving it to DB, and checking if we already have it. All of this needs to be tracked by the backend and monitored by frontend to show the progress.
So lets start with displaying everything, since we can use placeholders for that.
Our slice will have 3 relevant actions: IsbnSearch, SearchProgress and SearchResult.
scope 'search' do
get '/isbn', to: 'isbn_search.show'
get '/progress', to: 'search_progress.show'
get '/result', to: 'search_result.show'
end
First step will be the search:
module Main
module Actions
module IsbnSearch
class Show < Main::Action
params do
required(:isbn).filled(:string)
end
def handle(request, response)
halt 422, {errors: request.params.errors}.to_json unless request.params.valid?
Main::Workers::IsbnSearch.perform_async(request.params[:isbn])
response.render(view, isbn: request.params[:isbn])
end
end
end
end
end
# Spec, only showing the basic happy path example, for breviety
context "with good params" do
let(:params) { Hash[isbn: "978-0-306-40615-7"] }
let(:worker) { double(Main::Workers::IsbnSearch) }
it "works" do
Sidekiq::Testing.fake! do
response = subject.call(params)
allow(Main::Workers::IsbnSearch).to receive(:perform_async)
.with("978-0-306-40615-7")
.and_return(worker)
expect(response.status).to eq 200
end
end
end
Then we have the view object
module Main
module Views
module IsbnSearch
class Show < Main::View
config.layout = nil
expose :isbn do |isbn:|
{ type: isbn.size, identifier: isbn }
end
end
end
end
end
With the template:
<input hidden id="isbn-type" name="isbn[type]" value="<%= isbn[:type] %>">
<input hidden id="isbn-identifier"
name="isbn[identifier]"
value="<%= isbn[:identifier] %>">
<div
hx-trigger="done"
hx-get="/search/result"
hx-include="#isbn-type, #isbn-identifier"
hx-swap="outerHTML"
hx-target="this">
<h3 role="status" id="pblabel" tabindex="-1" autofocus>Searching</h3>
<div
hx-get="/search/progress"
hx-include="#isbn-type, #isbn-identifier"
hx-trigger="every 2000ms"
hx-target="this"
hx-swap="innerHTML">
<div class="w-64 h-5 mb-5 overflow-hidden bg-base-100 rounded-md shadow-inner"
role="progressbar"
aria-valuemin="0"
aria-valuemax="100"
aria-valuenow="122"
aria-labelledby="pblabel">
<div id="pb" class="float-left h-full text-white text-center bg-accent transition-width duration-2000"
style="width:0%"></div>
</div>
</div>
</div>
This is where the majority of HTMX magic comes in. We have 3 important div's:
1. The outer div, that will be replaced with the result of the search, and along with him, all divs inside (that are described below)
2. The progress div, that every 2 seconds checks the progress of the search, and updates the progress bar -> div below
3. The initial progress bar div, that is replaced with the progress bar div from the SearchProgress action, the div from point 2 makes the request.
All this happens based on the hx-target, hx-get and hx-swap attributes, hx-trigger is used to repeat the request every 2 seconds on the div that checks the progress, and for completing the process in case of the outer div.
- target tells HTMX where to put the response
- get is simply the request identifier, the get is the type and the value given is the URL
- swap is a method of replacement, so how to replace the target with the response
This is all rather basic stuff from HTMX, pretty much the introduction part of their docs. So if you think it is somewhat impressive, keep it mind this is just what is possible with the very basics. Although I do think that hanami routing and actions and view systems make it much easier to use (than for example rails would). In Hanami it is simply very easy to make the code very modular, composable and repeatable. I think HTMX and Hanami are a great fit together and I hope this whole example does show it.
So lets take a look at the SearchProgress action:
module SearchProgress
class Show < Main::Action
include Deps["redis"]
def handle(request, response)
if redis.hget("isbn_search", request.params[:isbn][:identifier]).to_i == 3
response.headers["HX-Trigger"] = "done"
end
response.render(view, isbn: request.params[:isbn])
end
end
end
HX-Trigger is a header that will tell the HTMX to trigger the done action, which will fire up the SearchResult action, and that will render the result of the search.
The header is based on the redis value, which is set by the worker (we will get there soon).
module SearchResult
class Show < Main::Action
include Deps["repositories.books"]
def handle(request, response)
response.render(
view,
book_found: books.by_isbn(type: request.params[:isbn][:type].to_i,
identifier: request.params[:isbn][:identifier])
)
end
end
end
Result is simple, we use the books repo to make a simple request, som rom and sequel are used here, but it is not really relevant to the topic at hand, so the repo implementation is omitted.
You probably noticed the inputs in the view, they just hold the values we need to keep finding the correct book based on the initial isbn (which later got an identifier on the back, that tells us if it is ISBN-10 or 13 to make some queries easier).
The templates are also important, here is the template for progress along with its view object:
<div class="w-64 h-5 mb-5 overflow-hidden bg-gray-300 rounded-md shadow-inner"
role="progressbar"
aria-valuemin="0"
aria-valuemax="100"
aria-valuenow="<%= search_progress %>"
aria-labelledby="pblabel">
<div id="pb" class="float-left h-full text-white text-center bg-accent transition-width duration-2000"
style="width: <%= search_progress %>%">
</div>
</div>
module Main
module Views
module SearchProgress
class Show < Main::View
config.layout = nil
include Deps["redis"]
expose :search_progress, decorate: false do |isbn:|
search_progress(isbn: isbn)
end
private
def search_progress(isbn:)
progress = redis.hget("isbn_search", isbn[:identifier])
case progress.to_i
when 1
40
when 2
80
when 3
100
else
10
end
end
end
end
end
end
Just a simple check of the redis value, and returning the correct value for the progress bar. Then the rest is taken care by CSS and tailwind styling. Thanks to sharing the id between the HTML elements that are "replaced", the transition is smooth, rather than the filler color of the bar just appearing. It flows.
Then we have the result that replaces the progress bar, it is a simple card with the book data, taken from the struct we got from the repository.
<div class="card lg:card-side bg-base-100 shadow-xl">
<figure class="w-1/2 h-1/2 max-w-96 max-h-96">
<img src="<%= book.image_url.gsub("http", "https") %>" alt="Cover" />
</figure>
<div class="card-body">
<h2 class="card-title">
<%= book.title %>
</h2>
<p>
<%= book.description[0..500] %>
</p>
<div class="card-actions justify-end">
<button class="btn btn-primary">See more</button>
</div>
</div>
</div>
module SearchResult
class Show < Main::View
config.layout = nil
expose :book, decorate: false do |book_found:|
book_found
end
end
end
Those are all the blocks we need in place to have a fully functional frontend. We just need the backend to implement all the logic that does the actual work, and monitors the progress.
Worker that works hard
For brevity's sake I will only show the basic spec, with the conditions we wanna meet.
We wanna take an ISBN, and delegate work to other objects, checking if the book was saved (this is a scenario where the ISBN is for a new book), and that redis status was updated correctly at the end.
context 'when called' do
let(:google_isbn_service) { double(Main::Services::GetGoogleIsbn) }
let(:parser) { double(Main::Services::GetGoogleIsbn) }
let(:parse_output) {
{ title: "Dune",
description: "A nice SF book",
image_url: "https://www.some.site",
published_date: "1965",
category: "SF",
language: "en",
authors: ["Frank Herbert"],
isbn_numbers: [
{ type: 10, identifier: "0441172717" },
{ type: 13, identifier: "9780441172719" }
]
}
}
it 'processes the job' do
expect(Main::Services::GetGoogleIsbn).to receive(:new).and_return(google_isbn_service)
expect(Main::Parsers::Google::Isbn).to receive(:new).and_return(parser)
expect(google_isbn_service).to receive(:call).with(isbn: "9780441172719").and_return({ body: "some body" })
expect(parser).to receive(:parse).with(json: { body: "some body" }).and_return(parse_output)
expect{Main::Workers::IsbnSearch.perform_async("9780441172719")}.to change{db[:books].count}.by(1)
expect(Hanami.app["redis"].hget("isbn_search", "9780441172719")).to eq("3")
end
end
In general sidekiq workers should only take the most basic input (strings etc.) and delegate most of the work to other objects, so that we can test them in isolation, handle errors better etc. This is the way I always preferred the workers/jobs to be coded, and it works great here too, cause we just list all the dependencies, use them one by one and monitor the process easily and clearly thanks to that.
For brevity's sake I've put everything in the perform method, but it in real life it could be a good idea to split it out more into methods, for readability and clearer error outputs.
module Main
module Workers
class IsbnSearch
include Sidekiq::Job
include Deps[
"services.get_google_isbn",
"redis",
"persistence.rom",
"repositories.books",
parser: "parsers.google.isbn"]
def perform(isbn)
redis.hset("isbn_search", { isbn => 1 })
output = parser.parse(json: get_google_isbn.call(isbn:))
redis.hset("isbn_search", { isbn => 2 })
if books.by_isbn(type: 10, identifier: isbn) || books.by_isbn(type: 13, identifier: isbn)
redis.hset("isbn_search", { isbn => 3 })
else
rom.relations[:books].transaction do
author = rom.relations[:authors].changeset(
:create, { name: output[:authors].join(', ') }
).commit
new_book = rom.relations[:books].changeset(
:create, output[:data]
).commit
redis.hset("isbn_search", { isbn => 3 })
end
end
end
end
end
end
Now this is a fully functional progress bar (the implementations of parser and get_google_isbn objects are not really relevant, those are implementation details not connected to progress bar directly).
This could be further improved and made into a far more reusable code, with redis communication being delegated to pub/sub system, that could be better coupled with places that do actually push the progress forward, rather than putting everything into a worker, that should be more of a simple delegator.
But maybe pub/sub system in hanami is a topic for a different episode?
This gives us a reactive progress bar, that updates every second and in the end, switches place with the ready content.
Summary
So this is a neat feature using Hanami and HTMX, backed by redis and sidekiq. I hope this was a nice mix of familiar and unfamiliar technologies, to get you interested in something new, like Hanami or HTMX.
