26
Searching, Sorting and Pagination in a Common Lisp web application
In this post, we are going to build a demo web application in Common Lisp with search, sort and pagination functionalities with a tabular data of randomly generated food dish names, ratings, price and cuisines.
As usual we are going to use Caveman for scaffolding our web application. If you want to know more about Caveman or why Caveman, you might want to take a look at my previous post
about Common Lisp and web development. Caveman is available in quicklisp, so you can install it with:
(ql:quickload :caveman2)And you can start a create a new project with Caveman like this:
(caveman2:make-project #P"~/quicklisp/local-projects/cl-tabular" :author "Rajasegar")Now take a look at our index route. Our index route is going to use some query params for sorting and pagination.
With Caveman you can parse the query params using the _parsed key and we are using custom defined function to
take the query parameter values from it like below:
(defun query-param (name parsed)
(cdr (assoc name parsed :test #'string=)))So for our route logic, we need the values of the following query parameters, start which is the starting offset
of the records for the page, direction which is either ascending or descending and the sort-by key based on which
column we are currently sorting the list.
(defroute "/" (&key _parsed)
(format t "_parsed = ~a~%" _parsed)
(let ((start (parse-integer (or (query-param "start" _parsed) "0")))
(direction (or (query-param "direction" _parsed) "asc"))
(sort-by (or (query-param "sort-by" _parsed) "name")))
(render #P"index.html"
(list
:foods (slice-list start (sort-list direction sort-by))
:total (length *foods*)
:pages (generate-pages)
:start start
:direction direction
:sort-by sort-by
:opposite-direction (get-opposite-direction direction)))))One thing important to note in the above code snippet is how we are sending data to the templates via the render function. We can construct our data using a list and the templates can easily access them via the keyword mapped to the data.
Say, for example, in the template you can loop through the list of foods like
{% for food in foods}
<p>{{food.name}}</p>
{% endfor %}So how are we sending the paginated list of foods to our home page. If you take a closer look, we are first sorting the entire list of foods using the sort-by parameter and then
we are slicing the list based on the start offset and returning them to the template.
:foods (slice-list start (sort-list direction sort-by))Let's take a look at our slice-list function on how we are slicing our list.
(defun slice-list (start)
(let ((new-list nil))
(dotimes (i 10)
(push (elt *foods* (+ i start)) new-list))
new-list))We construct a new temporary list by pushing the 10 items from the original list, starting from the start offset and then returning the new list to the page. The sort-list function is discussed in the later part of the post under Sorting.
Our index template is very big and has got three sections, the search form, the table and the pagination.
First we will take a look at the search form. So this is a simple form with an input field with the name query.
<form>
<div class="mb-4">
<div class="col-6">
<input
class="form-control form-control-lg"
type="text"
placeholder="Search dish name..."
name="query"
hx-post="/search?start=0&direction=asc&sort-by=name"
hx-trigger="keyup changed delay:500ms"
hx-target="#results">
</div>
</div>
</form>We are adding some custom attributes starting with hx- , these are actually some enhanced attributes for HTML using a library called htmx which allows you to access AJAX, CSS Transitions, WebSockets and Server Sent Events directly in HTML, using attributes, so you can build modern user interfaces with the simplicity and power of hypertext
Using htmx, we are sending a POST request to the url /search and the response will be swapped with the element with an id #results. Hence as soon as you start typing the search keywords the client will start sending the request to the server with a delay of 500 milli seconds and you get to see the results populated in the table.
Next comes our important UI component the table itself. The table will have four columns like Name, Rating, Price and Cuisine. When you click on the table headers, it will toggle the sort direction automatically from ascending to descending and vice-versa for a particular column value. We will show some up and down arrows to indicate the sorting direction.
<div id="results">
<p>{{total}} results found</p>
<table class="table table-striped">
<thead>
<tr class="table-dark">
<th><a href="/?start=0&sort-by=name&direction={{opposite-direction}}">Name
{% if sort-by == "name" and direction == "asc" %} ↑ {% endif %}
{% if sort-by == "name" and direction == "desc" %} ↓ {% endif %}
</a></th>
<th><a href="/?start=0&sort-by=rating&direction={{opposite-direction}}">Rating
{% if sort-by == "rating" and direction == "asc" %} ↑ {% endif %}
{% if sort-by == "rating" and direction == "desc" %} ↓ {% endif %}
</a></th>
<th><a href="/?start=0&sort-by=price&direction={{opposite-direction}}"> Price
{% if sort-by == "price" and direction == "asc" %} ↑ {% endif %}
{% if sort-by == "price" and direction == "desc" %} ↓ {% endif %}
</a></th>
<th><a href="/?start=0&sort-by=cuisine&direction={{opposite-direction}}">Cuisine
{% if sort-by == "cuisine" and direction == "asc" %} ↑ {% endif %}
{% if sort-by == "cuisine" and direction == "desc" %} ↓ {% endif %}
</a></th>
</tr>
</thead>
<tbody>
{% for food in foods %}
<tr>
<td>{{food.name}}</td>
<td>
{% ifequal food.rating 1 %}★{% endifequal %}
{% ifequal food.rating 2 %}★★{% endifequal %}
{% ifequal food.rating 3 %}★★★{% endifequal %}
{% ifequal food.rating 4 %}★★★★{% endifequal %}
{% ifequal food.rating 5 %}★★★★★{% endifequal %}
</td>
<td>
${{food.price}}
</td>
<td>{{food.cuisine}}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>Now we will take a look at our pagination component. This will be placed at the bottom of the table. We will also indicate the active page with a different background highlight if the record offset values are matching with the page and the url value. We will construct the links based on the pagination
data sent by the server for the route along with other things like direction and sort-by values.
<nav aria-label="Page navigation example">
<ul class="pagination">
{% for page in pages %}
<li class="page-item {% ifequal start page.start %} active {% endifequal %}" >
<a class="page-link" href="/?start={{page.start}}&direction={{direction}}&sort-by={{sort-by}}">{{page.id}}</a>
</li>
{% endfor %}
</ul>
</nav>Next we are going to take a look at our utility function to generate our pagination data. We are going to use a loop with 10 iterations to create the respective pagination data for the page and the start offset value for the table data. It will be something like for page 2, we will start with the record offset 10 and for page 3, it will be 20 and so on. Please make
note that our records for the first page start from 0 to 9, so the second page starts from 10 and so on.
We are also ensuring that the pagination data is in ascending order using the reverse function at the end while returning the output from the function, otherwise we will end up with pages in the descending order.
(defun generate-pages ()
"Generate pagination"
(let ((pages nil))
(dotimes (i 10)
(push (list :id (+ 1 i) :start (* 10 i)) pages))
(reverse pages)))The data for our table is just a random list of dishes, ratings, price and the cuisine. First we declare a global variable called *foods* and initialize the value to nil.
(defvar *foods* nil)Next we will create a list of dish names in a separate variable called *dishes*.
(defvar *dishes* '("Pizza"
"Noodles"
"Fried Rice"
"Roti"
"Lasagna"
"Churros"
"Tea"
"Soup"
"Egg roll"
"Salad"
"Burger"
"Rice"
"Curry"
"Bread"))Then, we will create a list of cuisine names in a variable called *cuisines*.
(defvar *cuisines* '("Indian"
"Chinese"
"Thai"
"Continental"
"Mexican"
"Indonesian"
"Japanese"
"Spanish"
"Italian"
"Greek"))Now it's time to combine all our dish names and cuisines to generate a list of dishes with random rating values and prices. So before pushing the generated values into our global foods variable, let's be sure to reset the variable to nil.
Then using a dotimes loop for 100 iterations we are going to generate a random record for dish. We are getting a random dish and cuisine form the previously created lists called dishes and cuisines respectively.
;; Clear the list
(setf *foods* nil)
;; Push 100 items into foods with random values
(dotimes (i 100)
(push (list :name (random-elt *dishes*)
:cuisine (random-elt *cuisines*)
:rating (+ 1 (random 5))
:price (+ 1 (random 100))) *foods*))For that we are using a custom defined function
called random-elt which will pick a random element from a list.
(defun random-elt (mylist)
(elt mylist (random (length mylist))))And then for the rating and price, we are using the standard library function called random to generate random numbers within a specified range. For example, (random 5) will generate random numbers between 0 and 4 and we are adding 1 to
ensure we are getting a non-zero value.
Sorting data in Common Lisp is pretty easy and straight-forward when it comes to lists. We are using an higher-order function called sort-list which will take two parameters, the sort direction either "asc" or "desc" and the sort-by which is the key based on which we sort the list. And based on the sort-by key we will delegate the sorting to the respective sort functions with the direction as an argument.
(defun sort-list (direction sort-by)
"Sort a list based on the direction and key"
(cond ((string= sort-by "name") (sort-list-by-name direction))
((string= sort-by "rating") (sort-list-by-rating direction))
((string= sort-by "price") (sort-list-by-price direction))
((string= sort-by "cuisine") (sort-list-by-cuisine direction))))Based on the direction, we will figure out the sort function to use, #'string> or #'string< for name and cuisine, and #'> or #'< for rating and price. We can still have one function for sorting all the columns if we can refactor, because this approach will not scale for large number of columns in the table.
(defun sort-list-by-name (direction)
"Sort a list by name"
(let ((sort-fn (if (string= direction "asc") #'string< #'string>)))
(sort (copy-list *foods*) sort-fn :key (lambda (plist) (getf plist :name)))))
(defun sort-list-by-rating (direction)
"Sort a list by rating"
(let ((sort-fn (if (string= direction "asc") #'< #'>)))
(sort (copy-list *foods*) sort-fn :key (lambda (plist) (getf plist :rating)))))
(defun sort-list-by-price (direction)
"Sort a list by price"
(let ((sort-fn (if (string= direction "asc") #'< #'>)))
(sort (copy-list *foods*) sort-fn :key (lambda (plist) (getf plist :price)))))
(defun sort-list-by-cuisine (direction)
"Sort a list by price"
(let ((sort-fn (if (string= direction "asc") #'string< #'string>)))
(sort (copy-list *foods*) sort-fn :key (lambda (plist) (getf plist :cuisine)))))Next we focus on the search route for our application.
The search route will take a query parameter called query itself, through which we will get the search keywords for the route. We will perform the search only based on the names of the dishes. We will use a utility function called filter-foods for this purpose.
(defroute ("/search" :method :POST) (&key _parsed)
(format t "_parsed = ~a~%" _parsed)
(let* ((query (cdr (assoc "query" _parsed :test #'string=)))
(filtered-foods (filter-foods query)))
(render #P"_search.html"
(list
:foods filtered-foods
:total (length filtered-foods)))))The filter-foods function takes the query as the parameter
and filter out the dishes which is not matching with the name of the dish. To filter out the food list we are using the remove-if function with a lambda wherein we match the name of the food with the query string using the search function with the test as #'char-equal. If it matches return NIL so that it cannot be removed from the list , otherwise we return T, so that it can be removed from the list and we would only get all the matching dish names.
(defun filter-foods (query)
"Filter foods based on the query with name"
(remove-if #'(lambda (food)
(let ((name (getf food :name)))
(if (search query name :test #'char-equal)
nil
t))) *foods*))<div id="results" >
<p><a href="/">Clear Search</a></p>
<p>{{total}} results found</p>
<table class="table table-striped">
<thead>
<tr class="table-dark">
<th> <a href="/?sort-by=name&direction=desc"> Name ↓</a></th>
<th> <a href="/?sort-by=stars&direction=desc"> Stars</a></th>
<th> <a href="/?sort-by=price&direction=desc"> Price</a></th>
<th> <a href="/?sort-by=category&direction=desc"> Category</a></th>
</tr>
</thead>
<tbody>
{% for food in foods %}
<tr>
<td>{{food.name}}</td>
<td>{{food.rating}}</td>
<td>{{food.price}}</td>
<td>{{food.cuisine}}</td>
</tr>
{% endfor %}
</tbody></table>
</div>And this is how the final app look like in action.
The source code for this application is hosted in Github.
If you are stuck with any step or anything is missing in this post, you can always refer to the
updated source code in Github.
26