consulting spotify in a better way

After my latest adventures writing a small spotify library and learning in the process a bit more about consult, its author, Daniel Mendler, was kind enough to comment on how i had implemented the asynchronous search using consult's API, showing me better ways.

You can read the full discussion in this issue over at codeberg (where you can also find the latest version of the library), but the gist of it, regarding consult, is as follows.

To define a new asynchronous consult command, one wants to use consult--read, passing to it a function that generates our dynamic list of completion candidates. To create that function, one can use a pipeline of closures that successively create and massage those candidates. In the case of espotify that layering might look like this¹:

(thread-first (consult--async-sink)
  (consult--async-refresh-immediate)
  (espotify--async-search type filter)
  (consult--async-throttle)
  (consult--async-split))

where we only have to implement espotify--asynch-search to construct the generator of completion candidates (more about it in a moment). The rest are helpers already provided by consult:

• consult--async-split: splits the input string, one part for async, one part for filtering
• consult--async-throttle: throttles the user input
• consult--async-refresh-immediate: refreshes when candidates are pushed
• consult--async-sink: collects the candidates and refreshes

Consult offers also a few more closure generators that we haven't used (yet):

• consult--async-map: transform candidates
• consult--async-refresh-timer: refreshes, when candidates are pushed, throttles with a timer
• consult--async-filter: filter candidates
• consult--async-process, a source generator handy when your candidates come from the output of executing a local process

Back to our candidates generator. It must be a function that takes a continuation closure (the async after you in the pipeline) and returns an action dispatcher, that is, a function takiing that action as its single argument (possibly passing its results, or simply delegating, to the next handler in the pipeline). So our dispatcher generator is going to look something like this template, where we display all possible actions to be dispatched:

(defun espotify--async-search (next-async ...)
  ;; return a dispatcher for new actions
  (lambda (action)
    (pcase action
      ((pred stringp) ...) ;; if the action is a string, it's the user input
      ((pred listp) ...)   ;; if a list, candidates to be appended
      ('setup ...)
      ('destroy ...)
      ('flush ..)
      ('get ...))))

For each action, we must decide whether to handle it ourselves or simply pass it to next-async, or maybe both. Or we could ask next-async to perform new actions for us. In our case, we only care about generating a list of tracks when given a query string that ends on a marker character², and making sure it reaches the top level. Thus, our async has only work to do when it receives a string, simplifying my original implemetation to:

(defun espotify--async-search (next type filter)
  (lambda (action)
    (pcase action
      ((pred stringp)
       (when (string-suffix-p "=" action)
         (espotify-search-all
          (lambda (items)  ;; search results callback
            (funcall next 'flush)
            (funcall next (mapcar #'spotify--format-item items)))
          (substring action 0 (- (length action)
                                 (length espotify-search-suffix)))
          type
          filter)))
      (_ (funcall next action)))))

As you can see, when we receive a search string, we launch an asynchronous search and, upon receiving its results, we flush the layer above us (so that it discards previous candidates) and pass the new candidate list to it. It is ultimately the closure returned by consult--async-sink the one keeping track of those candidates, and making them accessible to consult--read. The latter is expecting candidates to be strings (possibly with properties), while our search callback is receiving, via its items parameter, a list of alists: that's why we need to map over them with espotify--format-item. If we prefer, we can make that transformation explicit by simply returning items in that callback (via (funcall next items)) and inserting consult--async-map in our pipeline, which would now look like:

(thread-first (consult--async-sink)
  (consult--async-refresh-immediate)
  (consult--async-map #'espotify--format-item)
  (espotify--async-search type filter)
  (consult--async-throttle)
  (consult--async-split))

With all that, our code looks tidier and easier to understand (i at least understand much better its workings) ³. You can always check its latest version, in literate version, here (C-c C-v t in the org buffer will generate espotify.el for you).

Thanks Daniel!