Reflection on Bloodhound extensibility

Disclaimer: this logs describes my relation with Bloodhound and monocle, there is no criticism neither to their authors, or their work; it is the opposite, I would like to thanks them.

Few years ago, I had a project which was relying on ElasticSearch, in Haskell.

The only library available at the time was Bloodhound, and sadly, there were some missing and broken parts.

In 2022, I became the maintainer, I did a lot of work such as:

  • Reworking the module organization
  • Dropping support for old versions
  • Adding support for OpenSearch 1 and 2
  • Adding new requests
  • Strengthening type-safety (breaking the API in the meantime)
  • Refining MonadBH so it became mock-friendly

I also started a move to make it more extensible, such as it became easier to refine a part of an existing query, for instance, we can perform a search directly:

Client.searchByIndex index payload

Or change the base request, e.g. adding a query string such as:

let query = (Query.searchByIndex index payload) {BH.bhRequestQueryStrings = qs}
resp <- BH.tryEsError $ BH.performBHRequest query

This year, as part of my Hacktoberfest participation, I have planned to upgrade monocle Bloodhound's version.

Let's start with the good parts:

  • I have managed to have a green build, to be honest, at some point, I thought I won't make it
  • Requests hook was really useful to compare the queries and unify the differences
  • Request/Client split is somewhat useful
  • I forced me to extends ElasticSearch supports
  • I have made two releases

The bad parts now:

  • One of the big change I have introduced with both Request/Client split and MonadBH is that some Requests throw when the status code is invalid, forcing to explicitly handle to silence errors, and many calls were failing
  • Request/Client split is only marginally useful, most of the need exists in some existing requests which are using a too restrictive type.

As an example, median_absolute_deviation was not supported, forcing the consumer to get out of bloodhound:

medianDeviationDuration :: QEffects es => QueryFlavor -> Eff es Double
medianDeviationDuration qf = queryAggValue =<< searchBody qf deviation
 where
  deviation =
    Aeson.object
      [ "median_absolute_deviation"
          .= Aeson.object ["field" .= ("duration" :: Text)]
      ]

Instead of:

medianDeviationDuration :: QEffects es => QueryFlavor -> Eff es Double
medianDeviationDuration qf = queryAggValue =<< searchBody qf deviation
 where
  deviation =
    BH.MedianAbsoluteDeviationAgg
      $ BH.MedianAbsoluteDeviationAggregation (BH.FieldName "duration") Nothing

Note also the nesting which makes everything more complicated.

Consequently, unlike my last announce, I'll focus on breaking sum wrappers in type classes, unleashing extensibility.