/ Shapeless/Scala

Copying fields using Shapeless

Sometimes we need to copy fields from one case class to another. Often the two case classes are quite similar but we still need to copy each field manually. Shapeless can eliminate this boilerplate for us

First let's think why we may want to do this. We often face the scenario of collecting information in multiple steps, validating it along the way and performing the final operation when we have everything we need. Wizard style forms are a good example of this (booking a rail or air ticket). The ubiquitous case class is the usual data type employed but case classes are not ideal:

  1. If we use a single case class we have to accept that some fields will be Null until we reach the final page of the wizard (ouch!)

  2. Alternatively we can make the fields optional but this introduces it's own problems. A case class made up of entirely optional fields is pretty useless

  3. The best solution is usually to use one case class per "page" then process them all together at the end.

Option 3 works well but at some point we probably want to aggregate the data from the various case classes togeher into a single object or otherwise transform/reshape it. Either way we end up copying data from one case class to another.

The scenario

I'm going to walk through a simplified car hire example in which our hypothetical user chooses her pickup and dropoff location, dates etc, chooses the type of car she wants and finally enters her own data. The three step flow can be modelled using these case classes:

case class Location(pickup: String, dropOff: String, from: LocalDate, to: LocalDate)
case class Vehicle(vehicleCategory: String, automatic: Boolean, numDoors: Int)
case class Driver(driverAge: Int, nationality: String)

Our final reservation object will look like this:

case class Reservation(
  pickup: String,
  dropOff: String,
  from: LocalDate,
  to: LocalDate,
  vehicleCategory: String,
  automatic: Boolean,
  numDoors: Int,
  driverAge: Int,
  nationality: String,
  reservationConfirmed: Boolean
)

For the purposes of this post I'm going to focus on how we can copy data from the three individual objects along with a boolean to a Reservation object. I'll do this without copying the fields manually

Getting started with Shapeless

As usual we add Shapeless as an SBT dependency:

"com.chuusai" %% "shapeless" % "2.3.3"

HLists

The HList is at the core of Shapeless, it's like a tuple on steroids. Like a tuple an HList has an fixed number of elements which can be of different types. We can model our data using HLists:

import shapeless.{::, HNil}
type LocationH = String :: String :: LocalDate :: LocalDate :: HNil
type VehicleH = String :: Boolean :: Int :: HNil
type DriverH = Int :: String :: HNil

Note how the syntax is very similar to a normal list. We can model our final reservation using an HList also:

type ReservationH = String :: String :: LocalDate :: LocalDate :: String :: Boolean :: Int :: 
                    Int :: String :: Boolean :: HNil

We can create instances of our HLists using a very similar syntax:

val locationH: LocationH = "Malaga Airport" :: "Malaga Airport" :: LocalDate.of(2018,8,1) :: 
                           LocalDate.of(2018,8,10) :: HNil
val vehicleH: VehicleH = "Economy" :: false :: 4 :: HNil
val driverH: DriverH = 35 :: "British" :: HNil

Unlike tuples, HLists can be concatenated and this is where things get interesting. We can concatenate the three HLists together along with the final boolean flag (reservationConfirmed) and we get a ReservationH type:

val reservationH: ReservationH = locationH ++ vehicleH ++ driverH :+ false

We've found a way to concatenate our three pieces of data together but the resulting type is pretty messy. Like tuples if we want to extract a field the best we can do is to pattern match:

reservation match {
  case 
    pickup :: 
    dropOff :: 
    from :: 
    to :: 
    vehicleCategory ::
    automatic :: 
    numDoors :: 
    driverAge :: 
    nationality :: 
    reservationConfirmed 
    :: HNil => println(s"pickup location: $pickup")
}

From case classes to Hlists and back again

Fortunately Shapeless allows us to convert case classes to Hlists and vice versa:

import shapeless.{::, HNil, Generic}

val location = Location(pickup = "Malaga Airport", dropOff = "Malaga Airport", 
               from = LocalDate.of(2018,8,1), to = LocalDate.of(2018,8,10))
val vehicle = Vehicle(vehicleCategory = "Economy", automatic = false, numDoors = 4)
val driver = Driver(driverAge = 35, nationality = "British")

val locationH: LocationH = Generic[Location].to(location)
val vehicleH: VehicleH = Generic[Vehicle].to(vehicle)
val driverH: DriverH = Generic[Driver].to(driver)

val reservationH: ReservationH = locationH ++ vehicleH ++ driverH :+ false

The Generic trait allows us to map from a case class (actually any Product or Coproduct type) to an HList. Generic[Location].to(location) means "transform the location case class instance to an Hlist". Of course we can also go back again:

val reservation: Reservation = Generic[Reservation].from(reservationH)

We can actually get rid of our custom HList types (the compiler will infer them for us) and greatly simplify the code. By convention we typically use the term Repr for HList representations of case classes:

import shapeless.Generic

val location = Location(pickup = "Malaga Airport", dropOff = "Malaga Airport", 
               from = LocalDate.of(2018,8,1), to = LocalDate.of(2018,8,10))
val vehicle = Vehicle(vehicleCategory = "Economy", automatic = false, numDoors = 4)
val driver = Driver(driverAge = 35, nationality = "British")

val locationRepr = Generic[Location].to(location)
val vehicleRepr = Generic[Vehicle].to(vehicle)
val driverRepr = Generic[Driver].to(driver)

val reservationRepr = locationRepr ++ vehicleRepr ++ driverRepr :+ false
val reservation: Reservation = Generic[Reservation].from(reservationRepr)

Improving the design

Things are not quite as good as they seem. Lets modify our Location case class and swap the from and to dates around. We'll leave the rest of the code unchanged:

case class Location(pickup: String, dropOff: String, to: LocalDate, from: LocalDate)

Everything compiles but what happens when we get the reservation's from date:

val reservationRepr = locationRepr ++ vehicleRepr ++ driverRepr :+ false
val reservation: Reservation = Generic[Reservation].from(reservationRepr)
println(s"from date: ${reservation.from}")

2018-8-10 // Actually this is the to date

This weird behaviour happens because like tuples, HLists operate on the type and position of the fields, not the field names. As far as Shapeless is concerned we've just extracted two dates from the location and we pass two dates into the Reservation. However there is a fix for this ...

LabelledGeneric

Shapeless includes a utility called LabelledGeneric which creates custom types on the fly based on the field type and the field name. I won't go into the details but the end result is that the resulting Location HList looks something like this (conceptually):

PickupString :: DropOffString :: ToLocalDate :: FromLocalDate :: HNil

The relevant part of the Reservation HList would look something like

PickupString :: DropOffString :: FromLocalDate :: ToLocalDate :: HNil

In this case the types don't align and therefore the code won't compile:

import shapeless.LabelledGeneric

val locationRepr = LabelledGeneric[Location].to(location)
val vehicleRepr = LabelledGeneric[Vehicle].to(vehicle)
val driverRepr = LabelledGeneric[Driver].to(driver)

val reservationRepr = locationRepr ++ vehicleRepr ++ driverRepr :+ false
// This won't compile
val reservation: Reservation = LabelledGeneric[Reservation].from(reservationRepr)

However if we swap the Location's dates around to align with the Reservation:

case class Location(pickup: String, dropOff: String, from: LocalDate, to: LocalDate)

Well actually it still doesn't compile! The problem is the final reservationConfirmed field.

We're expecting a type of ReservationConfirmedBoolean but we're appending a simple Boolean:

val reservationRepr = locationRepr ++ vehicleRepr ++ driverRepr :+ false

We need to create an instance of this custom type on the fly. Helpfully shapeless includes a utility for this:

import shapeless.syntax.singleton._

// ('reservationConfirmed ->> false) creates an instance of our custom type on the fly
val reservationRepr = locationRepr ++ vehicleRepr ++ driverRepr :+ ('reservationConfirmed ->> false)
val reservation: Reservation = LabelledGeneric[Reservation].from(reservationRepr)

Auto aligning the types

We've eliminated the bug caused by the misaligned dates. If another developer refactors our Location or Reservation classes and moves fields around it won't compile. Needing to keep the fields aligned is a bit of a drag and it feels pretty brittle. Why can't Shapeless understand that from: LocalData on the Location is the same as from: LocalDate on the Reservation ? Well as usual there's a utility to align the types for us:

// We need this so we can get the HList type of the Reservation
val reservationGen = LabelledGeneric[Reservation]
val reservationRepr = locationRepr ++ vehicleRepr ++ driverRepr :+ ('reservationConfirmed ->> false)
val reservation: Reservation = LabelledGeneric[Reservation].from(reservationRepr.align[reservationGen.Repr])

the .align extension basically means "align this hlist to the hlist representation of the Reservation case class". Note how we need to create a LabelledGeneric instance of the Reservation so we can access it's Repr (Hlist) type

We can now go ahead and rearrange our Location, Vehicle and Driver case classes as much as we want. We just need to ensure that the correct field names and types are present. The order is no longer important:

case class Location(dropOff: String, to: LocalDate, from: LocalDate, pickup: String)
...
// this still compiles!
val reservationRepr = vehicleRepr ++ driverRepr ++ locationRepr :+ ('reservationConfirmed ->> false)
val reservation: Reservation = LabelledGeneric[Reservation].from(reservationRepr.align[reservationGen.Repr])

Summary

Shapeless provides two useful utilities for copying data between arbitrary case classes. HLists are a generic representation of strongly typed data, like tuples on steroids. LabelledGeneric lets of map case classes to and from HLists. Used together we get behaviour similar to BeanUtils copyProperties but with full type safety!