For the last couple of weeks, we have pressed ahead and covered a lot of ground concerning some rather advanced techniques, particularly ones related to pattern matching and extractors. Time to shift down a gear and look at one of the more fundamental idiosyncrasies of Scala: the Option type.

If you have participated in the Scala course at Coursera, you have already received a brief introduction to this type and seen it in use in the Map API. In this series, we have also used it when implementing our own extractors.

And yet, there is still a lot left to be explained about it. You may have wondered what all the fuss is about, what is so much better about options than other ways of dealing with absent values. You might also be at a loss how to actually work with the Option type in your own code. The goal of this part of the series is to do away with all these question marks and teach you all you really need to know about Option as an aspiring Scala novice.

The basic idea

If you have worked with Java at all in the past, it is very likely that you have come across a NullPointerException at some time (other languages will throw similarly named errors in such a case). Usually this happens because some method returns null when you were not expecting it and thus not dealing with that possibility in your client code. A value of null is often abused to represent an absent optional value.

Some languages treat null values in a special way or allow you to work safely with values that might be null. For instance, Groovy has the null-safe operator for accessing properties, so that foo?.bar?.baz will not throw an exception if either foo or its bar property is null, instead directly returning null. However, you are screwed if you forget to use this operator, and nothing forces you to do so.

Clojure basically treats its nil value like an empty thing, i.e. like an empty list if accessed like a list, or like an empty map if accessed like a map. This means that the nil value is bubbling up the call hierarchy. Very often this is okay, but sometimes this just leads to an exception much higher in the call hierchary, where some piece of code isn't that nil-friendly after all.

Scala tries to solve the problem by getting rid of null values altogether and providing its own type for representing optional values, i.e. values that may be present or not: the Option[A] trait.

Option[A] is a container for an optional value of type A. If the value of type A is present, the Option[A] is an instance of Some[A], containing the present value of type A. If the value is absent, the Option[A] is the object None.

By stating that a value may or may not be present on the type level, you and any other developers who work with your code are forced by the compiler to deal with this possibility. There is no way you may accidentally rely on the presence of a value that is really optional.

Option is mandatory! Do not use null to denote that an optional value is absent.

Creating an option

Usually, you can simply create an Option[A] for a present value by directly instantiating the Some case class:

val greeting: Option[String] = Some("Hello world")

Or, if you know that the value is absent, you simply assign or return the None object:

val greeting: Option[String] = None

However, time and again you will need to interoperate with Java libraries or code in other JVM languages that happily make use of null to denote absent values. For this reason, the Option companion object provides a factory method that creates None if the given parameter is null, otherwise the parameter wrapped in a Some:

val absentGreeting: Option[String] = Option(null) // absentGreeting will be None
val presentGreeting: Option[String] = Option("Hello!") // presentGreeting will be Some("Hello!")

Working with optional values

This is all pretty neat, but how do you actually work with optional values? It's time for an example. Let's do something boring, so we can focus on the important stuff.

Imagine you are working for one of those hipsterrific startups, and one of the first things you need to implement is a repository of users. We need to be able to find a user by their unique id. Sometimes, requests come in with bogus ids. This calls for a return type of Option[User] for our finder method. A dummy implementation of our user repository might look like this:

case class User(
  id: Int, 
  firstName: String,
  lastName: String, 
  age: Int,
  gender: Option[String])

object UserRepository {
  private val users = Map(1 -> User(1, "John", "Doe", 32, Some("male")),
                          2 -> User(2, "Johanna", "Doe", 30, None))
  def findById(id: Int): Option[User] = users.get(id)
  def findAll = users.values
}

Now, if you received an instance of Option[User] from the UserRepository and need to do something with it, how do you do that?

One way would be to check if a value is present by means of the isDefined method of your option, and, if that is the case, get that value via its get method:

val user1 = UserRepository.findById(1)
if (user1.isDefined) {
	println(user1.get.firstName)
} // will print "John"

This is very similar to how the Optional type in the Guava library is used in Java. If you think this is clunky and expect something more elegant from Scala, you're on the right track. More importantly, if you use get, you might forget about checking with isDefined before, leading to an exception at runtime, so you haven't gained a lot over using null.

You should stay away from this way of accessing options whenever possible!

Providing a default value

Very often, you want to work with a fallback or default value in case an optional value is absent. This use case is covered pretty well by the getOrElse method defined on Option:

val user = User(2, "Johanna", "Doe", 30, None)
println("Gender: " + user.gender.getOrElse("not specified")) // will print "not specified"

Please note that the default value you can specify as a parameter to the getOrElse method is a by-name parameter, which means that it is only evaluated if the option on which you invoke getOrElse is indeed None. Hence, there is no need to worry if creating the default value is costly for some reason or another – this will only happen if the default value is actually required.

Pattern matching

Some is a case class, so it is perfectly possible to use it in a pattern, be it in a regular pattern matching expression or in some other place where patterns are allowed. Let's rewrite the example above using pattern matching:

val user = User(2, "Johanna", "Doe", 30, None)
user.gender match {
  case Some(gender) => println("Gender: " + gender)
  case None => println("Gender: not specified")
}

Or, if you want to remove the duplicated println statement and make use of the fact that you are working with a pattern matching expression:

val user = User(2, "Johanna", "Doe", 30, None)
val gender = user.gender match {
  case Some(gender) => gender
  case None => "not specified"
}
println("Gender: " + gender)

You will hopefully have noticed that pattern matching on an Option instance is rather verbose, which is also why it is usually not idiomatic to process options this way. So, even if you are all excited about pattern matching, try to use the alternatives when working with options.

There is one quite elegant way of using patterns with options, which you will learn about in the section on for comprehensions, below.

Options can be viewed as collections

So far you haven't seen a lot of elegant or idiomatic ways of working with options. We are coming to that now.

I already mentioned that Option[A] is a container for a value of type A. More precisely, you may think of it as some kind of collection — some special snowflake of a collection that contains either zero elements or exactly one element of type A. This is a very powerful idea!

Even though on the type level, Option is not a collection type in Scala, options come with all the goodness you have come to appreciate about Scala collections like List, Set etc — and if you really need to, you can even transform an option into a List, for instance.

So what does this allow you to do?

Performing a side-effect if a value is present

If you need to perform some side-effect only if a specific optional value is present, the foreach method you know from Scala's collections comes in handy:

UserRepository.findById(2).foreach(user => println(user.firstName)) // prints "Johanna"

The function passed to foreach will be called exactly once, if the Option is a Some, or never, if it is None.

Mapping an option

The really good thing about options behaving like a collection is that you can work with them in a very functional way, and the way you do that is exactly the same as for lists, sets etc.

Just as you can map a List[A] to a List[B], you can map an Option[A] to an Option[B]. This means that if your instance of Option[A] is defined, i.e. it is Some[A], the result is Some[B], otherwise it is None.

If you compare Option to List, None is the equivalent of an empty list: when you map an empty List[A], you get an empty List[B], and when you map an Option[A] that is None, you get an Option[B] that is None.

Let's get the age of an optional user:

val age = UserRepository.findById(1).map(_.age) // age is Some(32)

flatMap and options

Let's do the same for the gender:

val gender = UserRepository.findById(1).map(_.gender) // gender is an Option[Option[String]]

The type of the resulting gender is Option[Option[String]]. Why is that?

Think of it like this: You have an Option container for a User, and inside that container you are mapping the User instance to an Option[String], since that is the type of the gender property on our User class.

These nested options are a nuisance? Why, no problem, like all collections, Option also provides a flatMap method. Just like you can flatMap a List[List[A]] to a List[B], you can do the same for an Option[Option[A]]:

val gender1 = UserRepository.findById(1).flatMap(_.gender) // gender is Some("male")
val gender2 = UserRepository.findById(2).flatMap(_.gender) // gender is None
val gender3 = UserRepository.findById(3).flatMap(_.gender) // gender is None

The result type is now Option[String]. If the user is defined and its gender is defined, we get it as a flattened Some. If either the use or its gender is undefined, we get a None.

To understand how this works, let's have a look at what happens when flat mapping a list of lists of strings, always keeping in mind that an Option is just a collection, too, like a List:

val names: List[List[String]] = 
  List(List("John", "Johanna", "Daniel"), List(), List("Doe", "Westheide"))
names.map(_.map(_.toUpperCase)) 
// results in List(List("JOHN", "JOHANNA", "DANIEL"), List(), List("DOE", "WESTHEIDE"))
names.flatMap(_.map(_.toUpperCase)) 
// results in List("JOHN", "JOHANNA", "DANIEL", "DOE", "WESTHEIDE")

If we use flatMap, the mapped elements of the inner lists are converted into a single flat list of strings. Obviously, nothing will remain of any empty inner lists.

To lead us back to the Option type, consider what happens if you map a list of options of strings:

val names: List[Option[String]] = List(Some("Johanna"), None, Some("Daniel"))
names.map(_.map(_.toUpperCase)) // List(Some("JOHANNA"), None, Some("DANIEL"))
names.flatMap(xs => xs.map(_.toUpperCase)) // List("JOHANNA", "DANIEL")

If you just map over the list of options, the result type stays List[Option[String]]. Using flatMap, all elements of the inner collections are put into a flat list: The one element of any Some[String] in the original list is unwrapped and put into the result list, whereas any None value in the original list does not contain any element to be unwrapped. Hence, None values are effectively filtered out.

With this in mind, have a look again at what flatMap does on the Option type.

Filtering an option

You can filter an option just like you can filter a list. If the instance of Option[A] is defined, i.e. it is a Some[A], and the predicate passed to filter returns true for the wrapped value of type A, the Some instance is returned. If the Option instance is already None or the predicate returns false for the value inside the Some, the result is None:

UserRepository.findById(1).filter(_.age > 30) // Some(user), because age is > 30
UserRepository.findById(2).filter(_.age > 30) // None, because age is <= 30
UserRepository.findById(3).filter(_.age > 30) // None, because user is already None

For comprehensions

Now that you know that an Option can be treated as a collection and provides map, flatMap, filter and other methods you know from collections, you will probably already suspect that options can be used in for comprehensions. Often, this is the most readable way of working with options, especially if you have to chain a lot of map, flatMap and filter invocations. If it's just a single map, that may often be preferrable, as it is a little less verbose.

If we want to get the gender for a single user, we can apply the following for comprehension:

for {
	user <- UserRepository.findById(1)
	gender <- user.gender
} yield gender // results in Some("male")

As you may know from working with lists, this is equivalent to nested invocations of flatMap. If the UserRepository already returns None or the Gender is None, the result of the for comprehension is None. For the user in the example, a gender is defined, so it is returned in a Some.

If we wanted to retrieve the genders of all users that have specified it, we could iterate all users, and for each of them yield a gender, if it is defined:

for {
	user <- UserRepository.findAll
	gender <- user.gender
} yield gender

Since we are effectively flat mapping, the result type is List[String], and the resulting list is List("male"), because gender is only defined for the first user.

Usage in the left side of a generator

Maybe you remember from part three of this series that the left side of a generator in a for comprehension is a pattern. This means that you can also patterns involving options in for comprehensions.

We could rewrite the previous example as follows:

for {
	User(_, _, _, _, Some(gender)) <- UserRepository.findAll
} yield gender

Using a Some pattern in the left side of a generator has the effect of removing all elements from the result collection for which the respective value is None.

Chaining options

Options can also be chained, which is a little similar to chaining partial functions. To do this, you call orElse on an Option instance, and pass in another Option instance as a by-name parameter. If the former is None, orElse returns the option passed to it, otherwise it returns the one on which it was called.

A good use case for this is finding a resource, when you have several different locations to search for it and an order of preference. In our example, we prefer the resource to be found in the config dir, so we call orElse on it, passing in an alternative option:

case class Resource(content: String)
val resourceFromConfigDir: Option[Resource] = None
val resourceFromClasspath: Option[Resource] = Some(Resource("I was found on the classpath"))
val resource = resourceFromConfigDir orElse resourceFromClasspath

This is usually a good fit if you want to chain more than just two options – if you simply want to provide a default value in case a given option is absent, the getOrElse method may be a better idea.

Summary

In this article, I hope to have given you everything you need to know about the Option type in order to use it for your benefit, to understand other people's Scala code and write more readable, functional code. The most important insight to take away from this post is that there is a very basic idea that is common to lists, sets, maps, options, and, as you will see in a future post, other data types, and that there is a uniform way of using these types, which is both elegant and very powerful.

In the following part of this series I am going to deal with idiomatic, functional error handling in Scala.