Explicit receiver parameters

I discovered that Java 8 has a language feature I'd never heard of before today!

Explicit receiver parameters

Consider a simple method in Java 7:

 public class Currency {
   public String getCode() { ... }
 }

In Java 8, it turns out there is a second way to write the method:

 public class Currency {
   public String getCode(Currency this) { ... }
 }

The same trick also works for methods with arguments:

 public class Currency {
   public int compareTo(Currency this, Currency other) { ... }
 }

So, what is going on here?

The relevant part of the Java Language Specification is here.

The receiver parameter is an optional syntactic device for an instance method or an inner class's constructor. For an instance method, the receiver parameter represents the object for which the method is invoked. For an inner class's constructor, the receiver parameter represents the immediately enclosing instance of the newly constructed object. Either way, the receiver parameter exists solely to allow the type of the represented object to be denoted in source code, so that the type may be annotated. The receiver parameter is not a formal parameter; more precisely, it is not a declaration of any kind of variable, it is never bound to any value passed as an argument in a method invocation expression or qualified class instance creation expression, and it has no effect whatsoever at run time.

The new feature is entirely optional, and exists to allow the type to be annotated at the point it is used:

 public class Currency {
   public int compareTo(@AnnotatedUsage Currency this, Currency other) { ... }
 }
 @Target(ElementType.TYPE_USE)
 public @interface AnnotatedUsage {}

The annotation is made available in reflection using Method::getAnnotatedReceiverType().

Looking to the future

I discovered this feature in the context of Project Valhalla, the effort to add value types to Java. One possible syntax being considered to allow different methods on List<int> from List<String> is to use receiver types:

 public class List<any T> {
   public int sum(List<int> this) { ... }
 }

This syntax would be used to define a sum() method that only applies when List is parameterized by int, and not when parameterized by anything else, such as String.

For more information, see this email to the Valhalla experts mailing list. But please note that the list is read-only, cannot be signed up to and is for pre-selected experts only. In addition, everything discussed there is very, very early in the process, and liable to change greatly before being released in a real version of Java (such as Java 10).

Summary

Java 8 has a new language feature which you've probably never heard of and will probably never use. But at least you now have some knowledge that you can use to prove you are a Java Guru!

Naming Optional query methods

In the last article I outlined a pragmatic approach to Java 8's Optional class. In this one I'm looking at how we should name query methods that might return Optional.

Convenience method naming

Consider a requirement to produce a tree data structure, perhaps something like XML DOM. The basic data structure might look something like this:

 public XmlElement {
   private final String name;
   private final String content;
   private final Map<String, String> attributes;
   private final List<XmlElement> children;
 }

(This article uses this as an example, but many other data structures or domain objects face similar issues, so don't get too caught up in the "XML"-ness of the example, and it's inability to represent every last subtlety of XML.)

The key question for this article is about the convenience methods this class exposes. Using standard best practices, an element with no children should return an empty list, and an element with no attributes should return an empty map. Similarly, we can use the empty string, "", when there is no content.

 public XmlElement {
   public String getName() {...}
   public String getContent() {...}
   public Map<String, String> getName() {...}
   public List<XmlElement> getChildren() {...}
 }

This approach is fine, and allows us to fully query this immutable data structure. But it would be nice to have some additional convenience methods. The main one to consider is a method to return the value of an attribute by name. The naive implementation would be:

 public XmlElement {
   public String getAttribute(String name) {
     return attributes.get(name);
   }
 }

But, this method can return null which the last article recommended against. The not so naive implementation would be:

 public XmlElement {
   public Optional<String> getAttribute(String name) {
     return Optional.ofNullable(attributes.get(name));
   }
 }

This is better, as null is no longer returned and it is now clear that there is a failure case to consider. However, what if the calling code knows that the data it is asking for is expected to be there, and is going to throw an exception if it isn't?

The "standard" approach in that case is to use the method Optional.orElseThrow() on the result of the last method. And that is good advice. But, equally, if the same call to orElseThrow() follows lots of calls to the new helper method, it may be a sign that the convenience method isn't helping as much as it should! As such, perhaps it is worthy of a helper method itself:

 public XmlElement {
   public String getAttribute(String name) {
     String value = attributes.get(name);
     if (value == null) {
       throw new IllegalArgumentException(
           "Unknown attribute '" + name + "' on element '" + this.name + "'"));
     }
     return value;
   }
 }

(Note that this follows the last article's recommendation to use null, not Optional, within the methods of a class.)

So, this is better for the many use cases where it is a failure case anyway if the attribute is missing. ie. where the caller knows that the attribute is defined to be mandatory, and it is necessarily an exception if it is missing.

In reality though, some attributes are going to be mandatory and some are optional. Which brings us to the point of this article, and design approach for a case like this, and the ensuing naming problem. My suggestion is to have two methods:

 public XmlElement {

   // throw exception if no attribute
   public String getAttribute(String name) { ... }

   // return empty optional if no attribute
   public Optional<String> findAttribute(String name) { ... }
 }

In this approach, there are two methods, and the caller can choose one method if they want the exception when the name is not found, and the other method if they want to handle the missing case.

The need for this approach is relatively rare - it needs to be a method that can be used in both ways, mandatory and optional. It also needs to be an API that will be called enough for the value of the two helper methods to outweigh the extra cost. But when it does crop up, it needs a good naming convention as the two methods cannot be overloads. And that is what I'm proposing here.

• Name the method that throws an exception when not found getXxx(String)
• Name the method that returns an optional findXxx(String)

With this approach, both methods are available, and have reasonable names. It seems to me, that when this kind of situation arises, the mandatory case is typically most common, and thus it gets the "get" name.

There are other possible naming approaches to this "near overload". But I've settled on this one as the right balance for my APIs.

Note however, that this is not a suggestion to name all methods returning an Optional something like findXxx(String). Only use it when appropriate, such as when paired with an exception throwing version do I think this makes sense.

Summary

This article outlines a naming approach for when you need overloaded convenience methods on an API, typically for accessing a collection, such as a map or list. It proposes adding one method, getXxx(String), that throws an exception when the key cannot be found, and a second method findXxx(String), that returns an empty optional when the key is not found.

Update 2015-10-15

The original version of this article proposed the name "getXxxOptional()". I've changed that to "findXxx()" based on the comments and experimentation with both options. Its clear to me now that "findXxx()" is a way better method name than "getXxxOptional()" for a situation like this.

Java SE 8 Optional, a pragmatic approach

The Optional classs in Java 8 is a useful tool to help developers manage data. But advice on how to use it varies. This is my take on one good approach to using Optional in Java 8.

Note that this article assumes you know what Optional is and how it works. See my previous article and other tutorials for more info. Also, be aware that Optional is a heavily argued topic, with some commentators liable to get rather too excited about its importance.

A pragmatic approach to Optional in Java 8

What follows is a specific approach to using Optional in Java 8 that I have found very useful. It should be considered that the approach has been developed in terms of writing a new application, rather than maintaining an existing one. There are five basic points:

1. Do not declare any instance variable of type Optional.
2. Use null to indicate optional data within the private scope of a class.
3. Use Optional for getters that access the optional field.
4. Do not use Optional in setters or constructors.
5. Use Optional as a return type for any other business logic methods that have an optional result.

For example:

  public class Address {
    private final String addressLine;  // never null
    private final String city;         // never null
    private final String postcode;     // optional, thus may be null

    // constructor ensures non-null fields really are non-null
    // optional field can just be stored directly, as null means optional
    public Address(String addressLine, String city, String postcode) {
      this.addressLine = Preconditions.chckNotNull(addressLine);
      this.city = Preconditions.chckNotNull(city);
      this.postcode = postcode;
    }

    // normal getters
    public String getAddressLine() { return addressLine; }
    public String getCity() { return city; }

    // special getter for optional field
    public Optional<String> getPostcode() {
      return Optional.ofNullable(postcode);
    }

    // return optional instead of null for business logic methods that may not find a result
    public static Optional<Address> findAddress(String userInput) {
      return ... // find the address, returning Optional.empty() if not found
    }
  }

The first thing to notice about this users of our address API are protected from receiving null. Calling getAddressLine() or getCity() will always return a non-null value, as the address object cannot hold null in those fields. Calling getPostcode() will return an Optional<String> instance that forces callers to at least think about the potential for missing data. Finally, findPostcode() also returns an Optional. None of these methods can return null.

Within the object, the developer is still forced to think about null and manage it using != null checks. This is reasonable, as the problem of null is constrained. The code will all be written and tested as a unit (you do write tests don't you?), so nulls will not cause many issues.

In essence, what this approach does is to focus on using Optional in return types at API boundaries, rather than within a class or on input. Compared to using it as a field, optional is now created on-the-fly. The key difference here is the lifetime of the Optional instance.

It is often the case that domain objects hang about in memory for a fair while, as processing in the application occurs, making each optional instance rather long-lived (tied to the lifetime of the domain object). By contrast, the Optional instance returned from the getter is likely to be very short-lived. The caller will call the getter, interpret the result, and then move on. If you know anything about garbage collection you'll know that the JVM handles these short-lived objects well. In addition, there is more potential for hotspot to remove the costs of the Optional instance when it is short lived. While it is easy to claim this is "premature optimization", as engineers it is our responsibility to know the limits and capabilities of the system we work with and to choose carefully the point where it should be stressed.

While it is a minor point, it should be noted that the class could be Serializable, something that is not possible if any field is Optional (as Optional does not implement Serializable).

The approach above does not use Optional for inputs, such as setters or constructors. While accepting Optional would work, it is my experience that having Optional on a setter or constructor is annoying for the caller, as they typically have the actual object. Forcing the caller to wrap the parameter in Optional is an annoyance I'd prefer not to inflict on users. (ie. convenience trumps strictness on input)

On the downside, this approach results in objects that are not beans. The return type of the getter does not match the type of the field, which can cause issues for some tools. Before adopting this approach, check that any tool you use can handle it, such as by directly accessing the field.

If adopted widely in an application, the problem of null tends to disappear without a big fight. Since each domain object refuses to return null, the application tends to never have null passed about. In my experience, adopting this approach tends to result in code where null is never used outside the private scope of a class. And importantly, this happens naturally, without it being a painful transition. Over time, you start to write less defensive code, because you are more confident that no variable will actually contain null.

The key to making this approach work beyond the basics is to learn the various methods on Optional. If you simply call Optional.get() you've missed the whole point of the class.

For example, here is some code that handles an XML parse where either "effectiveDate" or "relativeEffectiveDate" is present:

 AdjustableDate startDate = tradeEl.getChildOptional("effectiveDate")
   .map(el -> parseKnownDate(el))
   .orElseGet(() -> parseRelativeDate(tradeEl.getChildSingle("relativeEffectiveDate")));

Breaking this down, tradeEl.getChildOptional("effectiveDate") returns an Optional<XmlElement>. If the element was found, the map() function is invoked to parse the date. If the element was not found, the orElseGet() function is invoked to parse the relative date.

For a large enterprise-style codebase that uses this approach to Optional, see OpenGamma Strata, a modern open source toolkit for the finance industry.

See also Joda-Beans code generation, which can generate this pattern (and much more).

Finally, it should be noted that some future Java version, beyond Java 9, will probably support value types. In this future world, the costs associated with Optional will disappear, and using it far more widely will make sense. I simply argue that now is not the time for an "optional everywhere" approach.

Summary

This article outlines a pragmatic approach to using Optional in Java 8. If followed consistently on a new application, the problem of null tends to just fade away.

Any comments?

Converting from Joda-Time to java.time

What steps are needed to upgrade from Joda-Time to Java SE 8 date and time (JSR-310)?

From Joda-Time to java.time

Joda-Time has been a very successful date and time library, widely used and making a real difference to many applications over the past 12 years or so. But if you are moving your application to Java SE 8, its time to consider moving on to java.time, formerly known as JSR-310.

The java.time library contains many of the lessons learned from Joda-Time, including stricter null handling and a better approach to multiple calendar systems. I use the phraseology that java.time is "inspired by Joda-Time", rather than an exact derivation, however many concepts will be familiar.

Converting types

Classes not including a time-zone:

Joda-Time	java.time	Notes
LocalDate	LocalDate	Same concept - date without time or time-zone
YearMonthDay
LocalTime	LocalTime	Same concept - time without date or time-zone
TimeOfDay
LocalDateTime	LocalDateTime	Same concept - date and time without time-zone
MonthDay	MonthDay	Same concept - month and day without time-zone
YearMonth	YearMonth	Same concept - year and month without time-zone
-	Year	New concept - a value type for the year
-	Month	New concept - an enum for the month-of-year
-	DayOfWeek	New concept - an enum for the day-of-week
Partial	-	Not included in java.time

Classes including a time-zone or representing an instant:

Joda-Time	java.time	Notes
DateTime	ZonedDateTime	Same concept - Date and time with time-zone
	OffsetDateTime	New concept - Date and time with offset from Greenwich/UTC
MutableDateTime	-	Not included in java.time, use immutable ZonedDateTime instead
DateMidnight	-	Deprecated as a bad idea in Joda-Time, Use LocalDate or ZonedDateTime
-	OffsetTime	New concept - Time with offset from Greenwich/UTC
Instant	Instant	Same concept - Instantaneous point on timeline
DateTimeZone	ZoneId	Same concept - Identifier for a time-zone, such as Europe/Paris
	ZoneOffset	New concept - An offset from Greenwich/UTC, such as +02:00

Amounts of time:

Joda-Time	java.time	Notes
Duration	Duration	Same concept - Amount of time, based on fractional seconds
Period	Period and/or Duration	Similar concept - Amount of time Joda-Time Period includes years to milliseconds, java.time only year/month/day (see also (ThreeTen-Extra PeriodDuration)
MutablePeriod	-	Not included in java.time, use immutable Period or Duration instead
Years	-	Not included in java.time, use Period instead (or ThreeTen-Extra)
Months
Weeks
Days
Hours	-	Not included in java.time, use Duration instead (or ThreeTen-Extra)
Minutes
Seconds
Interval	-	Not included in java.time (see ThreeTen-Extra)
MutableInterval	-	Not included in java.time

Formatting:

Joda-Time	java.time	Notes
DateTimeFormatter	DateTimeFormatter	Same concept - an immutable formatter
DateTimeFormatterBuilder	DateTimeFormatterBuilder	Same concept - a builder of the formatter
DateTimeFormat	DateTimeFormatter	Concepts exposed as static methods on DateTimeFormatter
ISODateTimeFormat
PeriodFormatter	-	Not included in java.time (see ThreeTen-Extra for limited support)
PeriodFormatterBuilder
PeriodFormat
ISOPeriodFormat

Other classes and interfaces:

Joda-Time	java.time	Notes
-	Clock	New concept - Standard way to pass around the current time
Chronology	Chronology	Similar concept - Very different API and implementation
ISOChronology	IsoChronology	Similar concept - Very different API and implementation
DateTimeFieldType	ChronoField	Similar concept - Very different API and implementation
DateTimeField
DurationFieldType	ChronoUnit	Similar concept - Very different API and implementation
DurationField
PeriodType	-	Not included in java.time
Readable*	Temporal*	The Readable* interfaces are most closely replaced by the Temporal* interfaces It is strongly recommended to use the value types, not the temporal interfaces

Odds and Ends

In most cases, the table above will be sufficient to identify the class to convert to. After that, in most cases the method needed will be obvious. Some special cases are noted below.

Rounding. Joda-Time has considerable support for rounding, java.time has less, based only on truncation. See the truncatedTo() methods.

End of month. Joda-Time has the withMaximumValue() method on the property object to change a field to the last day of the month. java.time has a more general mechanism, see TemporalAdjusters.lastDayOfMonth().

Nulls. Joda-Time accepts null as an input and handles it with a sensible default value. java.time rejects null on input.

Construction. Joda-Time has a constructor that accepts an Object and performs type conversion. java.time only has factory methods, so conversion is a user problem, although a parse() method is provided for strings.

Time between. Joda-Time provides methods on amount classes such as Days.between(a, b). java.time provides a similar method on the ChronoUnit classes - ChronoUnuit.DAYS.between(a, b).

Additional fields. Joda-Time provides a fixed set of fields in DateTimeFieldType. java.time allows the set of fields to be extended by implementing TemporalUnit, see IsoFields, JulianFields and WeekFields.

Summary

This blog post has outlined the key mappings between Joda-Time and java.time in Java SE 8. If you are writing code in Java SE 8, its time to migrate to java.time, perhaps using ThreeTen-Extra to fill any gaps.

Optional in Java SE 8

Java SE 8 added a new class to handle nullability - Optional - but when should it be used?

Optional in Java SE 8

Optional is not a new concept, but an old one which often results in polarized opinions. It was added to Java 8 and is used in limited places at the moment.

The class itself is a simple box around an object, which has a special marker for empty instead of using null:

  Optional<String> optional = Optional.of("Hello");
  Optional<String> empty = Optional.empty();

The goal of the class is to reduce the need to use null. Instead, Optional.empty() takes the place of null. One way to think of it is as a collection that holds either zero or one objects.

When you want to get the data out of the optional, there is a nice simple method, get(), but this will throw an exception if the optional is empty. In most cases, you will want to call one of the other methods, for example to return a default value:

  Optional<Person> personResult = findPerson(name);
  Person person = personResult.orElse(Person.GUEST);

One approach to using Optional is to use it everywhere in your entire codebase and eliminate null altogether. I don't subscribe to this approach, and nor does Brian Goetz.

Of course, people will do what they want. But we did have a clear intention when adding this feature, and it was not to be a general purpose Maybe or Some type, as much as many people would have liked us to do so. Our intention was to provide a limited mechanism for library method return types where there needed to be a clear way to represent "no result", and using null for such was overwhelmingly likely to cause errors.

The key here is the focus on use as a return type. The class is definitively not intended for use as a property of a Java Bean. Witness to this is that Optional does not implement Serializable, which is generally necessary for widespread use as a property of an object.

Where it is used as a return type, Optional can work very well. Consider the case where you are finding the first matching element in a stream. In this case, what should the stream API do if the stream is empty? Returning Optional makes it clear that this case should be considered when using the API.

For me, the pleasant surprise about Optional is that it has three friends - OptionalInt, OptionalLong and OptionalDouble that wrap an int, long and double. I've made good use of these as return types where previously I would have used Integer, Long and Double. Not only is the increase in meaning helpful, but the ability to call orElse(0) to default the missing value is a useful feature for API users.

My only fear is that Optional will be overused. Please focus on using it as a return type (from methods that perform some useful piece of functionality) Please don't use it as the field of a Java-Bean.

(As a side note, one reason not to use it in this way is that it creates an extra object for the garbage collector. If used as a method return type, the extra object is typically short-lived, which causes the gc little trouble. But when used in a Java-Bean, the object is likely to be long-lived, with a memory/gc overhead. I was bitten by something very similar to this about 13 years ago, and I can categorically say that for server systems running at scale, you don't want lots of little object boxes within your beans. Interestingly, if Optional becomes a value type in a future Java version, then the memory/gc issue goes away.)

Optional in Java SE 8

Java 8 Optional, and its three primitive friends, is very useful when used as a return type from an operation where previously you might have returned null. As written, the class is is not intended for use as a Java-bean property - please don't use it for that.

StringJoiner in Java SE 8

Java SE 8 added a new class for joining strings - StringJoiner. But is it any good?

StringJoiner

Here is what the Javadoc of the new class says:

StringJoiner is used to construct a sequence of characters separated by a delimiter and optionally starting with a supplied prefix and ending with a supplied suffix.

Sounds good! Maybe we can finally stop using the Joiner class in Guava.

Unfortunately, the JDK StringJoiner is not a general purpose string joiner in the way that the Guava Joiner is. The first clue is in the API:

// constructors
 StringJoiner(CharSequence delimiter)
 StringJoiner(CharSequence delimiter, CharSequence prefix, CharSequence suffix)
 // methods
 StringJoiner setEmptyValue(CharSequence emptyValue)
 StringJoiner add(CharSequence newElement)
 StringJoiner merge(StringJoiner other)
 int length()
 String toString()

There are two constructors, one taking the separator (delimiter) and one allowing a prefix and suffix. There are then just 5 other methods. By contrast, the Guava version has 15 other methods on top of 2 factory methods.

But the real missing thing? A method to add multiple elements at once to the joiner!

Every time I want to join, I have a list, set or other iterable. With Guava I simply say:

 String joined = Joiner.on(", ").join(list);

StringJoiner has no equivalent method. You have to add the elements one by one using add(CharSequence)!

 StringJoiner joiner = new StringJoiner(", ");
 for (String str : list) {
   joiner.add(str);
 }
 String joined = joiner.toString();

I think we'd all agree that rather defeats the purpose of having a joiner at all!

However, it turns out that it is kind of possible to add multiple with the JDK, but you might not spot it:

 String joined = String.join(", ", list);

So, not too bad then?

Firstly, I don't expect the method to actually perform a useful join to be on String, I expect it to be on StringJoiner. The method on String is not referenced from StringJoiner at all.

Secondly, the method on String is static, whereas the Guava method is an instance method. This means that the Guava method can pickup additional state from the builder phase of the joiner, such as the ability to handle null. The Guava joiner can in fact handle Map joins as well thanks to its clever immutable instance-based design.

Thirdly, StringJoiner only works on CharSequence. By contrast, Guava's Joiner works on Object, which is much more useful in most circumstances.

Rationale

So, why was StringJoiner written this way?

Well, partly, it is just bad API design. But the reason why no-one noticed is because you are not supposed to actually use the class!

The whole StringJoiner API is designed to be a tool used as a Collector, the mutable reduction phase of the new Java SE 8 stream API. In this context StringJoiner itself is not visible:

 String joined = list.stream()
   .map(Object::toString)
   .collect(Collectors.joining(", "));

In the simple case, this is longer than Guava and less discoverable, plus I had to manually map to a string. However, in more advanced stream cases it is a great tool to have.

The other advantage of StringJoiner over Guava Joiner is that it handles prefixes and suffixes. This is actually really useful, the classic example being to output the '[' and ']' at the start and end of a list. Ideally, Guava would add prefix and suffix handling to their Joiner.

The good news is that some of the flaws in StringJoiner can be mitigated in a later JDK version. However, since StringJoiner is fundamentally stateful and mutable it will never be comparable to Guava's Joiner.

Commons-Lang

Amusingly, for many of the day-to-day tasks in string building, the class I developed in Commons-Lang over 12 years ago, StrBuilder is still the best option. It takes the concept of StringBuilder class and adds many additional methods. Relevant to this discussion is:

 return new StrBuilder()
   .append("something")
   .append(somethingElse)
   .appendWithSeparators(list, ", ")
   .toString();

Note how the joining occurs naturally within the middle of a fluent set of method calls. Neither Guava nor JDK joiners can be used in this way.

Summary

The Java SE 8 StringJoiner class is in my opinion nothing more than a behind-the-scenes tool. It should only be used indirectly from String.join() or Collectors.joining(). If you use it directly you are liable to be frustrated.

Personally, I plan to continue using the Guava joiner, unless I am performing a mutable reduction of a stream.

Iterable in Java SE 8

In the last few weeks I've finally had the chance to use lambdas and streams in Java SE 8 in anger. In doing so, I've found much to like, but some rough edges.

Iterable woes

The Iterable interface was added in Java SE 5 to enable the foreach loop to work. It was defined with a single method that returns an iterator:

 public interface Iterable<T> {
   Iterator<T> iterator();
 }

The simplicity of the interface was based around the primary use case, the foreach loop, which needed nothing more than a source of an iterator.

In Java SE 8, the interface changed, as it added two default methods:

 public interface Iterable<T> {
   Iterator<T> iterator();

   default void forEach(Consumer<? super T> action) {
     Objects.requireNonNull(action);
     for (T t : this) {
       action.accept(t);
     }
   }
   
   default Spliterator<T> spliterator() {
     return Spliterators.spliteratorUnknownSize(iterator(), 0);
   }
 }

Isn't this wonderful? Lots of new functionality for free! Er well, no.

Despite following the lambda-dev mailing lists, the problem I now have with Iterable isn't one I foresaw. Yet it took just a few days of active use of Java SE 8 to realise that Iterable is now a lot less useful than it was.

The problem as I see it, is that Iterable has been used for two different things in my code. The first, is as the parameter of a method, where it acted as a very abstract form of collection. The second is as a "common" interface on a domain object.

Receiving an Iterable parameter

The first case is where a method has been written to receive an Iterable parameter. In my code, I have defined many methods that take an Iterable, often alongside a varargs array:

   void addNames(Name... names);
   void addNames(Iterable<Name> names);

This pairing provides great flexibility to callers. If you have a set of actual name objects, you can use the varargs form. If you have an array of name objects, you can also use the varargs form. And if you have virtually any kind of collection, you can use the iterable form. (That abstract sense of just providing an iteration really broadens the kind of inputs allowed and blurs the line between collections and non-collections).

Code within the addName(Iterable) method will benefit to some degree from the new methods on Iterable. That is because the method receiving the Iterable knows nothing else about the input type other than it is an iterable.

Given this, the addition of the new forEach(Consumer) method is reasonable, as it provides additional looping options. But, in reality, there is little difference between these two:

   void addNames(Iterable<Name> names) {
     for (Name name : names) { ... }
   }
   void addNames(Iterable<Name> names) {
     names.forEach(() -> ...);
   }

Yes its a new method, but the actual new functionality is limited. Nevertheless, from this perspective it is a win.

By contrast, the addition of the new spliterator() method simply leads to frustration, as by itself the method has no value. What would have had value would have been a stream() method but there are good reasons why stream() was not added.

Instead, we have to write some rather ugly code to get the actually useful stream:

   void addNames(Iterable<Name> names) {
     StreamSupport.stream(names.spliterator(), false)
    .doMyWizzyStreamThing();
   }

As an aside, arrays manage this better:

   void addNames(Name[] names) {
     Stream.of(names)
    .doMyWizzyStreamThing();
   }

Now why oh why oh why is there no Stream.of(Iterable) method?

Actually, I know why, but that is a story for another day...

Streamable?

Perhaps you're thinking that the reason why Iterable isn't supported well is that there is a Streamable interface that you are supposed to use for this purpose instead?

Er, no. There was a Streamable interface, but it got removed during development. There is no JDK supplied abstraction for a type that provides streams.

Perhaps this is philosophical - you could just declare and call using a supplier:

   void addNames(Supplier<Stream<T>> streamSupplier) {
     streamSupplier.get()
    .doMyWizzyStreamThing();
   }
   Collection<T> coll = ...
   addNames(coll::stream));
   T[] array = ...
   addNames(Stream.of(array));

Definitely a different kind of use of the type system.

After much thought and playing with the feel of the options available, I decided to keep on declaring methods to take Iterable and varargs as that is friendliest to users of the API. Internally, the mapping has to be done from iterable to stream, which necessitates a convenience method in a helper for my sanity.

Iterable on domain objects

The second major way I use Iterable is where I had more difficulty.

I have found over the years that some domain objects are little more than a type-safe wrapper around a collection. Consider a FooSummary object that contains a list of FooItem. In many cases, I have made FooSummary implement Iterable<FooItem>:

 public class FooSummary implements Iterable<FooItem> {
   List<FooItem> items;
   @Override
   public Iterator<FooItem> iterator() {
     return items.iterator();
   }

Doing this allowed the object to be iterated over directly in a foreach loop.

 FooSummary summary = ...
 for (FooItem item : summary) { ... }

This approach is very useful for those domain objects that are primarily a collection of some other object, yet are distinct enough to have a class of their own.

However, this kind of approach relied on using Iterable as a kind of "common" interface, similar to Comparable or Serializable. These small "common" interfaces work on the basis that they provide a common language across a very broad spectrum of use cases. Comparable does nothing other than define the method needed to participate in comparisons. I used Iterable in exactly that sense - to accept the common language for things that can be iterated over, without any implication that they were actual collections.

In my opinion, usage of Iterable as a "common" interface has been all but destroyed by Java SE 8.

The FooSummary class is essentially a domain object. The methods it provides must make sense as a set in their own right. Adding the iterable() method was perfectly acceptable in exactly the same way as adding comparable() is. It is a well-known method with clearly defined properties and usage. However, in Java SE 8, the domain object has had two additional methods foisted upon it.

The issue is that from the perspective of a user of FooSummary, the two additional methods are net negative, rather than positive. While there is nothing wrong in theory with being able to call forEach(Consumer) or spliterator(), it needs to be a specific and separate decision to add them to the domain object.

In my cases, I definitely do not want those extra methods. As such, I have no choice but to stop treating Iterable as a "common" interface. It can now only be used by classes that really make sense to be treated as collections.

In practical terms, this change simply makes life for callers slightly more verbose:

 FooSummary summary = ...
 for (FooItem item : summary.getItems()) { ... }

Where the line between a collection and a non-collection could previously be blurred through Iterable, that option is not really present any more. As such, I feel an API design tool has been removed from my tool-box.

(Another example I played with was a Tuple interface abstracting over Pair and Triple. Again, while adding an iterator() method to Tuple is desirable, adding forEach(Consumer) and spliterator() was not. Here, none of the available options are very pleasing.)

Summary

The Iterable interface has had two default methods added in Java SE 8. These make reasonable sense when receiving an iterable. Unfortunately, the difficulty of obtaining an actual stream is rather annoying.

The more significant problem for me is that Iterable is no longer suitable for use as a "common" interface. The extra methods, and the threat of more in future releases mean that the use cases for Iterable have been reduced. It should now only be used for classes that really are collections, something that I find to be a significant reduction in usability.

Any thoughts on iterable, "common" interfaces and default methods? Has Iterable been ruined by the addition of default methods?

Turning off doclint in JDK 8 Javadoc

JDK 8 includes many updates, but one is I suspect going to cause quite a few complaints - doclint for Javadoc.

Javadoc doclint

Documentation is not something most developers like writing. With Java, we were fortunate to have the Javadoc toolset built in and easy to access from day one. As such, writing Javadoc is a standard part of most developers life.

The Javadoc comments in source code use a mixture of tags, starting with @, and HTML to allow the developer to express their comment and format it nicely.

Up to JDK 7, the Javadoc tool was pretty lenient. As a developer, you could write anything that vaguely resembled HTML and the tool would rarely complain beyond warnings. Thus you could have @link references that were inaccurate (such as due to refactoring) and the tool would simply provide a warning.

With JDK 8, a new part has been added to Javadoc called doclint and it changes that friendly behaviour. In particular, the tool aim to get conforming W3C HTML 4.01 HTML (despite the fact that humans are very bad at matching conformance wrt HTML).

With JDK 8, you are unable to get Javadoc unless your tool meets the standards of doclint. Some of its rules are:

• no self-closed HTML tags, such as <br /> or <a id="x" />
• no unclosed HTML tags, such as <ul> without matching </ul>
• no invalid HTML end tags, such as </br>
• no invalid HTML attributes, based on doclint's interpretation of W3C HTML 4.01
• no duplicate HTML id attribute
• no empty HTML href attribute
• no incorrectly nested headers, such as class documentation must have <h3>, not <h4>
• no invalid HTML tags, such as List<String> (where you forgot to escape using &lt;)
• no broken @link references
• no broken @param references, they must match the actual parameter name
• no broken @throws references, the first word must be a class name

Note that these are errors, not warnings. Break the rules and you get no Javadoc output.

In my opinion, this is way too strict to be the default. I have no problem with such a tool existing in Javadoc, but given the history of Javadoc, errors like this should be opt-in, not opt-out. Its far better to get slightly broken Javadoc than no Javadoc.

I also haven't been able to find a list of the rules, which makes life hard. At least we can see the source code to reverse engineer them.

Turning off doclint

The magic incantation you need is -Xdoclint:none. This goes on the command line invoking Javadoc.

Maven

If you are running from maven with maven-javadoc-plugin v3.0.0 or later, you need to use the doclint setting, as per the manual. Either add it as a global property:

  <properties>
    <doclint>none</doclint>
  </properties>

or add it to the maven-javadoc-plugin:

  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-javadoc-plugin</artifactId>
      <configuration>
        <doclint>none</doclint>
      </configuration>
    </plugin>
  </plugins>

When using earlier versions of maven-javadoc-plugin (before v3.0.0), you need to use a different setting:

  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-javadoc-plugin</artifactId>
      <configuration>
        <additionalparam>-Xdoclint:none</additionalparam>
      </configuration>
    </plugin>
  </plugins>

Ant

Ant also uses additionalparam to pass in -Xdoclint:none, see the manual.

Gradle

Gradle does not expose additionalparam but Tim Yates and Cedric Champeau advise of this solution:

  if (JavaVersion.current().isJava8Compatible()) {
    allprojects {
      tasks.withType(Javadoc) {
        options.addStringOption('Xdoclint:none', '-quiet')
      }
    }
  }

See also the Gradle manual.

Summary

I don't mind doclint existing, but there is no way that it should be turned on to error mode by default. Getting some Javadoc produced without hassle is far more important than pandering to the doclint style checks. In addition, it is very heavy handed with what it defines to be errors, rejecting plenty of HTML that works perfectly fine in a browser.

I've asked the maven team to disable doclint by default, and I'd suggest the same to Ant and Gradle. Unfortunately, the Oracle team seem convinced that they've made the right choice with errors by default and their use of strict HTML.

Comments welcome, but please note that non-specific "it didn't work for me" comments should be at Stack Overflow or Java Ranch, not here!

JDK helper Math methods

Update 2013-08-16: The three methods discussed below - incrementExact, decrementExact and negateExact - have all made it into JDK 8.

How far should simple helper methods go in the JDK? Specifically for Maths.

JDK helper Math methods

A discussion is occurring on the OpenJDK core-libs list to provide some maths helper methods. But there is a disagreement as to which methods should be provided.

The following methods appear to be going in (to java.math.Math, as static methods):

• addExact(int, int)
• subtractExact(int, int)
• multiplyExact(int, int)
• toIntExact(long)

All the methods are intended to throw an exception on overflow, so you would use addExact instead of a + b if you care that the result fits into an int.

Furthermore, some of these methods are designed such that the JVM may be able to provide better low-level implementations dependent on the processor. The same methods with long as the parameter type are also included.

The discussion has arisen around some additional methods:

• negateExact(int)
• incrementExact(int)
• decrementExact(int)

These are currently going to be rejected on the grounds that "they don't pull their weight in the API". So, this blog post is to canvas a wider opinion.

The proposed alternatives are either that the caller should just write the code themselves, by testing against Integer.MIN_VALUE, or to indirectly use one of the other methods. (If you're wondering how negating an integer could go wrong, read this.)

So, here are the three discussed options for negation:

  int x = ...
  
  // option A - dedicated method
  int a = Math.negateExact(x);
  
  // option B - code at the call-site
  if (x == Integer.MIN_VALUE) {
    throw new ArithmeticException();
  }
  int b = -x;
  
  // option C - indirectly use another API
  int a = Math.subtractExact(0, x);

And here are the options for increment:

  int x = ...
  
  // option A - dedicated method
  int a = Math.incrementExact(x);
  
  // option B - code at the call-site
  if (x == Integer.MAX_VALUE) {
    throw new ArithmeticException();
  }
  int b = x++;
  
  // option C - indirectly use another API
  int a = Math.addExact(x, 1);

For me, I find option A in both cases to be a lot clearer when reading the code, as it says three things - that I have understood the corner case, done something about it, and documented that thinking via the inserted method. Sure there is more code in the library (the JDK) to enable this, but isn't that really the point of libraries? To provide the common code so we don't have to? The alternatives are far less appealing to me.

The counter argument is that the more methods that the JDK provides, the harder it is for users to find things. And the JDK gets larger (bigger jar file) as a result.

So, dear reader, its over to you :-) How far is too far when adding methods to the JDK? Should negate, increment and decrement be provided in the JDK to help protect us against errors, or can/should we use the alternatives above?

The future is in the JEPs

If you want to know the most likely contents of Java 8 and beyond, the JEP process is the best place to look.

JDK Enhancement Proposals

JDK Enhancement Proposals (JEPs) are the simple descriptions of the tasks that are being considered for Java 8. Here is Brian Goetz's take:

JEP stands for "JDK Enhancement Proposal", and is part of our process for building a technical roadmap for the Java platform. Filing a JEP (what we used to called a "one pager") is the first step towards inclusion of a proposed feature in the JDK.

The JEP process document linked below outlines the states a JEP can go through, from Draft (we're just talking) to Submitted (I think this is good enough to review) to Candidate (Group/Area leads think the idea has enough merit to not toss out) to Funded (someone has actually committed resources to making it happen.) The last transition -- to Funded -- is the one at which this goes from being an idea to being part of the plan for some JDK release.
...
What JEP is not: it is not a "suggestion box" for drive-by requests. The JEP process is open to JDK *committers*, and it is unlikely a JEP will gain funding if the author is not prepared to contribute significant effort to the project's implementation or stewardship.

And the formal process document says:

The primary goal of this process is to produce a regularly-updated list of proposals to serve as the long-term Roadmap for JDK Release Projects and related efforts. The Roadmap should extend at least three years into the future so as to allow sufficient time for the most complex proposals to be investigated, defined, and implemented.
...
This process is open to every OpenJDK Committer. Decisions about specific proposals will be made in a transparent manner but are ultimately up to the OpenJDK Lead.

This process does not in any way supplant the Java Community Process. The JCP remains the governing body for all standard Java SE APIs and related interfaces. If a proposal accepted into this process intends to revise existing standard interfaces, or to define new ones, then a parallel effort to design, review, and approve those changes must be undertaken in the JCP, either as part of a Maintenance Review of an existing JSR or in the context of a new JSR.
...
That a particular JEP appears in the Roadmap means only that it is the proposal of record from a technical perspective. There is no guarantee that anyone will work on it, much less that its end result will appear in any JDK Release Project.

At the time of writing, there are 26 JEPs, plus 2 "meta" JEPs.

JEP	Title and Link	My comments
101	Generalized Target-Type Inference	More type inference
102	Process API Updates	Managing OS processes
103	Parallel Array Sorting	From the concurrency group
104	Annotations on Java Types	Annotations everywhere (JSR-305/308)
105	DocTree API	Compile AST access to javadoc
106	Add Javadoc to javax.tools	Programatic access to javadoc tool
107	Bulk Data Operations for Collections	Add lambda aware methods to collections
108	Collections Enhancements from Third-Party Libraries	New collection methods and collection classes
109	Enhance Core Libraries with Lambda	Add lambda aware methods to everything apart from collections
110	New HTTP Client	Replace HttpURLConnection
111	Additional Unicode Constructs for Regular Expressions	Enhance RegEx
112	Charset Implementation Improvements	Enhance CharSet
113	MS-SFU Kerberos 5 Extensions	Security
114	TLS Server Name Indication (SNI) Extension	Security
115	AEAD CipherSuites	Security
116	Extended Validation Certificates	Security
117	Remove the Annotation-Processing Tool (apt)	Replaced by JSR-269
118	Access to Parameter Names at Runtime	Reflective access to parameter names
119	javax.lang.model Implementation Backed by Core Reflection	Enhance low level compiler model
120	Repeating Annotations	Allow multiple annotations of the same type
121	Stronger Algorithms for Password-Based Encryption	Security
122	Remove the Permanent Generation	Major change to JVM memory usage
123	Configurable Secure Random-Number Generation	Better random numbers (Security)
124	Enhance the Certificate Revocation-Checking API	Security
125	Network Interface Aliases, Events, and Defaults	Such as listening to wifi and mobile networks
126	Lambda Expressions and Virtual Extension Methods	Main language change of Project lambda

Overall a good set of items so far, with access to parameter names and collection enhacements being of particular interest to me.

Since Oracle isn't accepting "drive by" enhancement requests, feel free to post your thoughts in the comments!