Scala parameters pattern (Spray routing example)

Question

Sorry about the vague title...wasn't sure how to characterize this.

I've seen/used a certain code construction in Scala for some time but I don't know how it works. It looks like this (example from Spray routing):

path( "foo" / Segment / Segment ) { (a,b) => {  // <-- What's this style with a,b?
...
}}

In this example, the Segements in the path are bound to a and b respectively inside the associated block. I know how to use this pattern but how does it work? Why didn't it bind something to "foo"?

I'm not so interested in how spray works for my purpose here, but what facility of Scala is this, and how would I write my own?

Hypomania

Answer 1

Senia's answer is helpful in understanding the Spray-routing directives and how they use HLists to do their work. But I get the impression you were really just interested in the Scala constructs used in

path( "foo" / Segment / Segment ) { (a,b) => ... }

It sounds as though you are interpreting this as special Scala syntax that in some way connects those two Segment instances to a and b. That is not the case at all.

path( "foo" / Segment / Segment )

is just an ordinary call to path with a single argument, an expression involving two calls to a / method. Nothing fancy, just an ordinary method invocation.

The result of that call is a function which wants another function -- the thing you want to happen when a matching request comes in -- as an argument. That's what this part is:

{ (a,b) => ... }

It's just a function with two arguments. The first part (the invocation of path) and the second part (what you want done when a matching message is received) are not syntactically connected in any way. They are completely separate to Scala. However, Spray's semantics connects them: the first part creates a function that will call the second part when a matching message is received.

Jahvist

I should confess that this is a somewhat simplified explanation with regard to what Spray is actually doing, but being more accurate would have made things less clear. I figured that since the focus was on whether this was Scala syntax I would go for clarity. – Jahvist

Simplified answer. Worked with clarity, for me.Thanks @Jahvist – Roentgenogram

Answer 2

This code is from a class that extends Directives. So all methods of Directives are in scope.

PathMatcher

There is no method / in String, so an implicit conversion is used to convert String to PathMatcher0 (PathMatcher[HNil]) with method /.

Method / takes a PathMatcher and returns a PathMatcher.

Segment is a PathMatcher1[String] (PathMatcher[String :: HNil]).

Method / of PathMatcher[HNil] with PathMatcher[String :: HNil] parameter returns a PathMatcher[String :: HNil].

Method / of PathMatcher[String :: HNil] with PathMatcher[String :: HNil] parameter returns a PathMatcher[String :: String :: HNil]. It's black magic from shapeless. See heterogenous lists concatenation; it is worth reading.

Directive

So you are calling method path with PathMatcher[String :: String :: HNil] as a parameter. It returns a Directive[String :: String :: HNil].

Then you are calling method apply on Directive with Function2[?, ?, ?] ((a, b) => ..) as a parameter. There is an appropriate implicit conversion (see black magic) for every Directive[A :: B :: C ...] that creates an object with method apply((a: A, b: B, c: C ...) => Route).

Parsing

PathMatcher contains rules for path parsing. It returns its result as an HList.

The "foo" matcher matches a String and ignores it (returns HNil).

The A / B matcher combines 2 matchers (A and B) separated by a "/" string. It concatenates the results of A and B using HList concatenation.

The Segment matcher matches a path segment and returns it as a String :: HNil.

So "foo" / Segment / Segment matches a path of 3 segments, ignores the first one and returns the remaining segments as String :: String :: HNil.

Then black magic allows you to use Function2[String, String, Route] ((String, String) => Route) to process String :: String :: HNil. Without such magic you would have to use the method like this: {case a :: b :: HNil => ...}.

Black magic

As @AlexIv noted:

There is an implicit conversion pimpApply for every Directive[A :: B :: C ...] that creates an object with method apply((a: A, b: B, c: C ...) => Route).

It accepts ApplyConverter implicitly. Type member In of ApplyConverter represents an appropriate function (A, B, C ...) => Route for every Directive[A :: B :: C ...].

There is no way to create such implicit values without macros or boilerplate code. So sbt-boilerplate is used for ApplyConverter generation. See ApplyConverterInstances.scala.

Eugenie

As for apply in Directive, there is an implicit conversion to Directive happly here – Grandiloquent

Actually for this they are using typeclass/magnet pattern ApplyConverter which is generated with this, there is a file ApplyConverterInstances.scala which is generted for all other cases – Grandiloquent

Answer 3

Some additional note to senia answer, which is really good.

When you write something like this:

path("foo" / Segment / Segment) { (a,b) => {...} }

you are calling apply method on Directive like senia wrote, but there is no apply method in Directive, so spray is using implicit conversion to happly method. As you can pimpApply is implemented with typeclass pattern ApplyConverter, which is defined only for Directive0 by default. As you can see it's companion object extends ApplyConverterInstances, which is generated with sbt-bolierplate plugin

Grandiloquent

Answer 4

1. Why don't you look into source?

2. As for me, it could be implemented as follows

   • method path takes arbitrary type parameter, some pattern-object of that type and a function from that type:

     def path[T](pattern:Pattern[T])(function:Function[T, `some other type like unit or any`])
     

   • the pattern is constructed with two tricks.

     • The String is either "pimped" to have method / or has an implicit convertion to Pattern[Nothing]

     • the Pattern[T] has method / that constructs another pattern with some new type. The method takes single argument (some ancestor of Segment). I guess — Pattern[T2]:

       trait Pattern[T] {
           ///
           def `/`[T2](otherPattern:Pattern[T2]):Pattern[(T,T2)] 
       }
       

3. So the first argument of path allows to determine the constructed type of pattern as being the pair. Thus we get proper type for the second argument.
4. The actual matching work is done inside path. I thought it was out of the questions scope.

Perkoff

Method path returns Directive (in this case Directive[String :: String :: HNil]) and there is no method apply(String, String) in Directive. It's not so simple. – Eugenie

Looking only at the given code I couldn't guess about some "black magic" behind the scene. Of course there are many implementations possible. The given answer is a simple possible implementation. – Perkoff

(a,b) => {...} is Function2, not Function1. So tricky part is how to use Function1 with path( "foo" / Segment ), Function2 with path( "foo" / Segment / Segment ) and so on. You could create 22 boilerplate methods, but it's not scala way. And the other problem: (a / b) / c and a / (b / c) should return the same result, so it's even more boilerplate code. And / is not the only combination method - there are other PathMatcher combinators. – Eugenie

My point is: you can't create such generic API this way. You'll have to use PartialFunctions as in scala parser combinators. – Eugenie

Yes. I've got your point. It cannot be generally constructed with tuples. Once I implemented something of the kind and I had similar problem of being unable to convert nested tuples (a, (b, c)) to an appropriate Function type. Thus I had to use an imperfect set of boilerplate methods. I see that scalaz' HList solves the problem. – Perkoff

"scalaz' HList"? Do you mean shapeless HList? – Eugenie

Yes, of course. It's shapeless' HList. scalaz and shapeless are listed in typelevel.org/projects and I've messed them. – Perkoff