Migrating from untyped macros

[stanch]

Hi,

Do you have any plans for migration from now deprecated untyped macros? I am not still sure if this is going to work, but maybe one could provide implicit views from Monad[A]/Functor[A] to A (e.g. with a @compileTimeOnly annotation), so that the code typechecks, and the macro would just disregard them, replacing with maps or binds. What do you think?

Nick

[aztek]

Hi,

It would certainly be great to get rid of untyped macros, but I can't think of how to do it right now.

To make your solution work, we would need to have implicit view for every instance of monad/functor separately, e.g. Option[A] => A (with the way things currently are) and somehow make it available only within context/workflow block so it won't mess with other code. Composition of functors/idioms won't work anymore (List[Option[A]] => A can't be resolved from List[A] => A and Option[A] => A). Also, powerful implicits like those might interfere with other implicits. For instance, Some("foo") + "bar" is valid Scala because of Predef.StringAdd implicit class and Option[String] => String simply wouldn't be called.

I'll keep trying to come up with something and, of course, all the ideas are highly appreciated.

[stanch]

To make your solution work, we would need to have implicit view for every instance of monad/functor separately, e.g. Option[A] => A (with the way things currently are) and somehow make it available only within context/workflow block so it won't mess with other code.

Well, providing the views for every functor is not that high a price to pay :) As for the messing, I believe @compileTimeOnly should keep them rather sane.

Composition of functors/idioms won't work anymore (List[Option[A]] => A can't be resolved from List[A] => A and Option[A] => A).

That’s true. Still can be done with an “explicit” implicit though, which is of course ugly.

For instance, Some("foo") + "bar" is valid Scala

This is an interesting matter. How do you know anyway—with the current implementation—that it’s not point(Some("foo").toString + "bar")? However, if part of the code typechecks like this and then fails typechecking somewhere midway because of wrong inferred types, things will break.

[stanch]

By the way, I just realized that the new macro annotations expand before type-checking! So conceptually a ported version could look like so:

@workflow[Option[_]] val x = Some(3) + 5

However, I just tried to pass the type argument through and it didn’t work quite well. It’s also unclear how to pass normal arguments, if we want @wokflow(zipList). @xeno-by, any thoughts on this?

[aztek]

How do you know anyway—with the current implementation—that it’s not point(Some("foo").toString + "bar")?

That's how rewriting works — we start with the most nested subexpression. Basically, the whole expression never gets to be typechecked as is (unless we walked through all of its subexpressions and left them untouched). In this case, Some("foo") is typechecked before Some("foo") + "bar", and rewritten with synthetic String argument to map.

[xeno-by]

What exactly didn't work?

[stanch]

@xeno-by

class Workflow[A] extends StaticAnnotation {
  def macroTransform(annottees: Any*) = macro WorkflowMacros.macroTransform[A]
}

object WorkflowMacros {
  def macroTransform[A: c.WeakTypeTag](c: MacroContext)(annottees: c.Expr[Any]*): c.Expr[Any] = {
    import c.universe._
    println(weakTypeOf[A])
    c.Expr[Any](Block(DefDef(Modifiers(), newTermName("x"), List(), List(), TypeTree(), Apply(Select(Literal(Constant(3)), newTermName("$plus")), List(Literal(Constant("asd"))))), Literal(Constant(()))))
  }
}

scala> import foo.Workflow
import foo.Workflow

scala> @Workflow[Int] val x = 4
x: Int = 4

So the annotation is not doing anything at all. Without the type arg it works fine:

class Workflow extends StaticAnnotation {
  def macroTransform(annottees: Any*) = macro WorkflowMacros.macroTransform
}

object WorkflowMacros {
  def macroTransform(c: MacroContext)(annottees: c.Expr[Any]*): c.Expr[Any] = {
    import c.universe._
    //println(weakTypeOf[A])
    c.Expr[Any](Block(DefDef(Modifiers(), newTermName("x"), List(), List(), TypeTree(), Apply(Select(Literal(Constant(3)), newTermName("$plus")), List(Literal(Constant("asd"))))), Literal(Constant(()))))
  }
}

scala>   import foo.Workflow
import foo.Workflow

scala> @Workflow val x = 4
x: String = 3asd

Another question is if we have a macro annotation with arguments (class Workflow(a: Int) extends StaticAnnotation), how do we access them in impls?

[aztek]

Another issue is that we need higher-kinded type as the argument. I seem to remember, that c.WeakTypeTag didn't work for that.

[stanch]

That's how rewriting works

I understand that, but what happens in this example?

workflow[Option] {
  Some(Success(42)) getOrElse "foobar"
}

is it option.map(x$1 ⇒ x$1.getOrElse("foobar"))(Some(Success(42)) or option.point(Some(Success(42)).getOrElse("foobar")) ? Maybe there’s no issue and I just need to sleep for a while :)

[aztek]

The first one. Method invocation is represented somewhat like Apply(Select('Some(Success(42))', 'getOrElse'), List("foobar")), so again, we start with the most nested subexpression: 42, then Success(42), then Some(Success(42)), that gets rewritten as synthetic x$1 : Success[Int] and getOrElse method will be taken from Try class, not Option.

Whereas

workflow[Option] {
  Some(Failure(new Exception)) getOrElse "foobar"
}

will produce Some("foobar").

[xeno-by]

@stanch

a) That's a bug: scalamacros/paradise#2. Probably would be easy to fix, but I don't have much time for that. If you need this fix badly, I can carry it out tomorrow. Otherwise, let's keep it on-demand.

2. c.prefix

[stanch]

@aztek It makes sense! Thanks for the explanation.

@xeno-by re 2) Lol now I remember that you suggested the same when we submitted https://issues.scala-lang.org/browse/SI-7736 :) So I’ll also try to get the type args with it and see how it goes.

Upd: even if I just put class Workflow[A] extends StaticAnnotation and never mention A again, the macro still does not expand.

[stanch]

Check this out!
stanch@506fcc7#L0R49
https://github.com/stanch/scala-workflow/blob/e0325cc641fd4ed952337a0aec797d2777a0247a/project/Build.scala

scala> @workflow(option) val x = Some("f") + Some("o")*2 + Some("bar")
x: Option[String] = Some(foobar)

However:

scala> @workflow(option) val x = Some(2) + 3
error: ambiguous reference to overloaded definition,
both method + in class Int of type (x: Char)Int
and  method + in class Int of type (x: Short)Int
match expected type ?
<console>:10: error: type mismatch;
 found   : Int(3)
 required: String
       @workflow(option) val x = Some(2) + 3
                                           ^

I am curious if this is an issue specific to how the tree is parsed/checked, e.g. https://github.com/stanch/scala-workflow/blob/506fcc75196406ce59ea5781630ea673768f8b50/core/src/main/scala/scala/workflow/package.scala#L195

[aztek]

Oh, cool!

I'm not sure about the error. Looks like the whole expression is typechecked at some point (you'll get the same error message without the annotation). Generated code is correct, so rewriter/typechecker worked well.

[stanch]

I think the first error happens somewhere in rewrite. The second one is just due to the failed annotation expansion. I will look into this today. We can also try to revive the context and $ duo in this fashion:

@context(option)
def foo(bar: Int) = {
  @$ val a = 4 // a = Some(4)
  a
}

Of course the annotations are not as good-looking as functions, and you can put them into fewer places. But on the other hand, I believe we can do

class async extends workflow(future)
@async val x = 5 // x = Future(5)

which is not bad!

Upd: The shown implementation of @async does not suffice because of how c.prefix works. I’m looking into it.