scalameta/paradise


Note, Scalameta-based macro annotations will soon be superseded by https://github.com/scalamacros/scalamacros, see also https://github.com/scalacenter/advisoryboard/pull/30. Scalameta-based macro annotations have several limitations such as not working with

In addition, efforts to support def macros and Dotty have been moved to scalamacros/scalamacros. New scalameta/paradise bug reports or feature requests will not be addressed. New scalameta/paradise pull requests to fix known issues will continue to be reviewed and we are happy to cut new releases with contributed fixes.

Setup build


It's possible to write macro annotations on Scalameta trees using the Scalameta paradise compiler plugin. To configure the Scalameta paradise plugin, you need to enable it in your build for both the projects that define macro annotation and the projects that use macro annotations
lazy val macroAnnotationSettings = Seq(
  addCompilerPlugin("org.scalameta" % "paradise" % "3.0.0-M10" cross CrossVersion.full),
  scalacOptions += "-Xplugin-require:macroparadise",
  scalacOptions in (Compile, console) ~= (_ filterNot (_ contains "paradise")) // macroparadise plugin doesn't work in repl yet.
)
// Requires scalaVersion 2.11.12 or 2.12.4
lazy val projectThatDefinesMacroAnnotations = project.settings(
  libraryDependencies += "org.scalameta" %% "scalameta" % "1.8.0" % Provided,
  macroAnnotationSettings
  // ... your other project settings
)
lazy val projectThatUsesMacroAnnotations = project.settings(
  macroAnnotationSettings,
  // ... your other project settings
)
These settings are already configured in the tutorial repo. Once you are setup, run
macros/test:run

Tips


Examples


Here are some example macro annotations.

Hello World

Here is an example macro annotation:
package scalaworld.macros

import scala.meta._
import scala.collection.immutable.Seq

class Main extends scala.annotation.StaticAnnotation {
  inline def apply(defn: Any): Any = meta {
    defn match {
      case q"object $name { ..$stats }" =>
        MainMacroImpl.expand(name, stats)
      case _ =>
        abort("@main must annotate an object.")
    }
  }
}

// This is an example how we can refactor the macro implementation into a utility
// function which can be used for unit testing, see MainUnitTest.
object MainMacroImpl {
  def expand(name: Term.Name, stats: Seq[Stat]): Defn.Object = {
    val main = q"def main(args: Array[String]): Unit = { ..$stats }"
    q"object $name { $main }"
  }
}
The annotation wraps the body of an object into a main function, serving a similar function as extending App.

Class2Map

Implement a Class2Map macro annotation that injects a toMap method that creates a Map[String, Any] from the fields of this class.

Solution:

package scalaworld.macros

import scala.collection.immutable.Seq
import scala.meta._

// Before:
// @Class2Map
// class Class2MapExample(a: Int, b: String)(c: List[Int]) {
// After:
// class Class2MapExample(a: Int, b: String)(c: List[Int]) {
//   def toMap: _root_.scala.collection.Map[String, Any] =
//     _root_.scala.collection.Map(("a", a), ("b", b), ("c", c))
// }

class Class2Map extends scala.annotation.StaticAnnotation {
  inline def apply(defn: Any): Any = meta {
    defn match {
      case cls @ Defn.Class(_, _, _, Ctor.Primary(_, _, paramss), template) =>
        val namesToValues: Seq[Term.Tuple] = paramss.flatten.map { param =>
          q"(${param.name.syntax}, ${Term.Name(param.name.value)})"
        }
        val toMapImpl: Term =
          q"_root_.scala.collection.Map[String, Any](..$namesToValues)"
        val toMap =
          q"def toMap: _root_.scala.collection.Map[String, Any] = $toMapImpl"
        val templateStats: Seq[Stat] = toMap +: template.stats.getOrElse(Nil)
        cls.copy(templ = template.copy(stats = Some(templateStats)))
      case _ =>
        println(defn.structure)
        abort("@Class2Map must annotate a class.")
    }
  }
}

WithApply

Implement a WithApply macro annotation that creates a apply method to construct an instance of the class (just like is created for case classes).

The challenge here is to handle the companion object correctly.

Solution:

package scalaworld.macros

import scala.collection.immutable.Seq
import scala.meta._

// Before:
// @WithApply
// class WithApplyExample(a: Int)(b: String)
// After:
// class WithApplyExample(a: Int)(b: String)
// object WithApplyExample {
//   def apply(a: Int)(b: String): WithApplyExample = new WithApplyExample(a)(b)
// }

class WithApply extends scala.annotation.StaticAnnotation {
  inline def apply(defn: Any): Any = meta {
    def createApply(
        name: Type.Name,
        paramss: Seq[Seq[Term.Param]]): Defn.Def = {
      val args = paramss.map(_.map(param => Term.Name(param.name.value)))
      q"""def apply(...$paramss): $name =
            new ${Ctor.Ref.Name(name.value)}(...$args)"""
    }
    defn match {
      // companion object exists
      case Term.Block(
          Seq(cls @ Defn.Class(_, name, _, ctor, _), companion: Defn.Object)) =>
        val applyMethod = createApply(name, ctor.paramss)
        val templateStats: Seq[Stat] =
          applyMethod +: companion.templ.stats.getOrElse(Nil)
        val newCompanion = companion.copy(
          templ = companion.templ.copy(stats = Some(templateStats)))
        Term.Block(Seq(cls, newCompanion))
      // companion object does not exists
      case cls @ Defn.Class(_, name, _, ctor, _) =>
        val applyMethod = createApply(name, ctor.paramss)
        val companion = q"object ${Term.Name(name.value)} { $applyMethod }"
        Term.Block(Seq(cls, companion))
      case _ =>
        println(defn.structure)
        abort("@WithApply must annotate a class.")
    }
  }
}

Debug

Open up Debug.scala and implement a Debug macro annotation for methods that:

Solution:

package scalaworld.macros

import scala.annotation.compileTimeOnly
import scala.meta._

// Before:
// @Debug
// def complicated(a: Int, b: String)(c: Int): Int = {
//   Thread.sleep(500)
//   a + b.length + c
// }
// After:
// def complicated(a: Int, b: String)(c: Int): Int = {
//   {
//     println("a" + ": " + a)
//     println("b" + ": " + b)
//     println("c" + ": " + c)
//   }
//   val start = System.currentTimeMillis()
//   val result = {
//     Thread.sleep(500)
//     a + b.length + c
//   }
//   val elapsed = System.currentTimeMillis() - start
//   println("Method " + "complicated" + " ran in " + elapsed + "ms")
//   result
// }
class Debug extends scala.annotation.StaticAnnotation {
  inline def apply(defn: Any): Any = meta {
    defn match {
      case defn: Defn.Def =>
        val printlnStatements = defn.paramss.flatten.map { param =>
          q"""println(
                ${param.name.syntax} + ": " +
                ${Term.Name(param.name.value)})"""
        }
        val body: Term = q"""
          { ..$printlnStatements }
          val start = _root_.java.lang.System.currentTimeMillis()
          val result = ${defn.body}
          val elapsed = _root_.java.lang.System.currentTimeMillis() - start
          println("Method " + ${defn.name.syntax} + " ran in " + elapsed + "ms")
          result
          """
        defn.copy(body = body)
      case _ =>
        abort("@Debug most annotate a def")
    }
  }
}

For extra credit:

generic

Implement a generic macro annotation to automatically derive a shapeless Generic[T] instance.

Note, macro annotations are purely syntactic. As a result, to find subclasses of a sealed trait, we depend on the assumption that all the subclasses are put under the companion class of the sealed trait. The implementation below looks inside the companion class and extracts definitions of classes which extend the sealed trait.

Solution:

package scalaworld.macros

import scala.collection.immutable.Seq
import scala.meta._

// Before:
// @generic
// case class Foo(i: Int, s: String)
//
// @generic
// sealed trait Bar
// object Bar {
//   case class Baz(i: Int)     extends Bar
//   case class Quux(s: String) extends Bar
// }
//
// After:
// // infix operators are used where possible, avoiding the syntax ::[A, B]
// case class Foo(i: Int, s: String)
// object Foo {
//   implicit val FooGeneric: _root_.shapeless.Generic[Foo] =
//     new _root_.shapeless.Generic[Foo] {
//       import shapeless._
//       type Repr = Int :: String :: HNil
//       def from(r: Repr): Foo = r match {
//         case i :: s :: HNil => new Foo(i, s)
//       }
//       def to(t: Foo): Repr = t.i :: t.s :: HNil
//     }
// }
// sealed trait Bar
// object Bar {
//   implicit val BarGeneric: _root_.shapeless.Generic[Bar] =
//     new _root_.shapeless.Generic[Bar] {
//       import shapeless._
//       type Repr = Baz :+: Quux :+: CNil
//       def from(r: Repr): Bar = r match {
//         case Inl(t)      => t
//         case Inr(Inl(t)) => t
//         case Inr(Inr(cnil)) => cnil.impossible
//       }
//       def to(t: Bar): Repr = t match {
//         case t: Baz  => Inl(t)
//         case t: Quux => Inr(Inl(t))
//       }
//     }
//   case class Baz(i: Int) extends Bar()
//   case class Quux(s: String) extends Bar()
// }

// This implementation contains quite a bit of boilerplate because we
// generate similar code in term, type and pattern position.
class generic extends scala.annotation.StaticAnnotation {
  inline def apply(defn: Any): Any = meta {
    defn match {
      // Sealed ADT, create coproduct Generic.
      case Term.Block(Seq(t @ ClassOrTrait(mods, name), companion: Defn.Object))
          if GenericMacro.isSealed(mods) =>
        val oldTemplStats = companion.templ.stats.getOrElse(Nil)
        val subTypes = oldTemplStats.collect {
          case t: Defn.Class if GenericMacro.inherits(name)(t) => t
        }
        val newStats =
          GenericMacro.mkCoproductGeneric(name, subTypes) +: oldTemplStats
        val newCompanion =
          companion.copy(templ = companion.templ.copy(stats = Some(newStats)))
        Term.Block(Seq(t, newCompanion))
      // Plain class with companion object, create HList Generic.
      case Term.Block(
          Seq(cls @ Defn.Class(_, name, _, ctor, _), companion: Defn.Object)) =>
        val newStats =
          GenericMacro.mkHListGeneric(name, ctor.paramss) +:
            companion.templ.stats.getOrElse(Nil)
        val newCompanion =
          companion.copy(templ = companion.templ.copy(stats = Some(newStats)))
        Term.Block(Seq(cls, newCompanion))
      // Plain class without companion object, create HList Generic.
      case cls @ Defn.Class(_, name, _, ctor, _) =>
        val companion =
          q"""object ${Term.Name(name.value)} {
                ${GenericMacro.mkHListGeneric(name, ctor.paramss)}
              }
           """
        Term.Block(Seq(cls, companion))
      case defn: Tree =>
        println(defn.structure)
        abort("@generic must annotate a class or a sealed trait/class.")
    }
  }
}

object GenericMacro {
  def mkCoproductTerm(depth: Int): Term =
    if (depth <= 0) q"Inl(t)"
    else q"Inr(${mkCoproductTerm(depth - 1)})"

  def mkCoproductPattern(depth: Int): Pat =
    if (depth <= 0) p"Inl(t)"
    else p"Inr(${mkCoproductPattern(depth - 1)})"

  // final unreachable case in `from` for coproduct generic.
  def mkCantHappen(depth: Int): Pat =
    if (depth <= 0) p"Inr(cnil)"
    else p"Inr(${mkCantHappen(depth - 1)})"

  def mkGeneric(
      name: Type.Name,
      repr: Type,
      to: Term,
      from: Seq[Case],
      importStat: Stat): Stat = {
    val reprTyp: Stat = q"type Repr = $repr"
    val toDef: Stat = q"def to(t: $name): Repr = $to"
    val fromDef: Stat =
      q"def from(r: Repr): $name = r match { ..case $from }"
    val implicitName = Pat.Var.Term(Term.Name(name.syntax + "Generic"))

    q"""implicit val $implicitName: _root_.shapeless.Generic[$name] =
            new _root_.shapeless.Generic[$name] {
              $importStat
              $reprTyp
              $toDef
              $fromDef
            }
       """
  }

  def mkCoproductGeneric(
      superName: Type.Name,
      subTypes: Seq[Defn.Class]): Stat = {
    val coproductType: Type = subTypes.foldRight[Type](t"CNil") {
      case (cls, accum) =>
        t"${cls.name} :+: $accum"
    }
    val coproductTermCases: Seq[Case] = subTypes.zipWithIndex.map {
      case (cls, i) =>
        p"case t: ${cls.name} => ${mkCoproductTerm(i)}"
    }
    val coproductTerm = q"t match { ..case $coproductTermCases }"
    val coproductPat: Seq[Case] = subTypes.zipWithIndex.map {
      case (cls, i) =>
        p"case ${mkCoproductPattern(i)} => t"
    }
    val cantHappen =
      p"""case ${mkCantHappen(subTypes.length - 1)} =>
              cnil.impossible
         """
    mkGeneric(
      superName,
      coproductType,
      coproductTerm,
      coproductPat :+ cantHappen,
      q"import shapeless.{CNil, :+:, Inr, Inl}")
  }

  def mkHListGeneric(name: Type.Name, paramss: Seq[Seq[Term.Param]]): Stat = {
    val params = paramss match {
      case params :: Nil => params
      case _ => abort("Can't create generic for curried functions yet.")
    }
    val hlistType: Type = params.foldRight[Type](t"HNil") {
      case (Term.Param(_, _, Some(decltpe: Type), _), accum) =>
        t"$decltpe :: $accum"
      case (param, _) =>
        abort(s"Unsupported parameter ${param.syntax}")
    }
    val hlistTerm: Term = params.foldRight[Term](q"HNil") {
      case (param, accum) =>
        q"t.${Term.Name(param.name.value)} :: $accum"
    }
    val hlistPat: Pat = params.foldRight[Pat](q"HNil") {
      case (param, accum) =>
        p"${Pat.Var.Term(Term.Name(param.name.value))} :: $accum"
    }
    val args = params.map(param => Term.Name(param.name.value))
    val patmat =
      p"case $hlistPat => new ${Ctor.Ref.Name(name.value)}(..$args)"
    mkGeneric(
      name,
      hlistType,
      hlistTerm,
      Seq(patmat),
      q"import shapeless.{::, HNil}")
  }

  def isSealed(mods: Seq[Mod]): Boolean = mods.exists(_.syntax == "sealed")

  // Poor man's semantic API, we check that X in `class Foo extends X`
  // matches syntactically the name of the annotated sealed type.
  def inherits(superType: Type.Name)(cls: Defn.Class): Boolean =
    cls.templ.parents.headOption.exists {
      case q"$parent()" => parent.syntax == superType.syntax
      case _ => false
    }
}

object ClassOrTrait {
  def unapply(any: Defn): Option[(Seq[Mod], Type.Name)] = any match {
    case t: Defn.Class => Some((t.mods, t.name))
    case t: Defn.Trait => Some((t.mods, t.name))
    case _ => None
  }
}

Testing macro annotations

See MainTest for an example of to both unit test and integration test a macro annotation.
package scalaworld.macros

import scala.meta._
import scala.meta.testkit._
import java.io.ByteArrayOutputStream
import java.io.PrintStream
import org.scalatest.FunSuite

// If you are doing complicated macro expansions, it's recommeded to unit test
// the trickiest bits instead of relying only on integration tests.
class MainUnitTest extends FunSuite {

  // TODO(olafur) this method should be exposed in testkit
  def assertStructurallyEqual(obtained: Tree, expected: Tree): Unit = {
    StructurallyEqual(obtained, expected) match {
      case Left(AnyDiff(x, y)) =>
        fail(s"""Not Structurally equal!:
                |obtained: $x
                |expected: $y
             """.stripMargin)
      case _ =>
    }
  }

  test("@Main creates a main method") {
    val obtained = MainMacroImpl.expand(
      q"AnswerToEverything",
      List(q"val x = 42", q"println(x)"))
    val expected =
      q"""
        object AnswerToEverything {
          def main(args: Array[String]): Unit = {
            val x = 42
            println(x)
          }
        }
       """
    assertStructurallyEqual(obtained, expected)
  }
}

// This is an integration tests because it requires running the macro expansion
// through the entire compiler pipeline, if you have a bug in your macro annotation
// the expanded code may not compile causing your test suite to not compile.
class MainIntegrationTest extends FunSuite {
  test("@Main creates a main method") {
    val out: ByteArrayOutputStream = new ByteArrayOutputStream()
    Console.withOut(new PrintStream(out)) {
      MainExample.main(Array())
    }
    assert(out.toString.stripLineEnd == "Hello Scalameta macros!")
  }
}

FAQ


How do I test a macro annotation?

You can either unit test or integration test your macro annotation. See Testing macro annotations

How do identify a particular annotation?

Maybe this help, https://stackoverflow.com/questions/43394357/scalameta-identify-particular-annotations

How do I get the type of a tree?

It is not possible to query for type information from a macro annotation. Macro annotations are purely syntactic, and there is no plan to add support for capabilities such as getting the fields/member of a type/class. Why? Because you hit on chicken vs. egg problems. There is a cyclic dependency between The generated definitions are necessary to type check the source file, and the type checker is necessary provide type information to the macro annotation.

Do I need to depend on Scalameta at runtime?

No. But your project needs a dependency on Scalameta. If you only use Scalameta at compile time, you can mark the dependency as % "provided" to exclude it from your runtime application.

How do I use macro annotations provided by a third-party library?

If your project depends on a library that provides macro annotations, you need to enable the `paradise` compiler plugin and declare a dependency on `scala-meta` so that macro annotations could be expanded:
addCompilerPlugin(
  ("org.scalameta" % "paradise" % paradiseVersion).cross(CrossVersion.full)
)

libraryDependencies +=
  "org.scalameta" %% "scalameta" % scalametaVersion % Provided

Here is a complete `settings` definition necessary and sufficient to enable dependent project to use the library (including workarounds for features that are being currently worked on):
lazy val enableMacroAnnotations: Seq[Def.Setting[_]] = Seq(
  addCompilerPlugin("org.scalameta" % "paradise" % paradiseVersion cross CrossVersion.full),
  libraryDependencies += "org.scalameta" %% "scalameta" % scalametaVersion % Provided,
  scalacOptions += "-Xplugin-require:macroparadise",
  // macroparadise plugin doesn't work in repl yet.
  scalacOptions in (Compile, console) ~= (_ filterNot (_ contains "paradise"))
)

How do I reuse code between macros?

If you try to call a method inside you macro class you get a "X not found" error.
class Argument(arg: Int) extends scala.annotation.StaticAnnotation {
  def helper(t: Any): Stat = ??? // utility method
  inline def apply(defn: Any): Any = meta {
    helper(defn) // ERROR: `helper` not found
    // Why? `this` is a Scalameta tree.
}
You can move the utility method to an external object.
package scalaworld.macros

import scala.meta._

object MacroUtil {
  def helper(defn: Any): Stat = q"class ReuseExample"
}

class Reuse extends scala.annotation.StaticAnnotation {
  inline def apply(defn: Any): Any = meta {
    MacroUtil.helper(defn)
  }
}

Incremental compiler is messing up / stale code

While editing the macro, it can be handy to keep this command running in SBT
~; macros/test:clean ; macros/test:run
Incremental compilation caches the macro expansion you need to clean the test project on every run.

My IDE/IntelliJ shows red squiggly marks

Your IDE might be all red like this

There are two possible workarounds:
  1. (Recommended if using IntelliJ) First, install the 2016.3 EAP. Then, select nightly or EAP update channel from Updates tab in Scala plugin settings Settings -> Languages and frameworks -> Scala -> Updates.
  2. (hack) Add import autocomplete._ to your file and a semicolon after inline, like this

    Remember to remove the semicolon when you run your macro.

New-style def macros

Scalameta doesn't yet provide a possibility to write new-style def macros, but we are working hard on implementing this functionality. Attend Eugene Burmako's talk at Scala eXchange 2016 to learn more about our progress.

Compatibility with traditional macros

At the moment, new-style macros can only take apart existing Scala syntax and generate new syntax (so called syntactic API). This corresponds to the functionality provided by traditional macro annotations that only use tree constructors and quasiquotes.

Even this limited functionality should be enough to port most of the existing macro annotations to Scalameta. Oleksandr Olgashko has ported a large subset of Simulacrum's @typeclass features to new-style macros, so we are confident that new-style macros are powerful enough to support even more complex annotations.

For new-style def macros, we are working on semantic API, which will provide compiler information such as type inference, name resolution and other functionality that requires typechecking. It is too early to tell how compatible this API will be with what is provided by scala.reflect. We will provide more information as the design of the semantic API shapes up.

Which versions of Scala do the Scalameta macros support?

2.11.x and 2.12.x.

';' expected but 'def' found. inline def apply

Be sure that the org.scalameta:paradise compiler plugin is enabled.

How do I pass an argument to the macro annotation?

You match on this as a Scalameta tree. For example:
package scalaworld.macros

import scala.meta._

class Argument(arg: Int) extends scala.annotation.StaticAnnotation {
  inline def apply(defn: Any): Any = meta {
    // `this` is a scala.meta tree.
    println(this.structure)
    val arg = this match {
      // The argument needs to be a literal like `1` or a string like `"foobar"`.
      // You can't pass in a variable name.
      case q"new $_(${Lit.Int(arg)})" => arg
      // Example if you have more than one argument.
      case q"new $_(${Lit.Int(arg)}, ${Lit.String(foo)})" => arg
      case _ => ??? // default value
    }
    println(s"Arg is $arg")
    defn.asInstanceOf[Stat]
  }
}

What's the status on new macros style with Dotty?

See https://github.com/scalacenter/advisoryboard/pull/30.

Will newstyle macros would remove the limitation of two tiered compilation?

No. The major focus of the new macro system is portability (same implementation running on scalac + dotty + intellij). See https://github.com/scalacenter/advisoryboard/pull/30.

Testing macro annotations


See MainTest for an example of to both unit test and integration test a macro annotation.
package scalaworld.macros

import scala.meta._
import scala.meta.testkit._
import java.io.ByteArrayOutputStream
import java.io.PrintStream
import org.scalatest.FunSuite

// If you are doing complicated macro expansions, it's recommeded to unit test
// the trickiest bits instead of relying only on integration tests.
class MainUnitTest extends FunSuite {

  // TODO(olafur) this method should be exposed in testkit
  def assertStructurallyEqual(obtained: Tree, expected: Tree): Unit = {
    StructurallyEqual(obtained, expected) match {
      case Left(AnyDiff(x, y)) =>
        fail(s"""Not Structurally equal!:
                |obtained: $x
                |expected: $y
             """.stripMargin)
      case _ =>
    }
  }

  test("@Main creates a main method") {
    val obtained = MainMacroImpl.expand(
      q"AnswerToEverything",
      List(q"val x = 42", q"println(x)"))
    val expected =
      q"""
        object AnswerToEverything {
          def main(args: Array[String]): Unit = {
            val x = 42
            println(x)
          }
        }
       """
    assertStructurallyEqual(obtained, expected)
  }
}

// This is an integration tests because it requires running the macro expansion
// through the entire compiler pipeline, if you have a bug in your macro annotation
// the expanded code may not compile causing your test suite to not compile.
class MainIntegrationTest extends FunSuite {
  test("@Main creates a main method") {
    val out: ByteArrayOutputStream = new ByteArrayOutputStream()
    Console.withOut(new PrintStream(out)) {
      MainExample.main(Array())
    }
    assert(out.toString.stripLineEnd == "Hello Scalameta macros!")
  }
}