diff --git a/CHANGES.md b/CHANGES.md new file mode 100644 index 0000000..0fa78b3 --- /dev/null +++ b/CHANGES.md @@ -0,0 +1,51 @@ +# Pillar Changes + +## 1.0.1 + +* Add a "destroy" method to drop a keyspace (iamsteveholmes) + +## 1.0.3 + +* Clarify documentation (pvenable) +* Update DataStax Cassandra driver to version 2.0.2 (magro) +* Update Scala to version 2.10.4 (magro) +* Add cross-compilation to Scala version 2.11.1 (magro) +* Shutdown cluster in migrate & initialize (magro) +* Transition support from StreamSend to Chris O'Meara (comeara) + +## 2.0.0 + +* Allow configuration of Cassandra port (fkoehler) +* Rework Migrator interface to allow passing a Session object when integrating Pillar as a library (magro, comeara) + +## 2.0.1 + +* Update a argot dependency to version 1.0.3 (magro) + +## 2.1.0 + +* Update DataStax Cassandra driver to version 3.0.0 (MarcoPriebe) +* Fix documentation issue where authored_at represented as seconds rather than milliseconds (jhungerford) +* Introduce PILLAR_SEED_ADDRESS environment variable (comeara) + +## 2.1.1 + +* Fix deduplicate error during merge, ref. issue #32 (ilovezfs) + +## 2.2.0 + +* Add feature to read registry from files (sadowskik) +* Add TLS/SSL support(bradhandy, comeara) +* Add authentication support (bradhandy, comeara) + +## 2.3.0 + +* Add multiple stages per migration (sadowskik) + +## 3.0.0 + +* Support Scala 2.12 (comeara) +* Split Pillar command line interface and core library into separate artifacts (comeara) +* Add SLF4J binding for command line interface (comeara) +* Update command-line interface to use command and sub-command structure (comeara) +* Remove RPM build (comeara) \ No newline at end of file diff --git a/README.md b/README.md index 625fe8d..be2aaf0 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,7 @@ # Pillar +[![Maven Central](https://img.shields.io/maven-central/v/com.chrisomeara/pillar_2.12.svg)][pillar_2.12] + Pillar manages migrations for your [Cassandra][cassandra] data stores. [cassandra]:http://cassandra.apache.org @@ -24,29 +26,29 @@ databases with one key difference: Pillar is completely independent from any app ### From Source This method requires [Simple Build Tool (sbt)][sbt]. -Building an RPM also requires [Effing Package Management (fpm)][fpm]. - - % sbt assembly # builds just the jar file in the target/ directory - - % sbt rh-package # builds the jar and the RPM in the target/ directory - % sudo rpm -i target/pillar-1.0.0-DEV.noarch.rpm -The RPM installs Pillar to /opt/pillar. +``` +% sbt assembly # builds a fat jar file in the target/ directory +``` [sbt]:http://www.scala-sbt.org -[fpm]:https://github.com/jordansissel/fpm ### Packages -Pillar is available at Maven Central under the GroupId com.chrisomeara and ArtifactId pillar_2.10 or pillar_2.11. The current version is 2.3.0. +Pillar is available at Maven Central under the GroupId com.chrisomeara and ArtifactId [pillar_2.10][pillar_2.10], +[pillar_2.11][pillar_2.11] or [pillar_2.12][pillar_2.12]. The current version is 3.0.0. #### sbt - libraryDependencies += "com.chrisomeara" % "pillar_2.10" % "2.3.0" +``` +libraryDependencies += "com.chrisomeara" %% "pillar" % "3.0.0" +``` #### Gradle - compile 'com.chrisomeara:pillar_2.10:2.3.0' +``` +compile 'com.chrisomeara:pillar_2.12:3.0.0' +``` ## Usage @@ -69,8 +71,8 @@ Here's the short version: 1. Write migrations, place them in conf/pillar/migrations/myapp. 1. Add pillar settings to conf/application.conf. - 1. % pillar -e development initialize myapp - 1. % pillar -e development migrate myapp + 1. % pillar initialize -e development myapp + 1. % pillar migrate -e development myapp #### Migration Files @@ -179,7 +181,6 @@ application.conf might look like the following: acceptance_test { cassandra-seed-address: ${?PILLAR_SEED_ADDRESS} cassandra-port: ${?PILLAR_PORT} - cassandra-keyspace-name: "pillar_acceptance_test" cassandra-keyspace-name: ${?PILLAR_KEYSPACE_NAME} cassandra-ssl: ${?PILLAR_SSL} cassandra-username: ${?PILLAR_USERNAME} @@ -211,89 +212,62 @@ $JAVA_OPTS are passed through to the JVM when using the pillar executable. #### The pillar Executable -The package installs to /opt/pillar by default. The /opt/pillar/bin/pillar executable usage looks like this: +The Pillar executable usage looks like this: - Usage: pillar [OPTIONS] command data-store + Usage: pillar [initialize|migrate] data-store - OPTIONS + Command: initialize [options] - -d directory - --migrations-directory directory The directory containing migrations + -e, --environment - -e env - --environment env environment + Command: migrate [options] - -t time - --time-stamp time The migration time stamp + -e, --environment - PARAMETERS + -t, --time-stamp - command migrate or initialize + -d, --migrations-directory - data-store The target data store, as defined in application.conf + data-store #### Examples Initialize the faker datastore development environment - % pillar -e development initialize faker + % pillar initialize -e development faker Apply all migrations to the faker datastore development environment - % pillar -e development migrate faker + % pillar migrate -e development faker ### Library -You can also integrate Pillar directly into your application as a library. -Reference the acceptance spec suite for details. +You can also integrate Pillar directly into your application as a library. Reference the [pillar-core][core] repository for +more information regarding Pillar library integration. -### Forks +[core]:https://github.com/comeara/pillar-core + +## Forks Several organizations and people have forked the Pillar code base. The most actively maintained alternative is the [Galeria-Kaufhof fork][gkf]. [gkf]:https://github.com/Galeria-Kaufhof/pillar -### Release Notes - -#### 1.0.1 - -* Add a "destroy" method to drop a keyspace (iamsteveholmes) - -#### 1.0.3 - -* Clarify documentation (pvenable) -* Update DataStax Cassandra driver to version 2.0.2 (magro) -* Update Scala to version 2.10.4 (magro) -* Add cross-compilation to Scala version 2.11.1 (magro) -* Shutdown cluster in migrate & initialize (magro) -* Transition support from StreamSend to Chris O'Meara (comeara) - -#### 2.0.0 - -* Allow configuration of Cassandra port (fkoehler) -* Rework Migrator interface to allow passing a Session object when integrating Pillar as a library (magro, comeara) - -#### 2.0.1 - -* Update a argot dependency to version 1.0.3 (magro) +## Change Log -#### 2.1.0 +Please reference the [Pillar Changes][changes] document. -* Update DataStax Cassandra driver to version 3.0.0 (MarcoPriebe) -* Fix documentation issue where authored_at represented as seconds rather than milliseconds (jhungerford) -* Introduce PILLAR_SEED_ADDRESS environment variable (comeara) +[changes]: CHANGES.md -#### 2.1.1 +## Upgrade Instructions -* Fix deduplicate error during merge, ref. issue #32 (ilovezfs) +Please reference the [Pillar Upgrades][upgrade] document. -#### 2.2.0 +[upgrade]: UPGRADE.md -* Add feature to read registry from files (sadowskik) -* Add TLS/SSL support(bradhandy, comeara) -* Add authentication support (bradhandy, comeara) -#### 2.3.0 -* Add multiple stages per migration (sadowskik) \ No newline at end of file +[pillar_2.10]: https://maven-badges.herokuapp.com/maven-central/com.chrisomeara/pillar_2.10 +[pillar_2.11]: https://maven-badges.herokuapp.com/maven-central/com.chrisomeara/pillar_2.11 +[pillar_2.12]: https://maven-badges.herokuapp.com/maven-central/com.chrisomeara/pillar_2.12 diff --git a/UPGRADE.md b/UPGRADE.md new file mode 100644 index 0000000..59c3dfb --- /dev/null +++ b/UPGRADE.md @@ -0,0 +1,32 @@ +# Pillar Upgrades + +## Version 2 to Version 3 + +When upgrading from version 2 to version 3, please consider the following. + +### Command Line Interface Change + +In version 2, the command line interface required the -e option, which specifies the environment, to come before the +subcommand. The subcommands are ```initialize``` and ```migrate```. + +Version 3 places the subcommand as the first argument, followed by all options, ending with the datastore. + +For example, in version 2, you might have run the following command to initialize your test keyspace: + + % pillar -e test initialize + +In version 3, that command becomes: + + % pillar initialize -e test + +### Code Repackaging + +In version 2, all the Scala classes were packaged in ```com.chrisomeara.pillar```. + +In version 3, the cli has been packaged in ```com.chrisomeara.pillar.cli``` and the core of pillar has been moved to +```com.chrisomeara.pillar.core```. + +### RPM Build Removal + +Version 3 removes the RedHat Package Manager build. If this feature interests you please open a new issue so that we +can discuss how to best re-implement. \ No newline at end of file diff --git a/build.sbt b/build.sbt index e69de29..a734e27 100644 --- a/build.sbt +++ b/build.sbt @@ -0,0 +1,48 @@ +lazy val pillarVersion = "3.0.0" + +organization := "com.chrisomeara" +name := "pillar" +version := pillarVersion +scalaVersion := "2.12.1" +crossScalaVersions := Seq("2.10.6", "2.11.8", "2.12.1") +homepage := Some(url("https://github.com/comeara/pillar")) +licenses := Seq("MIT license" -> url("http://www.opensource.org/licenses/mit-license.php")) +libraryDependencies ++= Seq( + "com.chrisomeara" %% "pillar-core" % pillarVersion, + "com.github.scopt" %% "scopt" % "3.5.0", + "com.typesafe" % "config" % "1.0.1", + "org.mockito" % "mockito-core" % "1.9.5" % "test", + "org.scalatest" %% "scalatest" % "3.0.1" % "test", + "org.slf4j" % "slf4j-simple" % "1.7.22" +) +publishMavenStyle := true +publishTo := { + val nexus = "https://oss.sonatype.org/" + if (isSnapshot.value) + Some("snapshots" at nexus + "content/repositories/snapshots") + else + Some("releases" at nexus + "service/local/staging/deploy/maven2") +} +publishArtifact in Test := false +pomIncludeRepository := { _ => false } +pomExtra := ( + + git@github.com:comeara/pillar.git + scm:git:git@github.com:comeara/pillar.git + + + + comeara + Chris O'Meara + https://github.com/comeara + + ) + +test in assembly := {} +assemblyMergeStrategy in assembly := { + case PathList("javax", "servlet", xs@_*) => MergeStrategy.first + case "META-INF/io.netty.versions.properties" => MergeStrategy.last + case x => + val oldStrategy = (assemblyMergeStrategy in assembly).value + oldStrategy(x) +} diff --git a/project/PillarBuild.scala b/project/PillarBuild.scala deleted file mode 100644 index fc8c4fe..0000000 --- a/project/PillarBuild.scala +++ /dev/null @@ -1,100 +0,0 @@ -import _root_.sbtassembly.Plugin.AssemblyKeys._ -import java.util.NoSuchElementException -import sbt._ -import Keys._ -import sbtassembly.Plugin.{MergeStrategy, PathList} -import xerial.sbt.Sonatype - -object PillarBuild extends Build { - val assemblyTestSetting = test in assembly := {} - val assemblyMergeStrategySetting = mergeStrategy in assembly <<= (mergeStrategy in assembly) { - (old) => { - case PathList("javax", "servlet", xs@_*) => MergeStrategy.first - case "META-INF/io.netty.versions.properties" => MergeStrategy.last - case x => old(x) - } - } - - val dependencies = Seq( - "com.datastax.cassandra" % "cassandra-driver-core" % "3.0.0", - "com.typesafe" % "config" % "1.0.1", - "org.clapper" %% "argot" % "1.0.3", - "org.mockito" % "mockito-core" % "1.9.5" % "test", - "org.scalatest" %% "scalatest" % "2.2.0" % "test" - ) - - val rhPackage = TaskKey[File]("rh-package", "Packages the application for Red Hat Package Manager") - val rhPackageTask = rhPackage <<= (sourceDirectory, target, assembly, version) map { - (sourceDirectory: File, targetDirectory: File, archive: File, versionId: String) => - val rootPath = new File(targetDirectory, "staged-package") - val subdirectories = Map( - "bin" -> new File(rootPath, "bin"), - "conf" -> new File(rootPath, "conf"), - "lib" -> new File(rootPath, "lib") - ) - subdirectories.foreach { - case (_, subdirectory) => IO.createDirectory(subdirectory) - } - IO.copyFile(archive, new File(subdirectories("lib"), "pillar.jar")) - val bashDirectory = new File(sourceDirectory, "main/bash") - bashDirectory.list.foreach { - script => - val destination = new File(subdirectories("bin"), script) - IO.copyFile(new File(bashDirectory, script), destination) - destination.setExecutable(true, false) - } - val resourcesDirectory = new File(sourceDirectory, "main/resources") - resourcesDirectory.list.foreach { - resource => - IO.copyFile(new File(resourcesDirectory, resource), new File(subdirectories("conf"), resource)) - } - val iterationId = try { sys.env("GO_PIPELINE_COUNTER") } catch { case e: NoSuchElementException => "DEV" } - "fpm -f -s dir -t rpm --package %s -n pillar --version %s --iteration %s -a all --prefix /opt/pillar -C %s/staged-package/ .".format(targetDirectory.getPath, versionId, iterationId, targetDirectory.getPath).! - - val pkg = file("%s/pillar-%s-%s.noarch.rpm".format(targetDirectory.getPath, versionId, iterationId)) - if(!pkg.exists()) throw new RuntimeException("Packaging failed. Check logs for fpm output.") - pkg - } - - lazy val root = Project( - id = "pillar", - base = file("."), - settings = Project.defaultSettings ++ sbtassembly.Plugin.assemblySettings ++ net.virtualvoid.sbt.graph.Plugin.graphSettings ++ Sonatype.sonatypeSettings - ).settings( - assemblyMergeStrategySetting, - assemblyTestSetting, - libraryDependencies := dependencies, - name := "pillar", - organization := "com.chrisomeara", - version := "2.3.0", - homepage := Some(url("https://github.com/comeara/pillar")), - licenses := Seq("MIT license" -> url("http://www.opensource.org/licenses/mit-license.php")), - scalaVersion := "2.10.6", - crossScalaVersions := Seq("2.10.6", "2.11.8"), - rhPackageTask - ).settings( - publishTo := { - val nexus = "https://oss.sonatype.org/" - if (isSnapshot.value) - Some("snapshots" at nexus + "content/repositories/snapshots") - else - Some("releases" at nexus + "service/local/staging/deploy/maven2") - }, - publishMavenStyle := true, - publishArtifact in Test := false, - pomIncludeRepository := { _ => false }, - pomExtra := ( - - git@github.com:comeara/pillar.git - scm:git:git@github.com:comeara/pillar.git - - - - comeara - Chris O'Meara - https://github.com/comeara - - - ) - ) -} diff --git a/project/assembly.sbt b/project/assembly.sbt new file mode 100644 index 0000000..39c1bb8 --- /dev/null +++ b/project/assembly.sbt @@ -0,0 +1 @@ +addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.3") diff --git a/project/build.properties b/project/build.properties index 43b8278..27e88aa 100644 --- a/project/build.properties +++ b/project/build.properties @@ -1 +1 @@ -sbt.version=0.13.11 +sbt.version=0.13.13 diff --git a/project/plugins.sbt b/project/plugins.sbt deleted file mode 100644 index 7c0a395..0000000 --- a/project/plugins.sbt +++ /dev/null @@ -1,5 +0,0 @@ -addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.9.0") - -addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.7.4") - -addSbtPlugin("org.xerial.sbt" % "sbt-sonatype" % "0.2.1") diff --git a/src/main/bash/pillar b/src/main/bash/pillar index d488191..c8d033e 100644 --- a/src/main/bash/pillar +++ b/src/main/bash/pillar @@ -7,6 +7,5 @@ export PILLAR_ROOT CLASS=com.chrisomeara.pillar.cli.App CLASS_PATH=${PILLAR_ROOT}/lib/pillar.jar:${PILLAR_ROOT}/conf -JAVA_OPTS="${JAVA_OPTS} -Dlog4j.configuration=pillar-log4j.properties" $JAVA -cp $CLASS_PATH $JAVA_OPTS $CLASS $* diff --git a/src/main/resources/application.conf b/src/main/resources/application.conf index 991d299..1d530c2 100644 --- a/src/main/resources/application.conf +++ b/src/main/resources/application.conf @@ -11,7 +11,6 @@ pillar.faker { acceptance_test { cassandra-seed-address: ${?PILLAR_SEED_ADDRESS} cassandra-port: ${?PILLAR_PORT} - cassandra-keyspace-name: "pillar_acceptance_test" cassandra-keyspace-name: ${?PILLAR_KEYSPACE_NAME} cassandra-ssl: ${?PILLAR_SSL} cassandra-username: ${?PILLAR_USERNAME} diff --git a/src/main/resources/pillar-log4j.properties b/src/main/resources/pillar-log4j.properties deleted file mode 100644 index b737cd5..0000000 --- a/src/main/resources/pillar-log4j.properties +++ /dev/null @@ -1,5 +0,0 @@ -log4j.rootLogger=INFO, default.out - -log4j.appender.default.out=org.apache.log4j.ConsoleAppender -log4j.appender.default.out.layout=org.apache.log4j.PatternLayout -log4j.appender.default.out.layout.ConversionPattern=%-5p %c: %m%n \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/AppliedMigrations.scala b/src/main/scala/com/chrisomeara/pillar/AppliedMigrations.scala deleted file mode 100644 index efcfeff..0000000 --- a/src/main/scala/com/chrisomeara/pillar/AppliedMigrations.scala +++ /dev/null @@ -1,27 +0,0 @@ -package com.chrisomeara.pillar - -import com.datastax.driver.core.Session -import com.datastax.driver.core.querybuilder.QueryBuilder -import scala.collection.JavaConversions -import java.util.Date - -object AppliedMigrations { - def apply(session: Session, registry: Registry): AppliedMigrations = { - val results = session.execute(QueryBuilder.select("authored_at", "description").from("applied_migrations")) - new AppliedMigrations(JavaConversions.asScalaBuffer(results.all()).map { - row => registry(MigrationKey(row.getTimestamp("authored_at"), row.getString("description"))) - }) - } -} - -class AppliedMigrations(applied: Seq[Migration]) { - def length: Int = applied.length - - def apply(index: Int): Migration = applied.apply(index) - - def iterator: Iterator[Migration] = applied.iterator - - def authoredAfter(date: Date): Seq[Migration] = applied.filter(migration => migration.authoredAfter(date)) - - def contains(other: Migration): Boolean = applied.contains(other) -} \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/CassandraMigrator.scala b/src/main/scala/com/chrisomeara/pillar/CassandraMigrator.scala deleted file mode 100644 index 87b59fa..0000000 --- a/src/main/scala/com/chrisomeara/pillar/CassandraMigrator.scala +++ /dev/null @@ -1,54 +0,0 @@ -package com.chrisomeara.pillar - -import java.util.Date - -import com.datastax.driver.core.Session -import com.datastax.driver.core.exceptions.AlreadyExistsException - -class CassandraMigrator(registry: Registry) extends Migrator { - override def migrate(session: Session, dateRestriction: Option[Date] = None) { - val appliedMigrations = AppliedMigrations(session, registry) - selectMigrationsToReverse(dateRestriction, appliedMigrations).foreach(_.executeDownStatement(session)) - selectMigrationsToApply(dateRestriction, appliedMigrations).foreach(_.executeUpStatement(session)) - } - - override def initialize(session: Session, keyspace: String, replicationOptions: ReplicationOptions = ReplicationOptions.default) { - executeIdempotentCommand(session, "CREATE KEYSPACE %s WITH replication = %s".format(keyspace, replicationOptions.toString())) - executeIdempotentCommand(session, - """ - | CREATE TABLE %s.applied_migrations ( - | authored_at timestamp, - | description text, - | applied_at timestamp, - | PRIMARY KEY (authored_at, description) - | ) - """.stripMargin.format(keyspace) - ) - } - - override def destroy(session: Session, keyspace: String) { - session.execute("DROP KEYSPACE %s".format(keyspace)) - } - - private def executeIdempotentCommand(session: Session, statement: String) { - try { - session.execute(statement) - } catch { - case _: AlreadyExistsException => - } - } - - private def selectMigrationsToApply(dateRestriction: Option[Date], appliedMigrations: AppliedMigrations): Seq[Migration] = { - (dateRestriction match { - case None => registry.all - case Some(cutOff) => registry.authoredBefore(cutOff) - }).filter(!appliedMigrations.contains(_)) - } - - private def selectMigrationsToReverse(dateRestriction: Option[Date], appliedMigrations: AppliedMigrations): Seq[Migration] = { - (dateRestriction match { - case None => List.empty[Migration] - case Some(cutOff) => appliedMigrations.authoredAfter(cutOff) - }).sortBy(_.authoredAt).reverse - } -} \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/ConfigurationException.scala b/src/main/scala/com/chrisomeara/pillar/ConfigurationException.scala deleted file mode 100644 index 2edac36..0000000 --- a/src/main/scala/com/chrisomeara/pillar/ConfigurationException.scala +++ /dev/null @@ -1,3 +0,0 @@ -package com.chrisomeara.pillar - -class ConfigurationException(message: String) extends RuntimeException(message) diff --git a/src/main/scala/com/chrisomeara/pillar/InvalidMigrationException.scala b/src/main/scala/com/chrisomeara/pillar/InvalidMigrationException.scala deleted file mode 100644 index bb6262f..0000000 --- a/src/main/scala/com/chrisomeara/pillar/InvalidMigrationException.scala +++ /dev/null @@ -1,3 +0,0 @@ -package com.chrisomeara.pillar - -class InvalidMigrationException(val errors: Map[String,String]) extends RuntimeException diff --git a/src/main/scala/com/chrisomeara/pillar/IrreversibleMigrationException.scala b/src/main/scala/com/chrisomeara/pillar/IrreversibleMigrationException.scala deleted file mode 100644 index 0692307..0000000 --- a/src/main/scala/com/chrisomeara/pillar/IrreversibleMigrationException.scala +++ /dev/null @@ -1,4 +0,0 @@ -package com.chrisomeara.pillar - -class IrreversibleMigrationException(migration: IrreversibleMigration) - extends RuntimeException(s"Migration ${migration.authoredAt.getTime}: ${migration.description} is not reversible") \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/Migration.scala b/src/main/scala/com/chrisomeara/pillar/Migration.scala deleted file mode 100644 index 0403a25..0000000 --- a/src/main/scala/com/chrisomeara/pillar/Migration.scala +++ /dev/null @@ -1,80 +0,0 @@ -package com.chrisomeara.pillar - -import java.util.Date -import com.datastax.driver.core.Session -import com.datastax.driver.core.querybuilder.QueryBuilder - -object Migration { - def apply(description: String, authoredAt: Date, up: Seq[String]): Migration = { - new IrreversibleMigration(description, authoredAt, up) - } - - def apply(description: String, authoredAt: Date, up: Seq[String], down: Option[Seq[String]]): Migration = { - down match { - case Some(downStatement) => - new ReversibleMigration(description, authoredAt, up, downStatement) - case None => - new ReversibleMigrationWithNoOpDown(description, authoredAt, up) - } - } -} - -trait Migration { - val description: String - val authoredAt: Date - val up: Seq[String] - - def key: MigrationKey = MigrationKey(authoredAt, description) - - def authoredAfter(date: Date): Boolean = { - authoredAt.after(date) - } - - def authoredBefore(date: Date): Boolean = { - authoredAt.compareTo(date) <= 0 - } - - def executeUpStatement(session: Session) { - up.foreach(session.execute) - insertIntoAppliedMigrations(session) - } - - def executeDownStatement(session: Session) - - protected def deleteFromAppliedMigrations(session: Session) { - session.execute(QueryBuilder. - delete(). - from("applied_migrations"). - where(QueryBuilder.eq("authored_at", authoredAt)). - and(QueryBuilder.eq("description", description)) - ) - } - - private def insertIntoAppliedMigrations(session: Session) { - session.execute(QueryBuilder. - insertInto("applied_migrations"). - value("authored_at", authoredAt). - value("description", description). - value("applied_at", System.currentTimeMillis()) - ) - } -} - -class IrreversibleMigration(val description: String, val authoredAt: Date, val up: Seq[String]) extends Migration { - def executeDownStatement(session: Session) { - throw new IrreversibleMigrationException(this) - } -} - -class ReversibleMigrationWithNoOpDown(val description: String, val authoredAt: Date, val up: Seq[String]) extends Migration { - def executeDownStatement(session: Session) { - deleteFromAppliedMigrations(session) - } -} - -class ReversibleMigration(val description: String, val authoredAt: Date, val up: Seq[String], val down: Seq[String]) extends Migration { - def executeDownStatement(session: Session) { - down.foreach(session.execute) - deleteFromAppliedMigrations(session) - } -} \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/MigrationKey.scala b/src/main/scala/com/chrisomeara/pillar/MigrationKey.scala deleted file mode 100644 index cd3b9ec..0000000 --- a/src/main/scala/com/chrisomeara/pillar/MigrationKey.scala +++ /dev/null @@ -1,5 +0,0 @@ -package com.chrisomeara.pillar - -import java.util.Date - -case class MigrationKey(authoredAt: Date, description: String) \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/Migrator.scala b/src/main/scala/com/chrisomeara/pillar/Migrator.scala deleted file mode 100644 index 69894f9..0000000 --- a/src/main/scala/com/chrisomeara/pillar/Migrator.scala +++ /dev/null @@ -1,23 +0,0 @@ -package com.chrisomeara.pillar - -import java.util.Date - -import com.datastax.driver.core.Session - -object Migrator { - def apply(registry: Registry): Migrator = { - new CassandraMigrator(registry) - } - - def apply(registry: Registry, reporter: Reporter): Migrator = { - new ReportingMigrator(reporter, apply(registry)) - } -} - -trait Migrator { - def migrate(session: Session, dateRestriction: Option[Date] = None) - - def initialize(session: Session, keyspace: String, replicationOptions: ReplicationOptions = ReplicationOptions.default) - - def destroy(session: Session, keyspace: String) -} \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/Parser.scala b/src/main/scala/com/chrisomeara/pillar/Parser.scala deleted file mode 100644 index 55a2978..0000000 --- a/src/main/scala/com/chrisomeara/pillar/Parser.scala +++ /dev/null @@ -1,132 +0,0 @@ -package com.chrisomeara.pillar - -import java.util.Date -import java.io.InputStream -import scala.collection.mutable -import scala.io.Source - -object Parser { - def apply(): Parser = new Parser - - private val MatchAttribute = """^-- (authoredAt|description|up|down|stage):(.*)$""".r -} - -class PartialMigration { - var description: String = "" - var authoredAt: String = "" - - var upStages = new mutable.MutableList[String]() - var downStages : Option[mutable.MutableList[String]] = None - - var currentUp = new mutable.MutableList[String]() - var currentDown: Option[mutable.MutableList[String]] = None - - def rotateUp() = { - upStages += currentUp.mkString("\n") - upStages = upStages.filterNot(line => line.isEmpty) - currentUp = new mutable.MutableList[String]() - } - - def rotateDown() = { - currentDown match { - case Some(currentDownLines) => - downStages match { - case None => downStages = Some(new mutable.MutableList[String]()) - case Some(_) => - } - - downStages = Some(downStages.get += currentDownLines.mkString("\n")) - case None => - } - - currentDown = None - } - - def validate: Option[Map[String, String]] = { - - rotateUp() - rotateDown() - - val errors = mutable.Map[String, String]() - - if (description.isEmpty) errors("description") = "must be present" - if (authoredAt.isEmpty) errors("authoredAt") = "must be present" - if (!authoredAt.isEmpty && authoredAtAsLong < 1) errors("authoredAt") = "must be a number greater than zero" - if (upStages.isEmpty) errors("up") = "must be present" - - if (errors.nonEmpty) Some(errors.toMap) else None - } - - def authoredAtAsLong: Long = { - try { - authoredAt.toLong - } catch { - case _:NumberFormatException => -1 - } - } - -} - -class Parser { - - import Parser.MatchAttribute - - trait ParserState - - case object ParsingAttributes extends ParserState - - case object ParsingUp extends ParserState - - case object ParsingDown extends ParserState - - case object ParsingUpStage extends ParserState - - case object ParsingDownStage extends ParserState - - def parse(resource: InputStream): Migration = { - val inProgress = new PartialMigration - var state: ParserState = ParsingAttributes - Source.fromInputStream(resource).getLines().foreach { - case MatchAttribute("authoredAt", authoredAt) => - inProgress.authoredAt = authoredAt.trim - case MatchAttribute("description", description) => - inProgress.description = description.trim - case MatchAttribute("up", _) => - state = ParsingUp - case MatchAttribute("down", _) => - inProgress.rotateUp() - inProgress.currentDown = Some(new mutable.MutableList[String]()) - state = ParsingDown - case MatchAttribute("stage", number) => - state match { - case ParsingUp => state = ParsingUpStage - case ParsingUpStage => inProgress.rotateUp() - case ParsingDown => state = ParsingDownStage - case ParsingDownStage => inProgress.rotateDown(); inProgress.currentDown = Some(new mutable.MutableList[String]()) - } - case cql => - if (!cql.isEmpty) { - - state match { - case ParsingUp | ParsingUpStage => inProgress.currentUp += cql - case ParsingDown | ParsingDownStage => inProgress.currentDown.get += cql - case other => - } - } - } - inProgress.validate match { - case Some(errors) => throw new InvalidMigrationException(errors) - case None => - - inProgress.downStages match { - case Some(downLines) => - if (downLines.forall(line => line.isEmpty)) { - Migration(inProgress.description, new Date(inProgress.authoredAtAsLong), inProgress.upStages, None) - } else { - Migration(inProgress.description, new Date(inProgress.authoredAtAsLong), inProgress.upStages, Some(downLines)) - } - case None => Migration(inProgress.description, new Date(inProgress.authoredAtAsLong), inProgress.upStages) - } - } - } -} diff --git a/src/main/scala/com/chrisomeara/pillar/PrintStreamReporter.scala b/src/main/scala/com/chrisomeara/pillar/PrintStreamReporter.scala deleted file mode 100644 index 39b9d40..0000000 --- a/src/main/scala/com/chrisomeara/pillar/PrintStreamReporter.scala +++ /dev/null @@ -1,28 +0,0 @@ -package com.chrisomeara.pillar - -import java.io.PrintStream -import java.util.Date - -import com.datastax.driver.core.Session - -class PrintStreamReporter(stream: PrintStream) extends Reporter { - override def initializing(session: Session, keyspace: String, replicationOptions: ReplicationOptions) { - stream.println(s"Initializing $keyspace") - } - - override def migrating(session: Session, dateRestriction: Option[Date]) { - stream.println(s"Migrating with date restriction $dateRestriction") - } - - override def applying(migration: Migration) { - stream.println(s"Applying ${migration.authoredAt.getTime}: ${migration.description}") - } - - override def reversing(migration: Migration) { - stream.println(s"Reversing ${migration.authoredAt.getTime}: ${migration.description}") - } - - override def destroying(session: Session, keyspace: String) { - stream.println(s"Destroying $keyspace") - } -} \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/Registry.scala b/src/main/scala/com/chrisomeara/pillar/Registry.scala deleted file mode 100644 index b01e996..0000000 --- a/src/main/scala/com/chrisomeara/pillar/Registry.scala +++ /dev/null @@ -1,74 +0,0 @@ -package com.chrisomeara.pillar - -import java.util.Date -import java.io.{FileInputStream, File} - -object Registry { - def apply(migrations: Seq[Migration]): Registry = { - new Registry(migrations) - } - - def fromDirectory(directory: File, reporter: Reporter): Registry = { - new Registry(parseMigrationsInDirectory(directory).map(new ReportingMigration(reporter, _))) - } - - def fromDirectory(directory: File): Registry = { - new Registry(parseMigrationsInDirectory(directory)) - } - - def fromFiles(files: Seq[File]): Registry = { - new Registry(parseMigrationsInFiles(filterExisting(files))) - } - - def fromFiles(files: Seq[File], reporter: Reporter): Registry = { - new Registry( - parseMigrationsInFiles(filterExisting(files)) - .map(new ReportingMigration(reporter, _)) - ) - } - - private def filterExisting(files : Seq[File]) : Seq[File] = { - files - .filterNot(file => file.isDirectory) - .filter(file => file.exists()) - } - - private def parseMigrationsInFiles(files: Seq[File]): Seq[Migration] = { - val parser = Parser() - - files.map { - file => - val stream = new FileInputStream(file) - try { - parser.parse(stream) - } finally { - stream.close() - } - }.toList - } - - private def parseMigrationsInDirectory(directory: File): Seq[Migration] = { - if (!directory.isDirectory) - return List.empty - - parseMigrationsInFiles(directory.listFiles()) - } -} - -class Registry(private var migrations: Seq[Migration]) { - migrations = migrations.sortBy(_.authoredAt) - - private val migrationsByKey = migrations.foldLeft(Map.empty[MigrationKey, Migration]) { - (memo, migration) => memo + (migration.key -> migration) - } - - def authoredBefore(date: Date): Seq[Migration] = { - migrations.filter(migration => migration.authoredBefore(date)) - } - - def apply(key: MigrationKey): Migration = { - migrationsByKey(key) - } - - def all: Seq[Migration] = migrations -} \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/ReplicationOptions.scala b/src/main/scala/com/chrisomeara/pillar/ReplicationOptions.scala deleted file mode 100644 index fe74ce1..0000000 --- a/src/main/scala/com/chrisomeara/pillar/ReplicationOptions.scala +++ /dev/null @@ -1,17 +0,0 @@ -package com.chrisomeara.pillar - -object ReplicationOptions { - val default = new ReplicationOptions(Map("class" -> "SimpleStrategy", "replication_factor" -> 3)) -} - -class ReplicationOptions(options: Map[String, Any]) { - override def toString: String = { - "{" + options.map { - case (key, value) => - value match { - case number: Int => "'%s':%d".format(key, number) - case string: String => "'%s':'%s'".format(key, string) - } - }.mkString(",") + "}" - } -} \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/Reporter.scala b/src/main/scala/com/chrisomeara/pillar/Reporter.scala deleted file mode 100644 index 369e131..0000000 --- a/src/main/scala/com/chrisomeara/pillar/Reporter.scala +++ /dev/null @@ -1,13 +0,0 @@ -package com.chrisomeara.pillar - -import java.util.Date - -import com.datastax.driver.core.Session - -trait Reporter { - def initializing(session: Session, keyspace: String, replicationOptions: ReplicationOptions) - def migrating(session: Session, dateRestriction: Option[Date]) - def applying(migration: Migration) - def reversing(migration: Migration) - def destroying(session: Session, keyspace: String) -} \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/ReportingMigration.scala b/src/main/scala/com/chrisomeara/pillar/ReportingMigration.scala deleted file mode 100644 index 18fbfae..0000000 --- a/src/main/scala/com/chrisomeara/pillar/ReportingMigration.scala +++ /dev/null @@ -1,20 +0,0 @@ -package com.chrisomeara.pillar - -import java.util.Date -import com.datastax.driver.core.Session - -class ReportingMigration(reporter: Reporter, wrapped: Migration) extends Migration { - val description: String = wrapped.description - val authoredAt: Date = wrapped.authoredAt - val up: Seq[String] = wrapped.up - - override def executeUpStatement(session: Session) { - reporter.applying(wrapped) - wrapped.executeUpStatement(session) - } - - def executeDownStatement(session: Session) { - reporter.reversing(wrapped) - wrapped.executeDownStatement(session) - } -} \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/ReportingMigrator.scala b/src/main/scala/com/chrisomeara/pillar/ReportingMigrator.scala deleted file mode 100644 index 08db78a..0000000 --- a/src/main/scala/com/chrisomeara/pillar/ReportingMigrator.scala +++ /dev/null @@ -1,22 +0,0 @@ -package com.chrisomeara.pillar - -import java.util.Date - -import com.datastax.driver.core.Session - -class ReportingMigrator(reporter: Reporter, wrapped: Migrator) extends Migrator { - override def initialize(session: Session, keyspace: String, replicationOptions: ReplicationOptions = ReplicationOptions.default) { - reporter.initializing(session, keyspace, replicationOptions) - wrapped.initialize(session, keyspace, replicationOptions) - } - - override def migrate(session: Session, dateRestriction: Option[Date] = None) { - reporter.migrating(session, dateRestriction) - wrapped.migrate(session, dateRestriction) - } - - override def destroy(session: Session, keyspace: String) { - reporter.destroying(session, keyspace) - wrapped.destroy(session, keyspace) - } -} \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/cli/App.scala b/src/main/scala/com/chrisomeara/pillar/cli/App.scala index 2d48370..03d135e 100644 --- a/src/main/scala/com/chrisomeara/pillar/cli/App.scala +++ b/src/main/scala/com/chrisomeara/pillar/cli/App.scala @@ -2,7 +2,7 @@ package com.chrisomeara.pillar.cli import java.io.File -import com.chrisomeara.pillar.{ConfigurationException, PrintStreamReporter, Registry, Reporter} +import com.chrisomeara.pillar.core.{ConfigurationException, PrintStreamReporter, Registry, Reporter} import com.datastax.driver.core.Cluster import com.typesafe.config.{Config, ConfigFactory} diff --git a/src/main/scala/com/chrisomeara/pillar/cli/Command.scala b/src/main/scala/com/chrisomeara/pillar/cli/Command.scala index 3a4eb36..fc3a5dc 100644 --- a/src/main/scala/com/chrisomeara/pillar/cli/Command.scala +++ b/src/main/scala/com/chrisomeara/pillar/cli/Command.scala @@ -1,6 +1,6 @@ package com.chrisomeara.pillar.cli -import com.chrisomeara.pillar.Registry +import com.chrisomeara.pillar.core.Registry import com.datastax.driver.core.Session case class Command(action: MigratorAction, session: Session, keyspace: String, timeStampOption: Option[Long], registry: Registry) \ No newline at end of file diff --git a/src/main/scala/com/chrisomeara/pillar/cli/CommandExecutor.scala b/src/main/scala/com/chrisomeara/pillar/cli/CommandExecutor.scala index 8d1effa..7b8158d 100644 --- a/src/main/scala/com/chrisomeara/pillar/cli/CommandExecutor.scala +++ b/src/main/scala/com/chrisomeara/pillar/cli/CommandExecutor.scala @@ -2,7 +2,7 @@ package com.chrisomeara.pillar.cli import java.util.Date -import com.chrisomeara.pillar.{Migrator, Registry, Reporter} +import com.chrisomeara.pillar.core.{Migrator, Registry, Reporter} object CommandExecutor { implicit private val migratorConstructor: ((Registry, Reporter) => Migrator) = Migrator.apply diff --git a/src/main/scala/com/chrisomeara/pillar/cli/CommandLineConfiguration.scala b/src/main/scala/com/chrisomeara/pillar/cli/CommandLineConfiguration.scala index d074aba..e9bfbe4 100644 --- a/src/main/scala/com/chrisomeara/pillar/cli/CommandLineConfiguration.scala +++ b/src/main/scala/com/chrisomeara/pillar/cli/CommandLineConfiguration.scala @@ -1,42 +1,32 @@ package com.chrisomeara.pillar.cli -import org.clapper.argot.ArgotParser -import org.clapper.argot.ArgotConverters._ +import scopt.OptionParser import java.io.File object CommandLineConfiguration { def buildFromArguments(arguments: Array[String]): CommandLineConfiguration = { - val parser = new ArgotParser("pillar") + val parser = new OptionParser[CommandLineConfiguration]("pillar") { - val commandParameter = parser.parameter[MigratorAction]("command", "migrate or initialize", optional = false) { - (commandString, _) => - commandString match { - case "initialize" => Initialize - case "migrate" => Migrate - case _ => parser.usage(s"$commandString is not a command") - } - } - val dataStoreConfigurationOption = parser.parameter[String]("data-store", "The target data store, as defined in application.conf", optional = false) - val migrationsDirectoryOption = parser.option[File](List("d", "migrations-directory"), "directory", "The directory containing migrations") { - (path, _) => - val directory = new File(path) - if (!directory.isDirectory) parser.usage(s"${directory.getAbsolutePath} is not a directory") - directory - } - val environmentOption = parser.option[String](List("e", "environment"), "env", "environment") - val timeStampOption = parser.option[Long](List("t", "time-stamp"), "time", "The migration time stamp") + cmd("initialize").action((_, c) => c.copy(command = Initialize)).children( + opt[String]('e', "environment").optional().action((e, c) => c.copy(environment = e)) + ) + + cmd("migrate").action((_, c) => c.copy(command = Migrate)).children( + opt[String]('e', "environment").optional().action((e, c) => c.copy(environment = e)), + opt[Long]('t', "time-stamp").optional().action((t, c) => c.copy(timeStampOption = Some(t))), + opt[File]('d', "migrations-directory").optional().action((d, c) => c.copy(migrationsDirectory = d)) + ) - parser.parse(arguments) + arg[String]("data-store").action((ds, c) => c.copy(dataStore = ds)) + } - CommandLineConfiguration( - commandParameter.value.get, - dataStoreConfigurationOption.value.get, - environmentOption.value.getOrElse("development"), - migrationsDirectoryOption.value.getOrElse(new File("conf/pillar/migrations")), - timeStampOption.value - ) + parser.parse(arguments, CommandLineConfiguration()) match { + case Some(configuration) => + configuration + case None => + throw new IllegalArgumentException("Unable to construct configuration from command line arguments") + } } } -case class CommandLineConfiguration(command: MigratorAction, dataStore: String, environment: String, migrationsDirectory: File, timeStampOption: Option[Long]) - +case class CommandLineConfiguration(command: MigratorAction = Noop, dataStore: String = "", environment: String = "development", migrationsDirectory: File = new File("conf/pillar/migrations"), timeStampOption: Option[Long] = None) diff --git a/src/main/scala/com/chrisomeara/pillar/cli/MigratorAction.scala b/src/main/scala/com/chrisomeara/pillar/cli/MigratorAction.scala index c0ddde0..f94f5a9 100644 --- a/src/main/scala/com/chrisomeara/pillar/cli/MigratorAction.scala +++ b/src/main/scala/com/chrisomeara/pillar/cli/MigratorAction.scala @@ -5,4 +5,6 @@ trait MigratorAction case object Migrate extends MigratorAction +case object Noop extends MigratorAction + case object Initialize extends MigratorAction diff --git a/src/test/resources/application.conf b/src/test/resources/application.conf new file mode 100644 index 0000000..9e112f9 --- /dev/null +++ b/src/test/resources/application.conf @@ -0,0 +1,11 @@ +# This file is used in test +pillar.faker { + acceptance_test { + cassandra-seed-address: ${?PILLAR_SEED_ADDRESS} + cassandra-port: ${?PILLAR_PORT} + cassandra-keyspace-name: pillar_acceptance_test + cassandra-ssl: ${?PILLAR_SSL} + cassandra-username: ${?PILLAR_USERNAME} + cassandra-password: ${?PILLAR_PASSWORD} + } +} \ No newline at end of file diff --git a/src/test/scala/com/chrisomeara/pillar/AcceptanceAssertions.scala b/src/test/scala/com/chrisomeara/pillar/AcceptanceAssertions.scala index e53674e..cc85953 100644 --- a/src/test/scala/com/chrisomeara/pillar/AcceptanceAssertions.scala +++ b/src/test/scala/com/chrisomeara/pillar/AcceptanceAssertions.scala @@ -2,9 +2,9 @@ package com.chrisomeara.pillar import com.datastax.driver.core.querybuilder.QueryBuilder import com.datastax.driver.core.{Metadata, Session} -import org.scalatest.matchers.ShouldMatchers +import org.scalatest.Matchers -trait AcceptanceAssertions extends ShouldMatchers { +trait AcceptanceAssertions extends Matchers { val session: Session val keyspaceName: String diff --git a/src/test/scala/com/chrisomeara/pillar/MigrationSpec.scala b/src/test/scala/com/chrisomeara/pillar/MigrationSpec.scala deleted file mode 100644 index 4587465..0000000 --- a/src/test/scala/com/chrisomeara/pillar/MigrationSpec.scala +++ /dev/null @@ -1,30 +0,0 @@ -package com.chrisomeara.pillar - -import org.scalatest.FunSpec -import org.scalatest.matchers.ShouldMatchers -import java.util.Date -import org.scalatest.mock.MockitoSugar - -class MigrationSpec extends FunSpec with ShouldMatchers with MockitoSugar { - describe(".apply") { - describe("without a down parameter") { - it("returns an irreversible migration") { - Migration.apply("description", new Date(), Seq("up")).getClass should be(classOf[IrreversibleMigration]) - } - } - - describe("with a down parameter") { - describe("when the down is None") { - it("returns a reversible migration with no-op down") { - Migration.apply("description", new Date(), Seq("up"), None).getClass should be(classOf[ReversibleMigrationWithNoOpDown]) - } - } - - describe("when the down is Some") { - it("returns a reversible migration with no-op down") { - Migration.apply("description", new Date(), Seq("up"), Some(Seq("down"))).getClass should be(classOf[ReversibleMigration]) - } - } - } - } -} diff --git a/src/test/scala/com/chrisomeara/pillar/ParserSpec.scala b/src/test/scala/com/chrisomeara/pillar/ParserSpec.scala deleted file mode 100644 index 2d6c140..0000000 --- a/src/test/scala/com/chrisomeara/pillar/ParserSpec.scala +++ /dev/null @@ -1,162 +0,0 @@ -package com.chrisomeara.pillar - -import org.scalatest.{FunSpec, BeforeAndAfter} -import org.scalatest.matchers.ShouldMatchers -import java.io.{ByteArrayInputStream, FileInputStream} -import java.util.Date - -class ParserSpec extends FunSpec with BeforeAndAfter with ShouldMatchers { - describe("#parse") { - describe("1370028262000_creates_events_table.cql") { - val migrationPath = "src/test/resources/pillar/migrations/faker/1370028262000_creates_events_table.cql" - - it("returns a migration object") { - val resource = new FileInputStream(migrationPath) - Parser().parse(resource).getClass should be(classOf[IrreversibleMigration]) - } - - it("assigns authoredAt") { - val resource = new FileInputStream(migrationPath) - Parser().parse(resource).authoredAt should equal(new Date(1370023262000L)) - } - - it("assigns description") { - val resource = new FileInputStream(migrationPath) - Parser().parse(resource).description should equal("creates events table") - } - - it("assigns up") { - val resource = new FileInputStream(migrationPath) - Parser().parse(resource).up should contain( - """CREATE TABLE events ( - | batch_id text, - | occurred_at uuid, - | event_type text, - | payload blob, - | PRIMARY KEY (batch_id, occurred_at, event_type) - |)""".stripMargin) - } - } - - describe("1469630066000_creates_users_groups_table.cql") { - val migrationPath = "src/test/resources/pillar/migrations/faker/1469630066000_creates_users_groups_table.cql" - - it("returns a migration object") { - val resource = new FileInputStream(migrationPath) - Parser().parse(resource).getClass should be(classOf[ReversibleMigration]) - } - - it("assigns authoredAt") { - val resource = new FileInputStream(migrationPath) - Parser().parse(resource).authoredAt should equal(new Date(1469630066000L)) - } - - it("assigns description") { - val resource = new FileInputStream(migrationPath) - Parser().parse(resource).description should equal("creates users and groups tables") - } - - it("assigns two up stages") { - val resource = new FileInputStream(migrationPath) - val migration = Parser().parse(resource) - - migration.up should contain( - """CREATE TABLE groups ( - | id uuid, - | name text, - | PRIMARY KEY (id) - |)""".stripMargin) - - migration.up should contain( - """CREATE TABLE users ( - | id uuid, - | group_id uuid, - | username text, - | password text, - | PRIMARY KEY (id) - |)""".stripMargin) - } - - it("assigns two down stages") { - val resource = new FileInputStream(migrationPath) - val migration = Parser().parse(resource).asInstanceOf[ReversibleMigration] - - migration.down should contain("""DROP TABLE users""".stripMargin) - migration.down should contain("""DROP TABLE groups""".stripMargin) - } - } - - describe("1370028263000_creates_views_table.cql") { - val migrationPath = "src/test/resources/pillar/migrations/faker/1370028263000_creates_views_table.cql" - - it("returns a migration object") { - val resource = new FileInputStream(migrationPath) - Parser().parse(resource).getClass should be(classOf[ReversibleMigration]) - } - - it("assigns down") { - val resource = new FileInputStream(migrationPath) - Parser().parse(resource).asInstanceOf[ReversibleMigration].down should contain("DROP TABLE views") - } - } - - describe("1370028264000_adds_user_agent_to_views_table.cql") { - val migrationPath = "src/test/resources/pillar/migrations/faker/1370028264000_adds_user_agent_to_views_table.cql" - - it("returns a migration object") { - val resource = new FileInputStream(migrationPath) - Parser().parse(resource).getClass should be(classOf[ReversibleMigrationWithNoOpDown]) - } - } - - describe("a migration missing an up stanza") { - val migrationContent = - """-- description: creates events table - |-- authoredAt: 1370023262""".stripMargin - - it("raises an InvalidMigrationException") { - val resource = new ByteArrayInputStream(migrationContent.getBytes) - val thrown = intercept[InvalidMigrationException] { - Parser().parse(resource) - } - thrown.errors("up") should equal("must be present") - } - } - - describe("a migration missing a description stanza") { - val migrationContent = "-- authoredAt: 1370023262" - - it("raises an InvalidMigrationException") { - val resource = new ByteArrayInputStream(migrationContent.getBytes) - val thrown = intercept[InvalidMigrationException] { - Parser().parse(resource) - } - thrown.errors("description") should equal("must be present") - } - } - - describe("a migration missing an authoredAt stanza") { - val migrationContent = "-- description: creates events table" - - it("raises an InvalidMigrationException") { - val resource = new ByteArrayInputStream(migrationContent.getBytes) - val thrown = intercept[InvalidMigrationException] { - Parser().parse(resource) - } - thrown.errors("authoredAt") should equal("must be present") - } - } - - describe("a migration with a bogus authored at stanza") { - val migrationContent = "-- authoredAt: a long, long time ago" - - it("raises an InvalidMigrationException") { - val resource = new ByteArrayInputStream(migrationContent.getBytes) - val thrown = intercept[InvalidMigrationException] { - Parser().parse(resource) - } - thrown.errors("authoredAt") should equal("must be a number greater than zero") - } - } - } -} diff --git a/src/test/scala/com/chrisomeara/pillar/PillarCommandLineAcceptanceSpec.scala b/src/test/scala/com/chrisomeara/pillar/PillarCommandLineAcceptanceSpec.scala index fcd2994..9feac84 100644 --- a/src/test/scala/com/chrisomeara/pillar/PillarCommandLineAcceptanceSpec.scala +++ b/src/test/scala/com/chrisomeara/pillar/PillarCommandLineAcceptanceSpec.scala @@ -1,13 +1,12 @@ package com.chrisomeara.pillar import com.datastax.driver.core.exceptions.InvalidQueryException -import org.scalatest.{BeforeAndAfter, FeatureSpec, GivenWhenThen} -import org.scalatest.matchers.ShouldMatchers +import org.scalatest.{BeforeAndAfter, FeatureSpec, GivenWhenThen, Matchers} import com.datastax.driver.core.Cluster import com.datastax.driver.core.querybuilder.QueryBuilder import com.chrisomeara.pillar.cli.App -class PillarCommandLineAcceptanceSpec extends FeatureSpec with GivenWhenThen with BeforeAndAfter with ShouldMatchers with AcceptanceAssertions { +class PillarCommandLineAcceptanceSpec extends FeatureSpec with GivenWhenThen with BeforeAndAfter with Matchers with AcceptanceAssertions { val seedAddress = sys.env.getOrElse("PILLAR_SEED_ADDRESS", "127.0.0.1") val username = sys.env.getOrElse("PILLAR_USERNAME", "cassandra") val password = sys.env.getOrElse("PILLAR_PASSWORD", "cassandra") @@ -33,7 +32,7 @@ class PillarCommandLineAcceptanceSpec extends FeatureSpec with GivenWhenThen wit Given("a non-existent keyspace") When("the migrator initializes the keyspace") - App().run(Array("-e", "acceptance_test", "initialize", "faker")) + App().run(Array("initialize", "-e", "acceptance_test", "faker")) Then("the keyspace contains a applied_migrations column family") assertEmptyAppliedMigrationsTable() @@ -47,13 +46,13 @@ class PillarCommandLineAcceptanceSpec extends FeatureSpec with GivenWhenThen wit scenario("all migrations") { Given("an initialized, empty, keyspace") - App().run(Array("-e", "acceptance_test", "initialize", "faker")) + App().run(Array("initialize", "-e", "acceptance_test", "faker")) Given("a migration that creates an events table") Given("a migration that creates a views table") When("the migrator migrates the schema") - App().run(Array("-e", "acceptance_test", "-d", "src/test/resources/pillar/migrations", "migrate", "faker")) + App().run(Array("migrate", "-e", "acceptance_test", "-d", "src/test/resources/pillar/migrations", "faker")) Then("the keyspace contains the events table") session.execute(QueryBuilder.select().from(keyspaceName, "events")).all().size() should equal(0) diff --git a/src/test/scala/com/chrisomeara/pillar/PillarLibraryAcceptanceSpec.scala b/src/test/scala/com/chrisomeara/pillar/PillarLibraryAcceptanceSpec.scala deleted file mode 100644 index f039c46..0000000 --- a/src/test/scala/com/chrisomeara/pillar/PillarLibraryAcceptanceSpec.scala +++ /dev/null @@ -1,236 +0,0 @@ -package com.chrisomeara.pillar - -import java.util.Date - -import com.datastax.driver.core.Cluster -import com.datastax.driver.core.exceptions.InvalidQueryException -import com.datastax.driver.core.querybuilder.QueryBuilder -import org.scalatest.{BeforeAndAfter, FeatureSpec, GivenWhenThen, Matchers} - -class PillarLibraryAcceptanceSpec extends FeatureSpec with GivenWhenThen with BeforeAndAfter with Matchers with AcceptanceAssertions { - val seedAddress = sys.env.getOrElse("PILLAR_SEED_ADDRESS", "127.0.0.1") - val username = sys.env.getOrElse("PILLAR_USERNAME", "cassandra") - val password = sys.env.getOrElse("PILLAR_PASSWORD", "cassandra") - val port = sys.env.getOrElse("PILLAR_PORT", "9042").toInt - val cluster = Cluster.builder().addContactPoint(seedAddress).withPort(port).withCredentials(username, password).build() - val keyspaceName = "test_%d".format(System.currentTimeMillis()) - val session = cluster.connect() - val migrations = Seq( - Migration("creates events table", new Date(System.currentTimeMillis() - 5000), - Seq(""" - |CREATE TABLE events ( - | batch_id text, - | occurred_at uuid, - | event_type text, - | payload blob, - | PRIMARY KEY (batch_id, occurred_at, event_type) - |) - """.stripMargin)), - Migration("creates views table", new Date(System.currentTimeMillis() - 3000), - Seq(""" - |CREATE TABLE views ( - | id uuid PRIMARY KEY, - | url text, - | person_id int, - | viewed_at timestamp - |) - """.stripMargin), - Some( Seq(""" - |DROP TABLE views - """.stripMargin))), - Migration("adds user_agent to views table", new Date(System.currentTimeMillis() - 1000), - Seq(""" - |ALTER TABLE views - |ADD user_agent text - """.stripMargin), None), // Dropping a column is coming in Cassandra 2.0 - Migration("adds index on views.user_agent", new Date(), - Seq(""" - |CREATE INDEX views_user_agent ON views(user_agent) - """.stripMargin), - Some( Seq(""" - |DROP INDEX views_user_agent - """.stripMargin))) - ) - val registry = Registry(migrations) - val migrator = Migrator(registry) - - after { - try { - session.execute("DROP KEYSPACE %s".format(keyspaceName)) - } catch { - case ok: InvalidQueryException => - } - } - - feature("The operator can initialize a keyspace") { - info("As an application operator") - info("I want to initialize a Cassandra keyspace") - info("So that I can manage the keyspace schema") - - scenario("initialize a non-existent keyspace") { - Given("a non-existent keyspace") - - When("the migrator initializes the keyspace") - migrator.initialize(session, keyspaceName) - - Then("the keyspace contains a applied_migrations column family") - assertEmptyAppliedMigrationsTable() - } - - scenario("initialize an existing keyspace without a applied_migrations column family") { - Given("an existing keyspace") - session.execute("CREATE KEYSPACE %s WITH replication = {'class': 'SimpleStrategy', 'replication_factor' : 1}".format(keyspaceName)) - - When("the migrator initializes the keyspace") - migrator.initialize(session, keyspaceName) - - Then("the keyspace contains a applied_migrations column family") - assertEmptyAppliedMigrationsTable() - } - - scenario("initialize an existing keyspace with a applied_migrations column family") { - Given("an existing keyspace") - migrator.initialize(session, keyspaceName) - - When("the migrator initializes the keyspace") - migrator.initialize(session, keyspaceName) - - Then("the migration completes successfully") - } - } - - feature("The operator can destroy a keyspace") { - info("As an application operator") - info("I want to destroy a Cassandra keyspace") - info("So that I can clean up automated tasks") - - scenario("destroy a keyspace") { - Given("an existing keyspace") - migrator.initialize(session, keyspaceName) - - When("the migrator destroys the keyspace") - migrator.destroy(session, keyspaceName) - - Then("the keyspace no longer exists") - assertKeyspaceDoesNotExist() - } - - scenario("destroy a bad keyspace") { - Given("a datastore with a non-existing keyspace") - - When("the migrator destroys the keyspace") - - Then("the migrator throws an exception") - evaluating { - migrator.destroy(session, keyspaceName) - } should produce[Throwable] - } - } - - feature("The operator can apply migrations") { - info("As an application operator") - info("I want to migrate a Cassandra keyspace from an older version of the schema to a newer version") - info("So that I can run an application using the schema") - - scenario("all migrations") { - Given("an initialized, empty, keyspace") - migrator.initialize(session, keyspaceName) - - Given("a migration that creates an events table") - Given("a migration that creates a views table") - - When("the migrator migrates the schema") - migrator.migrate(cluster.connect(keyspaceName)) - - Then("the keyspace contains the events table") - session.execute(QueryBuilder.select().from(keyspaceName, "events")).all().size() should equal(0) - - And("the keyspace contains the views table") - session.execute(QueryBuilder.select().from(keyspaceName, "views")).all().size() should equal(0) - - And("the applied_migrations table records the migrations") - session.execute(QueryBuilder.select().from(keyspaceName, "applied_migrations")).all().size() should equal(4) - } - - scenario("some migrations") { - Given("an initialized, empty, keyspace") - migrator.initialize(session, keyspaceName) - - Given("a migration that creates an events table") - Given("a migration that creates a views table") - - When("the migrator migrates with a cut off date") - migrator.migrate(cluster.connect(keyspaceName), Some(migrations(0).authoredAt)) - - Then("the keyspace contains the events table") - session.execute(QueryBuilder.select().from(keyspaceName, "events")).all().size() should equal(0) - - And("the applied_migrations table records the migration") - session.execute(QueryBuilder.select().from(keyspaceName, "applied_migrations")).all().size() should equal(1) - } - - scenario("skip previously applied migration") { - Given("an initialized keyspace") - migrator.initialize(session, keyspaceName) - - Given("a set of migrations applied in the past") - migrator.migrate(cluster.connect(keyspaceName)) - - When("the migrator applies migrations") - migrator.migrate(cluster.connect(keyspaceName)) - - Then("the migration completes successfully") - } - } - - feature("The operator can reverse migrations") { - info("As an application operator") - info("I want to migrate a Cassandra keyspace from a newer version of the schema to an older version") - info("So that I can run an application using the schema") - - scenario("reversible previously applied migration") { - Given("an initialized keyspace") - migrator.initialize(session, keyspaceName) - - Given("a set of migrations applied in the past") - migrator.migrate(cluster.connect(keyspaceName)) - - When("the migrator migrates with a cut off date") - migrator.migrate(cluster.connect(keyspaceName), Some(migrations(0).authoredAt)) - - Then("the migrator reverses the reversible migration") - val thrown = intercept[InvalidQueryException] { - session.execute(QueryBuilder.select().from(keyspaceName, "views")).all() - } - thrown.getMessage should include("views") - - And("the migrator removes the reversed migration from the applied migrations table") - val reversedMigration = migrations(1) - val query = QueryBuilder. - select(). - from(keyspaceName, "applied_migrations"). - where(QueryBuilder.eq("authored_at", reversedMigration.authoredAt)). - and(QueryBuilder.eq("description", reversedMigration.description)) - session.execute(query).all().size() should equal(0) - } - - scenario("irreversible previously applied migration") { - Given("an initialized keyspace") - migrator.initialize(session, keyspaceName) - - Given("a set of migrations applied in the past") - migrator.migrate(cluster.connect(keyspaceName)) - - When("the migrator migrates with a cut off date") - val thrown = intercept[IrreversibleMigrationException] { - migrator.migrate(cluster.connect(keyspaceName), Some(new Date(0))) - } - - Then("the migrator reverses the reversible migrations") - session.execute(QueryBuilder.select().from(keyspaceName, "applied_migrations")).all().size() should equal(1) - - And("the migrator throws an IrreversibleMigrationException") - thrown should not be null - } - } -} \ No newline at end of file diff --git a/src/test/scala/com/chrisomeara/pillar/PrintStreamReporterSpec.scala b/src/test/scala/com/chrisomeara/pillar/PrintStreamReporterSpec.scala deleted file mode 100644 index e723932..0000000 --- a/src/test/scala/com/chrisomeara/pillar/PrintStreamReporterSpec.scala +++ /dev/null @@ -1,55 +0,0 @@ -package com.chrisomeara.pillar - -import com.datastax.driver.core.Session -import org.scalatest._ -import org.scalatest.matchers.ShouldMatchers -import java.io.{ByteArrayOutputStream, PrintStream} -import java.util.Date - -import org.scalatest.mock.MockitoSugar - -class PrintStreamReporterSpec extends FunSpec with MockitoSugar with Matchers with OneInstancePerTest { - val session = mock[Session] - val migration = Migration("creates things table", new Date(1370489972546L), Seq("up"), Some(Seq("down"))) - val output = new ByteArrayOutputStream() - val stream = new PrintStream(output) - val reporter = new PrintStreamReporter(stream) - val keyspace = "myks" - - describe("#initializing") { - it("should print to the stream") { - reporter.initializing(session, keyspace, ReplicationOptions.default) - output.toString should equal("Initializing myks\n") - } - } - - describe("#migrating") { - describe("without date restriction") { - it("should print to the stream") { - reporter.migrating(session, None) - output.toString should equal("Migrating with date restriction None\n") - } - } - } - - describe("#applying") { - it("should print to the stream") { - reporter.applying(migration) - output.toString should equal("Applying 1370489972546: creates things table\n") - } - } - - describe("#reversing") { - it("should print to the stream") { - reporter.reversing(migration) - output.toString should equal("Reversing 1370489972546: creates things table\n") - } - } - - describe("#destroying") { - it("should print to the stream") { - reporter.destroying(session, keyspace) - output.toString should equal("Destroying myks\n") - } - } -} diff --git a/src/test/scala/com/chrisomeara/pillar/RegistrySpec.scala b/src/test/scala/com/chrisomeara/pillar/RegistrySpec.scala deleted file mode 100644 index f31f8a0..0000000 --- a/src/test/scala/com/chrisomeara/pillar/RegistrySpec.scala +++ /dev/null @@ -1,91 +0,0 @@ -package com.chrisomeara.pillar - -import org.scalatest.{FunSpec, BeforeAndAfter} -import org.scalatest.matchers.ShouldMatchers -import java.io.File -import org.scalatest.mock.MockitoSugar -import java.util.Date - -class RegistrySpec extends FunSpec with BeforeAndAfter with ShouldMatchers with MockitoSugar { - describe(".fromDirectory") { - describe("without a reporter parameter") { - describe("with a directory that exists and has migration files") { - it("returns a registry with migrations") { - val registry = Registry.fromDirectory(new File("src/test/resources/pillar/migrations/faker/")) - registry.all.size should equal(4) - } - } - - describe("with a directory that does not exist") { - it("returns an empty registry") { - val registry = Registry.fromDirectory(new File("bogus")) - registry.all.size should equal(0) - } - } - } - - describe("with a reporter parameter") { - val reporter = mock[Reporter] - it("returns a registry populated with reporting migrations") { - val registry = Registry.fromDirectory(new File("src/test/resources/pillar/migrations/faker/"), reporter) - registry.all.head.getClass should be(classOf[ReportingMigration]) - } - } - } - - describe(".fromFiles") { - describe("without a reporter parameter") { - describe("with migration files provided") { - it("returns a registry with migrations") { - val registry = Registry.fromFiles(Seq( - new File("src/test/resources/pillar/migrations/faker/1370028262000_creates_events_table.cql"), - new File("src/test/resources/pillar/migrations/faker/1370028263000_creates_views_table.cql") - )) - - registry.all.size should equal(2) - } - } - - describe("with directories and files provided") { - it("returns a registry with migrations only from files") { - - val registry = Registry.fromFiles(Seq( - new File("src/test/resources/pillar/migrations/faker"), - new File("src/test/resources/pillar/migrations/faker/1370028263000_creates_views_table.cql") - )) - - registry.all.size should equal(1) - } - } - - describe("with a file that does not exist"){ - it("returns an empty registry") { - val registry = Registry.fromFiles(Seq(new File("non existing file"))) - registry.all.size should equal(0) - } - } - } - - describe("with a reporter parameter") { - val reporter = mock[Reporter] - it("returns a registry populated with reporting migrations") { - val registry = Registry.fromFiles(Seq(new File("src/test/resources/pillar/migrations/faker/1370028263000_creates_views_table.cql")), reporter) - registry.all.head.getClass should be(classOf[ReportingMigration]) - } - } - } - - describe("#all") { - val now = new Date() - val oneSecondAgo = new Date(now.getTime - 1000) - val migrations = List( - Migration("test now", now, Seq("up")), - Migration("test just before", oneSecondAgo, Seq("up")) - ) - val registry = new Registry(migrations) - - it("sorts migrations by their authoredAt property ascending") { - registry.all.map(_.authoredAt) should equal(List(oneSecondAgo, now)) - } - } -} diff --git a/src/test/scala/com/chrisomeara/pillar/ReportingMigrationSpec.scala b/src/test/scala/com/chrisomeara/pillar/ReportingMigrationSpec.scala deleted file mode 100644 index 0d5f6b4..0000000 --- a/src/test/scala/com/chrisomeara/pillar/ReportingMigrationSpec.scala +++ /dev/null @@ -1,39 +0,0 @@ -package com.chrisomeara.pillar - -import org.scalatest.FunSpec -import org.scalatest.matchers.ShouldMatchers -import org.scalatest.mock.MockitoSugar -import org.mockito.Mockito._ -import com.datastax.driver.core.Session -import java.util.Date - -class ReportingMigrationSpec extends FunSpec with ShouldMatchers with MockitoSugar { - val reporter = mock[Reporter] - val wrapped = mock[Migration] - val migration = new ReportingMigration(reporter, wrapped) - val session = mock[Session] - - describe("#executeUpStatement") { - migration.executeUpStatement(session) - - it("reports the applying action") { - verify(reporter).applying(wrapped) - } - - it("delegates to the wrapped migration") { - verify(wrapped).executeUpStatement(session) - } - } - - describe("#executeDownStatement") { - migration.executeDownStatement(session) - - it("reports the reversing action") { - verify(reporter).reversing(wrapped) - } - - it("delegates to the wrapped migration") { - verify(wrapped).executeDownStatement(session) - } - } -} diff --git a/src/test/scala/com/chrisomeara/pillar/ReportingMigratorSpec.scala b/src/test/scala/com/chrisomeara/pillar/ReportingMigratorSpec.scala deleted file mode 100644 index 524fe3a..0000000 --- a/src/test/scala/com/chrisomeara/pillar/ReportingMigratorSpec.scala +++ /dev/null @@ -1,51 +0,0 @@ -package com.chrisomeara.pillar - -import com.datastax.driver.core.Session -import org.mockito.Mockito._ -import org.scalatest.FunSpec -import org.scalatest.mock.MockitoSugar - -class ReportingMigratorSpec extends FunSpec with MockitoSugar { - val reporter = mock[Reporter] - val wrapped = mock[Migrator] - val migrator = new ReportingMigrator(reporter, wrapped) - val session = mock[Session] - val keyspace = "myks" - - describe("#initialize") { - val replicationOptions = mock[ReplicationOptions] - migrator.initialize(session, keyspace, replicationOptions) - - it("reports the initialize action") { - verify(reporter).initializing(session, keyspace, replicationOptions) - } - - it("delegates to the wrapped migrator") { - verify(wrapped).initialize(session, keyspace, replicationOptions) - } - } - - describe("#migrate") { - migrator.migrate(session) - - it("reports the migrate action") { - verify(reporter).migrating(session, None) - } - - it("delegates to the wrapped migrator") { - verify(wrapped).migrate(session, None) - } - } - - describe("#destroy") { - migrator.destroy(session, keyspace) - - it("reports the destroy action") { - verify(reporter).destroying(session, keyspace) - } - - it("delegates to the wrapped migrator") { - verify(wrapped).destroy(session, keyspace) - } - } -} diff --git a/src/test/scala/com/chrisomeara/pillar/cli/CommandExecutorSpec.scala b/src/test/scala/com/chrisomeara/pillar/cli/CommandExecutorSpec.scala index 50aebb5..61f52ef 100644 --- a/src/test/scala/com/chrisomeara/pillar/cli/CommandExecutorSpec.scala +++ b/src/test/scala/com/chrisomeara/pillar/cli/CommandExecutorSpec.scala @@ -2,7 +2,7 @@ package com.chrisomeara.pillar.cli import java.util.Date -import com.chrisomeara.pillar.{Migrator, Registry, Reporter} +import com.chrisomeara.pillar.core.{Migrator, Registry, Reporter} import com.datastax.driver.core.Session import org.mockito.Mockito._ import org.scalatest.mock.MockitoSugar diff --git a/src/test/scala/com/chrisomeara/pillar/cli/CommandLineConfigurationSpec.scala b/src/test/scala/com/chrisomeara/pillar/cli/CommandLineConfigurationSpec.scala index bb30aa3..ee6b04f 100644 --- a/src/test/scala/com/chrisomeara/pillar/cli/CommandLineConfigurationSpec.scala +++ b/src/test/scala/com/chrisomeara/pillar/cli/CommandLineConfigurationSpec.scala @@ -1,9 +1,8 @@ package com.chrisomeara.pillar.cli -import org.scalatest.FunSpec -import org.scalatest.matchers.ShouldMatchers +import org.scalatest.{FunSpec, Matchers} -class CommandLineConfigurationSpec extends FunSpec with ShouldMatchers { +class CommandLineConfigurationSpec extends FunSpec with Matchers { describe(".buildFromArguments") { describe("command initialize") { describe("data-store faker") { @@ -29,19 +28,19 @@ class CommandLineConfigurationSpec extends FunSpec with ShouldMatchers { describe("environment test") { it("sets the environment") { - CommandLineConfiguration.buildFromArguments(Array("-e", "test", "initialize", "faker")).environment should equal("test") + CommandLineConfiguration.buildFromArguments(Array("initialize", "-e", "test", "faker")).environment should equal("test") } } describe("migrations-directory baz") { it("sets the migrations directory") { - CommandLineConfiguration.buildFromArguments(Array("-d", "src/test/resources/pillar/migrations", "initialize", "faker")).migrationsDirectory.getPath should equal("src/test/resources/pillar/migrations") + CommandLineConfiguration.buildFromArguments(Array("migrate", "-d", "src/test/resources/pillar/migrations", "faker")).migrationsDirectory.getPath should equal("src/test/resources/pillar/migrations") } } describe("time-stamp 1370028262") { it("sets the time stamp option") { - CommandLineConfiguration.buildFromArguments(Array("-t", "1370028262", "initialize", "faker")).timeStampOption should equal(Some(1370028262)) + CommandLineConfiguration.buildFromArguments(Array("migrate", "-t", "1370028262", "faker")).timeStampOption should equal(Some(1370028262)) } } }