Provides the tech.sourced.engine.Engine class, which is the main entry point
of all the analysis you might do using this library as well as some implicits
to make it easier to use.
Provides the tech.sourced.engine.Engine class, which is the main entry point
of all the analysis you might do using this library as well as some implicits
to make it easier to use. In particular, it adds some methods to be able to
join with other "tables" directly from any org.apache.spark.sql.DataFrame.
import tech.sourced.engine._
val engine = Engine(sparkSession, "/path/to/repositories")
If you don't want to import everything in the engine, even though it only exposes
what's truly needed to not pollute the user namespace, you can do it by just importing
the tech.sourced.engine.Engine class and the tech.sourced.engine.EngineDataFrame
implicit class.
import tech.sourced.engine.{Engine, EngineDataFrame}
val engine = Engine(sparkSession, "/path/to/repositories")
Provides the tech.sourced.engine.Engine class, which is the main entry point of all the analysis you might do using this library as well as some implicits to make it easier to use. In particular, it adds some methods to be able to join with other "tables" directly from any org.apache.spark.sql.DataFrame.
If you don't want to import everything in the engine, even though it only exposes what's truly needed to not pollute the user namespace, you can do it by just importing the tech.sourced.engine.Engine class and the tech.sourced.engine.EngineDataFrame implicit class.