1. scala初识
spark由scala编写，要解析scala，首先要对scala有基本的了解。
1.1 class vs object
A class is a blueprint for objects. Once you define a class, you can create objects from the class blueprint with the keyword new.
import java.io._
class Point(val xc: Int, val yc: Int) {
var x: Int = xc
var y: Int = yc
def move(dx: Int, dy: Int) {
x = x + dx
y = y + dy
println ("Point x location : " + x);
println ("Point y location : " + y);
}
}
object Test {
def main(args: Array[String]) {
val pt = new Point(10, 20);
// Move to a new location
pt.move(10, 10);
}
}类的继承，使用extend实现：
import java.io._
class Point(val xc: Int, val yc: Int) {
var x: Int = xc
var y: Int = yc
def move(dx: Int, dy: Int) {
x = x + dx
y = y + dy
println ("Point x location : " + x);
println ("Point y location : " + y);
}
}
class Location(override val xc: Int, override val yc: Int,
val zc :Int) extends Point(xc, yc){
var z: Int = zc
def move(dx: Int, dy: Int, dz: Int) {
x = x + dx
y = y + dy
z = z + dz
println ("Point x location : " + x);
println ("Point y location : " + y);
println ("Point z location : " + z);
}
}
object Test {
def main(args: Array[String]) {
val loc = new Location(10, 20, 15);
// Move to a new location
loc.move(10, 10, 5);
}
}单例对象
Scala is more object-oriented than Java because in Scala we cannot have static members. Instead, Scala has singleton objects. A singleton is a class that can have only one instance, i.e., object. You create singleton using the keywordobject instead of class keyword. Since you can't instantiate a singleton object, you can't pass parameters to the primary constructor. You already have seen all the examples using singleton objects where you called Scala's main method. Following is the same example of showing singleton:
import java.io._
class Point(val xc: Int, val yc: Int) {
var x: Int = xc
var y: Int = yc
def move(dx: Int, dy: Int) {
x = x + dx
y = y + dy
}
}
object Test {
def main(args: Array[String]) {
val point = new Point(10, 20)
printPoint
def printPoint{
println ("Point x location : " + point.x);
println ("Point y location : " + point.y);
}
}
}1.2 trait
A trait encapsulates method and field definitions, which can then be reused by mixing them into classes. Unlike class inheritance, in which each class must inherit from just one superclass, a class can mix in any number of traits. Traits are used to define object * by specifying the signature of the supported methods. Scala also allows traits to be partially implemented but traits may not have constructor parameters.
a trait is very similar to what we have abstract classes in Java. Below is a complete example to show the concept of traits:
trait Equal {
def isEqual(x: Any): Boolean
def isNotEqual(x: Any): Boolean = !isEqual(x)
}
class Point(xc: Int, yc: Int) extends Equal {
var x: Int = xc
var y: Int = yc
def isEqual(obj: Any) =
obj.isInstanceOf[Point] &&
obj.asInstanceOf[Point].x == x
}
object Test {
def main(args: Array[String]) {
val p1 = new Point(2, 3)
val p2 = new Point(2, 4)
val p3 = new Point(3, 3)
println(p1.isNotEqual(p2))
println(p1.isNotEqual(p3))
println(p1.isNotEqual(2))
}
}1.3 extractor
An extractor in Scala is an object that has a method called unapply as one of its members. The purpose of that unapply method is to match a value and take it apart. Often, the extractor object also defines a dual method apply for building values, but this is not required.
Following example shows an extractor object for email addresses:
object Test {
def main(args: Array[String]) {
println ("Apply method : " + apply("Zara", "gmail.com"));
println ("Unapply method : " + unapply("Zara@gmail.com"));
println ("Unapply method : " + unapply("Zara Ali"));
}
// The injection method (optional)
def apply(user: String, domain: String) = {
user +"@"+ domain
}
// The extraction method (mandatory)
def unapply(str: String): Option[(String, String)] = {
val parts = str split "@"
if (parts.length == 2){
Some(parts(0), parts(1))
}else{
None
}
}
}1.4 closure
A closure is a function, whose return value depends on the value of one or more variables declared outside this function.
object Test {
def main(args: Array[String]) {
println( "muliplier(1) value = " + multiplier(1) )
println( "muliplier(2) value = " + multiplier(2) )
}
var factor = 3
val multiplier = (i:Int) => i * factor
}i, is a formal parameter to the function. Hence, it is bound to a new value each time multiplier is called.
1.5 function
A function is a group of statements that together perform a task. You can divide up your code into separate functions. How you divide up your code among different functions is up to you, but logically, the division usually is so that each function performs a specific task.
Scala has both functions and methods and we use the terms method and function interchangeably with a minor difference. A Scala method is a part of a class which has a name, a signature, optionally some annotations, and some bytecode where as a function in Scala is a complete object which can be assigned to a variable. In other words, a function, which is defined as a member of some object, is called a method.
A function definition can appear anywhere in a source file and Scala permits nested function definitions, that is, function definitions inside other function definitions. Most important point to note is that Scala function's name can have characters like +, ++, ~, &,-, -- , \, /, : etc.
object Test {
def main(args: Array[String]) {
println( "Returned Value : " + addInt(5,7) );
}
def addInt( a:Int, b:Int ) : Int = {
var sum:Int = 0
sum = a + b
return sum
}
}
