improve scaladoc

Author: izhangzhihao

Layer/src/main/scala/com/thoughtworks/deeplearning/Layer.scala

@@ -11,6 +11,7 @@ object Layer {
 
     private[CloseableOnce] final class ClosingFlag {
       var closed = false
+
       @elidable(elidable.ASSERTION)
       def close() = {
         assert(!closed)
@@ -55,7 +56,6 @@ object Layer {
     * `Tape` is a intermediate data structure generated by the [[Layer#forward forward]] of neural network , which contains result of the `forward`, and can perform [[backward]] for back-propagation.
     *
     * @note [[close]] method must be called when this [[Tape]] will not be used any more.
-    *
     * @see [[https://en.wikipedia.org/wiki/Automatic_differentiation tape auto differentiation]]
     */
   trait Tape extends AutoCloseable {
@@ -76,14 +76,13 @@ object Layer {
 
     /**
       * Returns a new [[Layer.Tape Tape]] that shares the same [[value]] and [[backward]] behavior with this `Tape`.
+      *
       * @note The newly created `Tape` and this `Tape` must be [[close]]d independently.
       */
     def duplicate(): Tape.Aux[Data, Delta]
 
-
     protected def forceBackward(delta: Delta): Unit
 
-
     def isTrainable: Boolean
 
     /**
@@ -121,10 +120,11 @@ object Layer {
 }
 
 /**
-  * 一个`Layer`表示一个神经网络。每个`Layer`可以作为子网络被包含在其它`Layer`中，构成更复杂的神经网络。`Layer`的嵌套结构可以用来表示数学公式或粗粒度神经网络结构。
-  * 当神经网络被编写完成后，其中大部分元素都是占位符，当网络开始训练时数据才真正进入到网络。
+  * A `Layer` represents a neural network. Each `Layer` can be included in as a sub-network of another `Layer`, forming a more complex neural network. The nesting structure of `Layer` can be used to represent mathematical expression or Coarse-grain neural network structure.
+  * When a neural network is written, the most elements in it are placeholders. When network training begins, the data enter into the network.
+  *
+  * === Tree structure of `Layer` ===
   *
-  * === `Layer` 的树结构 ===
   * {{{
   * val myLayer: Layer.Aux[Tape.Aux[Double, Double], Tape.Aux[Double, Double]] = {
   *   Times(
@@ -137,55 +137,59 @@ object Layer {
   * }
   * }}}
   *
-  * 以上代码等价的数学公式可以用[[Symbolic]]写作：`(1.0 + x) * 2.0.toWeight`。`2.0.toWeight`表示一个变量，其初始值是`2`，在神经网络迭代时，其值会更新。
-  * [[com.thoughtworks.deeplearning.DifferentiableDouble.Layers.Times Times]]、[[com.thoughtworks.deeplearning.DifferentiableDouble.Layers.Plus Plus]]都是 case class，
-  * 因此`myLayer`是一个case class构成的嵌套结构的树。`Times`和`Plus`都是占位符。
+  * The above mathematical expression with equivalent codes can be written, by [[Symbolic]], as: `(1.0 + x) * 2.0.toWeight`. `2.0.toWeight` represents a variable, of which the initial value is `2`. The value updates during neural network iteration.
+  *
+  * Both [[com.thoughtworks.deeplearning.DifferentiableDouble.Layers.Times Times]] and [[com.thoughtworks.deeplearning.DifferentiableDouble.Layers.Plus Plus]] are of case class, therefore, `myLayer` is a tree in nested structure consisted of the case class. `Times` and `Plus` are placeholders.
   *
-  * [[com.thoughtworks.deeplearning.DifferentiableDouble.Layers.Weight Weight]]是一个包含权重的`Layer`，初始值是`2.0`。
-  * [[com.thoughtworks.deeplearning.Symbolic.Layers.Identity Identity]]是一个输入和输出相同的`Layer`，它会将输入原样返回。`Identity`在这里是`Input`的占位符。
-  * [[com.thoughtworks.deeplearning.Symbolic.Layers.Literal Literal]]是一个包含常量的`Layer`。
+  * [[com.thoughtworks.deeplearning.DifferentiableDouble.Layers.Weight Weight]] is a `Layer` containing weight, of which the initial value is `2.0`.
   *
-  * === 迭代 ===
+  * [[com.thoughtworks.deeplearning.Symbolic.Layers.Identity Identity]] is a `Layer` with equal input and output, which return the same input back. The `Identity` here is the placeholder of `Input`.
   *
-  * 网络每次训练称为一个迭代，分为[[forward]]和[[com.thoughtworks.deeplearning.Layer.Tape#backward backward]]两个阶段，构成一次完整的[[https://en.wikipedia.org/wiki/Backpropagation 反向传播]]流程。
+  * [[com.thoughtworks.deeplearning.Symbolic.Layers.Literal Literal]] is a `Layer` containing a constant.
   *
-  * ==== forward ====
+  * === Iteration ===
   *
-  * 在`Layer.Aux[A,B]`中调用`forward`时，`A`是输入类型，`B`是输出类型，`A`和`B`都是[[com.thoughtworks.deeplearning.Layer.Tape Tape]]。下面开始逐段解释代码：
+  * Each training of the network is called as an iteration, including two stages: [[forward]] and [[com.thoughtworks.deeplearning.Layer.Tape#backward backward]], forming a complete process of [[https://en.wikipedia.org/wiki/Backpropagation]]
   *
-  * 例如：
+  * ==== Forward ====
+  *
+  * When invoking `forward` in `Layer.Aux[A,B]`, `A` is input type, `B` is output type, and both `A` and `B` are [[com.thoughtworks.deeplearning.Layer.Tape Tape]]. Now, the codes are interpreted segment by segment as follows.
+  *
+  * For example:
   * {{{
   * val inputTape: Tape.Aux[Double, Double] = Literal(a)
   * val outputTape = myLayer.forward(inputTape)
   * }}}
   *
   *
-  * 当调用`myLayer.forward(inputData)`时，首先调用`Times`的`forward`，其伪代码如下：
+  * When invoking `myLayer.forward(inputData)`, `forward` of `Times` shall be invoked first, of which the pseudo codes are as follows:
   * {{{
   * final case class Times(operand1: Layer, operand2: Layer) extends Layer {
   *   def forward(inputData: Tape): Output = {
   *     val upstream1 = operand1.forward(input)
   *     val upstream2 = operand2.forward(input)
-  *     new Output(upstream1, upstream2)//这里忽略具体实现，而关注递归细节
+  *     new Output(upstream1, upstream2)// the concrete realization is ignored here, and recursion details are focused on
   *   }
   *   final class Output(upstream1: Tape, upstream2: Tape) extends Tape { ... }
   * }
   * }}}
   *
-  * 在`myLayer.operand1`是`Plus`,`myLayer.operand2`是`Weight`，因此，`upstream1`和`upstream2`分别是`operand1`和`operand2` `forward` 的结果。
+  * It is `Plus` at `myLayer.operand1`, and `Weight` at `myLayer.operand2`. Therefore, `upstream1` and `upstream2` are the results of `forward` of `operand1` and `operand2` respectively.
   *
-  * 以此类推，`Plus`的`forward`代码与`Times`的`forward`类似，当调用`Plus`的`forward`时，[[com.thoughtworks.deeplearning.DifferentiableDouble.Layers.Plus#operand1 operand1]]是`Literal`，[[com.thoughtworks.deeplearning.DifferentiableDouble.Layers.Plus.operand2 operand2]]是`Identity`，这时会各自调用`Literal`和`Identity`的`forward`。
+  * In a similar way, the `forward` code of `Plus` is similar to `forward` of `Times`. During the invoking for `forward` of `Plus`, [[com.thoughtworks.deeplearning.DifferentiableDouble.Layers.Plus#operand1 operand1]] is `Literal`, and [[com.thoughtworks.deeplearning.DifferentiableDouble.Layers.Plus.operand2 operand2]] is `Identity`. At this point, `forward` of `Literal` and `Identity` of each are invoked respectively.
+  *
+  * During the invoking for `forward` of `Identity`, the same input will be returned. The pseudo code for `forward` of `Identity` is as follows:
   *
-  * 当调用`Identity`的`forward`时会原样返回输入， `Identity`的`forward`的伪代码如下：
   * {{{
   * def forward(inputTape: Tape.Aux[Double, Double]) = inputTape
   * }}}
-  * 所以`Input`即 数学公式`(1.0 + x) * 2.0.toWeight` 中的`x`，这样`Input`就被传递给了神经网络。
+  * Therefore, `Input` is the `x` in mathematical expression `(1.0 + x) * 2.0.toWeight`, and in this way, `Input` is propagated to the neural network.
+  *
+  * The return value `outputTape` of `myLayer.forward` is in `Tape` type. Therefore, a tree consisted of `Tape` will be generated finally with the structure similar to that of `myLayer`.
   *
-  * `myLayer.forward`的返回值`outputTape` 是 `Tape`类型，所以最终会生成一棵`Tape`构成的树，结构和`myLayer`一样。
-  * 因此，通过层层传播 `myLayer.forward(inputTape)`最终被`Identity`原样返回，组合进新生成的`Tape`树。
+  * Therefore, via layer-by-layer propagation, the same `myLayer.forward(inputTape)` is finally returned by `Identity` and combined into the newly generated `Tape` tree.
   *
-  * `outputTape` 的包含`forward` 的计算结果，计算结果可以用来 `backward` 比如：
+  * The computation result including `forward` of `outputTape` can be used for `outputTape`, for example:
   * {{{
   *    try {
   *      val loss = outputTape.value
@@ -196,11 +200,10 @@ object Layer {
   *    }
   * }}}
   *
-  * `outputTape.value` 是数学公式 `(1.0 + x) * 2.0.toWeight` 的计算结果。
+  * `outputTape.value` is the computation result of mathematical expression `(1.0 + x) * 2.0.toWeight`
+  * ==== Backward ====
   *
-  * ==== backward ====
-  *
-  * `outputTape.backward`即`Times.Output`的`backward` ，伪代码如下：
+  * `outputTape.backward` is the `outputTape.backward` of `Times.Output`, of which the pseudo code is as follows:
   * {{{
   * case class Times(operand1: Layer, operand2: Layer) extends Layer {
   *   def forward = ...
@@ -215,20 +218,22 @@ object Layer {
   * }
   * }}}
   *
-  * `outputTape.upstream1`和`outputTape.upstream2`分别是`operand1`和`operand2` `forward` 的结果。然后`outputTape.upstream1`和`outputTape.upstream2`分别进行`backward`。
+  * `outputTape.upstream1` and `outputTape.upstream2` are the results of `forward` of `operand1` and `operand2` respectively, which are followed by `backward` of `outputTape.upstream1` and `outputTape.upstream2`.
+  *
   *
-  * 以此类推，`Plus`的`backward`代码与`Times`的`backward`类似，当调用`Plus`的`backward`时，`upstream1`和`upstream2`分别是`Literal`和`Identity` `forward`的结果，这时会各自调用`upstream1`和`upstream2`的`backward`。
+  * In a similar way, the `backward` code of `Plus`` is similar to `backward` of `Times`. During the invoking for `backward` of `Plus`, `upstream1` and `upstream2` are the results of `forward` of `Literal` and `Identity` respectively. At this point, `backward` of `upstream1` and `upstream2` of each are invoked respectively.
   *
-  * `Weight`在`backward`时会更新自己，参考[[com.thoughtworks.deeplearning.DifferentiableDouble.Optimizers.LearningRate#updateDouble updateDouble]]
+  * `Weight` updates during `backward`, refer to [[com.thoughtworks.deeplearning.DifferentiableDouble.Optimizers.LearningRate#updateDouble updateDouble]]
   *
   * === Aux & Symbolic API ===
   *
-  * `Layer.Aux[A,B]`表示`Input`的类型是`A`，`Output`的类型是`B`。`Tape.Aux[C,D]`表示`Data`的类型是`C`，`Delta`的类型是`D`。
-  * `Layer.Aux`和`Type.Aux`可以组合起来使用，比如`Layer.Aux[Tape.Aux[A,B]`,`Tape.Aux[C,D]]`可以用来表示一个`layer`的输入类型是一个`Tape`，这个`Tape`的数据类型为`A`，`delta`类型为`B`，`layer`的输出类型是一个`Tape`，这个`Tape`的数据类型为`C`，`delta`类型为`D`。
+  * `Layer.Aux[A,B]` represents that `Input` is of `A` type, and `Output` is of `B` type. `Tape.Aux[C,D]` represents that `Data` is of `C` type, and `Delta` is of `D` type.
+  *
+  * `Layer.Aux` and `Type.Aux` can be combined for use. For example, `Layer.Aux[Tape.Aux[A,B]`,`Tape.Aux[C,D]]` can be used to represent that the input type of a `layer` is a `Tape`, and the data type of this `Tape` is `A`, `delta` type is `B`; the output type of a `layer` is a `Tape`, and the data type of this `Tape` is `C`, `delta` type is `D`.
   *
-  * [[https://gigiigig.github.io/posts/2015/09/13/aux-pattern.html Aux]]是一种实现了[[https://www.scala-lang.org/files/archive/spec/2.12/03-types.html type refinement]]的设计模式，可以用来限制类型参数的范围。
+  * [[https://gigiigig.github.io/posts/2015/09/13/aux-pattern.html Aux]] is a design pattern which realized [[https://www.scala-lang.org/files/archive/spec/2.12/03-types.html type refinement]] and can be used to limit the range of type parameters.
   *
-  * 通常我们不会手写`Aux`类型，因为我们可以使用`Symbolic`实现同样的功能，例如在用于符号方法内部变量和返回值时：`Layer.Aux[Tape.Aux[INDArray, INDArray], Tape.Aux[INDArray, INDArray` 和 `INDArray @Symbolic` 是等价的，所以我们经常使用`Symbolic`来替代`Aux`的写法。
+  * Generally, we will not handwrite `Aux` type, because we can use `Symbolic` to acquire the same effect. For example, when used for symbolic method internal variable and return value: `Layer.Aux[Tape.Aux[INDArray, INDArray], Tape.Aux[INDArray, INDArray` and `INDArray @Symbolic` are equivalent, so we usually use `Symbolic` to replace the writing method of `Aux`.
   *
   * @see [[https://gigiigig.github.io/posts/2015/09/13/aux-pattern.html aux pattern]]
   * @see [[http://www.vlachjosef.com/aux-pattern-evolution/ aux pattern evolution]]

Symbolic/src/main/scala/com/thoughtworks/deeplearning/Symbolic.scala

@@ -14,30 +14,31 @@ import scala.language.{existentials, implicitConversions}
   * Combining with [[https://github.com/ThoughtWorksInc/implicit-dependent-type implicit-dependent-type]] compiler plugin,
   * it can be treated as a type [[http://www.scala-lang.org/files/archive/spec/2.12/11-annotations.html annotation]] in the form of `NativeOutput @Symbolic`, converting `NativeOutput` to a specific [[Layer]] type.
   *
-  * == `@Symbolic` 的三种用法 ==
+  * == Three usages of `@Symbolic` ==
   *
-  * === 用于符号方法的隐式参数类型 ===
+  * === Implicit parameter types used for symbol methods ===
   *
-  * 如果某个方法的隐式类型参数标注了`@Symbolic`，那么这个方法就是符号方法，`@Symbolic`所标注的隐式参数类型是这个符号方法的'''输入类型'''。
-  * 这种情况下，`NativeOutput @Symbolic`会被展开为：
-  * {{{Identity[NativeOutput, NativeOutput的导数类型]}}}
+  * In case that the implicit parameter of an method is marked with `@Symbolic`, then this method is symbol method. The implicit parameter type marked with `@Symbolic` is the '''input type''' of this symbol method.
+  * In this case, `NativeOutput @Symbolic` will be expanded as:
+  * {{{Identity[NativeOutput, Derivative type of NativeOutput]}}}
   *
-  * 例如：
+  * For example:
   *
   * {{{
   * def sumNetwork(implicit scores: INDArray @Symbolic): Double = {
   *   exp(scores).sum
   * }
   * }}}
   *
-  * 上述代码中，由于`INDArray`的[[Layer.Tape#Delta 导数类型]]也是`INDArray`，所以`sumNetwork`的输入类型`INDArray @Symbolic`展开后是`Identity[INDArray, INDArray]`。
+  * In the above code, because [[Layer.Tape#Delta the derivative type]] of `INDArray` is also `INDArray`, the input type `INDArray @Symbolic` of `sumNetwork`, once expanded, is `Identity[INDArray, INDArray]`
   *
-  * === 用于符号方法内部变量和返回值 ===
+  * === Used for symbol method internal variable and return value ===
   *
-  * 在符号方法内部和返回值处，`NativeOutput @Symbolic`会被展开为：
-  * {{{Layer.Aux[Tape.Aux[输入类型的值类型, 输入类型的导数类型], Tape.Aux[NativeOutput, NativeOutput的导数类型]]}}}
+  * A `NativeOutput @Symbolic` inside a symbol method, or at the return position of a symbol method, will be expanded as:
+  * {{{Layer.Aux[Tape.Aux[value type of input type, derivative type of input type], Tape.Aux[NativeOutput, derivative type of NativeOutput]]}}}
+  *
+  * For example:
   *
-  * 例如：
   *
   * {{{
   * def sumNetwork(implicit scores: INDArray @Symbolic): Double @Symbolic = {
@@ -47,32 +48,34 @@ import scala.language.{existentials, implicitConversions}
   * }
   * }}}
   *
-  * 上述代码中，`expScores`的类型`INDArray @Symbolic`展开后是：
+  * In the above code, the type `INDArray @Symbolic` of `expScores` is expanded as:
   * {{{Layer.Aux[Tape.Aux[INDArray, INDArray], Tape.Aux[INDArray, INDArray]]}}}
   *
-  * 而`result`的类型`Double @Symbolic`展开后是：
+  * The type `Double @Symbolic` of `result` is expanded as:
   * {{{Layer.Aux[Tape.Aux[INDArray, INDArray], Tape.Aux[Double, Double]]}}}
   *
-  * === 用于符号方法之外 ===
+  * === Used for cases excluding symbol method ===
   *
-  * 在符号方法之外，`(NativeInput => NativeOutput) @Symbolic`会被展开为：
-  * {{{Layer.Aux[Tape.Aux[NativeInput, NativeInput的导数类型], Tape.Aux[NativeOutput, NativeOutput的导数类型]]}}}
+  * `(NativeInput => NativeOutput) @Symbolic` outside a symbol method, will be expanded as:
+  * {{{Layer.Aux[Tape.Aux[NativeInput, derivative type of NativeInput], Tape.Aux[NativeOutput, derivative type of NativeOutput]]}}}
   *
-  * 例如：
+  * For example:
   *
   * {{{
   * val predictor: (INDArray => Double) @Symbolic = sumNetwork
   * }}}
   *
-  * 上述代码中，`predictor`的类型`(INDArray => Double) @Symbolic`展开后是：
+  * In the above code, type `(INDArray => Double) @Symbolic` of `predictor` is expanded as:
+  *
   * {{{Layer.Aux[Tape.Aux[INDArray, INDArray], Tape.Aux[Double, Double]]}}}
   *
-  * == 定制符号类型 ==
+  * == Custom symbol type ==
+  *
   *
-  * `@Symbolic`通过检查[[Symbolic.ToLiteral]]隐式值来确定原生类型和导数之间的映射关系。
-  * 因此，只要定义[[Symbolic.ToLiteral]]类型的隐式值，`@Symbolic`就可以支持定制符号类型。
+  * The `@Symbolic` determines the mapping relation between the primitive type and derivative by checking [[Symbolic.ToLiteral]] implicit value. Therefore, `@Symbolic` can be a custom symbol type once you define your own the implicit [[Symbolic.ToLiteral]].
+  *
+  * For example, if you want to support `Short @Symbolic`, using [[scala.Float Float]] as the derivative type of [[scala.Short Short]], then you can conduct the follows:
   *
-  * 比如，假如你希望支持`Short @Symbolic`，其中使用[[scala.Float Float]]作为[[scala.Short Short]]的导数类型，那么可以这样做：
   *
   * {{{
   * implicit object ShortToLiteral extends ToLiteral[Short] {
@@ -88,9 +91,8 @@ import scala.language.{existentials, implicitConversions}
   * val shortNetwork: (Short => Short) @Symbolic = makeShortNetwork
   * }}}
   *
-  * 这样一来`shortNetwork`的类型就会展开为：
+  * Thus, type of `shortNetwork` is expanded as:
   * {{{Layer.Aux[Tape.Aux[Short, Float], Tape.Aux[Short, Float]]}}}
-  *
   * @see [[Symbolic.Layers.Identity]]
   * @see [[Symbolic.ToLiteral]]
   * @see [[Layer.Tape#Delta]]
@@ -121,7 +123,7 @@ private[deeplearning] trait LowPrioritySymbolic { this: Symbolic.type =>
   *   val floatLayer: Float @Symbolic = 1.0f.toLayer
   *   floatLayer
   * }
-  *}}}
+  * }}}
   *
   * The second way is [[Symbolic.autoToLayer autoToLayer]], such as:
   * {{{