【DyNet學習】Arithmetic operations

dynet.cdiv(x, y)

Componentwise division

Divide an expressions component-wise by another, broadcasting dimensions (currently only of the second expression!) if necessary as follows:

• When number of dimensions differ, we add dimensions of size 1 to make the number of dimensions match
• Now, every dimensions is required to have matching size, or the dim size of the right expression must equal 1 (in which case it will be broadcasted)
• In the same way, the batch sizes must match, or the batch size of the right expression must equal 1 in which case it will be broadcasted
• The resulting tensor’s dimensionality is thus determined as the max of both inputs at every position

Parameters:	x (dynet.Expression) – The first input expression y (dynet.Expression) – The second input expression
Returns:	An expression where the ith element is equal to xiyi
Return type:	dynet.Expression

dynet.cmult(x, y)

Componentwise multiplication

Multiply two expressions component-wise, broadcasting dimensions if necessary as follows:

• When number of dimensions differ, we add dimensions of size 1 to make the number of dimensions match
• Now, every dimensions is required to have matching size, or one of the dimensions must equal 1 (in which case it will be broadcasted)
• In the same way, the batch dimension must match, or equal 1 in which case it will be broadcasted
• The resulting tensor’s dimensionality is thus determined as the max of both inputs at every position

Parameters:	x (dynet.Expression) – The first input expression y (dynet.Expression) – The second input expression
Returns:	An expression where the ith element is equal to xi×yi
Return type:	dynet.Expression

dynet.colwise_add(x, y)

Columnwise addition

Add vector y to each column of matrix x

Parameters:	x (dynet.Expression) – An MxN matrix y (dynet.Expression) – A length M vector
Returns:	An expression where y is added to each column of x
Return type:	dynet.Expression

dynet.squared_norm(x)

Squared norm

The squared norm of the values of x: ∥x∥22=∑ix2i

Parameters:	x (dynet.Expression) – Input expression
Returns:	∥x∥22=∑ix2i
Return type:	dynet.Expression

dynet.l2_norm(x)

L2 norm

The l2 norm of the values of x: ∥x∥2=∑ix2i−−−−−√

Parameters:	x (dynet.Expression) – Input expression
Returns:	∥x∥2=∑ix2i−−−−−√
Return type:	dynet.Expression

dynet.exp(x)

Natural exponent

Calculate elementwise yi=exi

Parameters:	x (dynet.Expression) – Input expression
Returns:	ex
Return type:	dynet.Expression

dynet.square(x)

Square

Calculate elementwise yi=x2i

Parameters:	x (dynet.Expression) – Input expression
Returns:	y=x2
Return type:	dynet.Expression

dynet.sqrt(x)

Square root

Calculate elementwise yi=xi−−√

Parameters:	x (dynet.Expression) – Input expression
Returns:	y=x−−√
Return type:	dynet.Expression

dynet.abs(x)

Absolute value

Calculate elementwise yi=|xi|

Parameters:	x (dynet.Expression) – Input expression
Returns:	y=|x|
Return type:	dynet.Expression

dynet.erf(x)

Gaussian error function

Elementwise calculation of the Gaussian error function yi=erf(xi)=1π√∫xi−xie−t2dt

Parameters:	x (dynet.Expression) – Input expression
Returns:	yi=erf(xi)
Return type:	dynet.Expression

dynet.cube(x)

Calculate elementwise yi=x3i

Parameters:	x (dynet.Expression) – Input expression
Returns:	y=x3
Return type:	dynet.Expression

dynet.log(x)

Natural logarithm

Elementwise calculation of the natural logarithm yi=ln(xi)

Parameters:	x (dynet.Expression) – Input expression
Returns:	yi=ln(xi)
Return type:	dynet.Expression

dynet.log_sigmoid(x)

Log sigmoid

Calculate elementwise log sigmoid function yi=ln(11+exi) This is more numerically stable than log(logistic(x))

Parameters:	x (dynet.Expression) – Input expression
Returns:	yi=ln(11+exi)
Return type:	dynet.Expression

dynet.lgamma(x)

Log gamma

Calculate elementwise log gamma function yi=ln(Γ(xi))

Parameters:	x (dynet.Expression) – Input expression
Returns:	yi=ln(Γ(xi))
Return type:	dynet.Expression

dynet.sin(x)

Sine

Elementwise calculation of the sine

Parameters:	x (dynet.Expression) – Input expression
Returns:	sin(x)
Return type:	dynet.Expression

dynet.cos(x)

Cosine

Elementwise calculation of the cosine

Parameters:	x (dynet.Expression) – Input expression
Returns:	cos(x)
Return type:	dynet.Expression

dynet.tan(x)

Tangent

Elementwise calculation of the tangent

Parameters:	x (dynet.Expression) – Input expression
Returns:	tan(x)
Return type:	dynet.Expression

dynet.asin(x)

Inverse sine

Elementwise calculation of the inverse sine

Parameters:	x (dynet.Expression) – Input expression
Returns:	sin−1(x)
Return type:	dynet.Expression

dynet.acos(x)

Inverse cosine

Elementwise calculation of the inverse cosine

Parameters:	x (dynet.Expression) – Input expression
Returns:	cos−1(x)
Return type:	dynet.Expression

dynet.atan(x)

Tangent

Elementwise calculation of the inverse tangent

Parameters:	x (dynet.Expression) – Input expression
Returns:	tan−1(x)
Return type:	dynet.Expression

dynet.sinh(x)

Hyperbolic sine

Elementwise calculation of the hyperbolic sine

Parameters:	x (dynet.Expression) – Input expression
Returns:	sinh(x)
Return type:	dynet.Expression

dynet.cosh(x)

Hyperbolic cosine

Elementwise calculation of the hyperbolic cosine

Parameters:	x (dynet.Expression) – Input expression
Returns:	cosh(x)
Return type:	dynet.Expression

dynet.tanh(x)

Hyperbolic tangent

Elementwise calculation of the hyperbolic tangent

Parameters:	x (dynet.Expression) – Input expression
Returns:	tanh(x)
Return type:	dynet.Expression

dynet.asinh(x)

Inverse hyperbolic sine

Elementwise calculation of the inverse hyperbolic sine

Parameters:	x (dynet.Expression) – Input expression
Returns:	sinh−1(x)
Return type:	dynet.Expression

dynet.acosh(x)

Inverse hyperbolic cosine

Elementwise calculation of the inverse hyperbolic cosine

Parameters:	x (dynet.Expression) – Input expression
Returns:	cosh−1(x)
Return type:	dynet.Expression

dynet.atanh(x)

Inverse hyperbolic tangent

Elementwise calculation of the inverse hyperbolic tangent

Parameters:	x (dynet.Expression) – Input expression
Returns:	tanh−1(x)
Return type:	dynet.Expression

dynet.logistic(x)

Logistic sigmoid function

Calculate elementwise yi=11+e−xi

Parameters:	x (dynet.Expression) – Input expression
Returns:	yi=11+e−xi
Return type:	dynet.Expression

dynet.rectify(x)

Rectifier (or ReLU, Rectified Linear Unit)

Calculate elementwise recitifer (ReLU) function yi=max(xi,0)

Parameters:	x (dynet.Expression) – Input expression
Returns:	yi=max(xi,0)
Return type:	dynet.Expression

dynet.elu(x, alpha=1.0)

Exponential Linear Unit (ELU)

Calculate elementwise the function

yi={xi,α×(exi−1),if x>0if x⩽0

Reference: Clevert et al., 2015

Parameters:	x (dynet.Expression) – Input expression alpha (number) – αα parameter
Returns:	ELU(xi,α)
Return type:	dynet.Expression

dynet.selu(x)

Scaled Exponential Linear Unit (SELU)

Calculate elementwise the function

yi=λ×{xi,α×(exi−1),if x>0if x⩽0

With

λα=1.0507009873554804934193349852946=1.6732632423543772848170429916717

Reference: Klambaouer et al., 2017

Parameters:	x (dynet.Expression) – Input expression
Returns:	SELU(xi)
Return type:	dynet.Expression

dynet.sparsemax(x)

Sparsemax

The sparsemax function (Martins et al. 2016), which is similar to softmax, but induces sparse solutions where most of the vector elements are zero. Note: This function is not yet implemented on GPU.

Parameters:	x (dynet.Expression) – Input expression
Returns:	The sparsemax of the scores
Return type:	dynet.Expression

dynet.softsign(x)

Softsign function

Calculate elementwise the softsign function yi=xi1+|xi|

Parameters:	x (dynet.Expression) – Input expression
Returns:	yi=xi1+|xi|
Return type:	dynet.Expression

dynet.pow(x, y)

Power function

Calculate an output where the ith element is equal to xyi

Parameters:	x (dynet.Expression) – The first input expression y (dynet.Expression) – The second input expression(scalar expression)
Returns:	xyi
Return type:	dynet.Expression

dynet.bmin(x, y)

Minimum

Calculate an output where the ith element is min(xi,yi)

Parameters:	x (dynet.Expression) – The first input expression y (dynet.Expression) – The second input expression
Returns:	min(xi,yi)
Return type:	dynet.Expression

dynet.bmax(x, y)

Maximum

Calculate an output where the ith element is max(xi,yi)

Parameters:	x (dynet.Expression) – The first input expression y (dynet.Expression) – The second input expression
Returns:	max(xi,yi)
Return type:	dynet.Expression

dynet.cumsum(x, d=0)

Cumulative sum along an arbitrary dimension

Computes the cumulative sum yi=∑j≤ixjalong an arbitrary dimension.

Parameters:	x (dynet.Expression) – Input expression d (int) – Dimension along which to compute the cumulative sums (default: 0)
Returns:	An expression with the same dimension as the input
Return type:	dynet.Expression