torchnmf.plca

class torchnmf.plca.BaseComponent(rank=None, W=None, H=None, Z=None, trainable_W=True, trainable_H=True, trainable_Z=True)[source]

Bases: torch.nn.modules.module.Module

Base class for all PLCA modules.

You can’t use this module directly. Your models should also subclass this class.

Parameters

rank (int) – size of hidden dimension
W (tuple or Tensor) – size or initial probabilities of template tensor W
H (tuple or Tensor) – size or initial probabilities of activation tensor H
Z (Tensor) – initial probabilities of latent vector Z
trainable_W (bool) – controls whether template tensor W is trainable when initial probabilities is given. Default: True
trainable_H (bool) – controls whether activation tensor H is trainable when initial probabilities is given. Default: True
trainable_Z (bool) – controls whether latent vector Z is trainable when initial probabilities is given. Default: True

W

the template tensor of the module if corresponding argument is given. If size is given, values are initialized randomly.

Type: Tensor or None

H

the activation tensor of the module if corresponding argument is given. If size is given, values are initialized randomly.

Type: Tensor or None

Z

the latent vector of the module if corresponding argument or rank is given. If rank is given, values are initialized uniformly.

Type: Tensor or None

extra_repr()[source]

fit(V, tol=0.0001, max_iter=200, verbose=False, W_alpha=1.0, H_alpha=1.0, Z_alpha=1.0)[source]

Learn a PLCA model for the data V by maximizing the following log probability of V and model params \(\theta\) using EM algorithm:

\[\begin{split}\mathcal{L} (\theta)= \sum_{k_1...k_N} v_{k_1...k_N}\log{\hat{v}_{k_1...k_N}} \\ + \sum_k (\alpha_{z,k} - 1) \log z_k \\ + \sum_{f_1...f_M} (\alpha_{w,f_1...f_M} - 1) \log w_{f_1...f_M} \\ + \sum_{\tau_1...\tau_L} (\alpha_{h,\tau_1...\tau_L} - 1) \log h_{\tau_1...\tau_L} \\\end{split}\]

Where \(\hat{V}\) is the reconstructed output, N is the number of dimensions of target tensor \(V\), M is the number of dimensions of tensor \(W\), and L is the number of dimensions of tensor \(H\). The last three terms come from Dirichlet prior assumption.

To invoke this function, attributes H, W and Z should be presented in this module.

Parameters

V (Tensor) – data tensor to be decomposed
tol (float) – tolerance of the stopping condition. Default: 1e-4
max_iter (int) – maximum number of iterations before timing out. Default: 200
verbose (bool) – whether to be verbose. Default: False
W_alpha (float) – hyper parameter of Dirichlet prior on W. Can be a scalar or a tensor that is broadcastable to W. Set it to one to have no regularization. Default: 1
H_alpha (float) – hyper parameter of Dirichlet prior on H. Can be a scalar or a tensor that is broadcastable to H. Set it to one to have no regularization. Default: 1
Z_alpha (float) – hyper parameter of Dirichlet prior on Z. Can be a scalar or a tensor that is broadcastable to Z. Set it to one to have no regularization. Default: 1

Returns

a length-2 tuple with first element is total number of iterations, and the second is the sum of tensor V

Return type

tuple

forward(H=None, W=None, Z=None, norm=None)[source]

An outer wrapper of self.reconstruct(H,W,Z).

Note

Should call the BaseComponent instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters

H (Tensor, optional) – input activation tensor H. If no tensor was given will use H from this module instead
W (Tensor, optional) – input template tensor W. If no tensor was given will use W from this module instead
Z (Tensor, optional) – input latent vector Z. If no tensor was given will use Z from this module instead
norm (float, optional) – a scaling value multiply on output before return. Default: 1

Returns

tensor

Return type

Tensor

kernel_size

out_channels

rank

static reconstruct(H, W, Z)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

class torchnmf.plca.PLCA(Vshape=None, rank=None, **kwargs)[source]

Bases: torchnmf.plca.BaseComponent

Probabilistic Latent Component Analysis (PLCA).

Estimate two marginals \(P(c|z)\) and \(P(n|z)\), which is the matrix W and H, and a prior \(P(z)\) which is the vector Z, that approximate the observed \(P(n,c)\), where \(P(n,c)\) is obtained via V / V.sum() so the total probabilities sum to 1.

More precisely:

\[P(n, c) \approx \sum_{z}P(c|z)P(z)P(n|z)\]

In matrix form:

\[V \approx H diag(Z) W^T\]

Its formulation is very similar to NMF, but introduce an extra latent vector to incorporate probabilities concept.

Note

If Vshape argument is given, the model will try to infer the size of W, H and Z, and override arguments passed through to BaseComponent.

Parameters

Vshape (tuple, optional) – size of target matrix V
rank (int, optional) – size of hidden dimension
**kwargs – arguments passed through to BaseComponent

Shape:

V: \((N, C)\)
W: \((C, R)\)
H: \((N, R)\)
Z: \((R,)\)

Examples:

>>> V = torch.rand(20, 30)
>>> m = PLCA(V.shape, 5)
>>> m.W.size()
torch.Size([30, 5])
>>> m.H.size()
torch.Size([20, 5])
>>> m.Z.size()
torch.Size([5])
>>> HZWt = m()
>>> HZWt.size()
torch.Size([20, 30])

class torchnmf.plca.SIPLCA(Vshape=None, rank=None, T=1, **kwargs)[source]

Bases: torchnmf.plca.BaseComponent

Shift Invariant Probabilistic Latent Component Analysis (SI-PLCA).

Estimate two marginals \(P(c,t|z)\) and \(P(n,l|z)\), which is the tensor W and H, and a prior \(P(z)\) which is the vector Z, that approximate the observed \(P(n,c,l)\), where \(P(n,c,l)\) is obtained via V / V.sum() so the total probabilities sum to 1.

More precisely:

\[P(n, c, l) \approx \sum_{z} \sum_{t} P(c,t|z)P(z)P(n,l-t|z)\]

Look at the paper: Shift-Invariant Probabilistic Latent Component Analysis by Paris Smaragdis and Bhiksha Raj (2007) for more details.

Note

If Vshape argument is given, the model will try to infer the size of W, H and Z, and override arguments passed through to BaseComponent.

Parameters

Vshape (tuple, optional) – size of target matrix V
rank (int, optional) – size of hidden dimension
T (int, optional) – size of the convolving window
**kwargs – arguments passed through to BaseComponent

Shape:

V: \((N, C, L_{out})\)
W: \((C, R, T)\)
H: \((N, R, L_{in})\)
Z: \((R,)\) where

\[L_{in} = L_{out} - T + 1\]

Examples:

>>> V = torch.rand(33, 50).unsqueeze(0)
>>> m = SIPLCA(V.shape, 16, 3)
>>> m.W.size()
torch.Size([33, 16, 3])
>>> m.H.size()
torch.Size([1, 16, 48])
>>> m.Z.size()
torch.Size([16])
>>> HZWt = m()
>>> HZWt.size()
torch.Size([1, 33, 50])

class torchnmf.plca.SIPLCA2(Vshape=None, rank=None, kernel_size=1, **kwargs)[source]

Bases: torchnmf.plca.BaseComponent

Shift Invariant Probabilistic Latent Component Analysis across 2 dimensions (SI-PLCA 2D).

Estimate two marginals \(P(c,k_1,k_2|z)\) and \(P(n,l,m|z)\), which is the tensor W and H, and a prior \(P(z)\) which is the vector Z, that approximate the observed \(P(n,c,l,m)\), where \(P(n,c,l,m)\) is obtained via V / V.sum() so the total probabilities sum to 1.

More precisely:

\[P(n,c,l,m) \approx \sum_{z} \sum_{k_1} \sum_{k_2} P(c,k_1,k_2|z)P(z)P(n,l-k_1,m-k_2|z)\]

Look at the paper: Shift-Invariant Probabilistic Latent Component Analysis by Paris Smaragdis and Bhiksha Raj (2007) for more details.

Note

If Vshape argument is given, the model will try to infer the size of W, H and Z, and override arguments passed through to BaseComponent.

Parameters

Vshape (tuple, optional) – size of target tensor V
rank (int, optional) – size of hidden dimension
kernel_size (int or tuple, optional) – size of the convolving kernel
**kwargs – arguments passed through to BaseComponent

Shape:

V: \((N, C, L_{out}, M_{out})\)
W: \((C, R, \text{kernel_size}[0], \text{kernel_size}[1])\)
H: \((N, R, L_{in}, M_{in})\)
Z: \((R,)\) where

\[L_{in} = L_{out} - \text{kernel_size}[0] + 1\]

\[M_{in} = M_{out} - \text{kernel_size}[1] + 1\]

Examples:

>>> V = torch.rand(33, 50).unsqueeze(0).unsqueeze(0)
>>> m = SIPLCA2(V.shape, 16, 3)
>>> m.W.size()
torch.Size([1, 16, 3, 3])
>>> m.H.size()
torch.Size([1, 16, 31, 48])
>>> m.Z.size()
torch.Size([16])
>>> HZWt = m()
>>> HZWt.size()
torch.Size([1, 1, 33, 50])

class torchnmf.plca.SIPLCA3(Vshape=None, rank=None, kernel_size=1, **kwargs)[source]

Bases: torchnmf.plca.BaseComponent

Shift Invariant Probabilistic Latent Component Analysis across 3 dimensions (SI-PLCA 3D).

Estimate two marginals \(P(c,k_1,k_2,k_3|z)\) and \(P(n,l,m,o|z)\), which is the tensor W and H, and a prior \(P(z)\) which is the vector Z, that approximate the observed \(P(n,c,l,m,o)\), where \(P(n,c,l,m,o)\) is obtained via V / V.sum() so the total probabilities sum to 1.

More precisely:

\[P(n,c,l,m,o) \approx \sum_{z} \sum_{k_1} \sum_{k_2} \sum_{k_3} P(c,k_1,k_2,k_3|z)P(z)P(n,l-k_1,m-k_2,o-k_3|z)\]

Look at the paper: Shift-Invariant Probabilistic Latent Component Analysis by Paris Smaragdis and Bhiksha Raj (2007) for more details.

Note

If Vshape argument is given, the model will try to infer the size of W, H and Z, and override arguments passed through to BaseComponent.

Parameters

Vshape (tuple, optional) – size of target tensor V
rank (int, optional) – size of hidden dimension
kernel_size (int or tuple, optional) – size of the convolving kernel
**kwargs – arguments passed through to BaseComponent

Shape:

V: \((N, C, L_{out}, M_{out}, O_{out})\)
W: \((C, R, \text{kernel_size}[0], \text{kernel_size}[1], \text{kernel_size}[2])\)
H: \((N, R, L_{in}, M_{in}, O_{in})\)
Z: \((R,)\) where

\[L_{in} = L_{out} - \text{kernel_size}[0] + 1\]

\[M_{in} = M_{out} - \text{kernel_size}[1] + 1\]

\[O_{in} = O_{out} - \text{kernel_size}[2] + 1\]

Examples:

>>> V = torch.rand(3, 64, 64, 100).unsqueeze(0)
>>> m = SIPLCA3(V.shape, 8, (5, 5, 20))
>>> m.W.size()
torch.Size([3, 8, 5, 5, 20])
>>> m.H.size()
torch.Size([1, 8, 60, 60, 81])
>>> m.Z.size()
torch.Size([8])
>>> HZWt = m()
>>> HZWt.size()
torch.Size([1, 3, 64, 64, 100])