torchnmf.nmf

class torchnmf.nmf.BaseComponent(rank=None, W=None, H=None, trainable_W=True, trainable_H=True)[source]

Bases: torch.nn.modules.module.Module

Base class for all NMF modules.

You can’t use this module directly. Your models should also subclass this class.

Parameters

  • rank (int) – size of hidden dimension

  • W (tuple or Tensor) – size or initial weights of template tensor W

  • H (tuple or Tensor) – size or initial weights of activation tensor H

  • trainable_W (bool) – controls whether template tensor W is trainable when initial weights is given. Default: True

  • trainable_H (bool) – controls whether activation tensor H is trainable when initial weights is given. Default: True

W

the template tensor of the module if corresponding argument is given. If size is given, values are initialized non-negatively.

Type

Tensor or None

H

the activation tensor of the module if corresponding argument is given. If size is given, values are initialized non-negatively.

Type

Tensor or None

extra_repr()[source]
fit(V, beta=1, tol=0.0001, max_iter=200, verbose=False, alpha=0, l1_ratio=0)[source]

Learn a NMF model for the data V by minimizing beta divergence.

To invoke this function, attributes H and W should be presented in this module.

Parameters

  • V (Tensor) – data tensor to be decomposed. Can be a sparse tensor returned by torch.sparse_coo_tensor()

  • beta (float) – beta divergence to be minimized, measuring the distance between V and the NMF model. Default: 1.

  • tol (float) – tolerance of the stopping condition. Default: 1e-4

  • max_iter (int) – maximum number of iterations before timing out. Default: 200

  • verbose (bool) – whether to be verbose. Default: False

  • alpha (float) – constant that multiplies the regularization terms. Set it to zero to have no regularization Default: 0

  • l1_ratio (float) – the regularization mixing parameter, with 0 <= l1_ratio <= 1. For l1_ratio = 0 the penalty is an elementwise L2 penalty (aka Frobenius Norm). For l1_ratio = 1 it is an elementwise L1 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2. Default: 0

Returns

total number of iterations

Return type

int

forward(H=None, W=None)[source]

An outer wrapper of self.reconstruct(H,W).

Note

Should call the BaseComponent instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters

  • H (Tensor, optional) – input activation tensor H. If no tensor was given will use H from this module instead

  • W (Tensor, optional) – input template tensor W. If no tensor was given will use W from this module instead

Returns

tensor

Return type

Tensor

kernel_size
out_channels
rank
static reconstruct(H, W)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

sparse_fit(V, beta=2, max_iter=200, verbose=False, sW=None, sH=None)[source]

Learn a NMF model for the data V by minimizing beta divergence with sparseness constraints proposed in Non-negative Matrix Factorization with Sparseness Constraints.

To invoke this function, attributes H and W should be presented in this module.

Note

Although the value range of beta is unrestricted, the original implementation only use Euclidean Distance (which means beta=2) as their loss function, and we have no gaurantee on other values besides 2.

Parameters

  • V (Tensor) – data tensor to be decomposed. Can be a sparse tensor returned by torch.sparse_coo_tensor()

  • beta (float) – beta divergence to be minimized, measuring the distance between V and the NMF model Default: 1.

  • max_iter (int) – maximum number of iterations before timing out. Default: 200

  • verbose (bool) – whether to be verbose. Default: False

  • sW (float or None) – the target sparseness for template tensor W , with 0 < sW < 1. Set it to None will have no constraint. Default: None

  • sH (float or None) – the target sparseness for activation tensor H , with 0 < sH < 1. Set it to None will have no constraint. Default: None

Returns

total number of iterations

Return type

int

class torchnmf.nmf.NMF(Vshape=None, rank=None, **kwargs)[source]

Bases: torchnmf.nmf.BaseComponent

Non-Negative Matrix Factorization (NMF).

Find two non-negative matrices (W, H) whose product approximates the non- negative matrix V: \(V \approx HW^T\).

This factorization can be used for example for dimensionality reduction, source separation or topic extraction.

Note

If Vshape argument is given, the model will try to infer the size of W and H, and override arguments passed through to BaseComponent.

Parameters

  • Vshape (tuple, optional) – size of target matrix V

  • rank (int, optional) – size of hidden dimension

  • **kwargs – arguments passed through to BaseComponent

Shape:

  • V: \((N, C)\)

  • W: \((C, R)\)

  • H: \((N, R)\)

Examples:

>>> V = torch.rand(20, 30)
>>> m = NMF(V.shape, 5)
>>> m.W.size()
torch.Size([30, 5])
>>> m.H.size()
torch.Size([20, 5])
>>> HWt = m()
>>> HWt.size()
torch.Size([20, 30])
class torchnmf.nmf.NMF2D(Vshape=None, rank=None, kernel_size=1, **kwargs)[source]

Bases: torchnmf.nmf.BaseComponent

Nonnegative Matrix Factor 2-D Deconvolution (NMF2D).

Find non-negative 3-dimensional tensor H and 4-dimensional tensor W whose 2D convolutional output approximates the non-negative 3-dimensional tensor V:

\[\mathbf{V} \approx \sum_{\tau} \sum_{\phi} \stackrel{\downarrow \phi}{\mathbf{W}^{\tau}} \stackrel{\rightarrow \tau}{\mathbf{H}^{\phi}}\]

More precisely:

\[V_{i,j,k} \approx \sum_{l=0}^{k_1-1} \sum_{m=0}^{k_2-1} \sum_{r=0}^{\text{rank}-1} W_{i,r,l,m} * H_{r, j-l,k-m}\]

Look at the paper: Nonnegative Matrix Factor 2-D Deconvolution for Blind Single Channel Source Separation by Schmidt et al. (2006) for more details.

Note

To match with PyTorch convention, an extra batch dimension is required for target tensor V.

Note

If Vshape argument is given, the model will try to infer the size of W and H, and override arguments passed through to BaseComponent.

Warning

Using sparse tensor as target when calling NMF2D.fit() or NMF2D.sparse_fit() is currently not supported.

Parameters

  • Vshape (tuple, optional) – size of target tensor V

  • rank (int, optional) – size of hidden dimension

  • kernel_size (int or tuple, optional) – size of the convolving kernel

  • **kwargs – arguments passed through to BaseComponent

Shape:

  • V: \((N, C, L_{out}, M_{out})\)

  • W: \((C, R, \text{kernel_size}[0], \text{kernel_size}[1])\)

  • H: \((N, R, L_{in}, M_{in})\) where

\[L_{in} = L_{out} - \text{kernel_size}[0] + 1\]
\[M_{in} = M_{out} - \text{kernel_size}[1] + 1\]

Examples:

>>> V = torch.rand(33, 50).unsqueeze(0).unsqueeze(0)
>>> m = NMF2D(V.shape, 16, 3)
>>> m.W.size()
torch.Size([1, 16, 3, 3])
>>> m.H.size()
torch.Size([1, 16, 31, 48])
>>> HWt = m()
>>> HWt.size()
torch.Size([1, 1, 33, 50])
class torchnmf.nmf.NMF3D(Vshape=None, rank=None, kernel_size=1, **kwargs)[source]

Bases: torchnmf.nmf.BaseComponent

Nonnegative Matrix Factor 3-D Deconvolution (NMF3D).

Find non-negative 4-dimensional tensor H and 5-dimensional tensor W whose 2D convolutional output approximates the non-negative 4-dimensional tensor V:

\[V_{i,j,k,l} \approx \sum_{m=0}^{k_1-1} \sum_{n=0}^{k_2-1} \sum_{u=0}^{k_3-1} \sum_{r=0}^{\text{rank}-1} W_{i,r,m,n,u} * H_{r,j-m,k-n,l-u}\]

Note

To match with PyTorch convention, an extra batch dimension is required for target tensor V.

Note

If Vshape argument is given, the model will try to infer the size of W and H, and override arguments passed through to BaseComponent.

Warning

Using sparse tensor as target when calling NMF3D.fit() or NMF3D.sparse_fit() is currently not supported.

Parameters

  • Vshape (tuple, optional) – size of target tensor V

  • rank (int, optional) – size of hidden dimension

  • kernel_size (int or tuple, optional) – size of the convolving kernel

  • **kwargs – arguments passed through to BaseComponent

Shape:

  • V: \((N, C, L_{out}, M_{out}, O_{out})\)

  • W: \((C, R, \text{kernel_size}[0], \text{kernel_size}[1], \text{kernel_size}[2])\)

  • H: \((N, R, L_{in}, M_{in}, O_{in})\) where

\[L_{in} = L_{out} - \text{kernel_size}[0] + 1\]
\[M_{in} = M_{out} - \text{kernel_size}[1] + 1\]
\[O_{in} = O_{out} - \text{kernel_size}[2] + 1\]

Examples:

>>> V = torch.rand(3, 64, 64, 100).unsqueeze(0)
>>> m = NMF3D(V.shape, 8, (5, 5, 20))
>>> m.W.size()
torch.Size([3, 8, 5, 5, 20])
>>> m.H.size()
torch.Size([1, 8, 60, 60, 81])
>>> HWt = m()
>>> HWt.size()
torch.Size([1, 3, 64, 64, 100])
class torchnmf.nmf.NMFD(Vshape=None, rank=None, T=1, **kwargs)[source]

Bases: torchnmf.nmf.BaseComponent

Non-negative Matrix Factor Deconvolution (NMFD).

Find non-negative matrix H and 3-dimensional tensor W whose convolutional output approximates the non- negative matrix V:

\[\mathbf{V} \approx \sum_{t=0}^{T-1} \mathbf{W}_{t} \cdot \stackrel{t \rightarrow}{\mathbf{H}}\]

More precisely:

\[V_{i,j} \approx \sum_{t=0}^{T-1} \sum_{r=0}^{\text{rank}-1} W_{i,r,t} * H_{r, j - t}\]

Look at the paper: Non-negative Matrix Factor Deconvolution; Extraction of Multiple Sound Sources from Monophonic Inputs by Paris Smaragdis (2004) for more details.

Note

To match with PyTorch convention, an extra batch dimension is required for target matrix V.

Note

If Vshape argument is given, the model will try to infer the size of W and H, and override arguments passed through to BaseComponent.

Warning

Using sparse tensor as target when calling NMFD.fit() or NMFD.sparse_fit() is currently not supported.

Parameters

  • Vshape (tuple, optional) – size of target matrix V

  • rank (int, optional) – size of hidden dimension

  • T (int, optional) – size of the convolving window

  • **kwargs – arguments passed through to BaseComponent

Shape:

  • V: \((N, C, L_{out})\)

  • W: \((C, R, T)\)

  • H: \((N, R, L_{in})\) where

\[L_{in} = L_{out} - T + 1\]

Examples:

>>> V = torch.rand(33, 50).unsqueeze(0)
>>> m = NMF(V.shape, 16, 3)
>>> m.W.size()
torch.Size([33, 16, 3])
>>> m.H.size()
torch.Size([1, 16, 48])
>>> HWt = m()
>>> HWt.size()
torch.Size([1, 33, 50])