TFGENZOO.flows.quantize module

class TFGENZOO.flows.quantize.LogitifyImage(corruption_level=1.0, alpha=0.05)[source]

Bases: TFGENZOO.flows.flowbase.FlowBase

Apply Tapani Raiko’s dequantization and express image in terms of logits

Sources:

Parameters

  • corrupution_level (float) – power of added random variable.

  • alpha (float) – parameter about transform close interval to open interval [0, 1] to (1, 0)

Note

We know many implementation on this quantization, but we use this formula. since many implementations use it.

  • forward preprocess (add noise)
    \[\begin{split}z &\leftarrow 255.0 x \ \because [0, 1] \rightarrow [0, 255] \\ z &\leftarrow z + \text{corruption_level} \times \epsilon \ where\ \epsilon \sim N(0, 1)\\ z &\leftarrow z / (\text{corruption_level} + 255.0)\\ z &\leftarrow z (1 - \alpha) + 0.5 \alpha \ \because \ [0, 1] \rightarrow (0, 1) \\ z &\leftarrow \log(z) - \log(1 -z)\end{split}\]
  • forward formula
    \[\begin{split}z &= logit(x (1 - \alpha) + 0.5 \alpha)\\ &= \log(x) - \log(1 - x)\\ LogDetJacobian &= sum(softplus(z) + softplus(-z) - softplus(\log(\cfrac{\alpha}{1 - \alpha})))\end{split}\]
  • inverse formula
    \[\begin{split}x &= logisitic(z)\\ &= 1 / (1 + exp( -z )) \\ x &= (x - 0.5 * \alpha) / (1.0 - \alpha)\\ InverseLogDetJacobian &= sum(-2 \log(logistic(z)) - z) + softplus(\log(\cfrac{\alpha}{1 - \alpha})))\end{split}\]

Examples

>>> import tensorflow as tf
>>> from TFGENZOO.flows import LogitifyImage
>>> li = LogitifyImage()
>>> li.build([None, 32, 32, 1])
>>> li.get_config()
{'name': 'logitify_image_1', ...}
>>> inputs = tf.keras.Input([32, 32, 1])
>>> li(inputs)
(<tf.Tensor 'logitify_image/Identity:0' shape=(None, 32, 32, 1) dtype=float32>,
<tf.Tensor 'logitify_image/Identity_1:0' shape=(None,) dtype=float32>)
>>> tf.keras.Model(inputs, li(inputs)).summary()
Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         [(None, 32, 32, 1)]       0
_________________________________________________________________
logitify_image (LogitifyImag ((None, 32, 32, 1), (None 1
=================================================================
Total params: 1
Trainable params: 0
Non-trainable params: 1
_________________________________________________________________
build(input_shape: tensorflow.python.framework.tensor_shape.TensorShape)[source]

Creates the variables of the layer (optional, for subclass implementers).

This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call.

This is typically used to create the weights of Layer subclasses.

Parameters

input_shape – Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).

forward(x: tensorflow.python.framework.ops.Tensor, **kwargs)[source]
get_config()[source]

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns

Python dictionary.

inverse(z: tensorflow.python.framework.ops.Tensor, **kwargs)[source]