CodeCollection2018
10/28/2018 - 5:55 AM

tf.nn.nce_loss注意

tf.nn.nce_loss损失函数用法注意事项

A common use case is to use this method for training, and calculate the full sigmoid loss for evaluation or inference. In this case, you must set`partition_strategy="div"` for the two losses to be consistent, as in the following example:

```python
if mode == "train":
  loss = tf.nn.nce_loss(
      weights=weights,
      biases=biases,
      labels=labels,
      inputs=inputs,
      ...,
      partition_strategy="div")
elif mode == "eval":
  logits = tf.matmul(inputs, tf.transpose(weights))
  logits = tf.nn.bias_add(logits, biases)
  labels_one_hot = tf.one_hot(labels, n_classes)
  loss = tf.nn.sigmoid_cross_entropy_with_logits(
      labels=labels_one_hot,
      logits=logits)
  loss = tf.reduce_sum(loss, axis=1)
```
nce_loss默认负采样使用log-uniform (Zipfian)。
同时如果其参数中num_true>1,就是有多个值是正确的,则会付给每个目标类"1/num_true"概率的值以保证全体概率值为1.
It would be useful to allow a variable number of target classes per example.  We hope to provide this functionality in a future release.For now, if you have a variable number of target classes, you can pad them out to a constant number by either repeating them or by padding with an otherwise unused class.
weights: A `Tensor` of shape `[num_classes, dim]`, or a list of `Tensor`
      objects whose concatenation along dimension 0 has shape
      [num_classes, dim].  The (possibly-partitioned) class embeddings.
  biases: A `Tensor` of shape `[num_classes]`.  The class biases.
  labels: A `Tensor` of type `int64` and shape `[batch_size,
      num_true]`. The target classes.
  inputs: A `Tensor` of shape `[batch_size, dim]`.  The forward
      activations of the input network.
  num_sampled: An `int`.  The number of classes to randomly sample per batch.
  num_classes: An `int`. The number of possible classes.
  num_true: An `int`.  The number of target classes per training example.