1. tf.data

1. 相比 feed_in和placeholder的优势：数据的一些操作(比如shuffle/batch/repeat/map)集成在tf中，所以效率高速度快，而且属于high-level api，使用方便

2. tf.data.Dataset

3. 输出一些dataset的types/shape以做sanity check

   print(xxxdataset.output_types)           # >> (tf.float32, tf.float32)print(xxxdataset.output_shapes)             # >> (TensorShape([]), TensorShape([]))

4. 有很多格式，就我做过大型视频和图像的经验，我推荐tf.data.TFRecordDataset(filenames)

5. 一些基本的数据操作

   # 准备数据dataset = tf.data.TFRecordDataset([file1, file2, file3, ...])# 数据操作dataset = dataset.shuffle(1000)dataset = dataset.repeat(100)dataset = dataset.batch(128)dataset = dataset.map(lambda x: tf.one_hot(x, 10)) #转化为 one-hot encoding# 取数据iterator = dataset.make_one_shot_iterator() # 一种获取iterator的方式，后面还有更通用的X, Y = iterator.get_next() # 如果上面batch过，一次就是取一个batch,否则就是一个sample(x,y)

6. 据Notes所说，tf.data比fedd_in和placehodler效率要高

7. 一个非常的编程实践

   iterator = tf.data.Iterator.from_structure(train_data.output_types,                                           train_data.output_shapes)img, label = iterator.get_next()train_init = iterator.make_initializer(train_data)  # initializer for train_datatest_init = iterator.make_initializer(test_data)  # initializer for train_data# ...sess.run(train_init) # 系统会自动加载training set的img,label# ...sess.run(test_init) # 加载的是 testing set的 img,labels# 最上面的 img, label = iterator.get_next() 完全不存在同名的冲突，因为为init的控制隔离

2. optimizer速记

1. 对梯度做些特殊修改

   # create an optimizer.optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1)# compute the gradients for a list of variables.grads_and_vars = optimizer.compute_gradients(loss, 
         
          )# grads_and_vars is a list of tuples (gradient, variable).  Do whatever you# need to the 'gradient' part, for example, subtract each of them by 1.subtracted_grads_and_vars = [(gv[0] - 1.0, gv[1]) for gv in grads_and_vars]# ask the optimizer to apply the subtracted gradients.optimizer.apply_gradients(subtracted_grads_and_vars)
         

2. 让某些变量不参与计算梯度的过程

   stop_gradient( input, name=None )

   • 应用场景举例：
     • When you train a GAN (Generative Adversarial Network) where no backprop should happen through the adversarial example generation process.
     • The EM algorithm where the M-step should not involve backpropagation through the output of the E-step

3. 手动对某些y=f(x)求偏导：

   tf.gradients(    ys,    xs,    grad_ys=None,    name='gradients',    colocate_gradients_with_ops=False,    gate_gradients=False,    aggregation_method=None,    stop_gradients=None)

   • 应用场景举例

     Technical detail: This is especially useful when training only parts of a model. For example, we can use tf.gradients() to take the derivative G of the loss w.r.t. to the middle layer. Then we use an optimizer to minimize the difference between the middle layer output M and M + G. This only updates the lower half of the network.(冻结某些层，只训练一些层，比如说：fine-tune过程)