Mgiyuk

It takes so much time to define an Adam optimizer in TensorFlow I use Adam optimizer to train a network, but I don't know why it takes so much time to just define the trainer. In TensorFlow, what does Adam optimizer do when we define it? Here is how I define the trainer. mse_loss = tf.reduce_sum(tf.squared_difference(generated_images, FD_placeholder)) / (batch_size * width * height) print(gen_variables) print(mse_loss) g_trainer = tf.train.AdamOptimizer(learning_rate=lr) print("aaaaa") g_trainer = g_trainer.minimize(mse_loss, var_list=gen_variables) print("aaaaa") lr is a placeholder for learning rate, it has type tf.float32, and shape=[ ]. generated_images and FD_placeholder have a shape (batch_size,64,64,1). I use batch_size = 2. width and height are equal to 64 gen_variables has shape (91,1,1) and dtype=tf.float32. Here is the output for these few lines of code. [<tf.Variable 'generator_model/g_w1:0' shape=(91, 1, 1) dtype=float32_ref>] Tensor(...

HDF5 reading and fit_generator multiprocessing error I'm trying to multiprocess the fit_generator. These are the problems that I face. trainable_model.fit_generator(load_random_cached_bottlenecks(BATCH_SIZE, label_map, training_addr_label_map, train_npy_dir, 'h5py', h5py_file_train),epochs = EPOCHS, steps_per_epoch=iterations_per_epoch_t, validation_data = load_random_cached_bottlenecks(BATCH_SIZE, label_map, validation_addr_label_map, val_npy_dir, 'h5py', h5py_file_val), validation_steps=iterations_per_epoch_v, workers = 1, callbacks = callback_list, use_multiprocessing = True, max_queue_size = 32) The main arguments that are causing problem: workers and use_multiprocessing . workers use_multiprocessing When worker=1 , use_multiprocessing=True/False runs with no problem. worker=1 use_multiprocessing=True/False If workers=5 , use_multiprocessing=True its throwing errors. The weird thing is its running, but at some random iteration I'm getting errors like work...

Read CSV file with features and labels in the same row in Tensorflow I have a .csv file with around 5000 rows and 3757 columns. The first 3751 columns of each row are the features and the last 6 columns are the labels. Each row is a set of features-labels pair. I'd like to know if there are built-in functions or any fast ways that I can: Basically I want to train a DNN model with 3751 features and 1 label and I'd like the output of the parsing function be fed into the following function for training: train_input_fn = tf.estimator.inputs.numpy_input_fn( x={"x": np.array(training_set.data)}, y=np.array(training_set.target), num_epochs=None, shuffle=True) I know some functions like "tf.contrib.learn.datasets.base.load_csv_without_header" can do similar things but it is already deprecated. 2 Answers 2 You could look into tf.data.Dataset 's input pi...

Plateauing loss in neural style transfer I am writing an implementation of style transfer by loading a vgg model from keras and supplying it to a tensorflow model. I am using an adam optimizer. The loss function is reducing but it is very slow and plateaus off at about 10 8 . Also the style loss is huge (order of 10 8 ) whereas content loss is much smaller(order of 10 5 ). This is weird as the paper for style transfer says to scale content loss down by a factor of 100 or 1000 when calculating total loss. I tried increasing the learning rate but that only makes the gradient overshoot. I suspect there must be a bug in my implementation but despite searching endlessly I have been unable to find what's wrong. Here's the code: # coding: utf-8 # In[1]: from keras.applications.vgg16 import VGG16 from keras.models import Model import tensorflow as tf import tensorflow.contrib.eager as tfe import numpy as np import matplotlib.pyplot as plt # In[2]: content_image_path = './skyline.jp...

keras list of Numpy arrays not the size model expected I am having trouble finding the correct way of passing multiple inputs to a model. The model has 2 inputs (256, 256, 3) (256, 256, 3) and 1 output (256, 256, 3) I am producing the images via ImageDataGenerator : ImageDataGenerator x_data_gen = ImageDataGenerator( horizontal_flip=True, validation_split=0.2) And I am producing the samples via a python generator: def image_sampler(datagen, batch_size, subset="training"): for imgs in datagen.flow_from_directory('data/r_cropped', batch_size=batch_size, class_mode=None, seed=1, subset=subset): g_y = noises = bw_images = for i in imgs: # append to expected output the original image g_y.append(i/255.0) noises.append(generate_noise(1, 256, 3)[0]) bw_images.append(iu_rgb2gray(i)) yield(np.array([noises, bw_images]), np.array(g_y)) When trying to train the model with: gene...

Tensorflow: How to call an operation in C++ from another operation I am writing a new operation in C++ and would like to make use of another operation. For example, I construct two new tensors and would like to do matrix multiplication in my operation so I would like to make use of the MatMulOp . However, the MatMulOp::Compute requires a OpKernelContext as input. MatMulOp MatMulOp::Compute OpKernelContext My question is how I can form such input to call this operation? By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Tensorflow custom estimator: 'Series' objects are mutable, thus they cannot be hashed Trying to create a custom classifier in Tensorflow like so def my_model_fn( features, # This is batch_features from input_fn labels, # This is batch_labels from input_fn mode, # An instance of tf.estimator.ModeKeys params # Additional configuration ): input_layer = tf.feature_column.input_layer(features, feature_columns=params['feature_columns']) (...) where params['feature_columns'] is defined as below, and is of type _NumericColumn params['feature_columns'] _NumericColumn feature_columns = [ tf.feature_column.numeric_column(training_examples['x']) ] params={'feature_columns': feature_columns, 'n_outputs': 1} When I try and construct the model, however, #construct model model = tf.estimator.Estimator( model_fn=my_model_fn, model_dir='te...

Restoring checkpoint in distributed tensorflow Using a setup similar to https://github.com/tensorflow/models/tree/master/inception, the chief worker automatically saves a checkpoint file periodically on the node this process is running on. I'm running two ps on two different nodes. Two workers are also running on the two nodes each, with one out of 4 workers being the chief. When restarting training without any modification, the Supervisor automatically tries to restore the last checkpoint file, but ends up giving an error that it could not find the ckpt on the second node (the node other than the chief worker), because the chief never saved the ckpt on the second node. W tensorflow/core/framework/op_kernel.cc:936] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for /home/muneebs/tf_train/model.ckpt-275 If I copy the ckpt directory to the second node, it restores fine. Is it a bug? Should the saver be initialized as sharded=True? If so, is t...

Tensorflow does not release the memory after session close I have two models. Both model A and B works with training and test when I run them separately. To be more efficient in training two models with same dataset, I put their running code together. A.training() A.close_session() # this closes session with sess.close() B.training() at B.training() it occurs Resource exhausted error! So it seems like it does not release the memory when I do the sess.close() after A.training(). This 'sess' is an attribute both A and B has separately as well. - meaning, it is being used as self.sess Is this a bug ? Is there a solution? . . I have googled and read some arguments and only closing session does not release the gpu memory though. How can I release the gpu memory so the next model can use it? By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and c...

Tensorflow: Unable to feed a string through to placeholder tensor I'm writing a function to compare the similarity of two strings, using Google's universal sentence encoder. Following the instructions in the notebook provided here I have the following method in my class that takes two sentences as input and prints the similarity between them. def tf_sim(self, text1, text2): # Reduce logging output. tf.logging.set_verbosity(tf.logging.ERROR) sim_input1 = tf.placeholder(tf.string, shape=(None), name="sim_input1") sim_input2 = tf.placeholder(tf.string, shape=(None), name="sim_input2") embedding1 = self.embed([sim_input1]) embedding2 = self.embed([sim_input2]) encode1 = tf.nn.l2_normalize(embedding1, axis=1) encode2 = tf.nn.l2_normalize(embedding2, axis=1) sim_scores = -tf.acos(tf.reduce_sum(tf.multiply(encode1, encode2), axis=1)) init_vars = tf.global_variables_initializer() init_tables = tf.tables_initializer() w...

How do I load categorical data from a numpy array into an Indicator or Embedding column? Using Tensorflow 1.8.0, we are running into an issue whenever we attempt to build a categorical column. Here is a full example demonstrating the problem. It runs as-is (using only numeric columns). Uncommenting the indicator column definition and data generates a stack trace ending in tensorflow.python.framework.errors_impl.InternalError: Unable to get element as bytes. tensorflow.python.framework.errors_impl.InternalError: Unable to get element as bytes. import tensorflow as tf import numpy as np def feature_numeric(key): return tf.feature_column.numeric_column(key=key, default_value=0) def feature_indicator(key, vocabulary): return tf.feature_column.indicator_column( tf.feature_column.categorical_column_with_vocabulary_list( key=key, vocabulary_list=vocabulary )) labels = ['Label1','Label2','Label3'] model = tf.estimator.DNNClassifier( feature_columns=[ ...

Correct way to get output of intermediate layer in Keras model? I have trained a model in Keras and want to extract the output from an intermediate layer. The model contains dropout layers and I want to be absolutely sure nothing is dropped when doing this. According to the documentation, a layer's output can be extracted like this: layer_name = 'my_layer' intermediate_layer_model = Model(inputs=model.input, outputs=model.get_layer(layer_name).output) intermediate_output = intermediate_layer_model.predict(data) However, docs also show how to do so with a Keras function: get_3rd_layer_output = K.function([model.layers[0].input, K.learning_phase()], [model.layers[3].output]) # output in test mode = 0 layer_output = get_3rd_layer_output([x, 0])[0] # output in train mode = 1 layer_output = get_3rd_layer_output([x, 1])[0] Here, the learning_phase() flag tells keras whether to actually use dropout and similar thin...