AttributeError When Updating Learning Rate in Keras Using K.set_value

Question

I'm trying to update the learning rate of my Keras model dynamically during training. I'm using the following code:

import tensorflow as tf
from tensorflow.keras import backend as K

model = keras.models.Sequential([keras.layers.Dense(10)])
model.compile(keras.optimizers.SGD(), loss='mse')

# Change learning rate to 0.001
K.set_value(model.optimizer.learning_rate, 0.001)

However, I get this error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-33-76f5edca812e> in <cell line: 0>()
----> 1 K.set_value(model.optimizer.learning_rate, 0.001)

/usr/local/lib/python3.11/dist-packages/keras/src/legacy/backend.py in set_value(x, value)
   1883 def set_value(x, value):
   1884     """DEPRECATED."""
-> 1885     value = np.asarray(value, dtype=x.dtype.name)
   1886     x.assign(value)
   1887 

AttributeError: 'str' object has no attribute 'name'

I'm using TensorFlow 2.18.0.

What is the correct way to update the learning rate dynamically in Keras for this version of TensorFlow?

Thank you for helping in advance!

You need to show more of your code. What is autoencoder_model? This is probably the reason for your problem. — Frede
– Frede, Commented Apr 1 at 8:20
Hello @codebysumit, please note that tf.keras.backend.set_value has been deprecated, as indicated in this documentation . Consequently, it will not function as expected. As an alternative, I successfully used the .assign method. My implementation, which is working correctly, can be found in this Gist. — Sagar
– Sagar, Commented Apr 1 at 10:46

codebysumit · Accepted Answer · 2025-04-02 13:05:07Z

After debugging, I found the issue.

In TensorFlow 2.18.0, model.optimizer.learning_rate is a Keras Variable object, and its dtype attribute returns a data type name in string format (e.g., "float32"). Since Python strings don’t have a name attribute, we encounter this error when using K.set_value().

Understanding the Error:

Let's inspect the dtype attribute:

print(type(model.optimizer.learning_rate.dtype))

print(model.optimizer.learning_rate.dtype)

Output:

<class 'str'>
float32

The error occurs because keras._legacy.backend.set_value() tries to access x.dtype.name, but in the latest TensorFlow version, the name attribute no longer exists in dtype.

Here’s the source code from K.set_value():

@keras_export("keras._legacy.backend.set_value")
def set_value(x, value):
    """DEPRECATED."""
    value = np.asarray(value, dtype=x.dtype.name)
    x.assign(value)

Better Alternative: Use .assign() Instead of K.set_value():

As @sagar pointed out in the comments, tf.keras.backend.set_value() is deprecated and may not function as expected. Instead, the recommended approach is using .assign() directly:

# Correct way to update the learning rate
model.optimizer.learning_rate.assign(0.001)

This approach works reliably in the latest TensorFlow versions without requiring a custom function.

Fix Custom Function: Rewrite set_value() for Newer TensorFlow Versions:

To resolve this, I rewrote the set_value() function to use x.dtype directly instead of x.dtype.name:

import numpy as np

def set_value(x, value):
    value = np.asarray(value, dtype=x.dtype)  # Use x.dtype instead of x.dtype.name
    x.assign(value)

Usage Example:

set_value(model.optimizer.learning_rate, 0.0055)
print(model.optimizer.learning_rate)

This ensures compatibility with newer versions of TensorFlow while avoiding the AttributeError.

This method correctly updates the learning rate without errors.

Hope this helps!

Collectives™ on Stack Overflow

AttributeError When Updating Learning Rate in Keras Using K.set_value

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related