Is there any way to convert pytorch tensor to tensorflow tensor

Question

https://github.com/taoshen58/BiBloSA/blob/ec67cbdc411278dd29e8888e9fd6451695efc26c/context_fusion/self_attn.py#L29

I need to use mulit_dimensional_attention from the above link which is implemented in TensorFlow but I am using PyTorch so can I Convert Pytorch Tensor to TensorFlow Tensor or I have to implement it in PyTorch.

code which I am trying to use here I have to pass 'rep_tensor' as TensorFlow tensor type but I have PyTorch tensor

def multi_dimensional_attention(rep_tensor, rep_mask=None, scope=None,
                                keep_prob=1., is_train=None, wd=0., activation='elu',
                                tensor_dict=None, name=None):

    # bs, sl, vec = tf.shape(rep_tensor)[0], tf.shape(rep_tensor)[1], tf.shape(rep_tensor)[2]

    ivec = rep_tensor.shape[2]
    with tf.variable_scope(scope or 'multi_dimensional_attention'):
        map1 = bn_dense_layer(rep_tensor, ivec, True, 0., 'bn_dense_map1', activation,
                              False, wd, keep_prob, is_train)
        map2 = bn_dense_layer(map1, ivec, True, 0., 'bn_dense_map2', 'linear',
                              False, wd, keep_prob, is_train)
        # map2_masked = exp_mask_for_high_rank(map2, rep_mask)

        soft = tf.nn.softmax(map2, 1)  # bs,sl,vec
        attn_output = tf.reduce_sum(soft * rep_tensor, 1)  # bs, vec

        # save attn
        if tensor_dict is not None and name is not None:
            tensor_dict[name] = soft

        return attn_output

iamanigeeit · Accepted Answer · 2020-06-08 10:24:32Z

You can convert a pytorch tensor to a numpy array and convert that to a tensorflow tensor and vice versa:

import torch
import tensorflow as tf

pytorch_tensor = torch.zeros(10)
np_tensor = pytorch_tensor.numpy()
tf_tensor = tf.convert_to_tensor(np_tensor)

That being said, if you want to train a model that uses a combination of pytorch and tensorflow that's going to be... awkward, slow, buggy and take a long time to write to say the least. Since the libraries have to figure out how to backpropagate the cost.

So unless the pytorch attention block you have is pre-trained, I'd recommend just sticking to one library or another, there's plenty of examples for implementing anything you want in either and plenty of pretrained models for both. Tensorflow is usually a bit faster but the speed differences aren't that significant and the kind of "hack" I presented above will likely make the whole thing slower than using either of the libraries standalone.

Eypros · Accepted Answer · 2023-05-17 08:45:03Z

3

@George solution is just fine but as of April 2023 at least the intermediate step of converting to numpy is not necessary (although I am not sure it is not used internally in the conversion procedure). So, it's a simple command like:

import torch
import tensorflow as tf

pytorch_tensor = torch.zeros(10)
tf_tensor = tf.convert_to_tensor(pytorch_tensor)

As for the comment of being awkward, slow etc. I have seen monumental works like Yolov5 mixing up this combination at some point.

edited May 17, 2023 at 8:45

answered Apr 5, 2023 at 10:13

Eypros

5,7437 gold badges54 silver badges89 bronze badges

Collectives™ on Stack Overflow

Is there any way to convert pytorch tensor to tensorflow tensor

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related