Advice

I want to implement in python some algorithms from a paper that allow for a pre-trained neural network to be modified (adding or removing neurons or layers) conserving (theoretically) the outputs of the network over the training data.

I've researched a bit on pytorch, keras and tensorflow and I'm not sure which one is more efficient for implementing this algorithms. For instance, in pytorch I've created a function that implements the neuron addition algorithm by cloning the pre-trained network to a bigger network and adjusting the weights. This approach seems rather inefficient to me.

I think that implementing this algorithms from scratch in numpy wouldn't be as useful for further research. Do you know of a library in which I could freely add and remove neurons and change weights while being efficient?

Improve this question

3 Replies 3

Stef
•

ai.stackexchange might also be a good place to ask.

Improve this answer
G-Man
•

Pytorch is generally the go to library to implement new architectures. It's very good for prototyping which is perfect for research. So you should start with Pytorch imo. Also, writing an algorithm from scratch is a lot of work and will probably be less efficient than Pytorch because you either don't use GPUs or use them but do not possess the optimized CUDA kernels Pytorch has access to.

Improve this answer
Mohammad Othman
•

You could use Pytorch or Tenserflow, thay have functions to do all what you need.

In my opinion, if your network architecture is big use Pytorh , otherwise use Tenseflow.

Improve this answer

Your Reply

By clicking “Post Your Reply”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.