PyTorch VS TensorFlow

原作者: Iliya Valchanov
转载自: quora

Let’s start from NumPy (you’ll see why a bit later).

You can build a machine learning algorithm even with NumPy, but creating a deep neural network is getting exponentially harder. NumPy is designed for (fast) computations, and not machine learning. It is usually used for machine learning together with a machine learning package.


PyTorch is similar to NumPy in the way that it manages computations, but has a strong GPU support. Similarly to NumPy, it also has a C (the programming language) backend, so they are both much faster than native Python libraries. NumPy could be GPU accelerated (with some extra code), but it doesn’t have this strong GPU support that PyTorch or TensorFlow do. Finally, PyTorch was specifically tailored for GPU functionality in Python.


TensorFlow, on the other hand, was written mainly in C++ and CUDA (NVIDIA’s language for programming GPUs), and was not specifically created for Python. It provides functionalities in C, C++, Java, Go (Google’s language), and there is community support for Haskell and Rust [1]. So, with TF, you are not restricted by Python. Even if the syntax differs a bit across languages, the concepts are the same.


Now, PyTorch has deep neural networks functionalities and that is why it is often compared with TensorFlow, sklearn, etc. Moreover, TensorFlow has a peculiar logic (with concepts like placeholders, sessions, etc.). So, for TensorFlow, you need to make that extra effort. Knowing NumPy (which is my underlying assumption for the dear reader), it is easier to switch to PyTorch than TensorFlow, that is why it is gaining popularity so fast.

As TensorFlow was used by Google for so long, it is very easy to deploy algorithms using it. So you can think about it as more product oriented. Logically, you want to be able to deploy the algorithms that you are creating (you can check out TensorFlow Serving[2] for more on that).

PyTorch, on the other hand, is more recent, so it does not have the full range of capabilities of other packages. I am sure that this will change and they are on their way to catch up. However, TensorFlow has this module tf.contrib. That’s code, contributed by the community, that will probably be embedded into the core TF. Point being, the TensorFlow community is stronger, because TF is older. I don’t know if PyTorch will catch up to the community of TF (that depends on the users and the adoption).


So, my verdict would be that TensorFlow has kind of stood the test of time (if you can use this expression for a framework that is not that old itself) and is still more widely adopted. It has more capabilities and a better scalability for larger projects (projects of the scale of Google). PyTorch is surely gaining momentum as it is easier to learn but doesn’t have the same integration. It is very good for small projects that need to be done quite fast, but is not optimal for product deployment.

Practical reasons to learn both

If you want to work in the industry, it is probable that a company may have some customized framework. On your CV, you will be expected to have experience with (a) machine learning package(s). In any case, they won’t expect you to have worked with all of them. If you have to choose a single one of them, it seems to me that going for “the easier to learn” package looks worse. So learn both or go with TF.