Your browser was unable to load all of the resources. They may have been blocked by your firewall, proxy or browser configuration.
Press Ctrl+F5 or Ctrl+Shift+R to have your browser try again.

pytorch support? #1

#1

Great project!

Given the stress on modular design, I'm wondering how hard it is to "swap-in" the pytorch library and the models.

  • Is the library tied deeply to tensorflow?
  • Is pytorch support planned in the roadmap?
  • replies 3
  • views 3.5K
  • likes 1
#2

thank you!
yes given the design PyTorch would fit well with Ludwig. When we started working on Ludwig pytorch was not mature so we decided to go for TensorFlow, but now we could think about supporting both.Ludwig ties with TensorFlow are not extremely deep, one would have to reimplement the models using pytorch functions instead of tf ones and one would have to substitute all calls to sess.run to fo calls to forward and backward from pytorch. Nothing really difficult or complex but surely time consuming (and a bit boring if you ask me). For this reason it's currently not on the roadmap, but on one hand it could enter it in the future and on the other hand, I would be open to consider contributions in this regard.

#3

I would be happy to contribute to make ludwig more modular. I think it would be easier (and perhaps quicker) to add pytorch backend once the different components of ludwig are 'decoupled'. I'm not sure if the components are fully decoupled at this point -- would be great if you could explain or point me to the current architecture doc.

In particular, having these components fully decoupled would be very useful:

  • data loaders
  • data preprocessors
  • model builders
  • model trainers
  • model loaders and savers

As an example, one may want to use their own data pre-processor (say, from another repository) but reuse the earlier dataloader and the later model builder/trainers/saver components from ludwig. I think, in general, the ability to swap in/out components of ludwig will be very useful and attractive to users. I'm hoping this 'decoupling' effort won't be so boring :slight_smile: Do let me know what you guys think.

#4

Hi Nishant,
you should take a look at the developer guide (it's not super rich yet, but we will add more stuff) and directly at the code, many things are already pretty well incapsulated.
But there is a bigger question that needs to be answered here, because embarking in the effort of making Ludwig work with both TensorFlow and PyTorch is non trivial with respect to the amount of work that it takes: why would one want that to begin with?
From the user point of view, what does training a model with PyTorch rather than TensorFlow change?
I'm not only asking you, but also asking myself. If we can answer these questions in a convincing way, then we can start thinking about that. Because it may be the case that abstracting out to match with both will make everything much more difficult to maintain down the line, like for instance dealing with features available in one framework and not in the other one, or also the differences in philosophy between declarative and imperative nature of the libraries (even with TF2.0 things will change).
Cheers.