DeepLearningKit – Open Source Deep Learning Framework for Apple iOS, OS X

We are introducing you to the new deep learning framework “DeepLearningKit”, for the Apple based OS which is developed in Metal and Swift.

By Amund Tveit.

Happy to announce that a(early) version of DeepLearningKit is available on:


0. What does DeepLearningKit do?

It currently allows using deep convolutional neural networks model trained in Caffe on Apple’s iOS, OS X and tvOS  (transformed from protobuffers into json with the tool at – a tutorial about this will come later).

1. Open Source License?

Apache 2.0

2. How to get started?

Have a look at Tutorial – Using DeepLearningKit with iOS for iPhone and iPad – it is about using a pre-trained CIFAR-10 Network in Network example.

3. What is DeepLearningKit developed in?

It is developed in Metal (for GPU Acceleration) and Swift (for app integration). I believe DeepLearningKit is the first (public) Deep Learning tool that is using the Metal compute API for GPUs (Metal is Apple’s recommended way to program GPUs)

4. More documentation

More tutorials and a paper describing DeepLearningKit will be made available on (+ for the paper)

5. I love developing for Apple’s [iOS,OS X or tvOS] and would like to contribute to this project, how?

Here are a few thoughts:

  • Performance Optimization wrt Metal (GPU): Metal is a very new API (in particular for GPGPU non-graphical processing), and there is probably ways of improve usage of it.
  • Performance Optimization wrt algorithms (e.g. shader functions for comvolution): see our paper for roadmap
  • Importers: develop model importers (in Swift) for convolutional neural networks) from other tools than Caffe, e.g. Torch,  TensorFlow, Theano, Nervana Systems, DeepLearning4J or Pylearn. HDF5 is an interesting format.
  • Training Support: our goal was to primarily support using already trained Deep Learning models (since in the long run people will probably not train their own DL models but rather pick them from a Deep Learning Model store or similar, see our paper for why), but it would still be great to train convolutional neural networks in DeepLearningKit itself.
  • Image Handling Support: DeepLearningKit is missing basic conversion from e.g. UIImage to RGB (the example network supports 32x32x3 CIFAR RGB Image Format, but has no conversion from UIImage to it). Check out e.g. Drawing Images From Pixel Data – In Swift and Image Processing in iOS Part 1: Raw Bitmap Modification for inspiration.
6. Is DeepLearningKit production ready for my mission critical app?

most likely not, but that doesn’t stop you from testing it out?

7. DeepLearningKit reminds me more about CUDA/GPU Libraries such as Nvidia’s cuDNN or  Facebook’s fbCunn rather than larger tools such as Torch, TensorFlow and Caffe, is that right?

You’re right, DeepLearningKit can be roughly seen as an early “metalDNN” with Swift packaging for loading and running models. (It currently doesn’t support Fast Fourier-based convolution such as Facebook’s fbcunn)

8. Who Developed and Open Sourced DeepLearningKit

DeepLearningKit was developed and open-sourced by the company Memkite – check out the About page for details

Bio: Amund Tveit, PhD, Co-Developer of DeepLearningKit for iOS, OS X & tvOS ( ), Ex-Googler, based in Norway.