MyCaffe 1.12.1.82

MyCaffe AI Platform (CUDA 11.8.0, cuDNN 8.8.0) with TFT version 1.12.1.82 ready!

MyCaffe now supports Temporal Fusion Transformer Models (TFT)! The MyCaffe AI Platform provides an easy AI solution for multiple AI disciplines, including:

• Classification with AlexNet, ResNet, VGG, NoisyNet, and Inception models • Classification with SiameseNet • Classification with TripletNet • Auto Encoders and DANN • Onnx AI Model Support (import and export) • Object detection with Single-Shot Multi Box (SSD) • Reinforcement Learning with Policy Gradient and Deep Q-Learning • Recurrent Learning with CharNet • Neural Style Transfer • Seq2Seq Models • Transformer Models • GPT Models • Temporal Fusion Transformer Models

Speed up AI training with the MyCaffe in-memory database that caches full datasets or drip-fed datasets into your local RAM on one side while feeding the training process on the other with label balanced data. Easily Train on multiple-GPUs with NCCL.

CUDA 11.8.0.522, cuDNN 8.8.0.121, nvapi 515, Windows 10-22H2/Windows 11-22H2, Driver 531.14

MyCaffe[1] (a complete C# re-write of CAFFE[2]) now supports Visual Studio 2022 and CUDA 11.8.0/cuDNN 8.8.0 and Windows 11!

When using TCC mode, we recommend that ALL headless GPUs are placed in TCC mode for we have experienced stability issues when using a mix of TCC and WDM modes with headless GPUs.

REQUIRED SOFTWARE to use MyCaffe: 1.) Download and install the full version of Microsoft SQL Express 2016 (or later). NOTE: The full version of SQL Express must be installed as opposed to the light version included in Visual Studio. Microsoft SQL Express can be downloaded from https://www.microsoft.com/en-us/sql-server/sql-server-downloads

This release of the MyCaffe AI Platform and Test Applications has the following new additions: • CUDA 11.8.0.522/cuDNN 8.8.0.121/nvapi 515/driver 531.14 • Windows 11 22H2 • Windows 10 22H2, OS Build 19045.2251, SDK 10.0.19041.0 • Added new GELU Layer. • Upgraded Google ProtoBuf to 3.23.2 • Upgraded NewtonJson to 13.0.3 • Added Temporal Fusion Transformer Support • Added custom token input to TokenizedDataPairsLayer. • Added new NumericTransformationLayer. • Added new CategoricalTransformationLayer. • Added new GluLayer (Gated Linear Unit) • Added new GrnLayer (Gated Residual Network) • Added new VarSelNetLayer • Added new MultiHeadAttentionInterpLayer • Added new ReshapeTemporalLayer • Added new QuartileLossLayer • Added new CudaDnn.channel_add function • Added new CudaDnn.max function • Added new CudaDnn.min function • Added new CudaDnn.max_bwd function. • Added new CudaDnn.percentile function. • Added new RNN8 support with new optimized functions. • Added new Blob.ToByteArray • Added new Blob.FromByteArray • Added new Blob.Percentile • Added new PropertyNames property to PropertySet. • Added new PropertyBlobNames property to PropertySet. • Added support for running TestMany on a model in TRAIN, TEST or RUN phase. • Added new Blob.SaveToNumpy with array of data. • Added new NumpyFile type. • Added new Net.FindLayers function. • Added TFT Test Data Downloader to MyCaffe Test Application • Renamed GrnLayer to GlobResNormLayer

The following bug fixes are in this release: • Fixed bug in Blob memory allocation when reshaping. • Fixed regression bug, optimizing ConvolutionLayer. • Fixed bug in LayerNormLayer when used on 2 axis blob. • Fixed bug where LayerNormParameter now persisted properly. • Fixed bug in ConvolutionBackwardBias • Fixed bug in Blob.SaveToNumpy.

Easily run Temporal Fusion Transformer Models[3], Language Translation Encoder/Decoder Transformer Models[4][5], minGPT[6], Single-Shot Multi-Box Nets[7][8], import/export ONNX AI Models, run Triplet Nets[9][10], run Siamese Nets[11][12], Neural Style, train Deep Q-Learning or Policy Gradient models to beat Pong or Cart-Pole, or create the CIFAR-10 and MNIST datasets using the MyCaffe Test Application which you can download from the MyCaffe GitHub site.

Schedule distributed AI work packages, or create and train these models by following step-by-step instructions in the SignalPop Tutorials. And, to see other cool examples that show what MyCaffe can do, see the SignalPop Examples.

If you would like to visually design, develop, test and debug your models, see the SignalPop AI Designer specifically designed to enhance your MyCaffe deep learning.
Happy ‘deep’ learning!

[1] MyCaffe: A Complete C# Re-Write of Caffe with Reinforcement Learning by D. Brown, 2018.

[2] Caffe: Convolutional Architecture for Fast Feature Embedding by Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, Trevor Darrell, 2014, arXiv:1408.5093

[3] Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting by Bryan Lim, Sercan O. Arik, Nicolas Loeff, and Tomas Pfister, 2019, arXiv: 1912.09363

[4] GitHub: devjwsong/transformer-translator-pytorch by Jaewoo (Kyle) Song, 2021, GitHub

[5] Attention Is All You Need by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin, 2017, arXiv:1706.03762

[6] GitHub: karpathy/minGPT by Andrej Karpathy, 2022

[7] SSD: Single Shot MultiBox Detector by Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg, 2016.

[8] GitHub: SSD: Single Shot MultiBox Detector, by weiliu89/caffe, 2016

[9] Deep metric learning using Triplet network by Elad Hoffer and Nir Ailon, 2018, arXiv:1412.6622

[10] In Defense of the Triplet Loss for Person Re-Identification by Alexander Hermans, Lucas Beyer and Bastian Liebe, 2017, arXiv:1703.07737v2

[11] Siamese Network Training with Caffe by Berkeley Artificial Intelligence (BAIR)

[12] Siamese Neural Network for One-shot Image Recognition by G. Koch, R. Zemel and R. Salakhutdinov, ICML 2015 Deep Learning Workshop, 2015.