## Tutorial

While Caffe is made for deep networks it can likewise represent “shallow” models like logistic regression for classification. We’ll do simple logistic regression on synthetic data that we’ll generate and save to HDF5 to feed vectors to Caffe. Once that model is done, we’ll add layers to improve accuracy. That’s what Caffe is about: define a model, experiment, and then deploy.

1 | import numpy as np |

Synthesize a dataset of 10,000 4-vectors for binary classification with 2 informative features and 2 noise features.

1 | X, y = sklearn.datasets.make_classification( |

```
data, (10000, 4) (10000,)
train, (7500, 4) (7500,)
test, (2500, 4) (2500,)
```

Learn and evaluate scikit-learn’s logistic regression with stochastic gradient descent (SGD) training. Time and check the classifier’s accuracy.

1 | %%timeit |

```
Accuracy: 0.781
Accuracy: 0.781
Accuracy: 0.781
Accuracy: 0.781
1 loop, best of 3: 372 ms per loop
```

Save the dataset to HDF5 for loading in Caffe.

1 | # Write out the data to HDF5 files in a temp directory. |

Let’s define logistic regression in Caffe through Python net specification. This is a quick and natural way to define nets that sidesteps manually editing the protobuf model.

1 | from caffe import layers as L |

Now, we’ll define our “solver” which trains the network by specifying the locations of the train and test nets we defined above, as well as setting values for various parameters used for learning, display, and “snapshotting”.

1 | from caffe.proto import caffe_pb2 |

Time to learn and evaluate our Caffeinated logistic regression in Python.

1 | %%timeit |

```
Accuracy: 0.770
Accuracy: 0.770
Accuracy: 0.770
Accuracy: 0.770
1 loop, best of 3: 195 ms per loop
```

Do the same through the command line interface for detailed output on the model and solving.

1 | !./build/tools/caffe train -solver examples/hdf5_classification/logreg_solver.prototxt |

```
I0224 00:32:03.232779 655 caffe.cpp:178] Use CPU.
I0224 00:32:03.391911 655 solver.cpp:48] Initializing solver from parameters:
train_net: "examples/hdf5_classification/logreg_auto_train.prototxt"
test_net: "examples/hdf5_classification/logreg_auto_test.prototxt"
......
I0224 00:32:04.087514 655 solver.cpp:406] Test net output #0: accuracy = 0.77
I0224 00:32:04.087532 655 solver.cpp:406] Test net output #1: loss = 0.593815 (* 1 = 0.593815 loss)
I0224 00:32:04.087541 655 solver.cpp:323] Optimization Done.
I0224 00:32:04.087548 655 caffe.cpp:222] Optimization Done.
```

If you look at output or the `logreg_auto_train.prototxt`

, you’ll see that the model is simple logistic regression.

We can make it a little more advanced by introducing a non-linearity between weights that take the input and weights that give the output – now we have a two-layer network.

That network is given in `nonlinear_auto_train.prototxt`

, and that’s the only change made in `nonlinear_logreg_solver.prototxt`

which we will now use.

The final accuracy of the new network should be higher than logistic regression!

1 | from caffe import layers as L |

1 | %%timeit |

```
Accuracy: 0.838
Accuracy: 0.837
Accuracy: 0.838
Accuracy: 0.834
1 loop, best of 3: 277 ms per loop
```

Do the same through the command line interface for detailed output on the model and solving.

1 | !./build/tools/caffe train -solver examples/hdf5_classification/nonlinear_logreg_solver.prototxt |

```
I0224 00:32:05.654265 658 caffe.cpp:178] Use CPU.
I0224 00:32:05.810444 658 solver.cpp:48] Initializing solver from parameters:
train_net: "examples/hdf5_classification/nonlinear_auto_train.prototxt"
test_net: "examples/hdf5_classification/nonlinear_auto_test.prototxt"
......
I0224 00:32:06.078208 658 solver.cpp:406] Test net output #0: accuracy = 0.8388
I0224 00:32:06.078225 658 solver.cpp:406] Test net output #1: loss = 0.382042 (* 1 = 0.382042 loss)
I0224 00:32:06.078234 658 solver.cpp:323] Optimization Done.
I0224 00:32:06.078241 658 caffe.cpp:222] Optimization Done.
```

1 | # Clean up (comment this out if you want to examine the hdf5_classification/data directory). |

## Reference

## History

- 20180102: created.