The last time we used a single hidden layer neural network, the results were promising, but this time we've taken a step further by implementing a Convolutional Neural Network (CNN). CNNs have become the backbone of modern computer vision systems and are especially effective for tasks like facial keypoint detection.
Below is an illustration of a convolutional neural network, showcasing its training process and data expansion techniques:
The image above demonstrates how convolution operations work within the network. The LeNet-5 architecture, one of the earliest CNN models, revolutionized the field by introducing three key concepts that help reduce the number of parameters while maintaining performance:
1. **Local Connectivity**: Neurons in a convolutional layer only connect to a small region of the previous layer, capturing local patterns.
2. **Weight Sharing**: The same set of weights (or filters) is applied across the entire input, reducing the total number of parameters.
3. **Pooling**: This step reduces spatial dimensions, making the model more efficient and robust to small translations in the input.
To better understand locality and weight sharing, imagine that each neuron in a convolutional layer is responsible for detecting specific features, such as edges or textures, within a small patch of the image. These patches are processed using shared weights, creating feature maps that capture relevant visual patterns.
When working with Lasagne, preparing the input becomes more complex than just flattening pixel values. Instead, the input should be structured as a 3D tensor in the form (channel, height, width). For grayscale images, this means a shape of (1, 96, 96), where 1 represents the single color channel.
To convert 2D data into this format, we use a helper function called `load2d`, which reshapes the input accordingly.
We designed a CNN with three convolutional layers followed by two fully connected layers. Each convolutional layer is paired with a max-pooling layer to reduce spatial dimensions. The number of filters doubles after each convolutional layer, starting with 32 filters in the first layer. The fully connected layers contain 500 neurons each.
Interestingly, even without explicit regularization techniques like dropout or L2 penalties, the small filter sizes (e.g., 3x3 or 2x2) naturally act as a form of regularization, helping prevent overfitting.
Here's the code for the network:
```python
Net2 = NeuralNet(
layers=[
('input', layers.InputLayer),
('conv1', layers.Conv2DLayer),
('pool1', layers.MaxPool2DLayer),
('conv2', layers.Conv2DLayer),
('pool2', layers.MaxPool2DLayer),
('conv3', layers.Conv2DLayer),
('pool3', layers.MaxPool2DLayer),
('hidden4', layers.DenseLayer),
('hidden5', layers.DenseLayer),
('output', layers.DenseLayer),
],
input_shape=(None, 1, 96, 96),
conv1_num_filters=32, conv1_filter_size=(3, 3), pool1_pool_size=(2, 2),
conv2_num_filters=64, conv2_filter_size=(2, 2), pool2_pool_size=(2, 2),
conv3_num_filters=128, conv3_filter_size=(2, 2), pool3_pool_size=(2, 2),
hidden4_num_units=500, hidden5_num_units=500,
output_num_units=30, output_nonlinearity=None,
update_learning_rate=0.01,
update_momentum=0.9,
regression=True,
max_epochs=1000,
verbose=1
)
```
After loading the data with `load2d()`, we trained the model:
```python
X, y = load2d()
Net2.fit(X, y)
```
Training this network is significantly more resource-intensive than the previous one. Each iteration is about 15 times slower, and running 1000 epochs can take over 20 minutes, even with a powerful GPU. However, the improvements in accuracy are well worth the wait.
Once the model was trained, we saved it using `pickle` for future use:
```python
import cPickle as pickle
with open('net2.pickle', 'wb') as f:
pickle.dump(net2, f, -1)
```
Looking at the output during training, we see that the model’s loss decreases steadily. The final RMSE (Root Mean Squared Error) was around 1.9, which is a significant improvement over the previous model.
For example:
```python
>>> np.sqrt(0.001566) * 48
1.8994904579913006
```
This shows that the new CNN-based model performs much better in detecting facial key points, demonstrating the power of deep learning in computer vision tasks.
Fiber Optic Cleaver
The fiber cutter is used to cut fiber as thin as hair. After hundreds of times of amplification, the cut fiber is observed to be flat before discharging the fuse.
The material of fiber is quartz, so the material of fiber cutting knife blade is required.
Adaptive fiber: single or multi-core quartz naked fiber;
Suitable for fiber cladding :100-250um diameter.
Fiber Optic Cleaver, Optical Fiber Cleaver, Fiber Optic Cutter, Hardware Networking Tools
NINGBO YULIANG TELECOM MUNICATIONS EQUIPMENT CO.,LTD. , https://www.yltelecom.com