Crack Segmentation

Tarkan Atak Kan, Nazlı As, Eray Yamaç

Earthquakes are disasters that are not known when they will occur. They cause great problems to societies on city-scale. Therefore, post-earthquake damage assessment is crucial in terms of providing social and economic recovery

Post-earthquake damage assessment may mainly be separated into two categories: detailed and rapid damage assessments to provide emergency management. Residual crack widths and directions are important parameters in damage evaluation.

Structural elements' damages are generally classified by determining the size and direction of the residual crack and evaluated whether the building poses a danger to be operational. In summary, the determination of residual crack plays a key role in detecting element damage.

True Mask Samples from the Dataset*

*same dataset used as in the study *https://github.com/khanhha/crack_segmentation#Dataset

---

After learning the general information about crack detection, it's time to learn about the data set we have.

This crack segmentation dataset contains approximately 11,200 images combined from 12 existing crack segmentation datasets. The name prefix of each image is assigned to the corresponding dataset to which the image belongs. There are also crack-free images that can be filtered with the "non-crack" pattern. All images in the dataset are resized to (224, 224).

First, let's start with the data preparation phase. Dataset is in two separate files train and test. Train and test files also contain two subfiles, images, and masks.

from keras.preprocessing.image import ImageDataGenerator
data_gen_args = dict(
                     rescale=1./255,
                     rotation_range=90,
                     width_shift_range=0.1,
                     height_shift_range=0.1,
                     zoom_range=0.2)
image_datagen = ImageDataGenerator(**data_gen_args)
mask_datagen = ImageDataGenerator(**data_gen_args)
seed = 1
image_generator = image_datagen.flow_from_directory(
    '../input/finalcrack/final crack/train/images',
    class_mode=None,
    color_mode='rgb',
    target_size=(224,224),
    batch_size=10,
    seed=seed)
mask_generator = mask_datagen.flow_from_directory(
    '../input/finalcrack/final crack/train/masks',
    class_mode=None,
    color_mode='grayscale',
    target_size=(224,224),
    batch_size=10,
    seed=seed)

train_generator = zip(image_generator, mask_generator)

There are various parameters in the data_gen_args dictionary. It is pre-prepared to apply these parameters separately to the images and mask subfiles in the training file. When taking photos from the training file with the flow_flow_directory() function, image_datagen and mask_datagen, ie the variables to which the parameters are applied, are used. The photos in the images file are taken in "RGB" format and the photos in the masks file are taken in "grayscale". Photos and masks taken in the desired format are zip-formed, and each photo is matched with its mask.

image_datagen = ImageDataGenerator(**data_gen_args)
mask_datagen = ImageDataGenerator(**data_gen_args)
seed = 1
image_generator = image_datagen.flow_from_directory(
    '../input/finalcrack/final crack/test/images',
    class_mode=None,
    color_mode='rgb',
    target_size=(224,224),
    batch_size=10,
    seed=seed)
mask_generator = mask_datagen.flow_from_directory(
    '../input/finalcrack/final crack/test/masks',
    class_mode=None,
    color_mode='grayscale',
    target_size=(224,224),
    batch_size=10,
    seed=seed)

test_generator = zip(image_generator, mask_generator)

The same operations that are done to get the training dataset are also done for the test dataset. Thus, we have created two different datasets train_generator and test_generator.

Now we can move on to the model creation part. We pre-create some functions that we will use frequently in the model. These functions are the double_conv_block function for the convolution layer and the upsample_block function for the decoder network architecture.

def double_conv_block(x,n_filters): 
	x=layers.Conv2D(n_filters,3,padding="same",activation="relu", 		 
				kernel_initializer="he_normal")(x)
	x=layers.Conv2D(n_filters,3,padding="same",activation="relu",
				  kernel_initializer="he_normal")(x)
    	return x
	
def upsample_block(x,conv_features,n_filters): 
	x=layers.Conv2DTranspose(n_filters,3,2,padding="same")(x)		
    	x=layers.concatenate([x,conv_features])
    	x=layers.Dropout(0.3)(x)
    	x=double_conv_block(x,n_filters)
    	return x

After creating the functions, we can move on to developing the U-Net model. By taking the encoder weights of the U-Net model from the VGG16 network architecture, we make the model have higher performance.

def build_unet_model():
       	
       inputs=layers.Input(shape=(224,224,3))
	   encoder = keras.applications.VGG16(include_top=False,
	   weights="imagenet",input_shape=(224,224,3),    
        input_tensor=inputs)							    	  
       
	   f1 = encoder.get_layer('block1_conv2').output
        f2 = encoder.get_layer('block2_conv2').output
        f3 = encoder.get_layer('block3_conv3').output
        f4 = encoder.get_layer('block4_conv3').output
       
        bottleneck = encoder.get_layer('block5_conv3').output

	   u6=upsample_block(bottleneck,f4,512)
        u7=upsample_block(u6,f3,256)
        u8=upsample_block(u7,f2,128)
        u9=upsample_block(u8,f1,64)
              
    	  outputs=layers.Conv2D(1,1,padding="same",activation="sigmoid")(u9)
    	  print(outputs.shape)
       
              
    	  unet_model = tf.keras.Model(inputs, outputs, name="U-Net")
	  return unet_model

First, we encode the incoming photos with the VGG16 network architecture. By performing this operation four times, we get the resulting f1, f2, f3, and f4 layers with the get_layer() function.

To decode the layers in the second part of the model, we send the f1, f2, f3, and f4 layers formed in the encode block as input to the upsample_block function and the u6, u7, u8, and u9 layers that we obtained as output in the decode block. In this function, the output occurs after Conv2dTranspose, concatenate and Dropout operations are performed.

After we build the model, we send it to the compile and fit functions so that the model starts to be trained.

unet_model = build_unet_model()
ad=tf.keras.optimizers.Adam(
    learning_rate=0.0001,
    beta_1=0.9,
    beta_2=0.999,
    epsilon=1e-07,
    amsgrad=False,
    name="Adam")
                
model = build_unet_model()
model.compile(optimizer=ad,
                  loss=dice_loss,
                  metrics=['accuracy'] )
model_history=model.fit(train_generator,steps_per_epoch=300,epochs=10,
validation_data=test_generator,validation_steps=30)
model.save('unet')

After training the model, the change of loss and accuracy values depending on epochs can be seen in the graphs below.

Let's apply the trained model to any crack photo.