We propose a neural preprocess approach for video-based gesture recognition system. Second-order neural network (SONN) and self-organizing map (SOM) are employed for extracting moving hand regions and for normalizing motion features respectively. The SONN is more robust to noise than frame difference technique. Obtained velocity feature vectors are translated into normalized feature space by the SOM with keeping their topology, and the transition of the activated node in the topological map is classified by DP matching. The topological nature of the SOM is quite suited to data normalization for the DP matching technique. Experimental results show that those neural networks effectively work on the gesture pattern recognition. The SONN shows its noise reduction ability for noisy backgrounds, and the SOM provides the robustness to spatial scaling of input images. The robustness of the SOM to spatial scaling is based on its robustness to velocity scaling.