We propose a novel estimation of collision response between a virtual and an arbitrary-shaped real objects for virtual realities. Collision is detected by using the overlap ratio between objects. A normal vector of a collided plane is estimated by the collided region, and collision response is calculated by the motion vector of the virtual object and the normal vector. For testing of the method, a sensory game using Web camera was implemented. As the results are successful beyond expectation, this method can be effectively utilized for interactive virtual systems.