We report the polarization symmetry of vertical photoluminescence from columnar InAs/GaAs quantum dots (QDs) that were formed by stacking small InAs islands directly in the growth direction. QD samples were grown with various stacking numbers using molecular beam epitaxy. We investigated the polarization dependence of the vertical photoluminescence intensity. The polarization dependence was enhanced by increasing the stacking number. We also found that the polarization direction shifted at random from the [1-10] direction both with the stacking number and the measurement position on the sample. Transmission electron microscopy observations suggested that the polarization features might be governed by problems in the growth process such as the bending of the stacking direction during the formation of QDs with a high aspect ratio. [DOI: 10.1380/ejssnt.2009.537]