For the data sampled from a low-dimensional nonlinear manifold embedded in a high-dimensional space, such as Swiss roll and S-curve, Self-Organizing Map (SOM) tends to get stuck in local minima and then yield topological defects in the final map. To avoid this problem and obtain more faithful visualization results, a variant of SOM, i.e. Dynamic Self-Organizing Map (DSOM), was presented in this paper. DSOM can dynamically increase the map size, as the training data set is expanded according to its intrinsic neighborhood structure, starting from a small neighborhood in which the data points can lie on or close to a linear patch. According to the locally Euclidean nature of the manifold, the map can be guided onto the manifold surface and then the global faithful visualization results can be achieved step by step. Experimental results show that DSOM can discover the intrinsic manifold structure of the data more faithfully than SOM. In addition, as a new manifold learning method, DSOM can obtain more concise visualization results and be less sensitive to the neighborhood size and the noise than typical manifold learning methods, such as Isometric Mapping (ISOMAP) and Locally Linear Embedding (LLE), which can also be verified by experimental results.