We investigated the photospheric and magnetic features accompanied by the Ellerman bombs (EBs) using the 1.6m New Solar Telescope at Big Bear Solar Observatory. On eight out of ten EBs we observed, we found unusual photospheric features of transverse motion with speeds of about 3.5 km/s. We also found upflows associated with the photospheric features. The photospheric features may be identified as newly emerging flux tubes or moving magnetic features. On the leading edge of the photospheric transverse moving features, we found the brightenings in TiO broadband filter images as well as far wings of Ha/Ca II lines that correspond to about 200K brightness temperature increase. We discuss the physical implication of our findings in the context of the magnetic reconnection associated with the EBs.