Let $FG$ be a group algebra of a group $G$ over a field $F$ and ${\mathcal U}(FG)$ the unit group of $FG$. It is a classical question to determine the structure of the unit group of the group algebra of a finite group over a finite field. In this article, the structure of the unit group of the group algebra of the non-abelian group $G$ with order $21$ over any finite field of characteristic $3$ is established. We also characterize the structure of the unit group of $FA_4$ over any finite field of characteristic $3$ and the structure of the unit group of $FQ_{12}$ over any finite field of characteristic $2$, where $Q_{12}=\langle x, y; x^6=1, y^2=x^3, x^y=x^{-1} \rangle $.