Deep neural networks (DNNs) demand significant amount of computation and memory access as the complexity of tasks and the number of data continue to grow. To ease the burden, resistive memory crossbar array (RCA) based neural network hardware accelerators are gaining attention due to its parallel vector-matrix multiplication capability and in-memory computing characteristics. However, there are many challenges to realize the potential of the RCA-based accelerator for large scale neural networks. In this talk, I will discuss some of the issues and introduce the solutions that my group has been working on. Based on our experiences, we believe that co-design of neural network and hardware is one of the keys to implement scalable RCA-based neural network hardware accelerator.