Little bit of crying and being sad will not harm your baby. However your baby is too small right now, you need to still take care of her/him.
Mental wellbeing is very important for you and your baby during entire pregnancy. Please don't wait for more than 2 weeks if this feeling continues. Consult an expert for this. If it is clinical depression you would need proper medicines to come out of this horrible phase.