Should I do limit breaks and awakenings on SR, R, and N units?

Yes you should. By doing limit breaks and awakenings on SR, R, and N units, you can complere record missions that counts how many times you perform them. You can get rank exp and lapis as rewards.

Since SR, R, and N are duplicated so many times, it's easy to limit break them. They also require small amount of materials for awakening compareted to high rarity units.

Note that you shouldn't use rainbow shards on them though.