Ok, so I figured out where the problem was (in case it is useful for anybody else). As I only ship from one location at a time, my second location was not ticked for “Fulfill online orders from this location” and that’s why it was not coming up as an option in my shipping profile settings.
So for setting up different rates/zones for different locations, all locations must be marked as fullfilling orders. Once the shipping settings for different location is done, if I wanted to untick "“Fulfill online orders from this location”, the shipping settings for that location would get deleted. The only way to keep the shipping settings is to make the location inactive. The problem with that option is that then I have to reassign the inventory in that location to the other location which I don’t want to do. So it seems, that my two options are to keep the inventory split between 2 locations and lose shipping settings. Or to keep the shipping settings and manually adjust the stock level for the location I am selling from at any given time. Not sure which one is worse.
Any solution for that?