Ash here from the Shopify team. Welcome to the Shopify Community!
It sounds like the problem here may be that you have a shipping zone which has Japan enabled as an eligible country for shipping. If you remove Japan entirely from your shipping settings, then customers with Japanese addresses will not be able to purchase from your store.
Head over to your Settings> Shipping, and you'll be able to see your active shipping profiles in the top section titled "Rates at checkout". If you have just one profile, click "Manage rates" to enter the profile and being editing. If you have multiple profiles, we're going to be looking at one that either shows Japan in it, or possibly Rest of the World, as that lumps in all countries you haven't specifically set up rates for.
Once you're in the "Manage rates" section, you should be able to edit the shipping zone that includes Japan, and remove Japan entirely from it. As long as Japan is not in any of your active shipping zones, then you should meet the requirements from Stripe mentioned in your post.
Let me know if you have any difficulty tracking down those rates, and feel free to send over screenshots of your shipping settings if you need assistance finding the areas I'm referring to.