Yes, you need a SIM (Subscriber Identity Module card) for your gadget. There are two ways. You can get it in your country. You can contact your carrier of cell phone, smartphone, or tablet. They may have a good option plan for usage in Japan. Another way, you can purchase it in Japan. Here is a list of vendors you can buy at various places. You can purchase SIM at airports and downtown.
Before you buy, please check your smartphon or tablet can handle a SIM designated for Japan. Please contact to your smartphon manufacturer and check the band.
You can buy it at airport or electrical big appliance stores but you can’t expect at all internatinal airport. it is better to do some homework before departure.