I went to my first Mensa Japan meetup a few months ago. It was an interesting experience. Not only was I the only black guy, I was the only non-Asian person (I think it was all Japanese natives but it is possible there were Korean or Chinese people there). I got a few surprised looks, sure, but otherwise the people were great. I met company presidents, software engineers, doctors, artists, and various other interesting people. It was about 90% men. The few ladies I spoke to were very friendly though.
The experience reinforced my sense that Japan is generally a very welcoming country for foreigners. I think this is largely because there are so few of them. With a comfortable 99% majority, native Japanese in their day to day lives do not have to think much about gaijin. Because it is such a safe and wealthy country, foreigners largely go out of their way to adopt Japanese norms. Most foreigners do not demand that Japan change itself to accommodate them. There are not enough foreigners to effectively agitate for such changes anyway.
Tatemae and Honne
However this does not mean that most Japanese people have zero racial bias. The thing is, race is considered a taboo subject, so people will not speak about it much openly. There is in Japan the concept of honne (本音） and tatemae (建前）. The former is people's honest opinion while the latter is a person's 'front' - their polite, politically correct view. On delicate social subjects, most of the time Japanese people will only give you their tatemae. This is why Youtube videos where they go around asking Japanese people about race are kind of silly. Very few Japanese people are willing to criticize other races on camera.
Learn their language fluently, make some close friends, and get drunk with a few. Then you can get their honne about other races. In particular the negative feeling toward other Asians will become clear. Even long before World War 2, there had been generations of bad blood between Chinese, Koreans, Japanese, and other Southeast Asians. Simply put, Japanese people do not consider these other Asians to be the same race as themselves. While there is a feeling of 'akogare' (admiration) toward whites and Europeans, the attitude toward Africans and Hispanics is not so charitable.
Tourists do not notice the casual racism in Japan because the people are so polite. They don't notice stores with 'Japanese Only' signs. They don't go through trying to rent an apartment and being told that their prospective landlord does not accept Africans or Brazilians. Since they don't know the language, they never understand the comments made about them on Subways by old ladies and salarymen.
My sense is that the average Japanese person does indeed harbor various racial biases. This does not bother me though for a few reasons. For one, the vast majority of Japanese people are great about treating everyone as individuals. It is rare that someone will openly insult or mistreat you merely because of your race, especially once they see you speak their language. Secondly, I did not move to Japan with any expectation that the people held the same American liberal egalitarian view of race. The thing is, if we define racism to mean simply "a belief in meaningful differences between ethnic groups," then racism is actually the default position for most of humanity. The wiser cultures (including Japan) are just better about not letting these beliefs taint their interactions with individuals of other races.
White westerners often forget that most of the world is racist. Go to Asia, the Middle East, South America, or Africa, and most of the people you meet will have racist opinions. It is only the majority white societies - Europe, Canada, Australia, USA - that have aggressively disavowed racism. And even among the white western nations I would argue that a large percentage of people, deep down, hold racist views. They cannot help it. Reality simply offers too much evidence to support them. The western world spent a lot of energy condemning racism, but never got around to debunking it. If we want racism to go the way of phrenology or geocentrism, we need to prove that it is false. Unfortunately, there is a lot of scientific evidence suggesting that it is not false. What's more, Brexit and Trump, are signs that the progressive narrative on race is breaking down, as whites are less and less scared of being branded racist.
So ultimately, yes, I think Japan is in some ways a racist society. You need look no further than their immigration policies for evidence. Sure, they admit lot of tourists and are making it easier to apply for permanent residency. However if you look at the data on the number of foreigners granted citizenship or long-term residency, it becomes obvious that Japan could take in far more people if they wanted to do so.
The fact is that Japan is one of the world's most peaceful and prosperous societies. Many tens of millions of people around the world would move there in a heartbeat were the borders truly open. This would inevitably result in the native population becoming gradually displaced, as we already see in many nations around the world. Even with the low birthrate Japan has still not opted for mass immigration as a solution for longterm stability (though this could gradually change as Prime Minister Abe is clearly trying to attract more skilled workers for certain industries).
Ultimately I respect Japan's right to maintain its borders and preserve its culture. After all, I moved to Japan because it is Japanese. If I wanted a multicultural society, I would have stayed in New York. I accept that as an immigrant, I am an outsider. I will never be fully accepted as a Japanese, even if I master the culture and become a citizen.
Furthermore, I have suffered discrimination, and I do not downplay the experience. I have been turned away from small quaint nightspots with a, "Sorry, no African please. Only Japanese. Try Roppongi." And it does sting. "Oh, you are affluent, smart, and speak our language fluently? Fuck off. You're still just a nigger." It genuinely hurts.
Yet still I am libertarian enough to respect the right of voluntary association. I think many libertarians want to believe that giving businesses the right to discriminate is not a problem because the free market will punish them. However Japan shows that this is not true. Here, the free market often rewards discrimination. Japanese people often prefer going to places where they don't have to deal with foreigners.
And again, I'm fine with that. I accept that Japanese people put their own culture and people first. They have every right to do so. If that is what is necessary to preserve the things that drew me here in the first place, then I am willing to tolerate it.