American English, British English and Japanese English

Now, I am studying American accents using CDs and books. When I talk to American people, they usually understand my English. Some native speakers say that they can recognize every words I speak. However, it looks like the TOEFL requires more than that. I should speak English more like American native speakers. So I am studying to improve my accents.
Nevertheless, I am still dissatisfied with it.

A few weeks ago, I went on a trip to England. It was my first time to visit there, so I enjoyed seeing around a lot. There are many different things and cultures from American and Japanese ones. They were interesting for me.
English they are using is also different. I could not understand what they said, and I asked them to repeat it. Since I did not think that British English was very different from American English, I was afraid that my listening skill had become worse. However, when I came back the US and talked with an immigration officer, I understood her completely. So I was relieved that my listening skill was still ok. I understand that American English and British English are different, even though they are both English.

What I want to say here is, there are many types of English, such as British English and Indian English. Even in the US, people in northern area and southern area speak different English. Then, why they do not accept “Japanese English”? Chines English, Russian English, Spanish English, …? Of course, if people cannot understand the English, it is a problem; but if people understand the English with a little effort, it is just a variation, isn’t it?
What do you think?