> Representing Chinese (no ASCII, many character points high up in the set) in utf-8, you begin wasting a lot of memory due to the ASCII compatibility.
That's not "a lot", that's only 33% more in the best case of purely CJK text.
OTOH any non-CJK characters are 50% smaller. It starts to even out if you add few foreign names or URLs to the text.
And for HTML, UTF-16 is just crazy. Makes HTML markup twice as expensive.
CJK websites don't use UTF-16, they use Shift-JIS or GBK, which are technically more like UTF-8.
How exactly does UTF-16 make HTML twice as expensive?
I guarantee you that there are very few apps that will double their memory usage if you start using UTF-16 text. Even if you start looking at bandwidth once you compress the text there is very little difference. (You are compressing your HTML right?)
The case for UTF-8 saving memory makes a lot of sense if you're writing embedded software, however in most stacks the amount of memory wasted by UTF-16 is trivial compared to the amount of memory wasted by the GC, bloated libraries, interpreted code, etc.
If you're using .NET or the JVM char is 16 bits wide anyway. The UTF-8 vs. UTF-16 debate is a perfect example of mircobenchmarking where theoretically there is a great case for saving a resource but in aggregate makes very little difference.
That's not "a lot", that's only 33% more in the best case of purely CJK text.
OTOH any non-CJK characters are 50% smaller. It starts to even out if you add few foreign names or URLs to the text.
And for HTML, UTF-16 is just crazy. Makes HTML markup twice as expensive.
CJK websites don't use UTF-16, they use Shift-JIS or GBK, which are technically more like UTF-8.