After searching through google help forum, many people talk that it's a bug. Some of their suggestion is using some kind of google chrome extension to reverse back google reader into its old UI while others said to choose another RSS feed service :D. Further reading, then I found a simple solution to get your feed or label accessible to public. So far I tried and it works.
To resolve the public feed issue in the new google reader interface, you can simply create a bundle. For example I use a label called MONITOR
Select your label, then choose create bundle. In the next page you can add or remove the RSS feed. Click save to create a new bundle.
Here is the bundle I created, you can see "add a link" there, that link will work like a public feed, which is accessible to others :D
I've tried that and it work fine..... :D