For me, I look at the big picture of "by themselves"...
When they're old enough/able to cross the street, walk home from school, run in to the store with $5 and grab a gallon of milk, etc.
(When they can fall and scrape a knee without having a meltdown... I've seen 12 year old screamers!)
When they're both responsible AND accountable for their actions. Overall, they must understand the consequences and threats/risks around them.
These situations are cumulative examples of the responsibility & behavior required to be entrusted with ones own well being.
I think of it like this:
If I wouldn't leave him alone at home, then I wouldn't expect that he'd be better off outside minding himself and his own business.
Oh, and never alone, always with a sibling or buddy.