I'll tag this thread for Staff assistance. Please subscribe to the thread so you are notified when they respond and please be patient while waiting.
If you cancel a domain, it is held by the domain registrar and may be sold in auction or otherwise made unavailable to you.
If you wish to use it with a different hosting platform, in most cases all you need to do is update the nameservers http://en.support.wordpress.com/domains/domain-management/ For a few platforms they require a bit more assistance (A Records/CNAMEs), so if that's the case in your situation, just let Staff know what needs to be updated and we'll take care of that for you.
You can also transfer the domain to a different registrar. There is no cost to do so but the new registrar will likely require you to renew it for at least one year with them.
Provided you are logged in under the same username account that purchased the upgrades, you ought to be able to request a refund from your dashboard > Store > My Upgrades
WordPress.com provides a 30-day refund on all upgrades except Domain Registrations, Domain Renewals, http://en.support.wordpress.com/domain-mapping/cancel-domain/ and Guided Transfers.
Note: It takes from 1 - 2 weeks for the refund to be received.