If I understand right, the hard part is purifying the radioactive material. Even if you have access to a uranium mine, there's a lot of work to filter the U-235 from the U-238 or to breed it into plutonium.
It's even harder if you start with other sources. But if you could figure out filtering it, a cubic kilometer of sea water should be enough for a bomb.
US government is very interested in any kind of uranium mining, processing, enrichment or plutonium breading. For example in 1944 US wanted to control world-wide uranium mining.
"The NSG was founded in response to India's first nuclear weapon test in May 1974. It first met in November 1975. The test demonstrated that certain non-weapons specific nuclear technology could be readily turned to weapons development."
Uranium is not even that rare, it's just that when chemistry fails at separating atoms, you have to use physics, and 3 ~proton~ (EDIT: neutron) masses is very little to work with
It's even harder if you start with other sources. But if you could figure out filtering it, a cubic kilometer of sea water should be enough for a bomb.