I think a divide between value and reference types in C# is inevitable. There are semantic differences between value and reference types in C# that I don't think you can elide in a way that's both reasonable for the tooling author and conceptually clear to the end user.
It's mostly an issue with poor escape analysis and the tendency to use heap allocation for data that could go on the stack. This isn't a requirement of the language/runtime though (I believe) so this could be improved, but maybe some property or feature of the language is what makes escape analysis hard?
I think it's more than that, though, much as it is in something like C++. A struct in C# can be considered a class in C++ with no virtual methods (where interfaces are handled either through static linkage in a local scope or via boxing, I forget the exact method), and this does have different implications with regard to dereferencing and that sort of thing. Think about an array of structs, which is represented internally as an array of n * sizeof(SomeStruct) versus an array of classes (which is an array of references).
(n.b.: when I write C#, it's for game development, and I am constantly thinking about structs versus classes and I'm very careful about them, so my perspective is probably not the same as your average C# developer.)