Hygro, your entire idea rests on the assumption that, if people are told "you are responsible for <this> area of government", then they will be both willing and able to educate themselves to a greater degree than the current crop of politicians would. Do you have any evidence for that? Or is it just a hunch? I mean, it sounds plausible, but it sounds a lot more plausible that this would end in tears. I admire your seemingly bottomless faith in humanity, that they would take time out of their busy lives -- work, family, socialising, relaxing, sleep -- to educate themselves on a topic that they have literally no prior interest in and have been forced to "take ownership of" by the government. Personally, I think it requires quite a leap of faith to believe that people would actually become better educated in, say, agricultural policy or fisheries or some other boringass government department that nobody has heretofore given two hoots about.
Secondly, you mentioned that even if this system was only a little bit better than the existing one, you would count that as a win. But this sort of marginal, incremental improvement is much more easily achieved through marginal, incremental improvements to the current system. Why bother scrapping the entire system for merely "better than now", when we can make small, easy changes to the way the current system works, which would still achieve "better than now" results? Not only is the risk/reward of your plan rather unfavourable, it is also far less realistic or achievable than marginal, incremental improvements to the current system.
Thirdly, what would happen to e.g. the civil service? What role would they play in the process? At the moment, the civil service often acts as a brake on "top down" change, by saying to government ministers, "err, actually, we can't just do that, because XYZ depends on that and so we'd have to change XYZ too". They provide input from the operational side, and also remind government of why things are the way they are. I'm not a conservative, risk averse guy, but that's exactly why I value these sort of people and this sort of advice. How would this interaction happen if a random selection of people from all over the country are in charge of voting on these decisions?
Finally, do you believe in leadership?