Well, that's kind of my point: if knowledge is only that which can be definitely proven, then most of what we keep in our heads isn't knowledge, it's experience, guesswork and intuition. I can't prove what caused the English Civil War, for example, I just have access to a number of accounts which offer varying degrees of consistency. Even more of a problem becomes something like, for example, France, which exists only insofar as a bunch of people pretend that it does. How could we possibly prove that France exists, when its existence consists in an historically specific game of make-believe? Does that mean that France doesn't exist? I see no easy resolution within this epistemological framework.
Of course, perhaps that's all we can ask for: perhaps "knowledge" outside of certain hard scientific fields is mere pretence. But that seems like exactly the sort of thing you wanted to avoid.