Very true. Luckily the staff at my local fromagerie is very good with matching recommendations for crackers and wine. I don't do nearly as well on my own.I find getting the right cracker to put under the cheese harder than getting the cheese itself!
And sometimes I'll simply use the crust of an artisan bread.