Do you mean inline/embedded into the actual KB article as an icon?
I agree this would be handy, but I suspect that it would be difficult to maintain in terms of version control. For example if the document was updated you'll need to update the KB as well. This might not sound too bad but if you have 100 KB articles then it becomes a nightmare.
You can embed videos using HTML. If your video file is stored somewhere accessible as a URL then just drop this into the HTML for the KB entry - note this didn't work for me as the main article, only the KB entries underneath.
You need to add some text...<BR><BR><EMBED type=video/wmv src=http://yourvideo.wmv autostart="false">
That gives you a nice little player inside the KB entry. You have to have text in the entry or it won't save, hence the wording in the example.