Since the introducing of the Froala editor and the embed.ly integration's possibility you may embed video in any post, discussion etc (any form with the Editor field). So the Video uploader was left for the direct upload and the file there can't be removed.
You are using Videos module, which is meant for uploaded videos, like your own YouTube. Inserting them via Froala effectively creates a description with a video embed. So, if the video is not uploaded then in most browsing pages we wouldn’t be able to show a preview (not possible from froala embeds as they are freeform and can be multiple).
So, just making the field non-compulsory isn’t going to fix the issue. Instead we should offer an option to either upload or embed a video and if embedded treat it as a main media file. This way we can control which video exactly is to be used for preview. And in this case froala toolbar would be better off without a video embed option.