Google has added a new feature and you can now add the controlList attribute.
In Chrome 55, prevent showing Download button for HTML 5 video - stackoverflow.com
<video width="512" height="380" controls controlsList="nodownload">
<source data-src="mov_bbb.ogg" type="video/mp4">
</video>