Remember that dragging an application to the dock does not create a copy of it, it just creates a link. If you drag the application from the dmg to the dock and then trash the downloaded dmg file, then Transcribe! will disappear from the dock the next time the trash is emptied.
If you are updating from an earlier version of Transcribe! then you can delete the old version by dragging it from the Applications folder to the trash.
If you try to run Transcribe! from within the dmg then it will produce a warning message describing the problem. The best thing to do is to delete all copies of the dmg file and empty the trash. If the left column in Finder is showing "transcribe" then that is the dmg and you should eject it by clicking the little eject triangle alongside the name, before deleting it.
With this version of Transcribe! you may also want to download GStreamer, mainly if you want to handle video.
Note that if you have already installed GStreamer then you don't need to install it again when you update Transcribe!
Click here for information about GStreamer.