Deleting a file and replacing it with a new version while retaining the same filename does not work, as the new version would automatically get a prefix (_0) attached to it each time a new version is uploaded (for example: filename_0.pdf; filename_1.pdf, etc.).
Use the Media Entity Download module. It allows linking content to media/{media_id}/download no matter the filename.
Caveats:
There are two community modules which add this ability to Drupal. Either can be used:
The major caveat with this approach is that the outdated versions of the files can be stuck in one of several places Varnish, Acquia Edge, Platform CDN, or even an external CDN. To get around this one of the following modules can be used in conjunction with either of the above:
The need for either of these approaches is usually only necessary on sites which may have a large number of files managed by a potentially large number of users. In this case, you may want to explore using one or both of the following modules as well. They have good track records of making files in Drupal easier to manage.
Drupal core doesn't currently support replacing uploaded files with the same filename.