Can I create templates for content entities added by contrib modules?

Yes. You can use Site Studio to create templates for almost any Content entity. You'll just need to implement some hooks to make it work.

If you want to use Site Studio to create templates for Content entities that aren't part of Core, for example, Paragraphs, then you can do so by implementing some hooks. Here's a developer guide Implementing hooks for custom content entities.