Is there a way to restrict to show the file details for some of the Community users

Is there a way to restrict to show the file details for some of the Community users

As You are using the xfilespro for community users, Default External Access for External Content Detail object should be public read/write. But if you want to expose ECD for few profiles or users from community then it can be allowed those and restrict others through Sharing-Settings rule.