Have you ever written a commandlet ? This is my point, regardless of how you feel in PS vs Bash or Python vs C# - exposing your application logic trough commandlets is incredibly simple and powerful - much simpler than doing a CLI app in other languages even with argument parsing frameworks in python and likes, commandlets let you pipe and return/accept .NET objects.

So my comment isn't PS everything or C# everything, it's if you're using C# to write a big fat monolithic app that has multiple isolated functionalities you can expose those fairly easily with PS.