Postman founder here. Postman let's you run tests outside of Postman through the newman tool (free) [1]. Just take collections outside Postman and run them anywhere you want. Collections can have Javascript scripts embedded inside them for request chaining (transferring data from one request to another) or just writing integration tests. Tons of possibilities and we have some great new stuff coming up around this.
We just push bug fixes now for the legacy app. The roadmap of legacy apps from Google is still not clear. I would strongly suggest the packaged app way for multiple reasons (folders, in-built documentation, syncing, real-time sharing, tests, request capture).
And yes, native apps are coming soon! Atom's Electron is a strong contender. We are exploring all possible options. :)
There are many better ways to do automatic testing. I've never used Paw for that, and I don't think it competes with e.g. postman on that front, if that's what you're asking.
However, for manual testing / having it write curl commands for you / making JSON really easy to work with by hand, it's fantastic, and I've converted a few people at work to it. Developer is also very responsive and nice to work with on Twitter.
Edit: you can also write extensions in JS for it. Not quite as accessible as e.g. Atom, but still decently hackable: https://luckymarmot.com/paw/extensions/
As the sibling comment says, there are probably better tools for automated testing. However, I haven't dove too deeply into this aspect of Paw, but I know that at the very least there is a "Run All" shortcut which runs all requests you have configured. The sidebar shows which requests succeeded and which failed. There may be some way to automate this. (It apparently supports extensions, if this functionality isn't built-in.)
