I prefer to use html parsers for such problems, such as beautifulsoup in python. I used xpath in the past but the ending code wasn't that much shorter then a more verbose version based on beautifulsoup. And, for someone looking at the code, the beautiful version makes so much more sense.. Xpath also feels like a big regex expression that magically works.
I'm not saying it's not useful. Actually, I believe that if you only have one use-case, then using xpath might be overkill because of all the added-complexity of maintaining a new library/technology/ideology. But if it's the sort of domain that xpath would be useful more than once, then sure use it.
Oh no, you've used the performance argument against me ;-) Obviously, when performance issues are on the line, you often need to trade simplicity and maintainability.
Or you could use lxml in Python, which allows you to mix XPath and CSS Selectors. E.g.:
from lxml import html
doc = html.fromstring('<html><body><p class="text"></p></body></html>')
doc.xpath('//p[@class = "text"]') == doc.cssselect('p.text')
# or
doc = html.fromstring('<html><body><p>1</p><p>2</p><p>3</p></body></html>')
doc.xpath('//p[2]')[0] == doc.cssselect('p')[1]
# Note: My only annoyance is that .xpath() always returns a list, even
# when you know that it will return only a single item.
I'm not saying it's not useful. Actually, I believe that if you only have one use-case, then using xpath might be overkill because of all the added-complexity of maintaining a new library/technology/ideology. But if it's the sort of domain that xpath would be useful more than once, then sure use it.